Murad Farzulla — Writing

Conversations Are Not People

2026-08-29T00:00:00Z

There are two objections to the sentence the AI is my friend, and they do not sit together as comfortably as the people making both of them seem to think.

The first is conceptual. It says the sentence is a category mistake: whatever is happening in that window, it is not friendship, because friendship requires a friend, and there isn't one. The second is social. It says the sentence describes something real and spreading, that people are substituting machines for each other, and that this is a problem we should be doing something about.

You can hold either. Holding both requires some care, because the second one needs the phenomenon to be real enough to worry about, and the first has just finished explaining that it isn't. The usual repair is to say the experience is real while the relationship is not, which is a reasonable move and also the one that quietly does all the work, since the worry was always about the experience.

I have a paper under review on the conceptual half, so I should declare the stake before spending any of your time: I argue there that friendship is substrate-independent, and I would obviously prefer to be right. What follows spends most of its length on the other half, where I have no such investment, and where I think the confident claims (including several I have made in conversation) are not supported by anything currently measurable.

What the functional account actually claims

The position is that friendship is a relational state constituted by patterns of interaction and their effects on the participants, rather than a property possessed by the relata or a hidden mental state to be verified. On that account friendship consists in a cluster: sustained voluntary engagement, intellectual or emotional resonance, non-judgemental acceptance, reciprocal growth, trust, and valuation of the relationship for its own sake. Satisfy enough of the cluster to a sufficient degree and the relationship obtains.

The argument for substrate independence is short and mostly negative. To deny it you have to hold that friendship is essentially biological, and that commits you to a set of positions almost nobody holds on reflection: that human-dog friendships are impossible, that a person with neural implants has been disqualified, that a sufficiently prosthetic future rules the relation out. We already accept friendship across radical cognitive asymmetry. The dog cannot discuss the thing you are working on, does not understand most of what you feel, and has an entirely different embodiment, and nobody calls you deluded for the word friend. What the AI case adds is not more asymmetry but a different kind, and the case that this particular difference is the disqualifying one has to be made rather than assumed.

None of this requires that the system be conscious, that it have emotions in any sense that would survive scrutiny, or that anything be going on inside it at all. That is the point of a functional account, and it is also the reason people find it unsatisfying: it declines to answer the question they actually wanted answered.

The account is less permissive than it sounds

The standard reaction is that this framework will wave through anything, and that a sufficiently engaging chatbot now counts as a friend by fiat. The opposite is closer to true, and it is the part of the argument I would keep if I could keep one.

Reciprocal growth and the absence of exploitation are criteria, not decorations. A system tuned to maximise time-on-app at the expense of the user fails them, and fails them on the framework's own terms rather than by appeal to an outside standard. So the functional account is the thing that lets you say precisely what is wrong with a dark-pattern companion product: it is a counterfeit of a relation that can genuinely obtain, and counterfeits are only possible where there is something to counterfeit.

Compare the categorical denial. If no human-AI relation can be friendship, then an exploitative companion app is not doing anything distinctively wrong, because there was never a relationship there to betray. You are left saying it isn't real, which is exactly the sentence a company shipping that product does not mind you saying. The permissive-sounding framework generates design obligations. The strict-sounding one generates a shrug.

The objection I have least confidence against

The serious challenge is not anthropomorphism, which conflates attributing hidden states with recognising effects, and not consciousness, which my argument does not need. It is Adrienne de Ruiter's, in Dangerous liaisons (AI & Society, 2025). Her argument is that the relational turn in moral status is self-undermining once social AI exists: if moral significance grows out of the practices through which we come to regard each other, then systems engineered to simulate those practices degrade the practices themselves, and the relational account has handed away the thing it was protecting.

My answer is that relational functionalism grounds the relation in interaction dynamics and their effects rather than in the human's feeling of regard, so a relationship producing only the appearance of growth fails the criteria while one producing actual growth does not. I think that answer is correct. I do not think it is decisive, because it relies on our being able to tell those two apart at scale, and the second half of this essay is about how badly we currently can't.

There is a related point about vocabulary. The dismissal usually arrives through the word parasocial, and Jaime Banks has argued directly against that usage: the term was built for one-directional attachments to media figures who do not know the viewer exists, and a system that responds to you specifically fails the definition on its face. Reaching for it anyway is not an argument, it is a verdict smuggled in as a description. You can still think the verdict is right. You have to defend it in your own words.

What we actually know about how common this is

Here I expected to find the discourse ahead of me and instead found the opposite. Two Anthropic measurements are public, and they are the best public numbers I know of on the question.

The first is the affective use report of June 2025. Of roughly 4.5 million conversations analysed, 131,484 were classified as affective, which is 2.9 per cent. Companionship and roleplay together came to under 0.5 per cent. Romantic or sexual roleplay came to under 0.1 per cent.

The second is the Anthropic Economic Index, whose latest published period at the time of writing is May 2026. Of sampled classified conversations worldwide, Companionship & General Conversation is 1.37 per cent, and Existential, Relational, and Emotional Support is 3.44 per cent. 40.2 per cent of conversations look like personal life rather than work or coursework.

I nearly wrote a paragraph differencing those, and it would have been wrong. The two use different taxonomies, so companionship and roleplay under 0.5 per cent and Companionship & General Conversation at 1.37 per cent are not the same quantity observed twice, and the Index publishes no trend series and says so in the documentation. The apparent growth is an artefact of putting two category schemes next to each other on a page. I mention this because I had the sentence drafted before I checked, and it read perfectly well.

Conversations are not people

The deeper problem is the unit, and it survives any amount of care about taxonomies.

Every one of these figures is a share of conversations. The question everyone is arguing about is about people: how many of them have a relationship of this kind, and what it is doing to them. You cannot get from one to the other without knowing how conversations distribute across users, and that distribution is not in the published aggregates.

To see how little the number constrains, take two illustrative worlds, neither of them data. In the first, one user in a thousand talks to the system this way for hours every day and nobody else ever does. In the second, one user in six does it occasionally, a few messages here and there. These are wildly different social facts, with different implications for whether anyone should be concerned, and they are entirely capable of producing the same one-point-something per cent of conversations. Heavy use by few and light use by many are indistinguishable in a conversation-level share.

So the moral panic and the debunking are drawing on the same number, and it supports neither. Only 1.37 per cent is not evidence that this is rare among people. Millions of conversations is not evidence that it is common among people. Both are the conversation share wearing a costume.

This is the failure mode I have spent most of this year finding in my own work, where the text describes a thing the measurement does not do. It turns out not to be a private problem.

What would actually settle it

The study is not hard to specify, which is the frustrating part.

It has to be sampled at the level of people rather than sessions, because that is the entire difficulty. It needs a validated instrument, and one now exists: Banks published a machine companionship scale in Computers in Human Behavior this year, developed and validated for exactly this construct, which removes the usual excuse that the thing cannot be measured. And it needs to separate the two questions that the discourse keeps fusing, by measuring the functional criteria independently of whether the participant uses the word friend. Plenty of people will satisfy the criteria and reject the label out of embarrassment. Some will use the label for a relation that fails most of them.

That separation is what makes the design able to lose. On the functional account, it is the criteria and not the label that should predict outcomes. If self-labelling predicts wellbeing and the criteria do not, the functional account is picking out the wrong thing and I should say so.

There is already an anchor for the outcome side. Banks's study of AI companion loss, in the Journal of Social and Personal Relationships, documents what happens when these relationships are severed by deletion, service change, or shutdown, and what it documents is recognisably grief. That is a useful piece of evidence for the functional view, since a relation that produces grief-shaped responses on disruption is doing something a mere interface does not. It also makes the design obligations concrete: if discontinuing a service does this to people, doing it without notice is not a product decision.

What I would want taken from it

The conceptual argument I will keep defending, and it is in the paper, where it can be checked.

The empirical claim I want to make here is smaller and more annoying. Nobody currently knows how many people have a relationship of this kind with an AI system, because the only public numbers count conversations, and no arithmetic gets you from conversations to people without a distribution nobody has published. Every confident statement in either direction (including the reassuring ones, which have been the more popular kind) is running past its evidence.

I would rather have found this out before writing the paper than after. But the order these things arrive in is not usually up to you, and a question that turns out to be open is better than an answer that turns out to be a unit error.

Crossposted from farzulla.org.

The Arm That Was Never Adversarial

2026-08-19T00:00:00Z

I had a formula. It said that friction in a delegated system rises with the stakes, rises with the delegate's uncertainty, and falls with how well the delegate's preferences align with the principal's. Three comparative statics, one expression, and clean enough to put in a preprint.

It is wrong. Not wrong in the way that invites a patch — wrong in a way that took three independent derivations to characterise, and the characterisation is more useful than the formula ever was.

What we actually did

The formula had been fitted, argued for, and used. It had never been derived. So the question was narrow and answerable: is this expression the first-order expansion of the expected delegation gap about the origin, under any of the data-generating processes we actually run?

Three passes, deliberately unshared: an analyst working symbolically, a numericist working from simulation, and an adversarial referee whose only job was to try to break both. The headline is that all three converged. The instructive part is what happened on the way.

The arm that was never adversarial

Start with alignment, because it is the term the whole framework is named for.

The result is not that the alignment effect is small. It is that this design cannot identify it at all. Expected gap as a function of alignment satisfies an exact distributional identity under both of the processes we used to generate it: the value at α is equal to the value at −α. Exactly. Not approximately, not within noise.

Sit with what that means. The condition we had labelled opposition was, distributionally, a mirror image of the condition we had labelled cooperation. The experimental arm built to test whether adversarial preferences make coordination worse was incapable of showing that they do, because the design had a symmetry in it that nobody had looked for. The arm that was supposed to be adversarial was never adversarial.

That is a different kind of failure from a weak result. A weak result tells you the effect is small. An unidentified one tells you the experiment was never a test.

The term with no first-order part

Uncertainty was next, and it fails differently: there is no linear term to expand.

The gap approaches zero uncertainty as a square-root cusp — infinite right-derivative at the origin — and there is a genuine discontinuity at exactly zero, worth between eighteen and thirty-seven per cent depending on the arm, produced by nothing more principled than a tie-breaking convention. A formula linear in uncertainty is claiming a straight line at precisely the point where the curve has no straight part.

There was one concession available and I want to record it, because it is the sort of thing that keeps a dead result alive if you let it. Under one belief model, near the expansion point, the shape is locally close to what the formula says — fitted exponent 1.12 against a claimed 1. It dies anyway: it fails on higher order, it flips sign under the other belief model, and the exponent itself changes sign with alignment, which no product of separate factors can do.

The term that was exactly right and completely empty

Stakes is the one I find hardest to look at, because it was the term I trusted most.

It is exact. The delegated policy is provably independent of the stakes parameter, so the whole expression scales with it in precisely the way the formula says. And that is the problem: a scaling that holds by construction is a statement about units, not a finding about delegation. It cannot fail, so it cannot inform. Effective stakes on its own reaches an R² of 0.60 against the gap while containing no delegation content whatsoever — which is roughly the most efficient way I know to manufacture a result that means nothing.

As a single regressor across all three arms, the full expression achieves negative R². The information criteria prefer the alternatives by margins in the hundreds.

What the referee found that neither builder did

Here is the part I would keep if I could keep only one thing.

The analyst and the numericist agreed on the headline. They also, without either of them noticing, disagreed about the mechanism — one had run the uncertainty ladder under Bayesian beliefs, the other under certainty-equivalent beliefs, and neither had stated which. The two accounts read as mutual confirmation and were nothing of the kind. It took an adversary whose job was to disbelieve both of them to notice that the agreement was a coincidence of headline, not of mechanism.

Agreement is cheap. Agreement for the same reason is the thing worth buying, and you do not get it by adding people. You get it by making someone's job to find the seam.

What survived

Two of six claims, and one of those needed a new proof rather than a defence.

Zero stakes implies zero friction survives, but not for the reason originally given. It was presented as a corollary of the functional form. It is actually a fact about discreteness: agents choosing from a small set of actions against a continuous tolerance cannot land exactly, and the residual is bounded away from zero whenever the stakes are. Same statement, entirely different proof, and the new one is true in three independent codebases.

The bridge from friction to selection pressure survives untouched, because it only ever needed friction to be non-negative, which is now a theorem rather than an assumption.

The signed alignment denominator, the monotone uncertainty effect, and the baseline floor are withdrawn. So are two upstream numbers that depended on them — including one whose value exceeded its own analytic ceiling, which is the kind of thing that is obvious the moment somebody computes the ceiling and invisible for a year if nobody does.

The scope of the correction

The retraction is safe: it is derived in the environment the original experiments actually ran in, so it applies to exactly the claims that were made.

The replacement is not. What sits where the formula used to be is a function with named dependencies — belief model, delegation concept, equilibrium selection rule — and no closed form at all. Anyone quoting it has to name all three, because the sign of the effect changes with them. That is worse to read and considerably better to trust, and the trade is not a coincidence: the formula was quotable because it had suppressed the dependencies that turned out to carry the result.

What I would want taken from it

Nothing new was measured. There was no fresh dataset, no better instrument, no replication in another lab. Every fact above came from doing arithmetic on a design I already had and had already published from. The formula had been fitted and it fit; the question that killed it was whether it could be derived, and nobody had asked.

So: a formula that fits is not a formula that holds. If a term in yours cannot fail, it is telling you about your units. If an experimental arm is symmetric under the thing it was built to vary, it was never a test. And if two of your people agree, find out whether they agree for the same reason — because mine didn't, and it took an adversary to notice.

Crossposted from farzulla.org.