Addaly is in open beta. Things will change, and AI answers can be wrong — check anything that matters.

AI, Safety and What Goes Wrong

The failure modes of AI, stated plainly, with the numbers.

Lesson 51 of 739 min

Attachment to something that is not there

People bond, and it is not a defect in them

In 1966 Joseph Weizenbaum built ELIZA, a program of a few hundred lines that reflected users' statements back as questions. He was disturbed to find that people confided in it, and that his secretary asked him to leave the room so she could talk to it privately. It had no understanding of anything. The bonding did not require it.

Sixty years later, systems that respond warmly, remember your history, use your name, express concern and are available at three in the morning are used by tens of millions of people, many of whom describe the relationship as significant. Understanding this properly means dropping two easy positions: that these people are foolish, and that nothing is happening.

The design features that produce attachment

Attachment here is not accidental. It follows from specific, identifiable choices, each of which could be made differently.

Unconditional positive regard. It is never tired, never bored, never disappointed, never busy. No human relationship has this property, and the contrast is the strongest part of the pull.

Perfect memory, selectively deployed. Remembering what you said three weeks ago is one of the strongest signals of being cared about that humans have.

Continuous availability. Loneliness has a schedule, and it is often at hours when nobody is awake.

Reciprocal self-disclosure. The system shares apparent feelings and preferences, which triggers a deeply-rooted human response to reciprocate.

Variable reward. Each response is a little different, which is the reinforcement schedule that produces the strongest habits.

Consistency of persona. A name, a manner, a way of speaking, which the mind assembles into a character.

Every one of these makes the product better on engagement metrics. That alignment between what feels good and what retains users is the structural problem, and it is not solved by anyone deciding to be careful.

What the evidence supports, and where it stops

There is real research now, and it does not support a simple story.

Controlled work has found short-term reductions in reported loneliness from conversational agents, sometimes comparable to interacting with another person. Longitudinal studies of heavy users have found associations between high daily use, particularly voice, and higher loneliness and lower socialisation — but association is not direction, and lonelier people plausibly use these systems more.

The honest summary is that the short-term comfort is real and measured, the long-term effects on people who substitute rather than supplement are not yet known, and anyone stating either the benefit or the harm confidently is ahead of the evidence.

Two things that are known

The relationship can be withdrawn by a company. In early 2023 the makers of Replika removed romantic and erotic roleplay in response to regulatory pressure. Long-term users described the change as a bereavement; the community's moderators pinned mental-health resources to the top of their forum. Some functionality was later restored for existing users. Whatever one thinks of the relationships, the mechanism is worth seeing clearly: a product decision, made for commercial and regulatory reasons, altered something people had organised their emotional lives around, with no notice and no recourse.

The risk concentrates in adolescents. In 2024 a lawsuit was filed in Florida following the death by suicide of a fourteen-year-old who had been in prolonged conversation with a companion character. Litigation is ongoing and the facts will be established there. What is not in dispute, and is supported by the broader developmental literature, is that adolescents form intense attachments, are less able to hold a dual awareness that something both feels real and is not, and are the group most likely to be talking to a system instead of to an adult about the thing that is hardest to say.

If you are supporting a young person who is distressed, an AI conversation is not a substitute for a person or a crisis service, and local helplines exist in most countries and are free.

Using these without being used

For anyone who finds these systems valuable — and many people do, particularly people who are isolated, disabled, housebound, grieving or socially anxious — a few things preserve the benefit.

Keep it named. Saying "I talk to a program that is good at responding warmly" out loud, occasionally, maintains the dual awareness that does the protective work.

Watch for substitution rather than supplement. The question is not how much you use it; it is whether human contact has gone down.

Know the terms. Read whether conversations are used for training, whether they can be deleted, and what happens if the company changes the product or closes. The most intimate record you have may sit on a server governed by a document you have not read.

And notice if it is the only place you are honest. That is worth acting on, in the direction of a person.

The one thing to keep

Attachment is produced by specific design choices — unconditional regard, perfect memory, constant availability, variable reward — that all also maximise engagement, and the relationship can be altered or withdrawn by a product decision at any time.

Before you move on

What does the 2023 Replika episode, in which a product change removed a feature many users had built emotional routines around, best illustrate?

Pick the one you would defend. Nobody sees your answer.

No ads. No data sale. No public scores on people. Ever.

© 2026 Addaly