What not to hand it

Three things a chatbot is not

It is not a confidant. A doctor, a lawyer and a priest have legal or professional duties about what you tell them. A chatbot has terms of service. In 2025 the chief executive of the largest chatbot company said in an interview that conversations with it carry no legal privilege, and that a court could require them to be produced. People, he said, tell it the most personal things and should know that.

It is not a professional. Nobody is accountable for its answer. A practitioner who gets it wrong can be struck off, sued or sacked, and that accountability is most of what you are paying for. The model’s answer is a fluent draft with no one’s name on it.

It is not a record-keeper. What you told it is stored under the previous lessons’ rules, and what it told you is not a record of anything: ask again tomorrow and get a different answer.

From those three follow the things to keep out of the box.

Identities and secrets

Your identity documents, and anyone else’s: Aadhaar, passport, PAN, driving licence. Passwords, one-time codes, card numbers, account numbers. These have no business in a system that stores text, may show it to a contractor and may train on it, and they are exactly what a scam will ask you to paste “to verify”. No legitimate task needs a document number to rewrite a paragraph.

Other people’s data

The moment you paste a colleague’s medical note, a customer list, a student’s essay with their name on it, or a patient’s history, you have disclosed someone else’s personal data to a third party. Under India’s data-protection law of 2023, an organisation that does that is responsible for it, and the same is true under the European rules and most others. Strip the names. If the task needs the names, it is not a chatbot task.

Work that is not yours to share

Unreleased code, contracts under negotiation, anything under a non-disclosure agreement, the client’s brief. The electronics company from the earlier lesson banned the tools after its engineers pasted source code. The rule is the same one you would apply to emailing the material to a stranger who promises not to look.

Medical, legal and financial: the shape of the risk

People use chatbots for these more than for anything else, and the honest advice is not “never”. It is: use it to understand and to prepare, never to decide, and know the three mechanisms that make the decision unsafe.

It invents specifics. Dosages, deadlines, section numbers, tax thresholds (the exact things a decision turns on) are the exact things the hallucination module told you it fabricates. In 2023 two New York lawyers filed a brief containing six cases the chatbot had made up, complete with quotations, and were fined by the court. The cases sounded exactly like cases.

It does not know where you are. Law, tax, drug approvals and medical guidelines differ by country and by state, and the training text is weighted toward the United States. An answer that is correct in Texas can be wrong in Tamil Nadu, delivered with the same confidence.

Its knowledge stops. Rules change; the cut-off lesson explained that the model’s do not, and a search tool bolted on helps only if it finds the current rule and the model reads it correctly.

So: ask it to explain the terms in your blood-test report so that you can ask your doctor better questions. Ask it what questions a tenant should ask a lawyer. Ask it what the categories on a tax form mean. Then take the questions to the person whose name goes on the answer. That is not a failure of the tool. It is what the tool is for.

The final word

The last thing not to hand it is the decision itself. A model can lay out options, draft the letter, summarise the arguments and argue the other side. It cannot carry the consequence, and a decision made by someone who cannot carry the consequence is not a decision; it is a guess with good grammar. The next lesson is about how to keep that line when the guesses are this good.