AI, Safety and What Goes WrongLesson 7 of 73

Module 1 · How these systems fail

Why people believe the machine

In this lesson

  • 4 min read
  • 6 reading sections
  • Interactive activity
  • Knowledge check
  • Lesson 7 of 9 in this chapter

This chapter’s goalSort a failure into design, capability or misuse, read a benchmark or a demo for what it leaves out, and set how much checking a use needs by what it costs if wrong.

This chapter’s media

The film explains chapter ideas. The audiobook narrates the complete teaching text.

FormatEnglishहिंदी
Chapter filmWatch2:47Not available
Full audiobookNot availableNot available
Horizon: fifteen years of believing the computer1999Horizon installed in Post Office branches2000 to 2014About 736 sub-postmasters prosecuted by the Post OfficeDec 2019High Court: bugs in Horizon could cause the shortfallsApr 2021Court of Appeal quashes 39 convictionsMay 2024An Act quashes convictions in England, Wales and Northern IrelandTwenty years from installation to a court test of the software, and four and a half more beforeParliament cleared the convictions by statute.
Twenty years from installation to a court test of the software, and four and a half more before Parliament cleared the convictions by statute.

Figure

A little at a time

Explore an idea, try it, then check what stays.

Step 1 of 9Free · no account needed
Lesson steps

A computer against a village postmaster

From 1999 the Post Office installed an accounting system called Horizon in its branches across the United Kingdom. It reported cash shortfalls that were not real. Sub-postmasters, often the most trusted person in a village, were told the computer showed money missing.

Your place and first choices stay on this browser. Classroom credit is recorded separately.

Stuck on something?Ask Rukmini, your AI tutor

How to cite: Addaly (2026) ‘Why people believe the machine’, in AI, Safety and What Goes Wrong. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-and-you/automation-bias (Accessed: 21 November 2026).