{"course":{"slug":"building-apps-with-ai","title":"Building Apps with AI","url":"https://addaly.com/learn/building-apps-with-ai","level":"beginner","track":"build"},"lesson":{"slug":"the-smallest-experiment","title":"The smallest experiment that settles it","url":"https://addaly.com/learn/building-apps-with-ai/the-smallest-experiment","position":26,"of":87,"minutes":8,"answer":"Predict the result before you run it, change one thing, and break the code you think is responsible to confirm the behaviour actually depends on it.","keyIdea":"Predict the result before you run it, change one thing, and break the code you think is responsible to confirm the behaviour actually depends on it."},"body":{"format":"markdown","text":"## Stop arguing, start testing\n\nYou think the overlap rule is broken. The model says it is not. Neither of you knows. The way out is an experiment small enough to run in a minute and specific enough that its result means one thing.\n\nThe method is four steps, and the third is the one people skip.\n\n1. **State the claim.** \"Booking 14:15 for a stylist who has 14:00–14:30 is refused.\"\n2. **State what each outcome would mean.** Refused: the rule works for overlaps starting inside an existing booking. Accepted: it does not.\n3. **Predict, out loud, before running it.** Write down what you expect.\n4. **Run it once, changing one thing.**\n\nStep 3 costs nothing and is what turns a click into evidence. If you predicted correctly, you understand the system. If you did not, you have just learned something, and the surprise is the valuable part. Without a prediction, every result feels like it confirms whatever you already believed.\n\n## Change one thing\n\nThe fastest way to waste an hour is to change three things and see whether the problem goes away. It might, and you will not know which change did it, and one of the other two may have broken something you have not looked at yet.\n\nOne variable per run. If you must change two, run twice.\n\n## The controls that catch a fake pass\n\nA test that passes tells you less than you think, because it may be passing for the wrong reason. Two cheap controls fix that.\n\n**The negative control.** Deliberately do the thing that should fail. Book a slot that definitely overlaps and confirm it is refused. If a validation always says yes, an always-yes function passes every positive test.\n\n**The break test.** Temporarily break the thing you believe is doing the work and confirm the behaviour changes. If you comment out the overlap check and bookings are *still* refused, something else is refusing them and your understanding is wrong.\n\nThat second one feels perverse and it is the single most reliable way to find out whether the code you are reading is the code that runs. Generated projects frequently contain two implementations, and you can spend an afternoon perfecting the one that nothing calls.\n\n## Six experiments worth running on any feature\n\n- **The second account.** Log in as a different user and try to reach the first user's data by changing an id in the URL.\n- **The empty case.** A brand new account with no data at all.\n- **The double press.** Submit twice, fast.\n- **The back button.** Do the action, press back, look at what state the page is in.\n- **The refresh.** Reload mid-flow. Does anything survive that should not, or vanish that should not?\n- **The wrong input.** An apostrophe in a name, an emoji, 500 characters, a leading zero on a phone number.\n\nEach takes under a minute. Together they find more real defects in generated code than reading the code does, because they exercise the paths nobody described in the prompt.\n\n## Write down what you did\n\nOne line per experiment, in a file in the repository.\n\n```\n2026-09-06 Overlap: 14:00-14:30 exists.\n  14:15 → refused (expected refused) OK\n  14:30 → accepted (expected accepted) OK\n  13:45 for 45min → ACCEPTED (expected refused) BUG\n```\n\nThat third line is a bug report, a reproduction and a test case in one, and it took eight seconds to write. Hand it to the model exactly as it is and you will get a far better fix than from any description of the problem.\n\n## When the experiment is expensive\n\nSome things cannot be tested by clicking: a race between two simultaneous requests, a failure that only happens under load, a bug that appears once a week. For those, the equivalent move is logging: add enough output that when it next happens you can see what the state was. The next lesson is about exactly that.\n\nWhat does not change is the discipline. One variable, a prediction first, and a negative control to prove your test can fail."},"check":{"question":"You disable the overlap check and overlapping bookings are still refused. What have you learned?","options":[{"text":"The overlap check is correct and was not the cause of the problem.","correct":false,"tempting":"Draws a conclusion about correctness from a test showing the code has no effect at all."},{"text":"Something other than the code you were reading is enforcing the rule, so your model of the system is wrong.","correct":true},{"text":"The change did not take effect because the server needs restarting.","correct":false,"tempting":"A real possibility worth ruling out, but treating it as the conclusion abandons the finding rather than following it."},{"text":"The database is enforcing a constraint, which is the correct place for it.","correct":false,"tempting":"Names one of several possible other sources and adopts it as fact without checking which one is actually responsible."}]},"prev":{"slug":"devtools-as-an-x-ray","title":"The browser devtools as an x-ray","url":"https://addaly.com/learn/building-apps-with-ai/devtools-as-an-x-ray"},"next":{"slug":"printing-what-you-cannot-see","title":"Printing what you cannot see","url":"https://addaly.com/learn/building-apps-with-ai/printing-what-you-cannot-see"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The smallest experiment that settles it', in Building Apps with AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/building-apps-with-ai/the-smallest-experiment (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The smallest experiment that settles it","courseTitle":"Building Apps with AI","module":{"index":3,"slug":"reading-and-checking","title":"Reading and checking what came back"},"answer":null,"keyIdea":"Predict the result before you run it, change one thing, and break the code you think is responsible to confirm the behaviour actually depends on it.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"stop-arguing-start-testing","runs":[{"text":"Stop arguing, start testing"}]},{"t":"p","runs":[{"text":"You think the overlap rule is broken. The model says it is not. Neither of you knows. The way out is an experiment small enough to run in a minute and specific enough that its result means one thing."}]},{"t":"p","runs":[{"text":"The method is four steps, and the third is the one people skip."}]},{"t":"ol","start":1,"items":[[{"text":"State the claim.","b":true},{"text":" “Booking 14:15 for a stylist who has 14:00–14:30 is refused.”"}],[{"text":"State what each outcome would mean.","b":true},{"text":" Refused: the rule works for overlaps starting inside an existing booking. Accepted: it does not."}],[{"text":"Predict, out loud, before running it.","b":true},{"text":" Write down what you expect."}],[{"text":"Run it once, changing one thing.","b":true}]]},{"t":"p","runs":[{"text":"Step 3 costs nothing and is what turns a click into evidence. If you predicted correctly, you understand the system. If you did not, you have just learned something, and the surprise is the valuable part. Without a prediction, every result feels like it confirms whatever you already believed."}]},{"t":"h2","id":"change-one-thing","runs":[{"text":"Change one thing"}]},{"t":"p","runs":[{"text":"The fastest way to waste an hour is to change three things and see whether the problem goes away. It might, and you will not know which change did it, and one of the other two may have broken something you have not looked at yet."}]},{"t":"p","runs":[{"text":"One variable per run. If you must change two, run twice."}]},{"t":"h2","id":"the-controls-that-catch-a-fake-pass","runs":[{"text":"The controls that catch a fake pass"}]},{"t":"p","runs":[{"text":"A test that passes tells you less than you think, because it may be passing for the wrong reason. Two cheap controls fix that."}]},{"t":"p","runs":[{"text":"The negative control.","b":true},{"text":" Deliberately do the thing that should fail. Book a slot that definitely overlaps and confirm it is refused. If a validation always says yes, an always-yes function passes every positive test."}]},{"t":"p","runs":[{"text":"The break test.","b":true},{"text":" Temporarily break the thing you believe is doing the work and confirm the behaviour changes. If you comment out the overlap check and bookings are "},{"text":"still","i":true},{"text":" refused, something else is refusing them and your understanding is wrong."}]},{"t":"p","runs":[{"text":"That second one feels perverse and it is the single most reliable way to find out whether the code you are reading is the code that runs. Generated projects frequently contain two implementations, and you can spend an afternoon perfecting the one that nothing calls."}]},{"t":"h2","id":"six-experiments-worth-running-on-any-feature","runs":[{"text":"Six experiments worth running on any feature"}]},{"t":"ul","items":[[{"text":"The second account.","b":true},{"text":" Log in as a different user and try to reach the first user’s data by changing an id in the URL."}],[{"text":"The empty case.","b":true},{"text":" A brand new account with no data at all."}],[{"text":"The double press.","b":true},{"text":" Submit twice, fast."}],[{"text":"The back button.","b":true},{"text":" Do the action, press back, look at what state the page is in."}],[{"text":"The refresh.","b":true},{"text":" Reload mid-flow. Does anything survive that should not, or vanish that should not?"}],[{"text":"The wrong input.","b":true},{"text":" An apostrophe in a name, an emoji, 500 characters, a leading zero on a phone number."}]]},{"t":"p","runs":[{"text":"Each takes under a minute. Together they find more real defects in generated code than reading the code does, because they exercise the paths nobody described in the prompt."}]},{"t":"h2","id":"write-down-what-you-did","runs":[{"text":"Write down what you did"}]},{"t":"p","runs":[{"text":"One line per experiment, in a file in the repository."}]},{"t":"code","lang":"","label":"Text","text":"2026-09-06 Overlap: 14:00-14:30 exists.\n  14:15 → refused (expected refused) OK\n  14:30 → accepted (expected accepted) OK\n  13:45 for 45min → ACCEPTED (expected refused) BUG"},{"t":"p","runs":[{"text":"That third line is a bug report, a reproduction and a test case in one, and it took eight seconds to write. Hand it to the model exactly as it is and you will get a far better fix than from any description of the problem."}]},{"t":"h2","id":"when-the-experiment-is-expensive","runs":[{"text":"When the experiment is expensive"}]},{"t":"p","runs":[{"text":"Some things cannot be tested by clicking: a race between two simultaneous requests, a failure that only happens under load, a bug that appears once a week. For those, the equivalent move is logging: add enough output that when it next happens you can see what the state was. The next lesson is about exactly that."}]},{"t":"p","runs":[{"text":"What does not change is the discipline. One variable, a prediction first, and a negative control to prove your test can fail."}]}],"check":{"question":[{"text":"You disable the overlap check and overlapping bookings are still refused. What have you learned?"}],"options":[{"text":[{"text":"The overlap check is correct and was not the cause of the problem."}],"correct":false,"tempting":[{"text":"Draws a conclusion about correctness from a test showing the code has no effect at all."}]},{"text":[{"text":"Something other than the code you were reading is enforcing the rule, so your model of the system is wrong."}],"correct":true,"tempting":null},{"text":[{"text":"The change did not take effect because the server needs restarting."}],"correct":false,"tempting":[{"text":"A real possibility worth ruling out, but treating it as the conclusion abandons the finding rather than following it."}]},{"text":[{"text":"The database is enforcing a constraint, which is the correct place for it."}],"correct":false,"tempting":[{"text":"Names one of several possible other sources and adopts it as fact without checking which one is actually responsible."}]}],"order":[0,1,2,3],"why":null},"prev":{"slug":"devtools-as-an-x-ray","title":"The browser devtools as an x-ray","minutes":8},"next":{"slug":"printing-what-you-cannot-see","title":"Printing what you cannot see","minutes":8}}}