{"course":{"slug":"ai-careers","title":"Working in AI","url":"https://addaly.com/learn/ai-careers","level":"beginner","track":"work"},"lesson":{"slug":"the-design-conversation","title":"\"How would you build this?\" answered well","url":"https://addaly.com/learn/ai-careers/the-design-conversation","position":54,"of":85,"minutes":9,"answer":"A design answer is a set of constraints, measurements and failure modes, not an architecture diagram.","keyIdea":"A design answer is a set of constraints, measurements and failure modes, not an architecture diagram."},"body":{"format":"markdown","text":"## The question you will get\n\n> \"We have a help centre with about four hundred articles. How would you build something that answers customer questions from it?\"\n\nThis has become a standard interview question at almost every level, including entry, because it is cheap to ask and reveals a great deal. It is not a test of whether you know the current fashionable stack.\n\n## The five steps, in this order\n\n**1. Ask who uses it, how often, and what a wrong answer costs.**\n\nAn internal tool for support agents who can see the source article is a completely different system from a public one answering questions about medication. The cost of a wrong answer sets everything else: how conservative the retrieval is, whether there is a human in the loop, whether it may answer at all when unsure. Beginners skip this and start naming tools. Ask it first, every time.\n\n**2. Say how you would measure it before you say how you would build it.**\n\n\"Before building anything I would take fifty real questions from the support queue, write down what a correct answer must contain, and score whatever we build against them. I would also score plain keyword search on the same fifty, so we know what the improvement actually is.\"\n\nThis one paragraph moves you above most candidates, because it demonstrates the habit that separates people who ship working systems from people who ship demos.\n\n**3. Data, and who owns it.**\n\nWhere do the articles live? How often do they change, and does anyone tell you when? Are they all current, or are a third of them from 2021 and contradicting each other? Is there anything in them that should not be shown to a customer? A system built over stale documents answers confidently and wrongly, and no model choice fixes that.\n\n**4. The simplest thing that could work.**\n\nSearch the articles, retrieve the top few passages, have a model answer using only those passages, and show the source link with every answer. Citations are not a feature; they are what makes the thing checkable.\n\nSay out loud that you would try plain keyword or full-text search as the baseline. Interviewers notice. It is frequently good enough on four hundred articles, and it is far easier to debug: when it retrieves the wrong document you can see exactly why.\n\n**5. Failure modes and cost.**\n\nWhat happens when retrieval finds nothing relevant? The answer should be \"it says it does not know and offers to hand over to a person\", not \"it answers anyway\". What happens at ten times the traffic? What does a query cost: a few thousand tokens in and a few hundred out is cents rather than dollars, but multiplied by fifty thousand queries a month it becomes a real budget line, and that arithmetic is worth doing out loud.\n\n## The senior tell: saying what you would not build\n\n> \"I would not fine-tune a model on the articles. They change weekly, fine-tuning bakes them in, and fixing a wrong answer would mean retraining. With retrieval I fix it by editing the article.\"\n\nExplaining a rejected option with its mechanism is the strongest single move available in this conversation, and it works at any experience level because it is reasoning rather than knowledge.\n\n## What loses\n\n- Naming five products in the first thirty seconds with no reason attached.\n- Jumping straight to a vector database before establishing what is being measured.\n- Never mentioning evaluation at all. This is the most common failure, and for AI roles it is close to disqualifying.\n- Designing for a million users when the company has four hundred articles and six support agents.\n\n## Building the whole thing free\n\nEverything in that design has a free path, which matters because the strongest possible answer is \"I built a small version of this\":\n\n- **Search:** PostgreSQL full-text search, SQLite FTS5, or DuckDB. If you need vectors, **pgvector** or **Chroma**, both free and self-hosted.\n- **The model:** a free API tier, or a small local model through **Ollama** on a laptop.\n- **Hosting:** a free **Hugging Face Space** with **Gradio**.\n- **Evaluation:** a CSV and a script.\n\nThe named paid products (Pinecone, Weaviate Cloud, managed retrieval services) buy operational convenience at scale. None of them is required to demonstrate that you understand the problem, and at four hundred articles none of them is required at all.\n\n## The limitation\n\nNobody expects a correct architecture in forty-five minutes, and there is no single right answer to grade against. What is being graded is order of thinking: constraints before components, measurement before building, failure modes before scaling.\n\n\"I do not know how that behaves at ten million documents, and here is how I would find out\" is a scoring answer. Confident invention is not, and interviewers who work on these systems can tell the difference immediately."},"check":{"question":"In a design conversation about an internal question-answering tool, which opening move is strongest?","options":[{"text":"Describe the architecture: documents, embeddings, a vector store, retrieval, a model, an interface.","correct":false,"tempting":"Answers the question that was asked but skips the constraints that determine whether any of it is the right shape, which is what is being graded."},{"text":"Ask which model provider the company already uses, so the design fits their stack.","correct":false,"tempting":"A sensible practical question that arrives several steps too early, before anyone has established what the system is for."},{"text":"Ask who uses it, how often, and what a wrong answer costs; then say how you would measure the system before describing how you would build it.","correct":true},{"text":"Propose starting with a small pilot on a subset of documents to learn before committing.","correct":false,"tempting":"Sounds prudent but defers the thinking rather than doing it, and a pilot with no defined measurement teaches nothing."}]},"prev":{"slug":"the-live-technical","title":"The live exercise, where silence is scored","url":"https://addaly.com/learn/ai-careers/the-live-technical"},"next":{"slug":"the-metrics-case","title":"\"How would you measure it?\"","url":"https://addaly.com/learn/ai-careers/the-metrics-case"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) '\"How would you build this?\" answered well', in Working in AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-careers/the-design-conversation (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"“How would you build this?” answered well","courseTitle":"Working in AI","module":{"index":6,"slug":"the-interview","title":"The interview, from first screen to signed offer"},"answer":null,"keyIdea":"A design answer is a set of constraints, measurements and failure modes, not an architecture diagram.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-question-you-will-get","runs":[{"text":"The question you will get"}]},{"t":"quote","runs":[{"text":"“We have a help centre with about four hundred articles. How would you build something that answers customer questions from it?”"}]},{"t":"p","runs":[{"text":"This has become a standard interview question at almost every level, including entry, because it is cheap to ask and reveals a great deal. It is not a test of whether you know the current fashionable stack."}]},{"t":"h2","id":"the-five-steps-in-this-order","runs":[{"text":"The five steps, in this order"}]},{"t":"p","runs":[{"text":"1. Ask who uses it, how often, and what a wrong answer costs.","b":true}]},{"t":"p","runs":[{"text":"An internal tool for support agents who can see the source article is a completely different system from a public one answering questions about medication. The cost of a wrong answer sets everything else: how conservative the retrieval is, whether there is a human in the loop, whether it may answer at all when unsure. Beginners skip this and start naming tools. Ask it first, every time."}]},{"t":"p","runs":[{"text":"2. Say how you would measure it before you say how you would build it.","b":true}]},{"t":"p","runs":[{"text":"“Before building anything I would take fifty real questions from the support queue, write down what a correct answer must contain, and score whatever we build against them. I would also score plain keyword search on the same fifty, so we know what the improvement actually is.”"}]},{"t":"p","runs":[{"text":"This one paragraph moves you above most candidates, because it demonstrates the habit that separates people who ship working systems from people who ship demos."}]},{"t":"p","runs":[{"text":"3. Data, and who owns it.","b":true}]},{"t":"p","runs":[{"text":"Where do the articles live? How often do they change, and does anyone tell you when? Are they all current, or are a third of them from 2021 and contradicting each other? Is there anything in them that should not be shown to a customer? A system built over stale documents answers confidently and wrongly, and no model choice fixes that."}]},{"t":"p","runs":[{"text":"4. The simplest thing that could work.","b":true}]},{"t":"p","runs":[{"text":"Search the articles, retrieve the top few passages, have a model answer using only those passages, and show the source link with every answer. Citations are not a feature; they are what makes the thing checkable."}]},{"t":"p","runs":[{"text":"Say out loud that you would try plain keyword or full-text search as the baseline. Interviewers notice. It is frequently good enough on four hundred articles, and it is far easier to debug: when it retrieves the wrong document you can see exactly why."}]},{"t":"p","runs":[{"text":"5. Failure modes and cost.","b":true}]},{"t":"p","runs":[{"text":"What happens when retrieval finds nothing relevant? The answer should be “it says it does not know and offers to hand over to a person”, not “it answers anyway”. What happens at ten times the traffic? What does a query cost: a few thousand tokens in and a few hundred out is cents rather than dollars, but multiplied by fifty thousand queries a month it becomes a real budget line, and that arithmetic is worth doing out loud."}]},{"t":"h2","id":"the-senior-tell-saying-what-you-would-not-build","runs":[{"text":"The senior tell: saying what you would not build"}]},{"t":"quote","runs":[{"text":"“I would not fine-tune a model on the articles. They change weekly, fine-tuning bakes them in, and fixing a wrong answer would mean retraining. With retrieval I fix it by editing the article.”"}]},{"t":"p","runs":[{"text":"Explaining a rejected option with its mechanism is the strongest single move available in this conversation, and it works at any experience level because it is reasoning rather than knowledge."}]},{"t":"h2","id":"what-loses","runs":[{"text":"What loses"}]},{"t":"ul","items":[[{"text":"Naming five products in the first thirty seconds with no reason attached."}],[{"text":"Jumping straight to a vector database before establishing what is being measured."}],[{"text":"Never mentioning evaluation at all. This is the most common failure, and for AI roles it is close to disqualifying."}],[{"text":"Designing for a million users when the company has four hundred articles and six support agents."}]]},{"t":"h2","id":"building-the-whole-thing-free","runs":[{"text":"Building the whole thing free"}]},{"t":"p","runs":[{"text":"Everything in that design has a free path, which matters because the strongest possible answer is “I built a small version of this”:"}]},{"t":"ul","items":[[{"text":"Search:","b":true},{"text":" PostgreSQL full-text search, SQLite FTS5, or DuckDB. If you need vectors, "},{"text":"pgvector","b":true},{"text":" or "},{"text":"Chroma","b":true},{"text":", both free and self-hosted."}],[{"text":"The model:","b":true},{"text":" a free API tier, or a small local model through "},{"text":"Ollama","b":true},{"text":" on a laptop."}],[{"text":"Hosting:","b":true},{"text":" a free "},{"text":"Hugging Face Space","b":true},{"text":" with "},{"text":"Gradio","b":true},{"text":"."}],[{"text":"Evaluation:","b":true},{"text":" a CSV and a script."}]]},{"t":"p","runs":[{"text":"The named paid products (Pinecone, Weaviate Cloud, managed retrieval services) buy operational convenience at scale. None of them is required to demonstrate that you understand the problem, and at four hundred articles none of them is required at all."}]},{"t":"h2","id":"the-limitation","runs":[{"text":"The limitation"}],"role":"limit"},{"t":"p","runs":[{"text":"Nobody expects a correct architecture in forty-five minutes, and there is no single right answer to grade against. What is being graded is order of thinking: constraints before components, measurement before building, failure modes before scaling."}]},{"t":"p","runs":[{"text":"“I do not know how that behaves at ten million documents, and here is how I would find out” is a scoring answer. Confident invention is not, and interviewers who work on these systems can tell the difference immediately."}]}],"check":{"question":[{"text":"In a design conversation about an internal question-answering tool, which opening move is strongest?"}],"options":[{"text":[{"text":"Describe the architecture: documents, embeddings, a vector store, retrieval, a model, an interface."}],"correct":false,"tempting":[{"text":"Answers the question that was asked but skips the constraints that determine whether any of it is the right shape, which is what is being graded."}]},{"text":[{"text":"Ask which model provider the company already uses, so the design fits their stack."}],"correct":false,"tempting":[{"text":"A sensible practical question that arrives several steps too early, before anyone has established what the system is for."}]},{"text":[{"text":"Ask who uses it, how often, and what a wrong answer costs; then say how you would measure the system before describing how you would build it."}],"correct":true,"tempting":null},{"text":[{"text":"Propose starting with a small pilot on a subset of documents to learn before committing."}],"correct":false,"tempting":[{"text":"Sounds prudent but defers the thinking rather than doing it, and a pilot with no defined measurement teaches nothing."}]}],"order":[1,0,2,3],"why":null},"prev":{"slug":"the-live-technical","title":"The live exercise, where silence is scored","minutes":8},"next":{"slug":"the-metrics-case","title":"“How would you measure it?”","minutes":10}}}