{"course":{"slug":"data-and-sql","title":"Data, SQL and Getting to the Answer","url":"https://addaly.com/learn/data-and-sql","level":"beginner","track":"build"},"lesson":{"slug":"right-vs-useful","title":"A right number and a useful number are not the same","url":"https://addaly.com/learn/data-and-sql/right-vs-useful","position":11,"of":66,"minutes":9,"answer":"A correct query answers the question you typed, which is rarely the question you were asked.","keyIdea":"A correct query answers the question you typed, which is rarely the question you were asked."},"body":{"format":"markdown","text":"## The query was correct. The answer was wrong.\n\nThis is the lesson that separates people who can write SQL from people you can trust with a decision. Your query can be flawless (correct joins, correct filters, no NULL traps) and still produce a number that misleads everyone in the room.\n\nHere is the shape of it, four ways.\n\n## The definition nobody agreed on\n\n\"How many active users do we have?\" You write the query. You return 41,200.\n\nBut nobody said what active means. Signed in during the last 30 days? Took any action at all? Has a live subscription? Opened the app once, ever, and never deleted the account? Those four numbers might be 41,200 and 12,400 and 6,900 and 380,000. All four are correct. Three of them will get someone fired.\n\nThe fix is not technical. Before the query, write the sentence: *\"one row of my result is one account that performed at least one action between 1 and 31 March, in the account's own time zone.\"* Send that sentence to whoever asked, before you send them a number.\n\n## The denominator that moved\n\nA team reports that conversion rate rose from 2.1% to 3.4% after a redesign. The numerator query is correct. The denominator query is correct. What happened is that an analytics script broke on mobile, so a third of the visits stopped being recorded. Fewer visits, same purchases, higher rate.\n\n> Figure: Four correct answers to \"how many active users do we have?\". Bars (accounts answering to the word \"active\"): Opened the app once, ever 380000; Signed in during the last 30 days 41200; Took an action in the last 30 days 12400; Has a live subscription 6900. Every one of these is produced by a query with no bug in it. The number that gets reported is decided by a definition nobody wrote down, and the gap between the largest and the smallest is a factor of fifty-five. Send the sentence before you send the number.\n\nWhenever a ratio moves, look at the top and the bottom separately. A ratio hides which half changed, and it is very often the half you were not thinking about.\n\n## Survivorship\n\nA subscription business computes churn as: of the customers currently in the customers table, what share have status `'cancelled'`. It returns 4%.\n\nThe problem is that customers who cancelled and were then purged, or archived, or moved to another table, are not in the denominator at all. You are measuring churn among the people who did not leave. Adding a date filter does not help; the missing people are missing before any filter runs.\n\nThe general form: whenever your population is defined by having survived, you cannot use it to measure survival.\n\n## The average that lands where nobody is\n\nMean order value on an Indian marketplace: ₹4,800. Ninety percent of orders are between ₹300 and ₹900. A handful of bulk business orders are ₹200,000 each. The mean is arithmetically perfect and describes no customer who has ever existed.\n\nReport the median. Report the 90th percentile next to it. If the median is ₹620 and the mean is ₹4,800, that gap is itself the finding, and it is more interesting than either number alone.\n\n## Time zones and late data, briefly\n\n\"Yesterday's revenue\" computed in UTC for a business in Asia/Kolkata moves the boundary by five and a half hours, which reliably shifts the evening rush into the wrong day. And today's number is always low, because payments settle, webhooks retry and refunds land days later. A dashboard whose last bar always dips has taught its whole company that things are getting worse.\n\n## What to do about all of this\n\n**Say the sentence first.** \"One row of my result is ____, counted over ____, between ____ and ____ in ____ time zone, excluding ____.\" If any blank is unfillable, go and ask.\n\n**Check the number against something you already know.** Last month's figure. A total from the finance system. Ten rows counted by hand. A number with nothing to compare it against is a number nobody can challenge, including you.\n\n**Try to break your own filter.** Change a condition so the result *should* move. If it does not move, your filter is not doing anything, and you have found that out for free instead of in a meeting.\n\n**Ship the query with the number.** Always. A figure in a message with no query behind it cannot be argued with, and things that cannot be argued with are how organisations end up confidently wrong for a year."},"check":{"question":"A subscription business calculates churn as: of the customers currently in the customers table, what share have status 'cancelled'. The SQL is correct and returns 4%. Why might 4% be the wrong answer to \"what is our churn rate\"?","options":[{"text":"The result has no date range, so it mixes every year together; adding `WHERE cancelled_at >= ...` fixes it","correct":false,"tempting":"Assumes a time window repairs the number, when the people who would change it are absent from the table entirely."},{"text":"Some customers marked 'cancelled' later returned, so the numerator overstates the loss","correct":false,"tempting":"Locates the error in how cancellation is labelled rather than in who made it into the table at all."},{"text":"Customers who cancelled and were later removed or archived are not in the denominator, so this measures churn only among people still on file","correct":true},{"text":"4% is a monthly figure and needs to be annualised before it can be reported as a churn rate","correct":false,"tempting":"Treats the problem as a units conversion, which would be a fair critique of a number that was measuring the right population."}]},"prev":{"slug":"the-grain-of-a-result","title":"Grain: the sentence to say before you write the query","url":"https://addaly.com/learn/data-and-sql/the-grain-of-a-result"},"next":{"slug":"sql-from-an-ai","title":"Asking an AI for SQL without being lied to","url":"https://addaly.com/learn/data-and-sql/sql-from-an-ai"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'A right number and a useful number are not the same', in Data, SQL and Getting to the Answer. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/data-and-sql/right-vs-useful (Accessed: 31 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"A right number and a useful number are not the same","courseTitle":"Data, SQL and Getting to the Answer","module":{"index":1,"slug":"tables-types-and-the-question","title":"Tables, types and the question you are actually asking"},"answer":null,"keyIdea":"A correct query answers the question you typed, which is rarely the question you were asked.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"bars","unit":"accounts answering to the word “active”","title":"Four correct answers to “how many active users do we have?”","series":[{"label":"Opened the app once, ever","value":380000},{"label":"Signed in during the last 30 days","value":41200,"highlight":true},{"label":"Took an action in the last 30 days","value":12400},{"label":"Has a live subscription","value":6900}],"caption":"Every one of these is produced by a query with no bug in it. The number that gets reported is decided by a definition nobody wrote down, and the gap between the largest and the smallest is a factor of fifty-five. Send the sentence before you send the number."},"alt":"Figure: Four correct answers to “how many active users do we have?”. Bars (accounts answering to the word “active”): Opened the app once, ever 380000; Signed in during the last 30 days 41200; Took an action in the last 30 days 12400; Has a live subscription 6900. Every one of these is produced by a query with no bug in it. The number that gets reported is decided by a definition nobody wrote down, and the gap between the largest and the smallest is a factor of fifty-five. Send the sentence before you send the number."}},"blocks":[{"t":"h2","id":"the-query-was-correct-the-answer-was-wrong","runs":[{"text":"The query was correct. The answer was wrong."}]},{"t":"p","runs":[{"text":"This is the lesson that separates people who can write SQL from people you can trust with a decision. Your query can be flawless (correct joins, correct filters, no NULL traps) and still produce a number that misleads everyone in the room."}]},{"t":"p","runs":[{"text":"Here is the shape of it, four ways."}]},{"t":"h2","id":"the-definition-nobody-agreed-on","runs":[{"text":"The definition nobody agreed on"}]},{"t":"p","runs":[{"text":"“How many active users do we have?” You write the query. You return 41,200."}]},{"t":"p","runs":[{"text":"But nobody said what active means. Signed in during the last 30 days? Took any action at all? Has a live subscription? Opened the app once, ever, and never deleted the account? Those four numbers might be 41,200 and 12,400 and 6,900 and 380,000. All four are correct. Three of them will get someone fired."}]},{"t":"p","runs":[{"text":"The fix is not technical. Before the query, write the sentence: "},{"text":"”one row of my result is one account that performed at least one action between 1 and 31 March, in the account’s own time zone.”","i":true},{"text":" Send that sentence to whoever asked, before you send them a number."}]},{"t":"h2","id":"the-denominator-that-moved","runs":[{"text":"The denominator that moved"}]},{"t":"p","runs":[{"text":"A team reports that conversion rate rose from 2.1% to 3.4% after a redesign. The numerator query is correct. The denominator query is correct. What happened is that an analytics script broke on mobile, so a third of the visits stopped being recorded. Fewer visits, same purchases, higher rate."}]},{"t":"p","runs":[{"text":"Whenever a ratio moves, look at the top and the bottom separately. A ratio hides which half changed, and it is very often the half you were not thinking about."}]},{"t":"h2","id":"survivorship","runs":[{"text":"Survivorship"}]},{"t":"p","runs":[{"text":"A subscription business computes churn as: of the customers currently in the customers table, what share have status "},{"text":"'cancelled'","code":true},{"text":". It returns 4%."}]},{"t":"p","runs":[{"text":"The problem is that customers who cancelled and were then purged, or archived, or moved to another table, are not in the denominator at all. You are measuring churn among the people who did not leave. Adding a date filter does not help; the missing people are missing before any filter runs."}]},{"t":"p","runs":[{"text":"The general form: whenever your population is defined by having survived, you cannot use it to measure survival."}]},{"t":"h2","id":"the-average-that-lands-where-nobody-is","runs":[{"text":"The average that lands where nobody is"}]},{"t":"p","runs":[{"text":"Mean order value on an Indian marketplace: ₹4,800. Ninety percent of orders are between ₹300 and ₹900. A handful of bulk business orders are ₹200,000 each. The mean is arithmetically perfect and describes no customer who has ever existed."}]},{"t":"p","runs":[{"text":"Report the median. Report the 90th percentile next to it. If the median is ₹620 and the mean is ₹4,800, that gap is itself the finding, and it is more interesting than either number alone."}]},{"t":"h2","id":"time-zones-and-late-data-briefly","runs":[{"text":"Time zones and late data, briefly"}]},{"t":"p","runs":[{"text":"“Yesterday’s revenue” computed in UTC for a business in Asia/Kolkata moves the boundary by five and a half hours, which reliably shifts the evening rush into the wrong day. And today’s number is always low, because payments settle, webhooks retry and refunds land days later. A dashboard whose last bar always dips has taught its whole company that things are getting worse."}]},{"t":"h2","id":"what-to-do-about-all-of-this","runs":[{"text":"What to do about all of this"}]},{"t":"p","runs":[{"text":"Say the sentence first.","b":true},{"text":" “One row of my result is ____, counted over ____, between ____ and ____ in ____ time zone, excluding ____.” If any blank is unfillable, go and ask."}]},{"t":"p","runs":[{"text":"Check the number against something you already know.","b":true},{"text":" Last month’s figure. A total from the finance system. Ten rows counted by hand. A number with nothing to compare it against is a number nobody can challenge, including you."}]},{"t":"p","runs":[{"text":"Try to break your own filter.","b":true},{"text":" Change a condition so the result "},{"text":"should","i":true},{"text":" move. If it does not move, your filter is not doing anything, and you have found that out for free instead of in a meeting."}]},{"t":"p","runs":[{"text":"Ship the query with the number.","b":true},{"text":" Always. A figure in a message with no query behind it cannot be argued with, and things that cannot be argued with are how organisations end up confidently wrong for a year."}]}],"check":{"question":[{"text":"A subscription business calculates churn as: of the customers currently in the customers table, what share have status ‘cancelled’. The SQL is correct and returns 4%. Why might 4% be the wrong answer to “what is our churn rate”?"}],"options":[{"text":[{"text":"The result has no date range, so it mixes every year together; adding "},{"text":"WHERE cancelled_at >= ...","code":true},{"text":" fixes it"}],"correct":false,"tempting":[{"text":"Assumes a time window repairs the number, when the people who would change it are absent from the table entirely."}]},{"text":[{"text":"Some customers marked ‘cancelled’ later returned, so the numerator overstates the loss"}],"correct":false,"tempting":[{"text":"Locates the error in how cancellation is labelled rather than in who made it into the table at all."}]},{"text":[{"text":"Customers who cancelled and were later removed or archived are not in the denominator, so this measures churn only among people still on file"}],"correct":true,"tempting":null},{"text":[{"text":"4% is a monthly figure and needs to be annualised before it can be reported as a churn rate"}],"correct":false,"tempting":[{"text":"Treats the problem as a units conversion, which would be a fair critique of a number that was measuring the right population."}]}],"order":[1,0,2,3],"why":null},"prev":{"slug":"the-grain-of-a-result","title":"Grain: the sentence to say before you write the query","minutes":8},"next":{"slug":"sql-from-an-ai","title":"Asking an AI for SQL without being lied to","minutes":8}}}