{"course":{"slug":"ai-for-small-business","title":"AI for a Small Business","url":"https://addaly.com/learn/ai-for-small-business","level":"beginner","track":"work"},"lesson":{"slug":"automl-and-what-it-hides","title":"AutoML: what it genuinely does, and what it cannot do for you","url":"https://addaly.com/learn/ai-for-small-business/automl-and-what-it-hides","position":75,"of":88,"minutes":9,"answer":"AutoML automates model selection and tuning, which was never the hard part, and leaves you the whole of the hard part: the question, the definition, the columns, and whether the score is real.","keyIdea":"AutoML automates model selection and tuning, which was never the hard part, and leaves you the whole of the hard part: the question, the definition, the columns, and whether the score is real."},"body":{"format":"markdown","text":"## What it is\n\nPoint a tool at a spreadsheet, say which column you want predicted, and it tries a range of methods, tunes them, and hands back the best with an accuracy figure. Paid versions exist on every cloud platform; free ones do the same job on a laptop.\n\nThis is a real convenience. Choosing between a dozen algorithms and tuning their settings used to take a specialist days, and it is now a button. For a small business it means you can get a competent model without learning what a gradient-boosted tree is.\n\n## What it automates\n\nModel selection. Hyperparameter tuning. Some encoding of categories, some handling of missing values. A cross-validation score.\n\nThat list is worth reading twice, because of what is not on it.\n\n## What it cannot do, and hands back to you unchanged\n\n**The question.** It cannot tell you whether predicting lapse is the useful question, or whether you should have been predicting which customers respond to a voucher, which is a different and usually more actionable thing.\n\n**The definition.** Someone has to decide what counts as lapsed. AutoML predicts whatever column you point at, including a badly defined one, with great efficiency.\n\n**The columns.** As the previous lessons said, features beat algorithms. AutoML does not know that a lengthening gap between orders matters; you have to compute that column and give it.\n\n**The leak.** This is the important one. AutoML will happily build a superb model on a leaking column and report 97%. It has no way to know that `deposit_paid_date` did not exist at prediction time, because that is a fact about your business, not about the data.\n\n**The threshold.** It reports a score at some default. The cut-off that matches what an intervention costs you is yours to set.\n\n**Whether to act.** A model with 30% precision might be worth acting on or might not, depending entirely on what a wrong flag costs. No tool computes that.\n\n## The free path, which is adequate\n\n**PyCaret**: a few lines of Python, compares a dozen models automatically, free and open source.\n**Orange**: visual, drag-and-drop, no code, free. Good for learning what the methods actually do because you can see the pipeline.\n**KNIME**: free desktop version, visual workflows, widely used in industry.\n**scikit-learn**: not AutoML, but its `RandomForestClassifier` with default settings is a strong baseline in about four lines, and beating it meaningfully is harder than people expect.\n\nPaid AutoML on the big clouds adds scale, deployment and monitoring. At small-business volumes you are paying for infrastructure you will not use.\n\n## Read the score it gives you, carefully\n\nAutoML reports a cross-validated figure, which means it split your data repeatedly and averaged the results. Two things to know about that number.\n\nIt is computed on the data you supplied, which is the past. It is an estimate of how the model would have done on history, and it assumes next year resembles last year. When a competitor opens or a product line changes, that assumption stops holding and the score does not update.\n\nAnd by default most tools split randomly, which is the time leak from the previous lesson. Look for the option to split by date, and use it for anything with a time dimension. In several popular tools it is not the default, and the difference in reported accuracy between the two settings is often larger than the difference between any two algorithms the tool compared.\n\n## The comparison that keeps you honest\n\nWhatever AutoML produces, compare it against two things.\n\nThe **rule**: sort by days since last order, take the top hundred. Measure its precision and recall on the same holdout.\n\nThe **simplest model**: logistic regression with five sensible columns. It also tells you, in plain numbers, which factors mattered and in which direction, which a boosted ensemble will not.\n\nIn small-business data, the gap between logistic regression and the best AutoML result is frequently a couple of percentage points. If that is the gap, take the simple model: you can explain it to your staff, debug it when it drifts, and see immediately when one of its inputs stops making sense.\n\n## When to use it\n\nWhen you have tens of thousands of rows, a defined target, features you built deliberately, and a rule-based baseline you want to beat. Then AutoML is a sensible hour's work.\n\nWhen you have four hundred rows and a hopeful question, it will still produce a model with a confident score, and the next lesson is about why that number is worse than nothing."},"check":{"question":"An owner runs AutoML on 8,000 customer records to predict lapse. It reports 91% accuracy and ranks `last_contact_type` as the most important feature. What should she do before acting?","options":[{"text":"Rerun with that feature removed, to confirm the remaining model still performs acceptably without relying so heavily on a single column.","correct":false,"tempting":"A reasonable robustness check, and it treats concentration as the problem when the question is what the column actually is."},{"text":"Compare the result against logistic regression on the same columns, since a simpler model performing similarly would be easier to explain and maintain.","correct":false,"tempting":"Good practice from this lesson, and it would reproduce the same fault if the column is contaminated."},{"text":"Find out when `last_contact_type` is recorded, since a contact prompted by her own suspicion would be reading her judgement back.","correct":true},{"text":"Check the accuracy against the base rate, because 91% may be close to what predicting \"nobody lapses\" would score on this dataset.","correct":false,"tempting":"Exactly the right reflex from the previous lesson, and here the named feature is the more specific warning sign to follow first."}]},"prev":{"slug":"leakage-the-flattering-mistake","title":"The mistake that makes a model look brilliant","url":"https://addaly.com/learn/ai-for-small-business/leakage-the-flattering-mistake"},"next":{"slug":"small-numbers-and-superstition","title":"When your data is too small, and what to do instead","url":"https://addaly.com/learn/ai-for-small-business/small-numbers-and-superstition"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'AutoML: what it genuinely does, and what it cannot do for you', in AI for a Small Business. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-for-small-business/automl-and-what-it-hides (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"AutoML: what it genuinely does, and what it cannot do for you","courseTitle":"AI for a Small Business","module":{"index":8,"slug":"your-own-numbers","title":"Learning from your own numbers"},"answer":null,"keyIdea":"AutoML automates model selection and tuning, which was never the hard part, and leaves you the whole of the hard part: the question, the definition, the columns, and whether the score is real.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"what-it-is","runs":[{"text":"What it is"}]},{"t":"p","runs":[{"text":"Point a tool at a spreadsheet, say which column you want predicted, and it tries a range of methods, tunes them, and hands back the best with an accuracy figure. Paid versions exist on every cloud platform; free ones do the same job on a laptop."}]},{"t":"p","runs":[{"text":"This is a real convenience. Choosing between a dozen algorithms and tuning their settings used to take a specialist days, and it is now a button. For a small business it means you can get a competent model without learning what a gradient-boosted tree is."}]},{"t":"h2","id":"what-it-automates","runs":[{"text":"What it automates"}]},{"t":"p","runs":[{"text":"Model selection. Hyperparameter tuning. Some encoding of categories, some handling of missing values. A cross-validation score."}]},{"t":"p","runs":[{"text":"That list is worth reading twice, because of what is not on it."}]},{"t":"h2","id":"what-it-cannot-do-and-hands-back-to-you-unchanged","runs":[{"text":"What it cannot do, and hands back to you unchanged"}]},{"t":"p","runs":[{"text":"The question.","b":true},{"text":" It cannot tell you whether predicting lapse is the useful question, or whether you should have been predicting which customers respond to a voucher, which is a different and usually more actionable thing."}]},{"t":"p","runs":[{"text":"The definition.","b":true},{"text":" Someone has to decide what counts as lapsed. AutoML predicts whatever column you point at, including a badly defined one, with great efficiency."}]},{"t":"p","runs":[{"text":"The columns.","b":true},{"text":" As the previous lessons said, features beat algorithms. AutoML does not know that a lengthening gap between orders matters; you have to compute that column and give it."}]},{"t":"p","runs":[{"text":"The leak.","b":true},{"text":" This is the important one. AutoML will happily build a superb model on a leaking column and report 97%. It has no way to know that "},{"text":"deposit_paid_date","code":true},{"text":" did not exist at prediction time, because that is a fact about your business, not about the data."}]},{"t":"p","runs":[{"text":"The threshold.","b":true},{"text":" It reports a score at some default. The cut-off that matches what an intervention costs you is yours to set."}]},{"t":"p","runs":[{"text":"Whether to act.","b":true},{"text":" A model with 30% precision might be worth acting on or might not, depending entirely on what a wrong flag costs. No tool computes that."}]},{"t":"h2","id":"the-free-path-which-is-adequate","runs":[{"text":"The free path, which is adequate"}],"role":"free"},{"t":"p","runs":[{"text":"PyCaret","b":true},{"text":": a few lines of Python, compares a dozen models automatically, free and open source. "},{"text":"Orange","b":true},{"text":": visual, drag-and-drop, no code, free. Good for learning what the methods actually do because you can see the pipeline. "},{"text":"KNIME","b":true},{"text":": free desktop version, visual workflows, widely used in industry. "},{"text":"scikit-learn","b":true},{"text":": not AutoML, but its "},{"text":"RandomForestClassifier","code":true},{"text":" with default settings is a strong baseline in about four lines, and beating it meaningfully is harder than people expect."}]},{"t":"p","runs":[{"text":"Paid AutoML on the big clouds adds scale, deployment and monitoring. At small-business volumes you are paying for infrastructure you will not use."}]},{"t":"h2","id":"read-the-score-it-gives-you-carefully","runs":[{"text":"Read the score it gives you, carefully"}]},{"t":"p","runs":[{"text":"AutoML reports a cross-validated figure, which means it split your data repeatedly and averaged the results. Two things to know about that number."}]},{"t":"p","runs":[{"text":"It is computed on the data you supplied, which is the past. It is an estimate of how the model would have done on history, and it assumes next year resembles last year. When a competitor opens or a product line changes, that assumption stops holding and the score does not update."}]},{"t":"p","runs":[{"text":"And by default most tools split randomly, which is the time leak from the previous lesson. Look for the option to split by date, and use it for anything with a time dimension. In several popular tools it is not the default, and the difference in reported accuracy between the two settings is often larger than the difference between any two algorithms the tool compared."}]},{"t":"h2","id":"the-comparison-that-keeps-you-honest","runs":[{"text":"The comparison that keeps you honest"}]},{"t":"p","runs":[{"text":"Whatever AutoML produces, compare it against two things."}]},{"t":"p","runs":[{"text":"The "},{"text":"rule","b":true},{"text":": sort by days since last order, take the top hundred. Measure its precision and recall on the same holdout."}]},{"t":"p","runs":[{"text":"The "},{"text":"simplest model","b":true},{"text":": logistic regression with five sensible columns. It also tells you, in plain numbers, which factors mattered and in which direction, which a boosted ensemble will not."}]},{"t":"p","runs":[{"text":"In small-business data, the gap between logistic regression and the best AutoML result is frequently a couple of percentage points. If that is the gap, take the simple model: you can explain it to your staff, debug it when it drifts, and see immediately when one of its inputs stops making sense."}]},{"t":"h2","id":"when-to-use-it","runs":[{"text":"When to use it"}]},{"t":"p","runs":[{"text":"When you have tens of thousands of rows, a defined target, features you built deliberately, and a rule-based baseline you want to beat. Then AutoML is a sensible hour’s work."}]},{"t":"p","runs":[{"text":"When you have four hundred rows and a hopeful question, it will still produce a model with a confident score, and the next lesson is about why that number is worse than nothing."}]}],"check":{"question":[{"text":"An owner runs AutoML on 8,000 customer records to predict lapse. It reports 91% accuracy and ranks "},{"text":"last_contact_type","code":true},{"text":" as the most important feature. What should she do before acting?"}],"options":[{"text":[{"text":"Rerun with that feature removed, to confirm the remaining model still performs acceptably without relying so heavily on a single column."}],"correct":false,"tempting":[{"text":"A reasonable robustness check, and it treats concentration as the problem when the question is what the column actually is."}]},{"text":[{"text":"Compare the result against logistic regression on the same columns, since a simpler model performing similarly would be easier to explain and maintain."}],"correct":false,"tempting":[{"text":"Good practice from this lesson, and it would reproduce the same fault if the column is contaminated."}]},{"text":[{"text":"Find out when "},{"text":"last_contact_type","code":true},{"text":" is recorded, since a contact prompted by her own suspicion would be reading her judgement back."}],"correct":true,"tempting":null},{"text":[{"text":"Check the accuracy against the base rate, because 91% may be close to what predicting “nobody lapses” would score on this dataset."}],"correct":false,"tempting":[{"text":"Exactly the right reflex from the previous lesson, and here the named feature is the more specific warning sign to follow first."}]}],"order":[1,2,0,3],"why":null},"prev":{"slug":"leakage-the-flattering-mistake","title":"The mistake that makes a model look brilliant","minutes":9},"next":{"slug":"small-numbers-and-superstition","title":"When your data is too small, and what to do instead","minutes":9}}}