{"course":{"slug":"machine-learning-foundations","title":"Machine Learning, Foundations","url":"https://addaly.com/learn/machine-learning-foundations","level":"intermediate","track":"start"},"lesson":{"slug":"how-much-data-is-enough","title":"How much data do I need?","url":"https://addaly.com/learn/machine-learning-foundations/how-much-data-is-enough","position":40,"of":92,"minutes":8,"answer":"Size the validation set from the precision you need (the standard error of a proportion at n=1,000 is about 1.1 points), and size the training set by events per feature rather than by total rows.","keyIdea":"Size the validation set from the precision you need (the standard error of a proportion at n=1,000 is about 1.1 points), and size the training set by events per feature rather than by total rows."},"body":{"format":"markdown","text":"## The answer, and then the useful ones\n\nThe honest answer is that it depends on how strong the signal is, and you cannot know that before you have some data. What follows are the approximations practitioners actually use, with their reasoning, so you can adapt them rather than recite them.\n\n## Rules of thumb, and where they come from\n\n**Ten to twenty rows per feature, for a linear model.** Below that, the coefficients are estimated so noisily that they are close to meaningless. It comes from the statistics of parameter estimation, and 30 features with 200 rows is a bad situation regardless of what the validation score says.\n\n**Ten to fifteen *events* per feature, for classification.** This is the version that matters, and the distinction is the one people miss. With 100,000 rows and 200 fraud cases, your effective sample size for learning what fraud looks like is 200, not 100,000. This is the epidemiologists' events-per-variable rule and it transfers directly.\n\n**A few hundred labelled examples per class, minimum, for anything non-trivial.** Under 50 per class you cannot reliably distinguish a good model from a lucky one, because your validation set for that class is a dozen rows.\n\n**For text classification with TF-IDF and a linear model**, a few thousand labelled documents usually gets you most of the achievable performance. For fine-tuning a pretrained transformer, a few hundred to a few thousand is often enough, because the model already knows the language and you are only teaching it your task.\n\n## The number that actually constrains you\n\nWork backwards from the validation set instead of forwards from the training set. Ask: *how precisely do I need to know my score?*\n\nFor a proportion, the standard error is roughly `sqrt(p * (1 - p) / n)`. At p = 0.85:\n\n- n = 100: standard error 3.6 points. The 95% interval spans about 14 points.\n- n = 1,000: 1.1 points. Interval about 4.5 points.\n- n = 10,000: 0.36 points. Interval about 1.4 points.\n\nSo if you need to detect a 1-point improvement, you need on the order of 10,000 validation rows and no fewer. If you have 300, you can detect large differences and nothing else, and running a 200-configuration search against those 300 rows is theatre.\n\nThis calculation takes one minute, it is the single most useful thing in this lesson, and almost nobody does it before starting.\n\n## When you have less than you need\n\nAll of these are real options, and none of them manufactures information.\n\n- **Use a simpler model.** Fewer parameters to estimate, less data required. On 400 rows, logistic regression frequently beats gradient boosting.\n- **Regularise harder.** Strong regularisation is how you spend a small dataset without overfitting it.\n- **Use cross-validation rather than a single split.** Every row does both jobs.\n- **Transfer learning.** Start from a model trained on a large related dataset and adapt. This is why a few hundred images can be enough for image classification today and were nowhere near enough in 2012.\n- **Augment.** For images and audio, generate variations: rotations, crops, noise. Genuinely effective, and it works because the transformations preserve the label. For tabular data there is no equivalent that reliably works; SMOTE is the usual attempt and module 3 explained its limits.\n- **Use rules for the part you understand** and a model for the rest, as module 1 argued.\n\n## The counter-argument to \"we need more data\"\n\nBefore anybody commissions a collection effort, run the learning curve from two lessons ago. Teams routinely ask for more data when their curve has been flat for the last three doublings, and what they actually need is a feature nobody has built.\n\nThe order that saves the most time: check the learning curve, check the baseline, check for leakage, look at 50 misclassified rows by hand. Then decide about data collection. Every one of those steps is cheaper than a quarter of collection, and any of them can change the answer.\n\nAnd when collection is the answer, be specific about what to collect. \"More data\" usually means one of three quite different things: more rows of the same kind, which helps only if the learning curve is still climbing; more rows of a specific underrepresented kind, which helps when a slice is failing; or more *columns* about the rows you already have, which is not a collection project at all but a data-access one. Teams that ask for the first and needed the third spend six months and land back where they started."},"check":{"question":"A team has 40,000 rows with a 0.5% positive rate and 60 features. They are surprised that a gradient boosting model performs poorly. What does the events-per-variable view say?","options":[{"text":"40,000 rows is ample for 60 features, so the problem must be feature quality rather than sample size.","correct":false,"tempting":"Counts total rows, which is the framing that makes this situation look comfortable; what constrains learning here is the number of positive cases."},{"text":"They have about 200 positive cases for 60 features (roughly 3 events per variable) so there is far too little information about the positive class to estimate 60 effects.","correct":true},{"text":"The class imbalance means accuracy is misleading, and switching to average precision would reveal that the model is fine.","correct":false,"tempting":"Metric choice does matter at this prevalence, but a better metric reveals performance rather than creating the events needed to learn from."},{"text":"Gradient boosting requires balanced classes, so undersampling the majority to 200 rows would fix it.","correct":false,"tempting":"Undersampling to match would leave 400 rows total and discard nearly all the information about what normal looks like."}]},"prev":{"slug":"the-validation-set-wears-out","title":"A validation set you have used two hundred times","url":"https://addaly.com/learn/machine-learning-foundations/the-validation-set-wears-out"},"next":{"slug":"when-more-parameters-stopped-being-worse","title":"Double descent, and the picture that stopped being complete","url":"https://addaly.com/learn/machine-learning-foundations/when-more-parameters-stopped-being-worse"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'How much data do I need?', in Machine Learning, Foundations. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/machine-learning-foundations/how-much-data-is-enough (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"How much data do I need?","courseTitle":"Machine Learning, Foundations","module":{"index":4,"slug":"generalisation","title":"Generalisation, and the discipline of held-out data"},"answer":null,"keyIdea":"Size the validation set from the precision you need (the standard error of a proportion at n=1,000 is about 1.1 points), and size the training set by events per feature rather than by total rows.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-answer-and-then-the-useful-ones","runs":[{"text":"The answer, and then the useful ones"}]},{"t":"p","runs":[{"text":"The honest answer is that it depends on how strong the signal is, and you cannot know that before you have some data. What follows are the approximations practitioners actually use, with their reasoning, so you can adapt them rather than recite them."}]},{"t":"h2","id":"rules-of-thumb-and-where-they-come-from","runs":[{"text":"Rules of thumb, and where they come from"}]},{"t":"p","runs":[{"text":"Ten to twenty rows per feature, for a linear model.","b":true},{"text":" Below that, the coefficients are estimated so noisily that they are close to meaningless. It comes from the statistics of parameter estimation, and 30 features with 200 rows is a bad situation regardless of what the validation score says."}]},{"t":"p","runs":[{"text":"**Ten to fifteen "},{"text":"events","i":true},{"text":" per feature, for classification.** This is the version that matters, and the distinction is the one people miss. With 100,000 rows and 200 fraud cases, your effective sample size for learning what fraud looks like is 200, not 100,000. This is the epidemiologists’ events-per-variable rule and it transfers directly."}]},{"t":"p","runs":[{"text":"A few hundred labelled examples per class, minimum, for anything non-trivial.","b":true},{"text":" Under 50 per class you cannot reliably distinguish a good model from a lucky one, because your validation set for that class is a dozen rows."}]},{"t":"p","runs":[{"text":"For text classification with TF-IDF and a linear model","b":true},{"text":", a few thousand labelled documents usually gets you most of the achievable performance. For fine-tuning a pretrained transformer, a few hundred to a few thousand is often enough, because the model already knows the language and you are only teaching it your task."}]},{"t":"h2","id":"the-number-that-actually-constrains-you","runs":[{"text":"The number that actually constrains you"}]},{"t":"p","runs":[{"text":"Work backwards from the validation set instead of forwards from the training set. Ask: "},{"text":"how precisely do I need to know my score?","i":true}]},{"t":"p","runs":[{"text":"For a proportion, the standard error is roughly "},{"text":"sqrt(p * (1 - p) / n)","code":true},{"text":". At p = 0.85:"}]},{"t":"ul","items":[[{"text":"n = 100: standard error 3.6 points. The 95% interval spans about 14 points."}],[{"text":"n = 1,000: 1.1 points. Interval about 4.5 points."}],[{"text":"n = 10,000: 0.36 points. Interval about 1.4 points."}]]},{"t":"p","runs":[{"text":"So if you need to detect a 1-point improvement, you need on the order of 10,000 validation rows and no fewer. If you have 300, you can detect large differences and nothing else, and running a 200-configuration search against those 300 rows is theatre."}]},{"t":"p","runs":[{"text":"This calculation takes one minute, it is the single most useful thing in this lesson, and almost nobody does it before starting."}]},{"t":"h2","id":"when-you-have-less-than-you-need","runs":[{"text":"When you have less than you need"}]},{"t":"p","runs":[{"text":"All of these are real options, and none of them manufactures information."}]},{"t":"ul","items":[[{"text":"Use a simpler model.","b":true},{"text":" Fewer parameters to estimate, less data required. On 400 rows, logistic regression frequently beats gradient boosting."}],[{"text":"Regularise harder.","b":true},{"text":" Strong regularisation is how you spend a small dataset without overfitting it."}],[{"text":"Use cross-validation rather than a single split.","b":true},{"text":" Every row does both jobs."}],[{"text":"Transfer learning.","b":true},{"text":" Start from a model trained on a large related dataset and adapt. This is why a few hundred images can be enough for image classification today and were nowhere near enough in 2012."}],[{"text":"Augment.","b":true},{"text":" For images and audio, generate variations: rotations, crops, noise. Genuinely effective, and it works because the transformations preserve the label. For tabular data there is no equivalent that reliably works; SMOTE is the usual attempt and module 3 explained its limits."}],[{"text":"Use rules for the part you understand","b":true},{"text":" and a model for the rest, as module 1 argued."}]]},{"t":"h2","id":"the-counter-argument-to-we-need-more-data","runs":[{"text":"The counter-argument to “we need more data”"}]},{"t":"p","runs":[{"text":"Before anybody commissions a collection effort, run the learning curve from two lessons ago. Teams routinely ask for more data when their curve has been flat for the last three doublings, and what they actually need is a feature nobody has built."}]},{"t":"p","runs":[{"text":"The order that saves the most time: check the learning curve, check the baseline, check for leakage, look at 50 misclassified rows by hand. Then decide about data collection. Every one of those steps is cheaper than a quarter of collection, and any of them can change the answer."}]},{"t":"p","runs":[{"text":"And when collection is the answer, be specific about what to collect. “More data” usually means one of three quite different things: more rows of the same kind, which helps only if the learning curve is still climbing; more rows of a specific underrepresented kind, which helps when a slice is failing; or more "},{"text":"columns","i":true},{"text":" about the rows you already have, which is not a collection project at all but a data-access one. Teams that ask for the first and needed the third spend six months and land back where they started."}]}],"check":{"question":[{"text":"A team has 40,000 rows with a 0.5% positive rate and 60 features. They are surprised that a gradient boosting model performs poorly. What does the events-per-variable view say?"}],"options":[{"text":[{"text":"40,000 rows is ample for 60 features, so the problem must be feature quality rather than sample size."}],"correct":false,"tempting":[{"text":"Counts total rows, which is the framing that makes this situation look comfortable; what constrains learning here is the number of positive cases."}]},{"text":[{"text":"They have about 200 positive cases for 60 features (roughly 3 events per variable) so there is far too little information about the positive class to estimate 60 effects."}],"correct":true,"tempting":null},{"text":[{"text":"The class imbalance means accuracy is misleading, and switching to average precision would reveal that the model is fine."}],"correct":false,"tempting":[{"text":"Metric choice does matter at this prevalence, but a better metric reveals performance rather than creating the events needed to learn from."}]},{"text":[{"text":"Gradient boosting requires balanced classes, so undersampling the majority to 200 rows would fix it."}],"correct":false,"tempting":[{"text":"Undersampling to match would leave 400 rows total and discard nearly all the information about what normal looks like."}]}],"order":[2,3,1,0],"why":null},"prev":{"slug":"the-validation-set-wears-out","title":"A validation set you have used two hundred times","minutes":8},"next":{"slug":"when-more-parameters-stopped-being-worse","title":"Double descent, and the picture that stopped being complete","minutes":8}}}