{"course":{"slug":"machine-learning-foundations","title":"Machine Learning, Foundations","url":"https://addaly.com/learn/machine-learning-foundations","level":"intermediate","track":"start"},"lesson":{"slug":"clustering-with-k-means","title":"k-means, and what it assumes about your data","url":"https://addaly.com/learn/machine-learning-foundations/clustering-with-k-means","position":63,"of":92,"minutes":9,"answer":"k-means minimises within-cluster squared distance, which assumes spherical clusters of similar size with no outliers and a known k, and it will return confident groupings from data that has no group structure at all.","keyIdea":"k-means minimises within-cluster squared distance, which assumes spherical clusters of similar size with no outliers and a known k, and it will return confident groupings from data that has no group structure at all."},"body":{"format":"markdown","text":"## The algorithm in four lines\n\n1. Pick k random points as initial centres.\n2. Assign every row to its nearest centre.\n3. Move each centre to the mean of the rows assigned to it.\n4. Repeat 2 and 3 until nothing moves.\n\nThat is the whole of k-means. It converges quickly, it is easy to implement, and it is the most used clustering algorithm by a wide margin.\n\n```python\nfrom sklearn.cluster import KMeans\nkm = KMeans(n_clusters=5, n_init=10, random_state=0).fit(X_scaled)\nkm.labels_, km.cluster_centers_, km.inertia_\n```\n\n`inertia_` is the sum of squared distances from each point to its assigned centre. It is what the algorithm minimises, and it will feature in the next lesson.\n\n## What it is quietly assuming\n\nEvery step above encodes an assumption, and each one fails on some real dataset in a way that produces confident nonsense.\n\n**Clusters are spherical.** Step 2 assigns by Euclidean distance to a centre, so the region belonging to each cluster is bounded by straight lines equidistant between centres. Two elongated, parallel groups get cut across rather than separated along their length. If your data has stretched or curved groups, k-means will not find them, and it will still return an answer.\n\n**Clusters are similar in size and density.** The mean is pulled towards wherever the points are, so a large dense cluster next to a small sparse one tends to have its centre annexe part of the small one.\n\n**Every point belongs somewhere.** There is no \"none of these\". An outlier 40 standard deviations away is assigned to whichever cluster is nearest and drags that centre towards it.\n\n**k is known.** The algorithm cannot tell you there are three groups when you asked for five. It will produce five.\n\n## Initialisation matters more than people expect\n\nk-means finds a local minimum, and different starting centres reach different ones. The classic failure is two centres landing inside one real cluster while a genuine second cluster shares a third.\n\n`k-means++` initialisation (the default in scikit-learn) chooses starting centres that are spread out, by picking each new centre with probability proportional to its squared distance from the nearest existing one. It reduces bad outcomes substantially, and `n_init=10` runs the whole thing ten times and keeps the best inertia, which reduces them further.\n\nDo not lower `n_init` to save time unless you have measured that it does not matter for your data.\n\n## Scaling, again\n\nk-means is a distance method, so a feature in rupees dominates one in years completely. Standardise before clustering, always.\n\nThere is a subtlety worth noticing. Standardising asserts that every feature should count equally in the distance, and that is itself a choice about what similarity means. If age matters twice as much as income for your purpose, you can say so by scaling age by two after standardising: clustering has no target to tell it what matters, so what matters is entirely encoded in how you scaled.\n\nThis is the deepest thing about unsupervised learning: **the answer depends on your definition of similar, and you supply that definition whether or not you notice.**\n\n## Variants worth knowing\n\n- **MiniBatchKMeans**: updates centres from small random batches. Much faster on large data, slightly worse clusters. Above a few hundred thousand rows, use it.\n- **Gaussian mixture models**: fit ellipse-shaped clusters with orientation, and give each point a probability of belonging to each cluster rather than a hard assignment. Strictly more flexible; k-means is the special case with spherical, equal-sized components and hard assignment.\n- **k-medoids**: uses actual data points as centres rather than means, so the centre is a real example you can look at, and it tolerates outliers better.\n\n## What clusters are actually for\n\nHonesty about the use case, because clustering is often run without one.\n\n- **Segmentation** for marketing or product decisions: the most common use, and the one most likely to produce groups that get named and then treated as real.\n- **Compression**: replace a row with its cluster id. Genuinely useful as a feature for a supervised model.\n- **Exploration**: look at the centres to understand what kinds of rows exist. Frequently the best value.\n- **Labelling triage**: cluster unlabelled data, label a few examples per cluster, and you have a rough labelled set for a fraction of the effort.\n\nWhat clusters are *not*: a discovery of real categories in the world. k-means will partition uniformly random noise into five tidy groups and report a respectable inertia. The groups will be reproducible, describable and entirely artefactual. Before believing any clustering result, run it on shuffled data and see whether the output looks meaningfully different."},"check":{"question":"A team clusters customer data into 4 groups and finds one enormous cluster and three tiny ones. The features include annual spend, which ranges from 0 to 40 million, alongside standardised behavioural features. What is the most likely cause?","options":[{"text":"Annual spend was not scaled, so distances are dominated by it and the three small clusters are just the extreme spenders split off from everyone else.","correct":true},{"text":"k=4 is too small, and increasing it would split the large cluster into meaningful groups.","correct":false,"tempting":"More clusters would subdivide the large group, but the lopsided result points at the distance metric rather than the number of centres."},{"text":"k-means++ initialisation placed three centres close together, and n_init should be raised.","correct":false,"tempting":"Poor initialisation does produce degenerate solutions, but n_init=10 with k-means++ makes this an unlikely explanation for a systematic pattern."},{"text":"The data has no cluster structure, so any partition will be arbitrary and unbalanced.","correct":false,"tempting":"Structureless data typically yields roughly balanced arbitrary partitions rather than one giant cluster and three tiny ones."}]},"prev":{"slug":"shap-and-partial-dependence","title":"Explaining one prediction, and the shape of one feature","url":"https://addaly.com/learn/machine-learning-foundations/shap-and-partial-dependence"},"next":{"slug":"how-many-clusters","title":"Choosing k, and why the elbow rarely helps","url":"https://addaly.com/learn/machine-learning-foundations/how-many-clusters"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'k-means, and what it assumes about your data', in Machine Learning, Foundations. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/machine-learning-foundations/clustering-with-k-means (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"k-means, and what it assumes about your data","courseTitle":"Machine Learning, Foundations","module":{"index":7,"slug":"learning-without-labels","title":"Learning without labels, and learning a representation"},"answer":null,"keyIdea":"k-means minimises within-cluster squared distance, which assumes spherical clusters of similar size with no outliers and a known k, and it will return confident groupings from data that has no group structure at all.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-algorithm-in-four-lines","runs":[{"text":"The algorithm in four lines"}]},{"t":"ol","start":1,"items":[[{"text":"Pick k random points as initial centres."}],[{"text":"Assign every row to its nearest centre."}],[{"text":"Move each centre to the mean of the rows assigned to it."}],[{"text":"Repeat 2 and 3 until nothing moves."}]]},{"t":"p","runs":[{"text":"That is the whole of k-means. It converges quickly, it is easy to implement, and it is the most used clustering algorithm by a wide margin."}]},{"t":"code","lang":"python","label":"Python","text":"from sklearn.cluster import KMeans\nkm = KMeans(n_clusters=5, n_init=10, random_state=0).fit(X_scaled)\nkm.labels_, km.cluster_centers_, km.inertia_"},{"t":"p","runs":[{"text":"inertia_","code":true},{"text":" is the sum of squared distances from each point to its assigned centre. It is what the algorithm minimises, and it will feature in the next lesson."}]},{"t":"h2","id":"what-it-is-quietly-assuming","runs":[{"text":"What it is quietly assuming"}]},{"t":"p","runs":[{"text":"Every step above encodes an assumption, and each one fails on some real dataset in a way that produces confident nonsense."}]},{"t":"p","runs":[{"text":"Clusters are spherical.","b":true},{"text":" Step 2 assigns by Euclidean distance to a centre, so the region belonging to each cluster is bounded by straight lines equidistant between centres. Two elongated, parallel groups get cut across rather than separated along their length. If your data has stretched or curved groups, k-means will not find them, and it will still return an answer."}]},{"t":"p","runs":[{"text":"Clusters are similar in size and density.","b":true},{"text":" The mean is pulled towards wherever the points are, so a large dense cluster next to a small sparse one tends to have its centre annexe part of the small one."}]},{"t":"p","runs":[{"text":"Every point belongs somewhere.","b":true},{"text":" There is no “none of these”. An outlier 40 standard deviations away is assigned to whichever cluster is nearest and drags that centre towards it."}]},{"t":"p","runs":[{"text":"k is known.","b":true},{"text":" The algorithm cannot tell you there are three groups when you asked for five. It will produce five."}]},{"t":"h2","id":"initialisation-matters-more-than-people-expect","runs":[{"text":"Initialisation matters more than people expect"}]},{"t":"p","runs":[{"text":"k-means finds a local minimum, and different starting centres reach different ones. The classic failure is two centres landing inside one real cluster while a genuine second cluster shares a third."}]},{"t":"p","runs":[{"text":"k-means++","code":true},{"text":" initialisation (the default in scikit-learn) chooses starting centres that are spread out, by picking each new centre with probability proportional to its squared distance from the nearest existing one. It reduces bad outcomes substantially, and "},{"text":"n_init=10","code":true},{"text":" runs the whole thing ten times and keeps the best inertia, which reduces them further."}]},{"t":"p","runs":[{"text":"Do not lower "},{"text":"n_init","code":true},{"text":" to save time unless you have measured that it does not matter for your data."}]},{"t":"h2","id":"scaling-again","runs":[{"text":"Scaling, again"}]},{"t":"p","runs":[{"text":"k-means is a distance method, so a feature in rupees dominates one in years completely. Standardise before clustering, always."}]},{"t":"p","runs":[{"text":"There is a subtlety worth noticing. Standardising asserts that every feature should count equally in the distance, and that is itself a choice about what similarity means. If age matters twice as much as income for your purpose, you can say so by scaling age by two after standardising: clustering has no target to tell it what matters, so what matters is entirely encoded in how you scaled."}]},{"t":"p","runs":[{"text":"This is the deepest thing about unsupervised learning: "},{"text":"the answer depends on your definition of similar, and you supply that definition whether or not you notice.","b":true}]},{"t":"h2","id":"variants-worth-knowing","runs":[{"text":"Variants worth knowing"}]},{"t":"ul","items":[[{"text":"MiniBatchKMeans","b":true},{"text":": updates centres from small random batches. Much faster on large data, slightly worse clusters. Above a few hundred thousand rows, use it."}],[{"text":"Gaussian mixture models","b":true},{"text":": fit ellipse-shaped clusters with orientation, and give each point a probability of belonging to each cluster rather than a hard assignment. Strictly more flexible; k-means is the special case with spherical, equal-sized components and hard assignment."}],[{"text":"k-medoids","b":true},{"text":": uses actual data points as centres rather than means, so the centre is a real example you can look at, and it tolerates outliers better."}]]},{"t":"h2","id":"what-clusters-are-actually-for","runs":[{"text":"What clusters are actually for"}]},{"t":"p","runs":[{"text":"Honesty about the use case, because clustering is often run without one."}]},{"t":"ul","items":[[{"text":"Segmentation","b":true},{"text":" for marketing or product decisions: the most common use, and the one most likely to produce groups that get named and then treated as real."}],[{"text":"Compression","b":true},{"text":": replace a row with its cluster id. Genuinely useful as a feature for a supervised model."}],[{"text":"Exploration","b":true},{"text":": look at the centres to understand what kinds of rows exist. Frequently the best value."}],[{"text":"Labelling triage","b":true},{"text":": cluster unlabelled data, label a few examples per cluster, and you have a rough labelled set for a fraction of the effort."}]]},{"t":"p","runs":[{"text":"What clusters are "},{"text":"not","i":true},{"text":": a discovery of real categories in the world. k-means will partition uniformly random noise into five tidy groups and report a respectable inertia. The groups will be reproducible, describable and entirely artefactual. Before believing any clustering result, run it on shuffled data and see whether the output looks meaningfully different."}]}],"check":{"question":[{"text":"A team clusters customer data into 4 groups and finds one enormous cluster and three tiny ones. The features include annual spend, which ranges from 0 to 40 million, alongside standardised behavioural features. What is the most likely cause?"}],"options":[{"text":[{"text":"Annual spend was not scaled, so distances are dominated by it and the three small clusters are just the extreme spenders split off from everyone else."}],"correct":true,"tempting":null},{"text":[{"text":"k=4 is too small, and increasing it would split the large cluster into meaningful groups."}],"correct":false,"tempting":[{"text":"More clusters would subdivide the large group, but the lopsided result points at the distance metric rather than the number of centres."}]},{"text":[{"text":"k-means++ initialisation placed three centres close together, and n_init should be raised."}],"correct":false,"tempting":[{"text":"Poor initialisation does produce degenerate solutions, but n_init=10 with k-means++ makes this an unlikely explanation for a systematic pattern."}]},{"text":[{"text":"The data has no cluster structure, so any partition will be arbitrary and unbalanced."}],"correct":false,"tempting":[{"text":"Structureless data typically yields roughly balanced arbitrary partitions rather than one giant cluster and three tiny ones."}]}],"order":[1,2,0,3],"why":null},"prev":{"slug":"shap-and-partial-dependence","title":"Explaining one prediction, and the shape of one feature","minutes":9},"next":{"slug":"how-many-clusters","title":"Choosing k, and why the elbow rarely helps","minutes":8}}}