{"course":{"slug":"machine-learning-foundations","title":"Machine Learning, Foundations","url":"https://addaly.com/learn/machine-learning-foundations","level":"intermediate","track":"start"},"lesson":{"slug":"feature-importance-and-its-lies","title":"Feature importance, and the ways it misleads","url":"https://addaly.com/learn/machine-learning-foundations/feature-importance-and-its-lies","position":61,"of":92,"minutes":9,"answer":"Impurity importance favours high-cardinality features and is computed on training data; permutation importance on validation data measures what you meant, but both collapse when features are correlated, so permute clustered groups rather than single columns.","keyIdea":"Impurity importance favours high-cardinality features and is computed on training data; permutation importance on validation data measures what you meant, but both collapse when features are correlated, so permute clustered groups rather than single columns."},"body":{"format":"markdown","text":"## The number everybody prints\n\n```python\nmodel.feature_importances_\n```\n\nOne line, a satisfying bar chart, and a plausible story about what drives the outcome. It is also, in its default form, systematically biased in ways that are easy to demonstrate and rarely mentioned.\n\n## What impurity importance actually measures\n\nFor a tree ensemble, the default importance sums the impurity decrease each feature achieved across all splits, weighted by how many rows passed through those nodes.\n\nThat definition contains two biases.\n\n**Bias towards high-cardinality features.** A feature with many distinct values offers many candidate thresholds, so by chance one of them will reduce impurity. Continuous features and high-cardinality categoricals therefore accumulate importance even when they carry no signal.\n\nThe demonstration is easy and worth running once. Add a column of random integers between 1 and 10,000 to your dataset and refit. It will appear in the top five features, above predictive binary columns. Nothing is broken; the metric is measuring what it says it measures, and what it measures is not what you wanted.\n\n> Figure: Impurity importance with a column of random noise added. Bars (share of total impurity decrease, computed on training rows): days_since_last_purchase 0.27; random integers 1 to 10,000 0.19; monthly_spend 0.16; tenure_months 0.12; is_verified (binary) 0.04; has_complained (binary) 0.03. The random column offers 10,000 candidate thresholds, so by chance some split reduces impurity, and it lands second: above two binary columns that genuinely predict the target. Nothing is broken. The metric measures what it says it measures, on training data, and that is not what you wanted.\n\n**Computed on training data.** Impurity importance reflects how the model used a feature to fit the training set, including the parts it overfitted. A feature that helped the model memorise noise scores highly.\n\n## Permutation importance\n\nA better default. Shuffle one feature's values in the *validation* set, breaking its relationship with the target while leaving the rest of the row intact, and measure how much the score drops.\n\n```python\nfrom sklearn.inspection import permutation_importance\nr = permutation_importance(model, X_val, y_val, n_repeats=10,\n                           scoring=\"roc_auc\", random_state=0)\n```\n\nThis measures what you probably meant: how much this model, on held-out data, depends on this feature. It works for any model, it uses the metric you care about, and it gives a standard deviation across repeats so you can see which differences are real.\n\nIt has one serious failure mode, and it is not optional knowledge.\n\n## The correlated-feature problem\n\nSuppose `height_cm` and `height_inches` are both in the model. Shuffle one, and the model reads the other and predicts fine. Score barely drops. Shuffle the other, same result. Both features get near-zero importance, and the conclusion \"height does not matter\" is exactly backwards.\n\nThis affects both methods, and it affects them in different directions: impurity importance splits the credit between correlated features, permutation importance zeroes both.\n\nTwo practical responses:\n\n1. **Cluster correlated features and permute the whole group at once.** Compute the correlation matrix, cluster it, and shuffle each cluster together. This tells you what the *group* is worth, which is usually the question.\n2. **Drop-column importance.** Remove a feature, refit, measure the drop. This is the most trustworthy method and costs one refit per feature, so it is affordable only with fast models and few columns.\n\nThe second also has a subtlety: if you drop `height_cm` and refit, the model learns to use `height_inches` instead and the score does not fall. The answer is the same: cluster first.\n\n## Importance is not effect, and not causation\n\nThree distinctions that get collapsed constantly.\n\n**Importance is not effect size.** \"Income is the most important feature\" does not tell you whether more income raises or lowers the prediction, or by how much. It says the model relies on it.\n\n**Importance is a property of the model, not the world.** Two models fitted on the same data can rank features differently, both legitimately, because they use different structure.\n\n**Importance is not causation.** Module 1 made this argument and it applies here with full force: a feature can be important because it is a consequence of the target. A high importance on a suspicious feature is a leakage alarm before it is anything else.\n\n## What importance is good for\n\nHaving spent the lesson on limits, the honest positive list.\n\n- **Finding leakage.** One feature far above the others on a hard problem is the most reliable leakage detector there is.\n- **Feature selection.** Drop the bottom half, refit, check whether the score moved. Often it does not, and you have halved your pipeline's maintenance cost.\n- **Sanity checking.** If your domain expert says the top feature should be irrelevant, one of you has learned something.\n- **Debugging a pipeline.** A feature with exactly zero importance across every tree is often not reaching the model at all.\n\nNone of those uses requires importance to be a measure of real-world influence. All of them are valuable. The failure is only ever in the last step, when a bar chart becomes a slide titled \"what drives customer churn\"."},"check":{"question":"A model contains both `total_spend` and `average_order_value`, which are strongly correlated. Permutation importance gives both near zero. What should you conclude?","options":[{"text":"Spending behaviour is genuinely unimportant to this model, and both columns can be dropped.","correct":false,"tempting":"Reads the number at face value; dropping both would likely cause a large score drop, which is exactly what the individual permutations failed to reveal."},{"text":"The permutation test was run on training data, which is why the importances collapsed.","correct":false,"tempting":"Running on training data inflates importances rather than zeroing them, and the described pattern points to redundancy between columns."},{"text":"Shuffling one leaves the other intact, so the model recovers the same information and the score barely moves; permute the correlated group together to see what spending is worth.","correct":true},{"text":"Near-zero permutation importance always indicates a feature that never reached the model, so the pipeline is dropping both columns.","correct":false,"tempting":"A plumbing fault is one cause of zero importance, but it would show equally in impurity importance and is ruled out by the columns being correlated in a way the model exploits."}]},"prev":{"slug":"support-vector-machines","title":"Support vector machines and the kernel trick","url":"https://addaly.com/learn/machine-learning-foundations/support-vector-machines"},"next":{"slug":"shap-and-partial-dependence","title":"Explaining one prediction, and the shape of one feature","url":"https://addaly.com/learn/machine-learning-foundations/shap-and-partial-dependence"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Feature importance, and the ways it misleads', in Machine Learning, Foundations. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/machine-learning-foundations/feature-importance-and-its-lies (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Feature importance, and the ways it misleads","courseTitle":"Machine Learning, Foundations","module":{"index":6,"slug":"trees-and-ensembles","title":"Trees, ensembles, and the algorithms that win on tables"},"answer":null,"keyIdea":"Impurity importance favours high-cardinality features and is computed on training data; permutation importance on validation data measures what you meant, but both collapse when features are correlated, so permute clustered groups rather than single columns.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"bars","unit":"share of total impurity decrease, computed on training rows","title":"Impurity importance with a column of random noise added","series":[{"label":"days_since_last_purchase","value":0.27},{"label":"random integers 1 to 10,000","value":0.19,"highlight":true},{"label":"monthly_spend","value":0.16},{"label":"tenure_months","value":0.12},{"label":"is_verified (binary)","value":0.04},{"label":"has_complained (binary)","value":0.03}],"caption":"The random column offers 10,000 candidate thresholds, so by chance some split reduces impurity, and it lands second: above two binary columns that genuinely predict the target. Nothing is broken. The metric measures what it says it measures, on training data, and that is not what you wanted."},"alt":"Figure: Impurity importance with a column of random noise added. Bars (share of total impurity decrease, computed on training rows): days_since_last_purchase 0.27; random integers 1 to 10,000 0.19; monthly_spend 0.16; tenure_months 0.12; is_verified (binary) 0.04; has_complained (binary) 0.03. The random column offers 10,000 candidate thresholds, so by chance some split reduces impurity, and it lands second: above two binary columns that genuinely predict the target. Nothing is broken. The metric measures what it says it measures, on training data, and that is not what you wanted."}},"blocks":[{"t":"h2","id":"the-number-everybody-prints","runs":[{"text":"The number everybody prints"}]},{"t":"code","lang":"python","label":"Python","text":"model.feature_importances_"},{"t":"p","runs":[{"text":"One line, a satisfying bar chart, and a plausible story about what drives the outcome. It is also, in its default form, systematically biased in ways that are easy to demonstrate and rarely mentioned."}]},{"t":"h2","id":"what-impurity-importance-actually-measures","runs":[{"text":"What impurity importance actually measures"}]},{"t":"p","runs":[{"text":"For a tree ensemble, the default importance sums the impurity decrease each feature achieved across all splits, weighted by how many rows passed through those nodes."}]},{"t":"p","runs":[{"text":"That definition contains two biases."}]},{"t":"p","runs":[{"text":"Bias towards high-cardinality features.","b":true},{"text":" A feature with many distinct values offers many candidate thresholds, so by chance one of them will reduce impurity. Continuous features and high-cardinality categoricals therefore accumulate importance even when they carry no signal."}]},{"t":"p","runs":[{"text":"The demonstration is easy and worth running once. Add a column of random integers between 1 and 10,000 to your dataset and refit. It will appear in the top five features, above predictive binary columns. Nothing is broken; the metric is measuring what it says it measures, and what it measures is not what you wanted."}]},{"t":"p","runs":[{"text":"Computed on training data.","b":true},{"text":" Impurity importance reflects how the model used a feature to fit the training set, including the parts it overfitted. A feature that helped the model memorise noise scores highly."}]},{"t":"h2","id":"permutation-importance","runs":[{"text":"Permutation importance"}]},{"t":"p","runs":[{"text":"A better default. Shuffle one feature’s values in the "},{"text":"validation","i":true},{"text":" set, breaking its relationship with the target while leaving the rest of the row intact, and measure how much the score drops."}]},{"t":"code","lang":"python","label":"Python","text":"from sklearn.inspection import permutation_importance\nr = permutation_importance(model, X_val, y_val, n_repeats=10,\n                           scoring=\"roc_auc\", random_state=0)"},{"t":"p","runs":[{"text":"This measures what you probably meant: how much this model, on held-out data, depends on this feature. It works for any model, it uses the metric you care about, and it gives a standard deviation across repeats so you can see which differences are real."}]},{"t":"p","runs":[{"text":"It has one serious failure mode, and it is not optional knowledge."}]},{"t":"h2","id":"the-correlated-feature-problem","runs":[{"text":"The correlated-feature problem"}]},{"t":"p","runs":[{"text":"Suppose "},{"text":"height_cm","code":true},{"text":" and "},{"text":"height_inches","code":true},{"text":" are both in the model. Shuffle one, and the model reads the other and predicts fine. Score barely drops. Shuffle the other, same result. Both features get near-zero importance, and the conclusion “height does not matter” is exactly backwards."}]},{"t":"p","runs":[{"text":"This affects both methods, and it affects them in different directions: impurity importance splits the credit between correlated features, permutation importance zeroes both."}]},{"t":"p","runs":[{"text":"Two practical responses:"}]},{"t":"ol","start":1,"items":[[{"text":"Cluster correlated features and permute the whole group at once.","b":true},{"text":" Compute the correlation matrix, cluster it, and shuffle each cluster together. This tells you what the "},{"text":"group","i":true},{"text":" is worth, which is usually the question."}],[{"text":"Drop-column importance.","b":true},{"text":" Remove a feature, refit, measure the drop. This is the most trustworthy method and costs one refit per feature, so it is affordable only with fast models and few columns."}]]},{"t":"p","runs":[{"text":"The second also has a subtlety: if you drop "},{"text":"height_cm","code":true},{"text":" and refit, the model learns to use "},{"text":"height_inches","code":true},{"text":" instead and the score does not fall. The answer is the same: cluster first."}]},{"t":"h2","id":"importance-is-not-effect-and-not-causation","runs":[{"text":"Importance is not effect, and not causation"}]},{"t":"p","runs":[{"text":"Three distinctions that get collapsed constantly."}]},{"t":"p","runs":[{"text":"Importance is not effect size.","b":true},{"text":" “Income is the most important feature” does not tell you whether more income raises or lowers the prediction, or by how much. It says the model relies on it."}]},{"t":"p","runs":[{"text":"Importance is a property of the model, not the world.","b":true},{"text":" Two models fitted on the same data can rank features differently, both legitimately, because they use different structure."}]},{"t":"p","runs":[{"text":"Importance is not causation.","b":true},{"text":" Module 1 made this argument and it applies here with full force: a feature can be important because it is a consequence of the target. A high importance on a suspicious feature is a leakage alarm before it is anything else."}]},{"t":"h2","id":"what-importance-is-good-for","runs":[{"text":"What importance is good for"}]},{"t":"p","runs":[{"text":"Having spent the lesson on limits, the honest positive list."}]},{"t":"ul","items":[[{"text":"Finding leakage.","b":true},{"text":" One feature far above the others on a hard problem is the most reliable leakage detector there is."}],[{"text":"Feature selection.","b":true},{"text":" Drop the bottom half, refit, check whether the score moved. Often it does not, and you have halved your pipeline’s maintenance cost."}],[{"text":"Sanity checking.","b":true},{"text":" If your domain expert says the top feature should be irrelevant, one of you has learned something."}],[{"text":"Debugging a pipeline.","b":true},{"text":" A feature with exactly zero importance across every tree is often not reaching the model at all."}]]},{"t":"p","runs":[{"text":"None of those uses requires importance to be a measure of real-world influence. All of them are valuable. The failure is only ever in the last step, when a bar chart becomes a slide titled “what drives customer churn”."}]}],"check":{"question":[{"text":"A model contains both "},{"text":"total_spend","code":true},{"text":" and "},{"text":"average_order_value","code":true},{"text":", which are strongly correlated. Permutation importance gives both near zero. What should you conclude?"}],"options":[{"text":[{"text":"Spending behaviour is genuinely unimportant to this model, and both columns can be dropped."}],"correct":false,"tempting":[{"text":"Reads the number at face value; dropping both would likely cause a large score drop, which is exactly what the individual permutations failed to reveal."}]},{"text":[{"text":"The permutation test was run on training data, which is why the importances collapsed."}],"correct":false,"tempting":[{"text":"Running on training data inflates importances rather than zeroing them, and the described pattern points to redundancy between columns."}]},{"text":[{"text":"Shuffling one leaves the other intact, so the model recovers the same information and the score barely moves; permute the correlated group together to see what spending is worth."}],"correct":true,"tempting":null},{"text":[{"text":"Near-zero permutation importance always indicates a feature that never reached the model, so the pipeline is dropping both columns."}],"correct":false,"tempting":[{"text":"A plumbing fault is one cause of zero importance, but it would show equally in impurity importance and is ruled out by the columns being correlated in a way the model exploits."}]}],"order":[3,0,1,2],"why":null},"prev":{"slug":"support-vector-machines","title":"Support vector machines and the kernel trick","minutes":9},"next":{"slug":"shap-and-partial-dependence","title":"Explaining one prediction, and the shape of one feature","minutes":9}}}