{"course":{"slug":"machine-learning-foundations","title":"Machine Learning, Foundations","url":"https://addaly.com/learn/machine-learning-foundations","level":"intermediate","track":"start"},"lesson":{"slug":"the-fit-transform-contract","title":"Fit on train, transform everywhere: the contract that keeps you honest","url":"https://addaly.com/learn/machine-learning-foundations/the-fit-transform-contract","position":29,"of":92,"minutes":8,"answer":"Every transformation that learns a parameter must learn it inside each cross-validation fold, which is why a Pipeline changes the score you get rather than merely tidying the code.","keyIdea":"Every transformation that learns a parameter must learn it inside each cross-validation fold, which is why a Pipeline changes the score you get rather than merely tidying the code."},"body":{"format":"markdown","text":"## Two verbs, and the rule between them\n\nEvery preprocessing object in scikit-learn has two methods, and the whole discipline of this module comes down to when you call which.\n\n- **`fit`** learns parameters from data. A mean. A median. A vocabulary. A set of PCA components.\n- **`transform`** applies those learned parameters to data.\n\nThe rule: **`fit` sees training data only. `transform` is applied to everything.**\n\n```python\nsc = StandardScaler()\nX_train_s = sc.fit_transform(X_train)   # learns AND applies\nX_test_s  = sc.transform(X_test)        # applies only\n```\n\nCalling `fit_transform` on the test set is the error. It computes a new mean from the test rows, so training and test are now on different scales, and the test rows have influenced their own transformation.\n\n## Why doing it by hand fails\n\nIt is not that manual preprocessing is impossible. It is that it fails under cross-validation, and cross-validation is where you will spend most of your time.\n\nWith 5-fold cross-validation, the training portion changes five times. The scaler's mean must be recomputed five times, from five different subsets. Do the scaling once before the loop and every fold's \"held-out\" portion contributed to the scaling, quietly, in all five folds, with no error and no warning.\n\n`Pipeline` exists to make this structurally impossible:\n\n```python\nfrom sklearn.pipeline import Pipeline\npipe = Pipeline([(\"sc\", StandardScaler()), (\"clf\", LogisticRegression())])\ncross_val_score(pipe, X, y, cv=5)     # refits the scaler inside each fold\n```\n\nThat is not a style preference. It changes the number you get.\n\n## ColumnTransformer, because real tables are mixed\n\nDifferent columns need different treatment, and `ColumnTransformer` routes them:\n\n```python\nfrom sklearn.compose import ColumnTransformer\nfrom sklearn.impute import SimpleImputer\n\nprep = ColumnTransformer([\n    (\"num\", Pipeline([\n        (\"imp\", SimpleImputer(strategy=\"median\", add_indicator=True)),\n        (\"sc\", StandardScaler()),\n    ]), numeric_cols),\n    (\"cat\", Pipeline([\n        (\"imp\", SimpleImputer(strategy=\"constant\", fill_value=\"missing\")),\n        (\"oh\", OneHotEncoder(handle_unknown=\"ignore\", min_frequency=20)),\n    ]), categorical_cols),\n], remainder=\"drop\")\n```\n\n`remainder=\"drop\"` is worth setting explicitly. The default drops unlisted columns, which is usually right, but writing it down means the next person can see that the omission was intentional rather than an oversight.\n\nOne caution: selecting columns by dtype (`selector(dtype_include=\"number\")`) is convenient and fragile. A column that is numeric in training and arrives as a string in production (because one row had \"N/A\" in it) silently routes down the wrong branch. Explicit column lists are more typing and fewer 3am incidents.\n\n## Writing your own step\n\nAnything you can express as a function of the columns can become a pipeline step:\n\n```python\nfrom sklearn.preprocessing import FunctionTransformer\nlog_step = FunctionTransformer(np.log1p, feature_names_out=\"one-to-one\")\n```\n\n`FunctionTransformer` is for stateless transformations: ones that learn nothing. If your step needs to remember something from training (a set of clip boundaries, a category list), write a small class with `fit` and `transform`. Inherit from `BaseEstimator` and `TransformerMixin` and it will work everywhere in scikit-learn, including inside a grid search.\n\n## The payoff at deployment\n\nA fitted pipeline is one object containing every learned parameter: the medians, the category lists, the scaler's means, the model's coefficients.\n\n```python\nimport joblib\njoblib.dump(pipe, \"model.joblib\")\n# in the service\npipe = joblib.load(\"model.joblib\")\npipe.predict_proba(new_rows_dataframe)\n```\n\nOne artefact, one `predict` call, and the serving code cannot apply a different median from the training code because there is only one copy of it. This eliminates an entire category of production bug, training-serving skew, by construction rather than by discipline, which is the only way it stays eliminated after you leave the team.\n\nTwo cautions on saved model files. Version pinning matters: a model saved under scikit-learn 1.3 may fail to load under 1.6, so record the version alongside the file. And never load a model file from a source you do not trust: restoring one of these files can execute arbitrary code, so a model downloaded from the internet should be treated exactly like a downloaded executable.\n\n## Inspecting what came out\n\nAfter one-hot encoding you have no idea which column is which. Ask:\n\n```python\npipe[:-1].get_feature_names_out()[:20]\n```\n\nThis is how you connect a coefficient or an importance back to a human-readable name, and it is the difference between \"feature 43 is important\" and \"applications from unverified accounts are important\".\n\nThe same call is your best defence against a silent column-order change. If an upstream table gains a column, a pipeline built from explicit lists will ignore it, while one built from positional indexing will shift every downstream feature by one and keep running. Print the first twenty names into your training log on every run; a diff between two runs then makes the change visible the day it happens rather than the week the metrics drift."},"check":{"question":"A team runs SMOTE oversampling on the full training set, then performs 5-fold cross-validation on the resampled data. Cross-validated recall is excellent; production recall is poor. What is the structural error?","options":[{"text":"SMOTE should be applied to the test set as well, so that train and test have matching class distributions.","correct":false,"tempting":"Resampling a test set would destroy its representativeness; the held-out data must reflect the real base rate."},{"text":"SMOTE synthesises rows before splitting, so synthetic points derived from a fold's training rows appear in that fold's validation portion: the model is evaluated on near-copies of what it trained on.","correct":true},{"text":"SMOTE only works with continuous features, and the categorical columns were interpolated meaninglessly.","correct":false,"tempting":"A real limitation of SMOTE on mixed data, but it would degrade both the cross-validated and production scores rather than creating a gap between them."},{"text":"Recall is the wrong metric for imbalanced data, and precision would have shown the true performance.","correct":false,"tempting":"Recall alone is incomplete, but the discrepancy between validation and production is caused by the resampling order, not by the metric chosen."}]},"prev":{"slug":"leakage-in-detail","title":"Leakage: a catalogue of the ways the answer gets in","url":"https://addaly.com/learn/machine-learning-foundations/leakage-in-detail"},"next":{"slug":"where-the-rows-came-from","title":"Where the rows came from, and who is missing","url":"https://addaly.com/learn/machine-learning-foundations/where-the-rows-came-from"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Fit on train, transform everywhere: the contract that keeps you honest', in Machine Learning, Foundations. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/machine-learning-foundations/the-fit-transform-contract (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Fit on train, transform everywhere: the contract that keeps you honest","courseTitle":"Machine Learning, Foundations","module":{"index":3,"slug":"the-data-is-the-job","title":"The data is the job"},"answer":null,"keyIdea":"Every transformation that learns a parameter must learn it inside each cross-validation fold, which is why a Pipeline changes the score you get rather than merely tidying the code.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"two-verbs-and-the-rule-between-them","runs":[{"text":"Two verbs, and the rule between them"}]},{"t":"p","runs":[{"text":"Every preprocessing object in scikit-learn has two methods, and the whole discipline of this module comes down to when you call which."}]},{"t":"ul","items":[[{"text":"fit","b":true,"code":true},{"text":" learns parameters from data. A mean. A median. A vocabulary. A set of PCA components."}],[{"text":"transform","b":true,"code":true},{"text":" applies those learned parameters to data."}]]},{"t":"p","runs":[{"text":"The rule: "},{"text":"fit","b":true,"code":true},{"text":" sees training data only. ","b":true},{"text":"transform","b":true,"code":true},{"text":" is applied to everything.","b":true}]},{"t":"code","lang":"python","label":"Python","text":"sc = StandardScaler()\nX_train_s = sc.fit_transform(X_train)   # learns AND applies\nX_test_s  = sc.transform(X_test)        # applies only"},{"t":"p","runs":[{"text":"Calling "},{"text":"fit_transform","code":true},{"text":" on the test set is the error. It computes a new mean from the test rows, so training and test are now on different scales, and the test rows have influenced their own transformation."}]},{"t":"h2","id":"why-doing-it-by-hand-fails","runs":[{"text":"Why doing it by hand fails"}]},{"t":"p","runs":[{"text":"It is not that manual preprocessing is impossible. It is that it fails under cross-validation, and cross-validation is where you will spend most of your time."}]},{"t":"p","runs":[{"text":"With 5-fold cross-validation, the training portion changes five times. The scaler’s mean must be recomputed five times, from five different subsets. Do the scaling once before the loop and every fold’s “held-out” portion contributed to the scaling, quietly, in all five folds, with no error and no warning."}]},{"t":"p","runs":[{"text":"Pipeline","code":true},{"text":" exists to make this structurally impossible:"}]},{"t":"code","lang":"python","label":"Python","text":"from sklearn.pipeline import Pipeline\npipe = Pipeline([(\"sc\", StandardScaler()), (\"clf\", LogisticRegression())])\ncross_val_score(pipe, X, y, cv=5)     # refits the scaler inside each fold"},{"t":"p","runs":[{"text":"That is not a style preference. It changes the number you get."}]},{"t":"h2","id":"columntransformer-because-real-tables-are-mixed","runs":[{"text":"ColumnTransformer, because real tables are mixed"}]},{"t":"p","runs":[{"text":"Different columns need different treatment, and "},{"text":"ColumnTransformer","code":true},{"text":" routes them:"}]},{"t":"code","lang":"python","label":"Python","text":"from sklearn.compose import ColumnTransformer\nfrom sklearn.impute import SimpleImputer\n\nprep = ColumnTransformer([\n    (\"num\", Pipeline([\n        (\"imp\", SimpleImputer(strategy=\"median\", add_indicator=True)),\n        (\"sc\", StandardScaler()),\n    ]), numeric_cols),\n    (\"cat\", Pipeline([\n        (\"imp\", SimpleImputer(strategy=\"constant\", fill_value=\"missing\")),\n        (\"oh\", OneHotEncoder(handle_unknown=\"ignore\", min_frequency=20)),\n    ]), categorical_cols),\n], remainder=\"drop\")"},{"t":"p","runs":[{"text":"remainder=\"drop\"","code":true},{"text":" is worth setting explicitly. The default drops unlisted columns, which is usually right, but writing it down means the next person can see that the omission was intentional rather than an oversight."}]},{"t":"p","runs":[{"text":"One caution: selecting columns by dtype ("},{"text":"selector(dtype_include=\"number\")","code":true},{"text":") is convenient and fragile. A column that is numeric in training and arrives as a string in production (because one row had “N/A” in it) silently routes down the wrong branch. Explicit column lists are more typing and fewer 3am incidents."}]},{"t":"h2","id":"writing-your-own-step","runs":[{"text":"Writing your own step"}]},{"t":"p","runs":[{"text":"Anything you can express as a function of the columns can become a pipeline step:"}]},{"t":"code","lang":"python","label":"Python","text":"from sklearn.preprocessing import FunctionTransformer\nlog_step = FunctionTransformer(np.log1p, feature_names_out=\"one-to-one\")"},{"t":"p","runs":[{"text":"FunctionTransformer","code":true},{"text":" is for stateless transformations: ones that learn nothing. If your step needs to remember something from training (a set of clip boundaries, a category list), write a small class with "},{"text":"fit","code":true},{"text":" and "},{"text":"transform","code":true},{"text":". Inherit from "},{"text":"BaseEstimator","code":true},{"text":" and "},{"text":"TransformerMixin","code":true},{"text":" and it will work everywhere in scikit-learn, including inside a grid search."}]},{"t":"h2","id":"the-payoff-at-deployment","runs":[{"text":"The payoff at deployment"}]},{"t":"p","runs":[{"text":"A fitted pipeline is one object containing every learned parameter: the medians, the category lists, the scaler’s means, the model’s coefficients."}]},{"t":"code","lang":"python","label":"Python","text":"import joblib\njoblib.dump(pipe, \"model.joblib\")\n# in the service\npipe = joblib.load(\"model.joblib\")\npipe.predict_proba(new_rows_dataframe)"},{"t":"p","runs":[{"text":"One artefact, one "},{"text":"predict","code":true},{"text":" call, and the serving code cannot apply a different median from the training code because there is only one copy of it. This eliminates an entire category of production bug, training-serving skew, by construction rather than by discipline, which is the only way it stays eliminated after you leave the team."}]},{"t":"p","runs":[{"text":"Two cautions on saved model files. Version pinning matters: a model saved under scikit-learn 1.3 may fail to load under 1.6, so record the version alongside the file. And never load a model file from a source you do not trust: restoring one of these files can execute arbitrary code, so a model downloaded from the internet should be treated exactly like a downloaded executable."}]},{"t":"h2","id":"inspecting-what-came-out","runs":[{"text":"Inspecting what came out"}]},{"t":"p","runs":[{"text":"After one-hot encoding you have no idea which column is which. Ask:"}]},{"t":"code","lang":"python","label":"Python","text":"pipe[:-1].get_feature_names_out()[:20]"},{"t":"p","runs":[{"text":"This is how you connect a coefficient or an importance back to a human-readable name, and it is the difference between “feature 43 is important” and “applications from unverified accounts are important”."}]},{"t":"p","runs":[{"text":"The same call is your best defence against a silent column-order change. If an upstream table gains a column, a pipeline built from explicit lists will ignore it, while one built from positional indexing will shift every downstream feature by one and keep running. Print the first twenty names into your training log on every run; a diff between two runs then makes the change visible the day it happens rather than the week the metrics drift."}]}],"check":{"question":[{"text":"A team runs SMOTE oversampling on the full training set, then performs 5-fold cross-validation on the resampled data. Cross-validated recall is excellent; production recall is poor. What is the structural error?"}],"options":[{"text":[{"text":"SMOTE should be applied to the test set as well, so that train and test have matching class distributions."}],"correct":false,"tempting":[{"text":"Resampling a test set would destroy its representativeness; the held-out data must reflect the real base rate."}]},{"text":[{"text":"SMOTE synthesises rows before splitting, so synthetic points derived from a fold’s training rows appear in that fold’s validation portion: the model is evaluated on near-copies of what it trained on."}],"correct":true,"tempting":null},{"text":[{"text":"SMOTE only works with continuous features, and the categorical columns were interpolated meaninglessly."}],"correct":false,"tempting":[{"text":"A real limitation of SMOTE on mixed data, but it would degrade both the cross-validated and production scores rather than creating a gap between them."}]},{"text":[{"text":"Recall is the wrong metric for imbalanced data, and precision would have shown the true performance."}],"correct":false,"tempting":[{"text":"Recall alone is incomplete, but the discrepancy between validation and production is caused by the resampling order, not by the metric chosen."}]}],"order":[0,1,3,2],"why":null},"prev":{"slug":"leakage-in-detail","title":"Leakage: a catalogue of the ways the answer gets in","minutes":10},"next":{"slug":"where-the-rows-came-from","title":"Where the rows came from, and who is missing","minutes":9}}}