{"course":{"slug":"machine-learning-foundations","title":"Machine Learning, Foundations","url":"https://addaly.com/learn/machine-learning-foundations","level":"intermediate","track":"start"},"lesson":{"slug":"fitting-a-line-by-hand","title":"Fitting a line to five points, by hand","url":"https://addaly.com/learn/machine-learning-foundations/fitting-a-line-by-hand","position":10,"of":92,"minutes":9,"answer":"Fitting means choosing parameter values that make a chosen error number small; the coefficient carries real units, and a linear model will extrapolate far outside its data without any signal that it is guessing.","keyIdea":"Fitting means choosing parameter values that make a chosen error number small; the coefficient carries real units, and a linear model will extrapolate far outside its data without any signal that it is guessing."},"body":{"format":"markdown","text":"## Five houses\n\nSize in square metres, price in lakhs.\n\n```text\nsize   price\n 50     32\n 70     44\n 90     51\n110     67\n130     74\n```\n\nWe want `price = w * size + b`. Two numbers. Everything about model fitting is visible in finding them.\n\n## Guess, and measure how wrong you are\n\nStart with `w = 0.5`, `b = 0`. Predictions are 25, 35, 45, 55, 65. The errors (actual minus predicted) are 7, 9, 6, 12, 9.\n\nMean squared error:\n\n```text\n(49 + 81 + 36 + 144 + 81) / 5 = 391 / 5 = 78.2\n```\n\nNow try `w = 0.55`, `b = 3`. Predictions: 30.5, 41.5, 52.5, 63.5, 74.5. Errors: 1.5, 2.5, -1.5, 3.5, -0.5. MSE = `(2.25 + 6.25 + 2.25 + 12.25 + 0.25) / 5 = 4.65`.\n\nThe second model is better and we know it by a number, not by looking. That number is the entire mechanism. Everything else in this module is about finding good values of `w` and `b` faster than by guessing.\n\n## Which direction is downhill?\n\nA useful thing about MSE is that you can compute, for the parameters you are currently holding, which way to move them. For one row the loss is `(y - (w*x + b))^2`, and the amount the loss changes when you nudge `w` is\n\n```text\nd/dw = -2 * x * (y - prediction)\n```\n\nRead that without the calculus. It says: the push on `w` is proportional to the error, and scaled by `x`. A row where you are badly wrong pushes hard. A row with a large `x` pushes harder than one with a small `x` for the same error, which is exactly why unscaled features cause trouble later.\n\nFor the first house at `w = 0.5, b = 0`: error is 7, `x` is 50, so the derivative is `-2 * 50 * 7 = -700`. Negative, meaning increasing `w` reduces the loss. Averaged over all five rows, and moved a small step at a time, this is gradient descent, which you will meet properly in two lessons.\n\n## Doing it in one line\n\n```python\nimport numpy as np\nfrom sklearn.linear_model import LinearRegression\n\nX = np.array([[50], [70], [90], [110], [130]])\ny = np.array([32, 44, 51, 67, 74])\n\nm = LinearRegression().fit(X, y)\nm.coef_, m.intercept_        # array([0.525]), 6.15\nm.predict([[100]])           # array([58.65])\n```\n\nThe best possible line for these five points is `price = 0.525 * size + 6.15`, MSE 3.585. Our hand guess of `0.55` and `3` got to 4.65, respectable, and it shows that near the optimum the loss surface is fairly flat, which is a property we will come back to.\n\n## What the two numbers mean\n\n`w = 0.525` means: **within this data**, each additional square metre is associated with 0.525 lakh more in price. Units are lakhs per square metre. A coefficient always has units, and stating them is the fastest way to catch a nonsense model.\n\n`b = 6.15` is the predicted price of a house of zero square metres, which does not exist. The intercept is usually not interpretable on its own; it is the number that positions the line vertically. If you centre your features first (subtract the mean from each) the intercept becomes the prediction at the average house, which is interpretable.\n\n## Two limitations you can see with five points\n\n**Extrapolation is a guess dressed as arithmetic.** Ask this model for a 400 square metre house and it answers 216. It has never seen a house above 130 and has no way to know the relationship flattens, or that a house that large is a different market. The model will not warn you. Nothing in a linear model warns you when you leave the range of the training data, and this is the failure that most often reaches production quietly.\n\n**Five points cannot tell you the relationship is linear.** They are consistent with a line. They are also consistent with a gentle curve, and with the true relationship being a step at 100 square metres. With five points you cannot distinguish these; with five hundred you often can. The straight line is an assumption you imposed, not a finding.\n\n## Try changing one number\n\nChange the last price from 74 to 140, say a penthouse crept into the data, and refit. The coefficient jumps from 0.525 to about 0.855 and the whole line tilts. One row in five moved the answer by 63%.\n\nThat sensitivity is the squared-error property from the previous module, made visible. With 5,000 rows one outlier moves little. With 50 rows it moves a lot. It is worth knowing which regime you are in before you defend a coefficient in a meeting.\n\n## Why this simple thing survives\n\nLinear regression is 220 years old and still the first model to fit on a new problem. It trains in milliseconds, gives a number per feature, extrapolates predictably rather than catastrophically, and needs no tuning. As the baseline lesson argued, you need something to beat. Half the time on tabular data, the fancy model beats this by less than anyone expected, and that fact is worth discovering in the first hour rather than the last."},"check":{"question":"You fit a line to 40 houses between 50 and 130 square metres and get 0.525 lakh per square metre. A colleague uses it to price a 400 square metre villa. What is the specific defect in that use?","options":[{"text":"The model was fitted with MSE, which is sensitive to outliers, and a 400 square metre house is an outlier.","correct":false,"tempting":"Correctly recalls that MSE is outlier-sensitive, but that concerns rows in the training data, not a prediction requested outside its range."},{"text":"The coefficient is only valid for the range of sizes observed; outside it the model applies a slope it has no evidence for, and it produces the extrapolation with no indication of doing so.","correct":true},{"text":"The intercept of 6.15 becomes meaningless at large sizes, so the prediction is offset by that amount.","correct":false,"tempting":"Fixes on the intercept, which contributes the same fixed 6.15 at every size and is not what makes the villa prediction unreliable."},{"text":"Linear regression requires at least 100 rows to be valid, and 40 is too few for any prediction.","correct":false,"tempting":"Invents a sample-size rule; 40 rows give a usable but uncertain fit, and no row count would make extrapolation to 400 safe."}]},"prev":{"slug":"a-toolkit-that-costs-nothing","title":"A working setup that costs nothing","url":"https://addaly.com/learn/machine-learning-foundations/a-toolkit-that-costs-nothing"},"next":{"slug":"the-closed-form-and-why-we-leave-it","title":"There is an exact answer, and we usually refuse it","url":"https://addaly.com/learn/machine-learning-foundations/the-closed-form-and-why-we-leave-it"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Fitting a line to five points, by hand', in Machine Learning, Foundations. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/machine-learning-foundations/fitting-a-line-by-hand (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Fitting a line to five points, by hand","courseTitle":"Machine Learning, Foundations","module":{"index":2,"slug":"linear-models-and-fitting","title":"Linear models and the machinery of fitting"},"answer":null,"keyIdea":"Fitting means choosing parameter values that make a chosen error number small; the coefficient carries real units, and a linear model will extrapolate far outside its data without any signal that it is guessing.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"five-houses","runs":[{"text":"Five houses"}]},{"t":"p","runs":[{"text":"Size in square metres, price in lakhs."}]},{"t":"code","lang":"text","label":"Text","text":"size   price\n 50     32\n 70     44\n 90     51\n110     67\n130     74"},{"t":"p","runs":[{"text":"We want "},{"text":"price = w * size + b","code":true},{"text":". Two numbers. Everything about model fitting is visible in finding them."}]},{"t":"h2","id":"guess-and-measure-how-wrong-you-are","runs":[{"text":"Guess, and measure how wrong you are"}]},{"t":"p","runs":[{"text":"Start with "},{"text":"w = 0.5","code":true},{"text":", "},{"text":"b = 0","code":true},{"text":". Predictions are 25, 35, 45, 55, 65. The errors (actual minus predicted) are 7, 9, 6, 12, 9."}]},{"t":"p","runs":[{"text":"Mean squared error:"}]},{"t":"code","lang":"text","label":"Text","text":"(49 + 81 + 36 + 144 + 81) / 5 = 391 / 5 = 78.2"},{"t":"p","runs":[{"text":"Now try "},{"text":"w = 0.55","code":true},{"text":", "},{"text":"b = 3","code":true},{"text":". Predictions: 30.5, 41.5, 52.5, 63.5, 74.5. Errors: 1.5, 2.5, -1.5, 3.5, -0.5. MSE = "},{"text":"(2.25 + 6.25 + 2.25 + 12.25 + 0.25) / 5 = 4.65","code":true},{"text":"."}]},{"t":"p","runs":[{"text":"The second model is better and we know it by a number, not by looking. That number is the entire mechanism. Everything else in this module is about finding good values of "},{"text":"w","code":true},{"text":" and "},{"text":"b","code":true},{"text":" faster than by guessing."}]},{"t":"h2","id":"which-direction-is-downhill","runs":[{"text":"Which direction is downhill?"}]},{"t":"p","runs":[{"text":"A useful thing about MSE is that you can compute, for the parameters you are currently holding, which way to move them. For one row the loss is "},{"text":"(y - (w*x + b))^2","code":true},{"text":", and the amount the loss changes when you nudge "},{"text":"w","code":true},{"text":" is"}]},{"t":"code","lang":"text","label":"Text","text":"d/dw = -2 * x * (y - prediction)"},{"t":"p","runs":[{"text":"Read that without the calculus. It says: the push on "},{"text":"w","code":true},{"text":" is proportional to the error, and scaled by "},{"text":"x","code":true},{"text":". A row where you are badly wrong pushes hard. A row with a large "},{"text":"x","code":true},{"text":" pushes harder than one with a small "},{"text":"x","code":true},{"text":" for the same error, which is exactly why unscaled features cause trouble later."}]},{"t":"p","runs":[{"text":"For the first house at "},{"text":"w = 0.5, b = 0","code":true},{"text":": error is 7, "},{"text":"x","code":true},{"text":" is 50, so the derivative is "},{"text":"-2 * 50 * 7 = -700","code":true},{"text":". Negative, meaning increasing "},{"text":"w","code":true},{"text":" reduces the loss. Averaged over all five rows, and moved a small step at a time, this is gradient descent, which you will meet properly in two lessons."}]},{"t":"h2","id":"doing-it-in-one-line","runs":[{"text":"Doing it in one line"}]},{"t":"code","lang":"python","label":"Python","text":"import numpy as np\nfrom sklearn.linear_model import LinearRegression\n\nX = np.array([[50], [70], [90], [110], [130]])\ny = np.array([32, 44, 51, 67, 74])\n\nm = LinearRegression().fit(X, y)\nm.coef_, m.intercept_        # array([0.525]), 6.15\nm.predict([[100]])           # array([58.65])"},{"t":"p","runs":[{"text":"The best possible line for these five points is "},{"text":"price = 0.525 * size + 6.15","code":true},{"text":", MSE 3.585. Our hand guess of "},{"text":"0.55","code":true},{"text":" and "},{"text":"3","code":true},{"text":" got to 4.65, respectable, and it shows that near the optimum the loss surface is fairly flat, which is a property we will come back to."}]},{"t":"h2","id":"what-the-two-numbers-mean","runs":[{"text":"What the two numbers mean"}]},{"t":"p","runs":[{"text":"w = 0.525","code":true},{"text":" means: "},{"text":"within this data","b":true},{"text":", each additional square metre is associated with 0.525 lakh more in price. Units are lakhs per square metre. A coefficient always has units, and stating them is the fastest way to catch a nonsense model."}]},{"t":"p","runs":[{"text":"b = 6.15","code":true},{"text":" is the predicted price of a house of zero square metres, which does not exist. The intercept is usually not interpretable on its own; it is the number that positions the line vertically. If you centre your features first (subtract the mean from each) the intercept becomes the prediction at the average house, which is interpretable."}]},{"t":"h2","id":"two-limitations-you-can-see-with-five-points","runs":[{"text":"Two limitations you can see with five points"}]},{"t":"p","runs":[{"text":"Extrapolation is a guess dressed as arithmetic.","b":true},{"text":" Ask this model for a 400 square metre house and it answers 216. It has never seen a house above 130 and has no way to know the relationship flattens, or that a house that large is a different market. The model will not warn you. Nothing in a linear model warns you when you leave the range of the training data, and this is the failure that most often reaches production quietly."}]},{"t":"p","runs":[{"text":"Five points cannot tell you the relationship is linear.","b":true},{"text":" They are consistent with a line. They are also consistent with a gentle curve, and with the true relationship being a step at 100 square metres. With five points you cannot distinguish these; with five hundred you often can. The straight line is an assumption you imposed, not a finding."}]},{"t":"h2","id":"try-changing-one-number","runs":[{"text":"Try changing one number"}]},{"t":"p","runs":[{"text":"Change the last price from 74 to 140, say a penthouse crept into the data, and refit. The coefficient jumps from 0.525 to about 0.855 and the whole line tilts. One row in five moved the answer by 63%."}]},{"t":"p","runs":[{"text":"That sensitivity is the squared-error property from the previous module, made visible. With 5,000 rows one outlier moves little. With 50 rows it moves a lot. It is worth knowing which regime you are in before you defend a coefficient in a meeting."}]},{"t":"h2","id":"why-this-simple-thing-survives","runs":[{"text":"Why this simple thing survives"}]},{"t":"p","runs":[{"text":"Linear regression is 220 years old and still the first model to fit on a new problem. It trains in milliseconds, gives a number per feature, extrapolates predictably rather than catastrophically, and needs no tuning. As the baseline lesson argued, you need something to beat. Half the time on tabular data, the fancy model beats this by less than anyone expected, and that fact is worth discovering in the first hour rather than the last."}]}],"check":{"question":[{"text":"You fit a line to 40 houses between 50 and 130 square metres and get 0.525 lakh per square metre. A colleague uses it to price a 400 square metre villa. What is the specific defect in that use?"}],"options":[{"text":[{"text":"The model was fitted with MSE, which is sensitive to outliers, and a 400 square metre house is an outlier."}],"correct":false,"tempting":[{"text":"Correctly recalls that MSE is outlier-sensitive, but that concerns rows in the training data, not a prediction requested outside its range."}]},{"text":[{"text":"The coefficient is only valid for the range of sizes observed; outside it the model applies a slope it has no evidence for, and it produces the extrapolation with no indication of doing so."}],"correct":true,"tempting":null},{"text":[{"text":"The intercept of 6.15 becomes meaningless at large sizes, so the prediction is offset by that amount."}],"correct":false,"tempting":[{"text":"Fixes on the intercept, which contributes the same fixed 6.15 at every size and is not what makes the villa prediction unreliable."}]},{"text":[{"text":"Linear regression requires at least 100 rows to be valid, and 40 is too few for any prediction."}],"correct":false,"tempting":[{"text":"Invents a sample-size rule; 40 rows give a usable but uncertain fit, and no row count would make extrapolation to 400 safe."}]}],"order":[2,1,0,3],"why":null},"prev":{"slug":"a-toolkit-that-costs-nothing","title":"A working setup that costs nothing","minutes":8},"next":{"slug":"the-closed-form-and-why-we-leave-it","title":"There is an exact answer, and we usually refuse it","minutes":8}}}