The loss is where you state what a mistake costs

Fitting means minimising something you chose

A model has parameters. Training searches for parameter values that make one number small. That number is the loss, and the choice of loss is not a technicality: it is the sentence “this is what being wrong costs me”, written in maths so an optimiser can act on it.

Two people with the same data and the same algorithm can get different models because they encoded different costs. Neither is wrong. Only one matches the situation.

Squared error punishes the outlier

For a numeric target, the default is mean squared error:

mse = ((y_true - y_pred) ** 2).mean()

Squaring does something specific. An error of 10 costs 100; an error of 100 costs 10,000. One badly wrong row can outweigh a hundred nearly right ones. Fit a line to house prices with MSE and a single mansion drags the whole line upward, worsening every ordinary prediction, because moving 5 units towards the mansion buys more loss reduction than 5 units of error added across many small houses.

Mean absolute error does not do that:

mae = np.abs(y_true - y_pred).mean()

An error of 100 costs 100. Outliers get no special weight.

There is a clean way to remember which you are asking for. Minimising squared error fits the mean; minimising absolute error fits the median. If your data is [1, 2, 3, 4, 100] and you must predict one constant, MSE gives you 22, MAE gives you 3. Ask yourself which of those two answers you actually want a delivery-time estimate to be.

Huber loss is the compromise: squared near zero, linear far away, with a threshold you set. It is a good default for regression on messy real data and it is one line in scikit-learn.

For classification, the loss is about probabilities

You might think classification should minimise the number of wrong labels. It does not, and the reason is mechanical: the count of errors is a step function. Nudge a parameter slightly and the count either does not move at all or jumps by one. There is no slope to follow, so gradient descent has nothing to work with.

So classification minimises log loss, also called cross-entropy, over predicted probabilities:

logloss = -(y * np.log(p) + (1 - y) * np.log(1 - p)).mean()

Read what this does at two rows. The true label is 1. You predict 0.9, and the loss is -log(0.9) = 0.105. You predict 0.6, and it is 0.511. You predict 0.01, and it is 4.6. Now the ugly one: you predict 0.0001 and the truth is 1, and the loss is 9.2. Log loss is asymmetric in a way that matters: being confidently wrong is punished savagely, and as your prediction approaches 0 for a positive case the loss goes to infinity. That is the property that makes models trained on log loss produce probabilities you can partly trust, and it is why a single confident mistake can dominate a validation score.

The loss is not the metric

This trips up almost everyone once. The loss is what training optimises; the metric is what you report and decide with. They are often different, deliberately.

You cannot train on “precision at a threshold of 0.7”: it is not differentiable and the threshold is not part of the model. So you train on log loss to get honest probabilities, and then you choose the threshold afterwards using precision and recall, which is exactly what module 5 does. The gap between loss and metric is normal. What is not normal is failing to notice that closing the gap in loss did nothing for the metric, which happens more often than the textbooks admit.

Weighting is how you say some rows matter more

Every major library accepts sample_weight. Multiply each row’s loss contribution by a number and the optimiser will trade accuracy on cheap rows for accuracy on expensive ones.

This is the cleanest tool available for imbalance and for unequal costs. If missing a fraudulent transaction costs 200 times what a false alarm costs, that is not a threshold trick: you can say it directly, in the loss, and the model will spend its capacity accordingly. class_weight='balanced' in scikit-learn is the same mechanism with the weights set to the inverse class frequency.

The limitation nobody puts on the slide

A loss function is a scalar. Real decisions have costs that do not collapse into one number, and when you force them to, you have made an ethical claim with a division sign. A model that decides who gets a loan is trading false rejections against false approvals, and those two harms land on different people. Writing sample_weight=200 does not resolve that; it records your answer to it in a place where nobody will read it again.

So state the loss in words next to the code. “A missed fraud costs us roughly 200 times a false alarm, based on average chargeback of ₹18,000 against about ₹90 of review time.” Someone can then argue with the 200. They cannot argue with a number they never saw.