{"course":{"slug":"machine-learning-foundations","title":"Machine Learning, Foundations","url":"https://addaly.com/learn/machine-learning-foundations","level":"intermediate","track":"start"},"lesson":{"slug":"recommendation-by-factorisation","title":"Recommendation: learning a vector per user and per item","url":"https://addaly.com/learn/machine-learning-foundations/recommendation-by-factorisation","position":70,"of":92,"minutes":9,"answer":"Matrix factorisation learns a vector per user and per item by fitting only the observed cells, which makes it an embedding trained on interactions, and its recommendations shape the next dataset.","keyIdea":"Matrix factorisation learns a vector per user and per item by fitting only the observed cells, which makes it an embedding trained on interactions, and its recommendations shape the next dataset, so offline metrics are always measured on data the previous model helped create."},"body":{"format":"markdown","text":"## The matrix with holes in it\n\nUsers down the rows, items across the columns, ratings in the cells. Almost every cell is empty: a user who has rated 40 films out of 50,000 leaves 99.92% of their row blank. Recommendation is the problem of guessing the missing values.\n\n## Two families\n\n**Content-based.** Describe items by their attributes (genre, price, text) and recommend items similar to what the user liked. Works for a brand-new item with no interaction history, which is its main advantage. Tends to be narrow: someone who watched three thrillers gets only thrillers.\n\n**Collaborative filtering.** Ignore item attributes entirely and use the pattern of who interacted with what. \"People who liked what you liked also liked this.\" It finds connections no attribute list would suggest, and it fails completely for a new item or a new user, since neither has any pattern yet.\n\n## Matrix factorisation, which is an embedding\n\nThe central technique. Assume the ratings matrix `R` (users by items) can be approximated by the product of two thin matrices:\n\n```text\nR (100,000 x 50,000)  is approximately  U (100,000 x 50) x V' (50 x 50,000)\n```\n\nEach user becomes a vector of 50 numbers; each item becomes a vector of 50 numbers; a predicted rating is the dot product of the two.\n\nThose 50 dimensions are learned, not designed. They may turn out to correspond loosely to things a human would name (how much action, how recent, how mainstream) but nobody specified them. This is the same construction as the embeddings in the last two lessons, with the training signal being interaction rather than co-occurrence in text.\n\nThe compression is the point. 5 billion possible cells are described by 7.5 million learned numbers, and the reduction forces the model to find structure rather than memorise.\n\n## Fitting it on observed cells only\n\nThe critical detail. You minimise error over the cells you *have*, not over the whole matrix:\n\n```text\nminimise  sum over observed (r_ui - u_i . v_u)^2  +  lambda * (||u||^2 + ||v||^2)\n```\n\nTreating missing cells as zero would tell the model that every unwatched film was disliked, which is false: most were never seen. The regularisation term is not optional here: with 50 dimensions per user and some users having rated four items, the model will otherwise fit those four perfectly and generalise to nothing.\n\nAlternating least squares is the classic fitting method: hold the item vectors fixed and solve for the user vectors, which is an ordinary least squares problem, then swap. It parallelises well. The free library is `implicit`, and `surprise` is the friendlier one for explicit ratings.\n\n## Explicit and implicit feedback are different problems\n\n**Explicit**: the user gave a rating. Rare, biased towards strong opinions, and you know what a low value means.\n\n**Implicit**: clicks, views, purchases, watch time. Abundant, and there are **no negatives**. A film you never watched might be one you would love. Treating absence as dislike is wrong, and the standard solution is to treat observed interactions as positives with a confidence weight, and unobserved ones as low-confidence negatives rather than definite ones.\n\nMost real systems are implicit, and getting this framing right matters more than the choice of algorithm.\n\n## The cold start problem\n\nThe permanent difficulty. A new user has no vector. A new item has no vector. Collaborative filtering has nothing to work with.\n\nThe usual answers: fall back to popularity for new users, use content features for new items, ask a few questions during onboarding, and use a hybrid that combines both families. There is no clean solution: this is a structural property of the approach rather than a gap in the technique.\n\n## Popularity bias and the feedback loop\n\nA recommender trained on interactions recommends what people interacted with, which causes more interaction with those items, which strengthens the pattern. Popular items become more popular because they were recommended, and the long tail becomes unreachable.\n\nThis is the feedback loop from module 3 in its most visible form, and it has two costs: the catalogue effectively shrinks, and the offline evaluation becomes untrustworthy, because your test data was generated by the previous recommender's choices rather than by the users' free preferences.\n\nMitigations exist and all cost something: inject randomness, penalise popularity in the ranking, optimise for coverage as well as accuracy, and measure diversity as an explicit metric alongside precision. None of them is free, and choosing how much to spend is a product decision.\n\n## Evaluating honestly\n\nRMSE on held-out ratings is the classic metric and it is a poor one, because users only rate things they chose to consume: the held-out cells are not a random sample of the matrix.\n\nBetter: ranking metrics on held-out interactions, split **by time** rather than randomly, since recommending yesterday's purchase using tomorrow's data is the temporal leakage from module 3. Precision at 10, recall at 10, and NDCG are the standard set. And note that all of them are still measured on a population shaped by the old recommender, which is why the definitive test remains an online experiment."},"check":{"question":"A team builds a recommender on purchase data and fills unpurchased items with zeros before factorising. Why is this a mistake?","options":[{"text":"Zeros make the matrix dense, so factorisation becomes computationally infeasible at this scale.","correct":false,"tempting":"Density does raise the cost substantially, but the fundamental problem is what the zeros assert rather than what they cost to process."},{"text":"Zeros should be replaced with the item's mean rating so the model has an unbiased starting estimate.","correct":false,"tempting":"Mean imputation is a reasonable instinct for missing numeric data, but it still treats absence as an observation rather than as unknown."},{"text":"It asserts that every item a user never bought was actively disliked, when almost all of them were simply never seen; the fit should use observed cells only, weighting unobserved ones as weak rather than definite negatives.","correct":true},{"text":"Zeros break the regularisation term, since the norm of the user vectors is computed over all cells.","correct":false,"tempting":"The penalty applies to the learned vectors rather than to matrix entries, so filling cells does not affect it directly."}]},"prev":{"slug":"embeddings-in-practice","title":"Building a working semantic search with no GPU","url":"https://addaly.com/learn/machine-learning-foundations/embeddings-in-practice"},"next":{"slug":"finding-what-does-not-belong","title":"Anomaly detection, and the base rate that ruins it","url":"https://addaly.com/learn/machine-learning-foundations/finding-what-does-not-belong"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Recommendation: learning a vector per user and per item', in Machine Learning, Foundations. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/machine-learning-foundations/recommendation-by-factorisation (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Recommendation: learning a vector per user and per item","courseTitle":"Machine Learning, Foundations","module":{"index":7,"slug":"learning-without-labels","title":"Learning without labels, and learning a representation"},"answer":null,"keyIdea":"Matrix factorisation learns a vector per user and per item by fitting only the observed cells, which makes it an embedding trained on interactions, and its recommendations shape the next dataset, so offline metrics are always measured on data the previous model helped create.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-matrix-with-holes-in-it","runs":[{"text":"The matrix with holes in it"}]},{"t":"p","runs":[{"text":"Users down the rows, items across the columns, ratings in the cells. Almost every cell is empty: a user who has rated 40 films out of 50,000 leaves 99.92% of their row blank. Recommendation is the problem of guessing the missing values."}]},{"t":"h2","id":"two-families","runs":[{"text":"Two families"}]},{"t":"p","runs":[{"text":"Content-based.","b":true},{"text":" Describe items by their attributes (genre, price, text) and recommend items similar to what the user liked. Works for a brand-new item with no interaction history, which is its main advantage. Tends to be narrow: someone who watched three thrillers gets only thrillers."}]},{"t":"p","runs":[{"text":"Collaborative filtering.","b":true},{"text":" Ignore item attributes entirely and use the pattern of who interacted with what. “People who liked what you liked also liked this.” It finds connections no attribute list would suggest, and it fails completely for a new item or a new user, since neither has any pattern yet."}]},{"t":"h2","id":"matrix-factorisation-which-is-an-embedding","runs":[{"text":"Matrix factorisation, which is an embedding"}]},{"t":"p","runs":[{"text":"The central technique. Assume the ratings matrix "},{"text":"R","code":true},{"text":" (users by items) can be approximated by the product of two thin matrices:"}]},{"t":"code","lang":"text","label":"Text","text":"R (100,000 x 50,000)  is approximately  U (100,000 x 50) x V' (50 x 50,000)"},{"t":"p","runs":[{"text":"Each user becomes a vector of 50 numbers; each item becomes a vector of 50 numbers; a predicted rating is the dot product of the two."}]},{"t":"p","runs":[{"text":"Those 50 dimensions are learned, not designed. They may turn out to correspond loosely to things a human would name (how much action, how recent, how mainstream) but nobody specified them. This is the same construction as the embeddings in the last two lessons, with the training signal being interaction rather than co-occurrence in text."}]},{"t":"p","runs":[{"text":"The compression is the point. 5 billion possible cells are described by 7.5 million learned numbers, and the reduction forces the model to find structure rather than memorise."}]},{"t":"h2","id":"fitting-it-on-observed-cells-only","runs":[{"text":"Fitting it on observed cells only"}]},{"t":"p","runs":[{"text":"The critical detail. You minimise error over the cells you "},{"text":"have","i":true},{"text":", not over the whole matrix:"}]},{"t":"code","lang":"text","label":"Text","text":"minimise  sum over observed (r_ui - u_i . v_u)^2  +  lambda * (||u||^2 + ||v||^2)"},{"t":"p","runs":[{"text":"Treating missing cells as zero would tell the model that every unwatched film was disliked, which is false: most were never seen. The regularisation term is not optional here: with 50 dimensions per user and some users having rated four items, the model will otherwise fit those four perfectly and generalise to nothing."}]},{"t":"p","runs":[{"text":"Alternating least squares is the classic fitting method: hold the item vectors fixed and solve for the user vectors, which is an ordinary least squares problem, then swap. It parallelises well. The free library is "},{"text":"implicit","code":true},{"text":", and "},{"text":"surprise","code":true},{"text":" is the friendlier one for explicit ratings."}]},{"t":"h2","id":"explicit-and-implicit-feedback-are-different-problems","runs":[{"text":"Explicit and implicit feedback are different problems"}]},{"t":"p","runs":[{"text":"Explicit","b":true},{"text":": the user gave a rating. Rare, biased towards strong opinions, and you know what a low value means."}]},{"t":"p","runs":[{"text":"Implicit","b":true},{"text":": clicks, views, purchases, watch time. Abundant, and there are "},{"text":"no negatives","b":true},{"text":". A film you never watched might be one you would love. Treating absence as dislike is wrong, and the standard solution is to treat observed interactions as positives with a confidence weight, and unobserved ones as low-confidence negatives rather than definite ones."}]},{"t":"p","runs":[{"text":"Most real systems are implicit, and getting this framing right matters more than the choice of algorithm."}]},{"t":"h2","id":"the-cold-start-problem","runs":[{"text":"The cold start problem"}]},{"t":"p","runs":[{"text":"The permanent difficulty. A new user has no vector. A new item has no vector. Collaborative filtering has nothing to work with."}]},{"t":"p","runs":[{"text":"The usual answers: fall back to popularity for new users, use content features for new items, ask a few questions during onboarding, and use a hybrid that combines both families. There is no clean solution: this is a structural property of the approach rather than a gap in the technique."}]},{"t":"h2","id":"popularity-bias-and-the-feedback-loop","runs":[{"text":"Popularity bias and the feedback loop"}]},{"t":"p","runs":[{"text":"A recommender trained on interactions recommends what people interacted with, which causes more interaction with those items, which strengthens the pattern. Popular items become more popular because they were recommended, and the long tail becomes unreachable."}]},{"t":"p","runs":[{"text":"This is the feedback loop from module 3 in its most visible form, and it has two costs: the catalogue effectively shrinks, and the offline evaluation becomes untrustworthy, because your test data was generated by the previous recommender’s choices rather than by the users’ free preferences."}]},{"t":"p","runs":[{"text":"Mitigations exist and all cost something: inject randomness, penalise popularity in the ranking, optimise for coverage as well as accuracy, and measure diversity as an explicit metric alongside precision. None of them is free, and choosing how much to spend is a product decision."}]},{"t":"h2","id":"evaluating-honestly","runs":[{"text":"Evaluating honestly"}]},{"t":"p","runs":[{"text":"RMSE on held-out ratings is the classic metric and it is a poor one, because users only rate things they chose to consume: the held-out cells are not a random sample of the matrix."}]},{"t":"p","runs":[{"text":"Better: ranking metrics on held-out interactions, split "},{"text":"by time","b":true},{"text":" rather than randomly, since recommending yesterday’s purchase using tomorrow’s data is the temporal leakage from module 3. Precision at 10, recall at 10, and NDCG are the standard set. And note that all of them are still measured on a population shaped by the old recommender, which is why the definitive test remains an online experiment."}]}],"check":{"question":[{"text":"A team builds a recommender on purchase data and fills unpurchased items with zeros before factorising. Why is this a mistake?"}],"options":[{"text":[{"text":"Zeros make the matrix dense, so factorisation becomes computationally infeasible at this scale."}],"correct":false,"tempting":[{"text":"Density does raise the cost substantially, but the fundamental problem is what the zeros assert rather than what they cost to process."}]},{"text":[{"text":"Zeros should be replaced with the item’s mean rating so the model has an unbiased starting estimate."}],"correct":false,"tempting":[{"text":"Mean imputation is a reasonable instinct for missing numeric data, but it still treats absence as an observation rather than as unknown."}]},{"text":[{"text":"It asserts that every item a user never bought was actively disliked, when almost all of them were simply never seen; the fit should use observed cells only, weighting unobserved ones as weak rather than definite negatives."}],"correct":true,"tempting":null},{"text":[{"text":"Zeros break the regularisation term, since the norm of the user vectors is computed over all cells."}],"correct":false,"tempting":[{"text":"The penalty applies to the learned vectors rather than to matrix entries, so filling cells does not affect it directly."}]}],"order":[1,3,0,2],"why":null},"prev":{"slug":"embeddings-in-practice","title":"Building a working semantic search with no GPU","minutes":9},"next":{"slug":"finding-what-does-not-belong","title":"Anomaly detection, and the base rate that ruins it","minutes":8}}}