{"course":{"slug":"hugging-face","title":"Hugging Face, End to End","url":"https://addaly.com/learn/hugging-face","level":"intermediate","track":"create"},"lesson":{"slug":"similarity-and-distance","title":"Cosine, dot product, and the normalisation that connects them","url":"https://addaly.com/learn/hugging-face/similarity-and-distance","position":51,"of":75,"minutes":8,"answer":"On unit-length vectors cosine, dot product and Euclidean distance rank identically, so normalising at encode time removes the choice, and because each model spreads its space differently.","keyIdea":"On unit-length vectors cosine, dot product and Euclidean distance rank identically, so normalising at encode time removes the choice, and because each model spreads its space differently, similarity scores are for ranking rather than for thresholds copied from anyone else."},"body":{"format":"markdown","text":"## Three measures, and when they differ\n\n**Cosine similarity** is the cosine of the angle between two vectors. It ignores length entirely and ranges from −1 to 1, though in practice text embeddings rarely go below about 0 because the vectors occupy a narrow cone of the space.\n\n**Dot product** multiplies element-wise and sums. It is cosine multiplied by both vectors' lengths, so it rewards longer vectors.\n\n**Euclidean distance** is straight-line distance. Smaller is better, which inverts every comparison and is a routine source of reversed results.\n\nThe connection that makes this simple: **on unit-length vectors, ranking by cosine, dot product and Euclidean distance gives the identical order.** Normalise once at encode time and the choice stops mattering:\n\n```python\nv = model.encode(texts, normalize_embeddings=True)\nscores = v @ q          # dot product == cosine, and it is the fastest\n```\n\nIf you do not normalise, dot product favours documents whose vectors happen to be long, which correlates weakly with length and topic frequency, so unnormalised dot-product search has a subtle bias toward a particular kind of document and nobody can see it in the results.\n\n**Normalise everything at encode time, then use dot product.** That one rule removes the entire category.\n\n## What the numbers mean, which is less than you think\n\nA cosine of 0.82 between two texts means nothing on its own. Embedding models differ in how they spread their space: one model's \"clearly related\" is 0.85, another's is 0.55. Absolute thresholds do not transfer between models, and a threshold copied from a tutorial written for a different model is a wrong threshold.\n\nTwo consequences:\n\n1. **Rank, do not threshold**, wherever you can. \"Top five\" is portable; \"above 0.8\" is not.\n2. **When you must threshold** (deduplication, a cutoff for \"no good answer\") derive it from your own data. Encode a few hundred known-related and known-unrelated pairs, plot the two distributions, and put the threshold where they separate. That takes fifteen minutes and produces a number that is true for your model and your text.\n\n## The high floor\n\nEncode two unrelated sentences with a typical model and you may see 0.3 or 0.4 rather than 0. This is normal. Text embeddings occupy a narrow region of the space, so nothing is ever truly orthogonal to anything else.\n\nIt means a similarity of 0.5 is not \"half related\". Calibrate on your own pairs before you interpret any score, and be suspicious of any product that shows users a raw similarity as a percentage: it reads as a confidence and is not one.\n\n## When Euclidean is the right choice\n\nFor clustering algorithms that assume Euclidean geometry (k-means in particular) use Euclidean distance on normalised vectors, which is equivalent to cosine for ranking and is what the algorithm's mathematics expects. For pure retrieval, dot product on normalised vectors is faster and equivalent.\n\nVector databases ask you to choose a metric when you create an index. Choose the one matching how you normalised, and write it down, because changing it later means rebuilding the index.\n\n## A practical check\n\n```python\nimport numpy as np\nv = model.encode(sample_texts)\nprint(\"norms:\", np.linalg.norm(v, axis=1)[:5])\n```\n\nIf those print as 1.0, the vectors are normalised. If they print as 8.3, 11.2, 6.7, they are not, and any dot-product search over them is length-biased. This one line catches a bug that otherwise shows up as \"search is a bit worse than I expected\" and is never diagnosed.\n\n## Maximal marginal relevance, briefly\n\nTop-k by similarity often returns five near-identical chunks, because if one paragraph matches, its neighbours do too. Maximal marginal relevance picks the next result by balancing similarity to the query against dissimilarity to what is already selected:\n\n```python\n# pick k items trading off relevance (to q) against redundancy (to chosen)\nscore = lam * sim_to_query[i] - (1 - lam) * max(sim(i, j) for j in chosen)\n```\n\nWith `lam` around 0.7 you get results that cover the question rather than repeating one part of it. For retrieval feeding a language model this matters a great deal: five copies of one paragraph waste the context window that five different paragraphs would have filled.\n\n## Do this now\n\nEncode your corpus twice, once normalised and once not, and compare the top five for three queries under dot product. Where the results differ, look at the lengths of the documents that moved. That is the bias, made visible."},"check":{"question":"A tutorial's threshold of 0.8 for \"related\" produces almost no matches when applied with a different embedding model. Why?","options":[{"text":"The new model returns Euclidean distances rather than cosine similarities, so the comparison direction is inverted.","correct":false,"tempting":"Would produce matches on everything rather than almost nothing, and the metric is chosen by the caller."},{"text":"Models spread their embedding space differently, so the score at which texts are genuinely related varies by model.","correct":true},{"text":"The new model's vectors are unnormalised, so cosine similarity cannot exceed the shorter vector's length.","correct":false,"tempting":"Cosine ignores length by construction, so normalisation does not cap it."},{"text":"Higher-dimensional spaces push all pairwise similarities toward zero, so thresholds must scale with dimension count.","correct":false,"tempting":"Invokes a general geometric intuition that does not describe how trained text embeddings distribute."}]},"prev":{"slug":"choosing-an-embedding-model","title":"Picking the model, and reading MTEB without being fooled","url":"https://addaly.com/learn/hugging-face/choosing-an-embedding-model"},"next":{"slug":"chunking-text-for-retrieval","title":"Chunking decides what your search can find","url":"https://addaly.com/learn/hugging-face/chunking-text-for-retrieval"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Cosine, dot product, and the normalisation that connects them', in Hugging Face, End to End. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/hugging-face/similarity-and-distance (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Cosine, dot product, and the normalisation that connects them","courseTitle":"Hugging Face, End to End","module":{"index":6,"slug":"embeddings-and-retrieval","title":"Embeddings, and search that understands"},"answer":null,"keyIdea":"On unit-length vectors cosine, dot product and Euclidean distance rank identically, so normalising at encode time removes the choice, and because each model spreads its space differently, similarity scores are for ranking rather than for thresholds copied from anyone else.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"three-measures-and-when-they-differ","runs":[{"text":"Three measures, and when they differ"}]},{"t":"p","runs":[{"text":"Cosine similarity","b":true},{"text":" is the cosine of the angle between two vectors. It ignores length entirely and ranges from −1 to 1, though in practice text embeddings rarely go below about 0 because the vectors occupy a narrow cone of the space."}]},{"t":"p","runs":[{"text":"Dot product","b":true},{"text":" multiplies element-wise and sums. It is cosine multiplied by both vectors’ lengths, so it rewards longer vectors."}]},{"t":"p","runs":[{"text":"Euclidean distance","b":true},{"text":" is straight-line distance. Smaller is better, which inverts every comparison and is a routine source of reversed results."}]},{"t":"p","runs":[{"text":"The connection that makes this simple: "},{"text":"on unit-length vectors, ranking by cosine, dot product and Euclidean distance gives the identical order.","b":true},{"text":" Normalise once at encode time and the choice stops mattering:"}]},{"t":"code","lang":"python","label":"Python","text":"v = model.encode(texts, normalize_embeddings=True)\nscores = v @ q          # dot product == cosine, and it is the fastest"},{"t":"p","runs":[{"text":"If you do not normalise, dot product favours documents whose vectors happen to be long, which correlates weakly with length and topic frequency, so unnormalised dot-product search has a subtle bias toward a particular kind of document and nobody can see it in the results."}]},{"t":"p","runs":[{"text":"Normalise everything at encode time, then use dot product.","b":true},{"text":" That one rule removes the entire category."}]},{"t":"h2","id":"what-the-numbers-mean-which-is-less-than-you-think","runs":[{"text":"What the numbers mean, which is less than you think"}]},{"t":"p","runs":[{"text":"A cosine of 0.82 between two texts means nothing on its own. Embedding models differ in how they spread their space: one model’s “clearly related” is 0.85, another’s is 0.55. Absolute thresholds do not transfer between models, and a threshold copied from a tutorial written for a different model is a wrong threshold."}]},{"t":"p","runs":[{"text":"Two consequences:"}]},{"t":"ol","start":1,"items":[[{"text":"Rank, do not threshold","b":true},{"text":", wherever you can. “Top five” is portable; “above 0.8” is not."}],[{"text":"When you must threshold","b":true},{"text":" (deduplication, a cutoff for “no good answer”) derive it from your own data. Encode a few hundred known-related and known-unrelated pairs, plot the two distributions, and put the threshold where they separate. That takes fifteen minutes and produces a number that is true for your model and your text."}]]},{"t":"h2","id":"the-high-floor","runs":[{"text":"The high floor"}]},{"t":"p","runs":[{"text":"Encode two unrelated sentences with a typical model and you may see 0.3 or 0.4 rather than 0. This is normal. Text embeddings occupy a narrow region of the space, so nothing is ever truly orthogonal to anything else."}]},{"t":"p","runs":[{"text":"It means a similarity of 0.5 is not “half related”. Calibrate on your own pairs before you interpret any score, and be suspicious of any product that shows users a raw similarity as a percentage: it reads as a confidence and is not one."}]},{"t":"h2","id":"when-euclidean-is-the-right-choice","runs":[{"text":"When Euclidean is the right choice"}]},{"t":"p","runs":[{"text":"For clustering algorithms that assume Euclidean geometry (k-means in particular) use Euclidean distance on normalised vectors, which is equivalent to cosine for ranking and is what the algorithm’s mathematics expects. For pure retrieval, dot product on normalised vectors is faster and equivalent."}]},{"t":"p","runs":[{"text":"Vector databases ask you to choose a metric when you create an index. Choose the one matching how you normalised, and write it down, because changing it later means rebuilding the index."}]},{"t":"h2","id":"a-practical-check","runs":[{"text":"A practical check"}]},{"t":"code","lang":"python","label":"Python","text":"import numpy as np\nv = model.encode(sample_texts)\nprint(\"norms:\", np.linalg.norm(v, axis=1)[:5])"},{"t":"p","runs":[{"text":"If those print as 1.0, the vectors are normalised. If they print as 8.3, 11.2, 6.7, they are not, and any dot-product search over them is length-biased. This one line catches a bug that otherwise shows up as “search is a bit worse than I expected” and is never diagnosed."}]},{"t":"h2","id":"maximal-marginal-relevance-briefly","runs":[{"text":"Maximal marginal relevance, briefly"}]},{"t":"p","runs":[{"text":"Top-k by similarity often returns five near-identical chunks, because if one paragraph matches, its neighbours do too. Maximal marginal relevance picks the next result by balancing similarity to the query against dissimilarity to what is already selected:"}]},{"t":"code","lang":"python","label":"Python","text":"# pick k items trading off relevance (to q) against redundancy (to chosen)\nscore = lam * sim_to_query[i] - (1 - lam) * max(sim(i, j) for j in chosen)"},{"t":"p","runs":[{"text":"With "},{"text":"lam","code":true},{"text":" around 0.7 you get results that cover the question rather than repeating one part of it. For retrieval feeding a language model this matters a great deal: five copies of one paragraph waste the context window that five different paragraphs would have filled."}]},{"t":"h2","id":"do-this-now","runs":[{"text":"Do this now"}],"role":"try"},{"t":"p","runs":[{"text":"Encode your corpus twice, once normalised and once not, and compare the top five for three queries under dot product. Where the results differ, look at the lengths of the documents that moved. That is the bias, made visible."}]}],"check":{"question":[{"text":"A tutorial’s threshold of 0.8 for “related” produces almost no matches when applied with a different embedding model. Why?"}],"options":[{"text":[{"text":"The new model returns Euclidean distances rather than cosine similarities, so the comparison direction is inverted."}],"correct":false,"tempting":[{"text":"Would produce matches on everything rather than almost nothing, and the metric is chosen by the caller."}]},{"text":[{"text":"Models spread their embedding space differently, so the score at which texts are genuinely related varies by model."}],"correct":true,"tempting":null},{"text":[{"text":"The new model’s vectors are unnormalised, so cosine similarity cannot exceed the shorter vector’s length."}],"correct":false,"tempting":[{"text":"Cosine ignores length by construction, so normalisation does not cap it."}]},{"text":[{"text":"Higher-dimensional spaces push all pairwise similarities toward zero, so thresholds must scale with dimension count."}],"correct":false,"tempting":[{"text":"Invokes a general geometric intuition that does not describe how trained text embeddings distribute."}]}],"order":[2,1,0,3],"why":null},"prev":{"slug":"choosing-an-embedding-model","title":"Picking the model, and reading MTEB without being fooled","minutes":9},"next":{"slug":"chunking-text-for-retrieval","title":"Chunking decides what your search can find","minutes":9}}}