{"course":{"slug":"open-models","title":"The Open Model Ecosystem","url":"https://addaly.com/learn/open-models","level":"intermediate","track":"deep"},"lesson":{"slug":"open-embedding-models","title":"Embeddings you can host yourself","url":"https://addaly.com/learn/open-models/open-embedding-models","position":15,"of":71,"minutes":10,"answer":"An embedding model is small, cheap and CPU-friendly, but its query and passage prefixes, normalisation, truncation limit and pooling are set by the model card.","keyIdea":"An embedding model is small, cheap and CPU-friendly, but its query and passage prefixes, normalisation, truncation limit and pooling are set by the model card, and getting any of them wrong degrades retrieval silently rather than raising an error."},"body":{"format":"markdown","text":"## What the model gives you\n\nAn embedding model turns a piece of text into a fixed-length list of numbers (384, 768 or 1024 of them, typically) arranged so that texts with similar meaning sit close together. Closeness is measured with cosine similarity, which is the dot product of two vectors after both have been scaled to length one.\n\nThat is the whole idea, and it is the machinery under semantic search, retrieval-augmented generation, deduplication, clustering, recommendation and topic discovery. It is also the cheapest useful thing in this entire course: an embedding model is 20 to 600 million parameters, runs happily on a CPU, and embedding a hundred thousand paragraphs is minutes rather than hours.\n\n## The families\n\n- **all-MiniLM-L6-v2.** 22 million parameters, 384 dimensions, Apache 2.0, and still the right default for a first system. It embeds thousands of sentences a second on a laptop CPU.\n- **BGE (BAAI)**, **E5 (Microsoft)** and **GTE (Alibaba)** are the three families that trade the top of the retrieval charts. Small, base and large sizes; multilingual variants; permissive licences.\n- **nomic-embed-text.** Apache 2.0, 8192-token context, which matters when your documents do not chunk neatly.\n- **jina-embeddings.** Long context, multilingual, strong on code.\n- **Qwen3-Embedding.** A recent family in several sizes with strong multilingual results.\n\nBigger is not automatically better here. The jump from MiniLM to a base-sized model is usually worth it; the jump from base to large often buys two points of recall for four times the compute and twice the storage.\n\n## The four details that decide whether it works\n\n**Prefixes.** This is the mistake that silently halves quality. Several families are trained asymmetrically: queries and documents get different prefixes. E5 wants `query: ` and `passage: `. BGE wants an instruction prefix on the query only. If you embed both sides identically, nothing errors and retrieval quietly gets worse.\n\n```python\nfrom sentence_transformers import SentenceTransformer\nm = SentenceTransformer(\"intfloat/multilingual-e5-base\")\nq = m.encode([\"query: how do I cancel my order\"], normalize_embeddings=True)\nd = m.encode([\"passage: To cancel an order, open ...\"], normalize_embeddings=True)\n```\n\nRead the model card for the exact strings. They differ per family and getting them wrong is undetectable without a measurement.\n\n**Normalisation.** Cosine similarity assumes unit-length vectors. Most libraries offer `normalize_embeddings=True`; many vector databases assume you have already done it. Do it once, at write time, and be consistent.\n\n**Sequence length.** Most embedding models truncate at 512 tokens, and truncation is silent. A 900-token chunk becomes an embedding of its first half, and the second half is unsearchable. Either chunk to fit the model or pick a long-context model deliberately.\n\n**Pooling.** Whether the vector comes from the CLS token or the mean of all token vectors is decided by the model, not by you. Use the library's own wrapper rather than calling the transformer directly, or you will pick the wrong one.\n\n## Matryoshka, and cheap storage\n\nNewer models are trained so that the first N dimensions of the vector are usable on their own. Truncating a 1024-dimension vector to 256 keeps most of the retrieval quality and cuts index size and search cost by four. If your card mentions Matryoshka representation learning, this is available; test the truncation on your own data rather than assuming the published loss figure transfers.\n\nThe arithmetic matters at scale. A million chunks at 1024 dimensions in float32 is 4 GB of index. The same million at 256 dimensions is 1 GB, and quantised to 8-bit integers it is 256 MB, which fits in memory on an ordinary server.\n\n## MTEB, and how to read it\n\nMTEB is the standard embedding leaderboard, covering retrieval, classification, clustering and reranking across many datasets. Two cautions. Models are routinely trained on the training splits of MTEB tasks, so the leaderboard increasingly measures familiarity with those datasets. And the tasks are mostly English or translated English; performance in your language is a separate question the aggregate score hides.\n\nThe evaluation that decides your choice takes an hour: fifty real queries from your own logs, the correct document for each marked by hand, and recall at 10 measured for three candidate models. Whichever wins on your fifty queries wins, whatever the leaderboard says.\n\n## Where it goes wrong\n\nEmbeddings capture topical similarity, not logical relation. The sentence *the payment succeeded* and *the payment failed* sit very close together because they are about the same thing. Any system where the difference between two near-identical sentences matters needs something after retrieval to tell them apart, which is the next lesson."},"check":{"question":"A team switches to an E5 embedding model, embeds queries and documents with identical code and no prefixes, and finds retrieval quality is worse than the MiniLM model it replaced. What is the most likely explanation?","options":[{"text":"E5 is asymmetric and expects distinct query and passage prefixes; without them the two sides are embedded into mismatched regions of the space.","correct":true},{"text":"E5 produces larger vectors, so cosine similarity becomes less discriminative at higher dimensions.","correct":false,"tempting":"Higher dimensions do change distance behaviour in general, but retrieval models are trained at their dimension and do not degrade for that reason."},{"text":"The MiniLM index was built with normalised vectors and the new one was not, so the scores are on different scales.","correct":false,"tempting":"A real bug worth checking, but it would affect thresholds rather than ranking, since cosine ranking is unchanged by a consistent scale."},{"text":"E5 is trained mainly on English, so it underperforms on any corpus containing other languages.","correct":false,"tempting":"Would apply to the monolingual variant only, and would not explain a drop against MiniLM, which is also English-centric."}]},"prev":{"slug":"open-code-models","title":"Models that write code","url":"https://addaly.com/learn/open-models/open-code-models"},"next":{"slug":"rerankers-and-the-retrieval-stack","title":"Rerankers, and why retrieval is two stages","url":"https://addaly.com/learn/open-models/rerankers-and-the-retrieval-stack"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Embeddings you can host yourself', in The Open Model Ecosystem. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/open-models/open-embedding-models (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Embeddings you can host yourself","courseTitle":"The Open Model Ecosystem","module":{"index":2,"slug":"the-map-of-the-ecosystem","title":"The map, and how to read it"},"answer":null,"keyIdea":"An embedding model is small, cheap and CPU-friendly, but its query and passage prefixes, normalisation, truncation limit and pooling are set by the model card, and getting any of them wrong degrades retrieval silently rather than raising an error.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"what-the-model-gives-you","runs":[{"text":"What the model gives you"}]},{"t":"p","runs":[{"text":"An embedding model turns a piece of text into a fixed-length list of numbers (384, 768 or 1024 of them, typically) arranged so that texts with similar meaning sit close together. Closeness is measured with cosine similarity, which is the dot product of two vectors after both have been scaled to length one."}]},{"t":"p","runs":[{"text":"That is the whole idea, and it is the machinery under semantic search, retrieval-augmented generation, deduplication, clustering, recommendation and topic discovery. It is also the cheapest useful thing in this entire course: an embedding model is 20 to 600 million parameters, runs happily on a CPU, and embedding a hundred thousand paragraphs is minutes rather than hours."}]},{"t":"h2","id":"the-families","runs":[{"text":"The families"}]},{"t":"ul","items":[[{"text":"all-MiniLM-L6-v2.","b":true},{"text":" 22 million parameters, 384 dimensions, Apache 2.0, and still the right default for a first system. It embeds thousands of sentences a second on a laptop CPU."}],[{"text":"BGE (BAAI)","b":true},{"text":", "},{"text":"E5 (Microsoft)","b":true},{"text":" and "},{"text":"GTE (Alibaba)","b":true},{"text":" are the three families that trade the top of the retrieval charts. Small, base and large sizes; multilingual variants; permissive licences."}],[{"text":"nomic-embed-text.","b":true},{"text":" Apache 2.0, 8192-token context, which matters when your documents do not chunk neatly."}],[{"text":"jina-embeddings.","b":true},{"text":" Long context, multilingual, strong on code."}],[{"text":"Qwen3-Embedding.","b":true},{"text":" A recent family in several sizes with strong multilingual results."}]]},{"t":"p","runs":[{"text":"Bigger is not automatically better here. The jump from MiniLM to a base-sized model is usually worth it; the jump from base to large often buys two points of recall for four times the compute and twice the storage."}]},{"t":"h2","id":"the-four-details-that-decide-whether-it-works","runs":[{"text":"The four details that decide whether it works"}]},{"t":"p","runs":[{"text":"Prefixes.","b":true},{"text":" This is the mistake that silently halves quality. Several families are trained asymmetrically: queries and documents get different prefixes. E5 wants "},{"text":"query: ","code":true},{"text":" and "},{"text":"passage: ","code":true},{"text":". BGE wants an instruction prefix on the query only. If you embed both sides identically, nothing errors and retrieval quietly gets worse."}]},{"t":"code","lang":"python","label":"Python","text":"from sentence_transformers import SentenceTransformer\nm = SentenceTransformer(\"intfloat/multilingual-e5-base\")\nq = m.encode([\"query: how do I cancel my order\"], normalize_embeddings=True)\nd = m.encode([\"passage: To cancel an order, open ...\"], normalize_embeddings=True)"},{"t":"p","runs":[{"text":"Read the model card for the exact strings. They differ per family and getting them wrong is undetectable without a measurement."}]},{"t":"p","runs":[{"text":"Normalisation.","b":true},{"text":" Cosine similarity assumes unit-length vectors. Most libraries offer "},{"text":"normalize_embeddings=True","code":true},{"text":"; many vector databases assume you have already done it. Do it once, at write time, and be consistent."}]},{"t":"p","runs":[{"text":"Sequence length.","b":true},{"text":" Most embedding models truncate at 512 tokens, and truncation is silent. A 900-token chunk becomes an embedding of its first half, and the second half is unsearchable. Either chunk to fit the model or pick a long-context model deliberately."}]},{"t":"p","runs":[{"text":"Pooling.","b":true},{"text":" Whether the vector comes from the CLS token or the mean of all token vectors is decided by the model, not by you. Use the library’s own wrapper rather than calling the transformer directly, or you will pick the wrong one."}]},{"t":"h2","id":"matryoshka-and-cheap-storage","runs":[{"text":"Matryoshka, and cheap storage"}]},{"t":"p","runs":[{"text":"Newer models are trained so that the first N dimensions of the vector are usable on their own. Truncating a 1024-dimension vector to 256 keeps most of the retrieval quality and cuts index size and search cost by four. If your card mentions Matryoshka representation learning, this is available; test the truncation on your own data rather than assuming the published loss figure transfers."}]},{"t":"p","runs":[{"text":"The arithmetic matters at scale. A million chunks at 1024 dimensions in float32 is 4 GB of index. The same million at 256 dimensions is 1 GB, and quantised to 8-bit integers it is 256 MB, which fits in memory on an ordinary server."}]},{"t":"h2","id":"mteb-and-how-to-read-it","runs":[{"text":"MTEB, and how to read it"}]},{"t":"p","runs":[{"text":"MTEB is the standard embedding leaderboard, covering retrieval, classification, clustering and reranking across many datasets. Two cautions. Models are routinely trained on the training splits of MTEB tasks, so the leaderboard increasingly measures familiarity with those datasets. And the tasks are mostly English or translated English; performance in your language is a separate question the aggregate score hides."}]},{"t":"p","runs":[{"text":"The evaluation that decides your choice takes an hour: fifty real queries from your own logs, the correct document for each marked by hand, and recall at 10 measured for three candidate models. Whichever wins on your fifty queries wins, whatever the leaderboard says."}]},{"t":"h2","id":"where-it-goes-wrong","runs":[{"text":"Where it goes wrong"}]},{"t":"p","runs":[{"text":"Embeddings capture topical similarity, not logical relation. The sentence "},{"text":"the payment succeeded","i":true},{"text":" and "},{"text":"the payment failed","i":true},{"text":" sit very close together because they are about the same thing. Any system where the difference between two near-identical sentences matters needs something after retrieval to tell them apart, which is the next lesson."}]}],"check":{"question":[{"text":"A team switches to an E5 embedding model, embeds queries and documents with identical code and no prefixes, and finds retrieval quality is worse than the MiniLM model it replaced. What is the most likely explanation?"}],"options":[{"text":[{"text":"E5 is asymmetric and expects distinct query and passage prefixes; without them the two sides are embedded into mismatched regions of the space."}],"correct":true,"tempting":null},{"text":[{"text":"E5 produces larger vectors, so cosine similarity becomes less discriminative at higher dimensions."}],"correct":false,"tempting":[{"text":"Higher dimensions do change distance behaviour in general, but retrieval models are trained at their dimension and do not degrade for that reason."}]},{"text":[{"text":"The MiniLM index was built with normalised vectors and the new one was not, so the scores are on different scales."}],"correct":false,"tempting":[{"text":"A real bug worth checking, but it would affect thresholds rather than ranking, since cosine ranking is unchanged by a consistent scale."}]},{"text":[{"text":"E5 is trained mainly on English, so it underperforms on any corpus containing other languages."}],"correct":false,"tempting":[{"text":"Would apply to the monolingual variant only, and would not explain a drop against MiniLM, which is also English-centric."}]}],"order":[1,0,3,2],"why":null},"prev":{"slug":"open-code-models","title":"Models that write code","minutes":9},"next":{"slug":"rerankers-and-the-retrieval-stack","title":"Rerankers, and why retrieval is two stages","minutes":9}}}