{"course":{"slug":"building-with-ai","title":"Building with AI","url":"https://addaly.com/learn/building-with-ai","level":"intermediate","track":"build"},"lesson":{"slug":"choosing-an-embedding-model","title":"Choosing an embedding model: dimensions, languages and the leaderboard trap","url":"https://addaly.com/learn/building-with-ai/choosing-an-embedding-model","position":35,"of":80,"minutes":9,"answer":"Shortlist embedding models from the leaderboard but choose on recall over your own fifty queries, and check the embedder's context limit before anything else.","keyIdea":"Shortlist embedding models from the leaderboard but choose on recall over your own fifty queries, and check the embedder's context limit before anything else, because a chunk longer than that limit is silently cut and the cut half is never searched."},"body":{"format":"markdown","text":"## The menu, as it stands\n\nHosted: OpenAI `text-embedding-3-small` and `-large`, Cohere `embed` (strong on many languages), Voyage, Google's Gemini embedding. Open weights: `bge-m3` (multilingual, 8,192-token input, 1,024 dimensions), `multilingual-e5-large`, `nomic-embed-text`, the `gte` family, and Qwen's embedding models. The open ones run through `sentence-transformers` on a CPU, through Ollama with `ollama pull nomic-embed-text`, or through Hugging Face's hosted inference. The names will have moved by the time you read this; the way of choosing will not.\n\n## The leaderboard trap\n\nMTEB, the Massive Text Embedding Benchmark, ranks models by an average over dozens of tasks (retrieval, classification, clustering, similarity) across many datasets. It is the right place to start and the wrong place to stop.\n\nThe mechanism is simple. An average over fifty tasks hides the variance on any one of them. Two models one point apart on the average can be ten points apart, in either direction, on a corpus like yours. And the datasets are public, so models get trained on data that resembles the test, which inflates scores without improving anything you will experience. The top of the table changes monthly, mostly by fractions.\n\nUse it to shortlist three models that fit your constraints on language, size and licence. Then run the bake-off at the end of this lesson. Never choose on the average.\n\n## The limit that silently cuts your data\n\nEvery embedding model has a maximum input length, and it is usually shorter than people assume. `all-MiniLM-L6-v2` truncates at 256 word pieces. Most of the `bge` and `e5` base models take 512. `bge-m3` and the hosted models take 8,192.\n\nTruncation is silent. Send a 900-token chunk to a 256-token model and you get a perfectly good vector for the first quarter of the chunk. The remaining three-quarters is not in the index at all; a question answered in the last paragraph of that chunk will never retrieve it. Teams discover this weeks later as \"retrieval is bad for some documents\" when it is in fact \"retrieval is bad for the ends of all documents\". Check the limit, and make your chunker respect it.\n\n## Dimensions, and the Matryoshka trick\n\nMore dimensions cost storage, memory and index-build time, not API money. They buy a little quality, with diminishing returns past a thousand or so.\n\nSeveral recent models are trained with Matryoshka representation learning, which arranges the vector so that the first N dimensions are usable on their own. OpenAI reports that `text-embedding-3-large` cut to 256 dimensions still beats its earlier 1,536-dimension `ada-002` on MTEB. If your model supports it, storing 256 or 512 dimensions and keeping the full vector only for a final re-scoring pass is a real saving on a large corpus.\n\n## Languages, and the test that matters\n\nIf your users write in Hindi, Tamil, Bengali or a mix of English and any of them, an English-only model is not a weaker choice, it is a broken one: Devanagari through `MiniLM` produces near-random vectors. `bge-m3` and `multilingual-e5` are trained on those scripts. But run one more test that no leaderboard covers: transliterated Hinglish in Latin script (\"password reset kaise karu\") against English documentation. That is what a large share of Indian users actually type, and it is almost absent from training data. Some multilingual models handle it acceptably; some do not. Only your test will tell you.\n\n## Speed, and where the embedding runs\n\nAt query time a hosted call costs a network round trip, typically 50 to 150 milliseconds. A small local model on CPU answers in about five. That difference matters when retrieval is one stage in a pipeline with a latency budget.\n\nAt ingestion time the shape reverses. `MiniLM` on a laptop CPU embeds a few hundred chunks per second; a million-chunk corpus is an hour or two. `bge-large` on the same CPU manages perhaps ten to twenty a second, which is a day. That is where a GPU, or a hosted API at a fraction of a cent per thousand chunks, earns its place.\n\n## Lock-in, and the field you must store\n\nVectors from two different models are not comparable, even at the same dimension count. Switching models means re-embedding the entire corpus. Hosted models are retired (`ada-002` was superseded) and open models get new versions.\n\nSo store the model name and version alongside every vector, keep the raw chunk text so you can always re-embed, and know the re-embed cost before you need it: five million tokens is ten cents hosted or a few hours on a CPU. That is cheap. Discovering that you deleted the source text is not.\n\n## The bake-off\n\nFifty real queries, each paired with the chunk that answers it. Three shortlisted models. For each, embed the corpus, run the fifty queries, and record recall at five: the fraction of queries whose answering chunk appears in the top five. One afternoon, one number per model, and a decision you can defend to anyone who asks why you did not pick the one at the top of the table."},"check":{"question":"A team chunks documents at about 900 tokens and embeds them with all-MiniLM-L6-v2. Facts near the end of long documents are rarely retrieved. What is the mechanism?","options":[{"text":"The model truncates input at 256 word pieces, so the tail of every chunk is never embedded and cannot be found.","correct":true},{"text":"384 dimensions is too few to represent a long chunk faithfully, so detail from later in the chunk is compressed away and gradually lost.","correct":false,"tempting":"Attributes to dimension count a failure that is complete rather than gradual; the tail is not compressed, it is absent."},{"text":"Long chunks dilute the vector, so the fix is to add a reranker over the top fifty results.","correct":false,"tempting":"A reranker can only reorder what the first stage returned; it cannot see text the embedder never processed."},{"text":"The chunks are too short, and 900 tokens leaves too little context for the model to place the facts.","correct":false,"tempting":"Points in the wrong direction; the chunk is already far longer than the model can read."}]},"prev":{"slug":"chunking-and-embeddings","title":"Chunking and embeddings in practice","url":"https://addaly.com/learn/building-with-ai/chunking-and-embeddings"},"next":{"slug":"vector-search-without-a-vector-database","title":"Vector search: when NumPy is enough, and what an index trades away","url":"https://addaly.com/learn/building-with-ai/vector-search-without-a-vector-database"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Choosing an embedding model: dimensions, languages and the leaderboard trap', in Building with AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/building-with-ai/choosing-an-embedding-model (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Choosing an embedding model: dimensions, languages and the leaderboard trap","courseTitle":"Building with AI","module":{"index":4,"slug":"retrieval-and-the-knowledge-problem","title":"Retrieval and the knowledge problem"},"answer":null,"keyIdea":"Shortlist embedding models from the leaderboard but choose on recall over your own fifty queries, and check the embedder’s context limit before anything else, because a chunk longer than that limit is silently cut and the cut half is never searched.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-menu-as-it-stands","runs":[{"text":"The menu, as it stands"}]},{"t":"p","runs":[{"text":"Hosted: OpenAI "},{"text":"text-embedding-3-small","code":true},{"text":" and "},{"text":"-large","code":true},{"text":", Cohere "},{"text":"embed","code":true},{"text":" (strong on many languages), Voyage, Google’s Gemini embedding. Open weights: "},{"text":"bge-m3","code":true},{"text":" (multilingual, 8,192-token input, 1,024 dimensions), "},{"text":"multilingual-e5-large","code":true},{"text":", "},{"text":"nomic-embed-text","code":true},{"text":", the "},{"text":"gte","code":true},{"text":" family, and Qwen’s embedding models. The open ones run through "},{"text":"sentence-transformers","code":true},{"text":" on a CPU, through Ollama with "},{"text":"ollama pull nomic-embed-text","code":true},{"text":", or through Hugging Face’s hosted inference. The names will have moved by the time you read this; the way of choosing will not."}]},{"t":"h2","id":"the-leaderboard-trap","runs":[{"text":"The leaderboard trap"}]},{"t":"p","runs":[{"text":"MTEB, the Massive Text Embedding Benchmark, ranks models by an average over dozens of tasks (retrieval, classification, clustering, similarity) across many datasets. It is the right place to start and the wrong place to stop."}]},{"t":"p","runs":[{"text":"The mechanism is simple. An average over fifty tasks hides the variance on any one of them. Two models one point apart on the average can be ten points apart, in either direction, on a corpus like yours. And the datasets are public, so models get trained on data that resembles the test, which inflates scores without improving anything you will experience. The top of the table changes monthly, mostly by fractions."}]},{"t":"p","runs":[{"text":"Use it to shortlist three models that fit your constraints on language, size and licence. Then run the bake-off at the end of this lesson. Never choose on the average."}]},{"t":"h2","id":"the-limit-that-silently-cuts-your-data","runs":[{"text":"The limit that silently cuts your data"}],"role":"limit"},{"t":"p","runs":[{"text":"Every embedding model has a maximum input length, and it is usually shorter than people assume. "},{"text":"all-MiniLM-L6-v2","code":true},{"text":" truncates at 256 word pieces. Most of the "},{"text":"bge","code":true},{"text":" and "},{"text":"e5","code":true},{"text":" base models take 512. "},{"text":"bge-m3","code":true},{"text":" and the hosted models take 8,192."}]},{"t":"p","runs":[{"text":"Truncation is silent. Send a 900-token chunk to a 256-token model and you get a perfectly good vector for the first quarter of the chunk. The remaining three-quarters is not in the index at all; a question answered in the last paragraph of that chunk will never retrieve it. Teams discover this weeks later as “retrieval is bad for some documents” when it is in fact “retrieval is bad for the ends of all documents”. Check the limit, and make your chunker respect it."}]},{"t":"h2","id":"dimensions-and-the-matryoshka-trick","runs":[{"text":"Dimensions, and the Matryoshka trick"}]},{"t":"p","runs":[{"text":"More dimensions cost storage, memory and index-build time, not API money. They buy a little quality, with diminishing returns past a thousand or so."}]},{"t":"p","runs":[{"text":"Several recent models are trained with Matryoshka representation learning, which arranges the vector so that the first N dimensions are usable on their own. OpenAI reports that "},{"text":"text-embedding-3-large","code":true},{"text":" cut to 256 dimensions still beats its earlier 1,536-dimension "},{"text":"ada-002","code":true},{"text":" on MTEB. If your model supports it, storing 256 or 512 dimensions and keeping the full vector only for a final re-scoring pass is a real saving on a large corpus."}]},{"t":"h2","id":"languages-and-the-test-that-matters","runs":[{"text":"Languages, and the test that matters"}]},{"t":"p","runs":[{"text":"If your users write in Hindi, Tamil, Bengali or a mix of English and any of them, an English-only model is not a weaker choice, it is a broken one: Devanagari through "},{"text":"MiniLM","code":true},{"text":" produces near-random vectors. "},{"text":"bge-m3","code":true},{"text":" and "},{"text":"multilingual-e5","code":true},{"text":" are trained on those scripts. But run one more test that no leaderboard covers: transliterated Hinglish in Latin script (“password reset kaise karu”) against English documentation. That is what a large share of Indian users actually type, and it is almost absent from training data. Some multilingual models handle it acceptably; some do not. Only your test will tell you."}]},{"t":"h2","id":"speed-and-where-the-embedding-runs","runs":[{"text":"Speed, and where the embedding runs"}]},{"t":"p","runs":[{"text":"At query time a hosted call costs a network round trip, typically 50 to 150 milliseconds. A small local model on CPU answers in about five. That difference matters when retrieval is one stage in a pipeline with a latency budget."}]},{"t":"p","runs":[{"text":"At ingestion time the shape reverses. "},{"text":"MiniLM","code":true},{"text":" on a laptop CPU embeds a few hundred chunks per second; a million-chunk corpus is an hour or two. "},{"text":"bge-large","code":true},{"text":" on the same CPU manages perhaps ten to twenty a second, which is a day. That is where a GPU, or a hosted API at a fraction of a cent per thousand chunks, earns its place."}]},{"t":"h2","id":"lock-in-and-the-field-you-must-store","runs":[{"text":"Lock-in, and the field you must store"}]},{"t":"p","runs":[{"text":"Vectors from two different models are not comparable, even at the same dimension count. Switching models means re-embedding the entire corpus. Hosted models are retired ("},{"text":"ada-002","code":true},{"text":" was superseded) and open models get new versions."}]},{"t":"p","runs":[{"text":"So store the model name and version alongside every vector, keep the raw chunk text so you can always re-embed, and know the re-embed cost before you need it: five million tokens is ten cents hosted or a few hours on a CPU. That is cheap. Discovering that you deleted the source text is not."}]},{"t":"h2","id":"the-bake-off","runs":[{"text":"The bake-off"}]},{"t":"p","runs":[{"text":"Fifty real queries, each paired with the chunk that answers it. Three shortlisted models. For each, embed the corpus, run the fifty queries, and record recall at five: the fraction of queries whose answering chunk appears in the top five. One afternoon, one number per model, and a decision you can defend to anyone who asks why you did not pick the one at the top of the table."}]}],"check":{"question":[{"text":"A team chunks documents at about 900 tokens and embeds them with all-MiniLM-L6-v2. Facts near the end of long documents are rarely retrieved. What is the mechanism?"}],"options":[{"text":[{"text":"The model truncates input at 256 word pieces, so the tail of every chunk is never embedded and cannot be found."}],"correct":true,"tempting":null},{"text":[{"text":"384 dimensions is too few to represent a long chunk faithfully, so detail from later in the chunk is compressed away and gradually lost."}],"correct":false,"tempting":[{"text":"Attributes to dimension count a failure that is complete rather than gradual; the tail is not compressed, it is absent."}]},{"text":[{"text":"Long chunks dilute the vector, so the fix is to add a reranker over the top fifty results."}],"correct":false,"tempting":[{"text":"A reranker can only reorder what the first stage returned; it cannot see text the embedder never processed."}]},{"text":[{"text":"The chunks are too short, and 900 tokens leaves too little context for the model to place the facts."}],"correct":false,"tempting":[{"text":"Points in the wrong direction; the chunk is already far longer than the model can read."}]}],"order":[1,3,0,2],"why":null},"prev":{"slug":"chunking-and-embeddings","title":"Chunking and embeddings in practice","minutes":8},"next":{"slug":"vector-search-without-a-vector-database","title":"Vector search: when NumPy is enough, and what an index trades away","minutes":9}}}