{"course":{"slug":"fine-tuning","title":"Fine-Tuning, and When Not To","url":"https://addaly.com/learn/fine-tuning","level":"advanced","track":"deep"},"lesson":{"slug":"fine-tuning-an-embedding-model","title":"Fine-tuning an embedding model","url":"https://addaly.com/learn/fine-tuning/fine-tuning-an-embedding-model","position":58,"of":84,"minutes":10,"answer":"Embedding models train on query-passage pairs with in-batch negatives.","keyIdea":"Embedding models train on query-passage pairs with in-batch negatives, which makes batch size the real hyperparameter (gradient caching buys a 512-wide batch on a small card) and mined hard negatives must be taken from below the top ranks, because the top ones are often correct answers you would be teaching the model to reject."},"body":{"format":"markdown","text":"## The highest return in this course\n\nAn embedding model is 20 to 350 million parameters. It trains in minutes on free hardware. It needs hundreds of pairs rather than thousands of examples. And because it sets the recall ceiling for everything downstream, improving it improves every generator you ever put behind it.\n\nThis is the cheapest substantial win available in applied fine-tuning, and it is the one most often skipped in favour of training the expensive model that sits after it.\n\n## The data is pairs\n\nYou need queries paired with the passage that answers them. Nothing else is required:\n\n```json\n{\"anchor\": \"how long do refunds take\", \"positive\": \"Refunds are processed within 5 working days of the returned item arriving at our warehouse…\"}\n```\n\nThree ways to get them:\n\n- **Click logs.** A search query and the result somebody opened. Free, plentiful, noisy.\n- **Generated.** For each passage in your corpus, ask a model to write three questions it answers. Cheap, and it covers your whole corpus rather than only what people have searched for. Filter by checking that your current retriever ranks the passage highly for the generated question; if it does not, keep the pair, because that is exactly the case worth learning.\n- **Existing question-answer content.** FAQs, support macros, documentation with headings phrased as questions.\n\nFive hundred pairs is enough to see a real change. Five thousand is comfortable.\n\n## The loss, and why batch size is the hyperparameter\n\nThe standard objective is multiple-negatives ranking loss. For each anchor in a batch, its own positive should score higher than *every other positive in the batch*, which serve as negatives for free.\n\nThe consequence is unusual and important: **the batch size is the number of negatives**. A batch of 16 asks the model to pick the right passage out of 16. A batch of 128 asks it to pick out of 128, which is a much harder task and produces a much better model.\n\nSo unlike almost everywhere else in this course, here you want the batch as large as you can fit. And when you cannot fit it, there is a technique for exactly this:\n\n```python\nfrom sentence_transformers.losses import CachedMultipleNegativesRankingLoss\nloss = CachedMultipleNegativesRankingLoss(model, mini_batch_size=16)\n```\n\nGradient caching computes embeddings in small chunks without gradients, then replays them, giving the gradient of an enormous batch with the memory of a small one. It makes batch sizes of 512 or more reachable on a single modest card.\n\n## Hard negatives, and the trap in them\n\nIn-batch negatives are random, so most are trivially irrelevant. The model learns to separate refunds from shipping and never learns to separate a refund from an exchange.\n\nHard negatives fix this: for each query, retrieve the top results with your current model and keep the ones that are *not* the gold passage.\n\n```python\nfrom sentence_transformers.util import mine_hard_negatives\ntriplets = mine_hard_negatives(dataset, model, range_min=10, range_max=50,\n                               max_score=0.8, num_negatives=3)\n```\n\nThose `range_min` and `max_score` arguments are the trap being handled. The very top results are frequently *also correct* (your corpus has several passages about refunds) and training on a correct passage as a negative teaches the model to push apart two things that belong together. Mining from rank 10 to 50 rather than from rank 1, and capping similarity, avoids most of it. Read twenty mined negatives by hand before training on fifty thousand.\n\n> Figure: In-batch negatives: the batch size is the difficulty. Bars (passages the model must choose between): Batch of 16 16; Batch of 64 64; Batch of 128 128; Batch of 512, gradient cached 512. Each anchor's positive must outscore every other positive in the batch. Unlike almost everywhere else in this course you want the batch as large as it will go, and gradient caching reaches 512 on one modest card.\n\n## Matryoshka, if you will truncate\n\nMatryoshka loss trains the model so that the first 64, 128 and 256 dimensions of its output are each independently useful. You then store 768-dimensional vectors for reranking and search over 128-dimensional ones, cutting index memory by six times for a small quality cost. It is one extra wrapper around your loss and it is worth it on any index above a few million vectors.\n\n## Evaluate on retrieval, not on loss\n\n```python\nfrom sentence_transformers.evaluation import InformationRetrievalEvaluator\nev = InformationRetrievalEvaluator(queries, corpus, relevant_docs)\n```\n\nRecall at 5 and 10, NDCG at 10, on held-out queries against your **whole corpus**, not against a small sample of it. Retrieving the right passage from 200 candidates is a different and much easier task than retrieving it from 200,000, and a number from the first tells you nothing about the second.\n\n## The free path, end to end\n\nBase models: `all-MiniLM-L6-v2` at 22M parameters for speed, `bge-base-en-v1.5` or `e5-base-v2` at around 110M for quality, and multilingual variants where you need them. All free, all on the Hub.\n\nTraining 5,000 pairs on a 110M model takes a few minutes on a free T4, and the 22M model trains on a laptop CPU. The resulting model is a few hundred megabytes, serves at thousands of embeddings per second, and runs anywhere.\n\nThere is no cheaper intervention in this field with a comparable effect on a retrieval system's quality."},"check":{"question":"A team mines hard negatives by taking the top 3 retrieved passages that are not the labelled gold passage, and retrieval quality gets worse after training. Why?","options":[{"text":"Three negatives per query is too few, so the in-batch negatives dominate and the hard negatives have no effect.","correct":false,"tempting":"Argues the hard negatives were too weak to matter, which cannot explain a decline relative to not using them at all."},{"text":"Mining with the current model creates a feedback loop in which the model reinforces its own ranking errors.","correct":false,"tempting":"Names a plausible-sounding self-reinforcement story, but mining from the current model is standard practice and works when false negatives are excluded."},{"text":"Hard negatives require a triplet loss rather than multiple-negatives ranking loss, and mixing them produces an inconsistent objective.","correct":false,"tempting":"Invents an incompatibility; multiple-negatives ranking loss accepts explicit negatives alongside in-batch ones by design."},{"text":"The top-ranked non-gold passages are frequently also relevant, so training pushes apart passages that should be close together.","correct":true}]},"prev":{"slug":"multi-gpu-without-tears","title":"Multi-GPU without tears","url":"https://addaly.com/learn/fine-tuning/multi-gpu-without-tears"},"next":{"slug":"training-a-reranker","title":"Training a reranker","url":"https://addaly.com/learn/fine-tuning/training-a-reranker"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Fine-tuning an embedding model', in Fine-Tuning, and When Not To. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/fine-tuning/fine-tuning-an-embedding-model (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Fine-tuning an embedding model","courseTitle":"Fine-Tuning, and When Not To","module":{"index":7,"slug":"beyond-text-generation","title":"Fine-tuning beyond a chat model"},"answer":null,"keyIdea":"Embedding models train on query-passage pairs with in-batch negatives, which makes batch size the real hyperparameter (gradient caching buys a 512-wide batch on a small card) and mined hard negatives must be taken from below the top ranks, because the top ones are often correct answers you would be teaching the model to reject.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure 7.1","figure":{"type":"bars","unit":"passages the model must choose between","title":"In-batch negatives: the batch size is the difficulty","series":[{"label":"Batch of 16","value":16},{"label":"Batch of 64","value":64},{"label":"Batch of 128","value":128},{"label":"Batch of 512, gradient cached","value":512,"highlight":true}],"caption":"Each anchor’s positive must outscore every other positive in the batch. Unlike almost everywhere else in this course you want the batch as large as it will go, and gradient caching reaches 512 on one modest card."},"alt":"Figure: In-batch negatives: the batch size is the difficulty. Bars (passages the model must choose between): Batch of 16 16; Batch of 64 64; Batch of 128 128; Batch of 512, gradient cached 512. Each anchor’s positive must outscore every other positive in the batch. Unlike almost everywhere else in this course you want the batch as large as it will go, and gradient caching reaches 512 on one modest card."}},"blocks":[{"t":"h2","id":"the-highest-return-in-this-course","runs":[{"text":"The highest return in this course"}]},{"t":"p","runs":[{"text":"An embedding model is 20 to 350 million parameters. It trains in minutes on free hardware. It needs hundreds of pairs rather than thousands of examples. And because it sets the recall ceiling for everything downstream, improving it improves every generator you ever put behind it."}]},{"t":"p","runs":[{"text":"This is the cheapest substantial win available in applied fine-tuning, and it is the one most often skipped in favour of training the expensive model that sits after it."}]},{"t":"h2","id":"the-data-is-pairs","runs":[{"text":"The data is pairs"}]},{"t":"p","runs":[{"text":"You need queries paired with the passage that answers them. Nothing else is required:"}]},{"t":"code","lang":"json","label":"JSON","text":"{\"anchor\": \"how long do refunds take\", \"positive\": \"Refunds are processed within 5 working days of the returned item arriving at our warehouse…\"}"},{"t":"p","runs":[{"text":"Three ways to get them:"}]},{"t":"ul","items":[[{"text":"Click logs.","b":true},{"text":" A search query and the result somebody opened. Free, plentiful, noisy."}],[{"text":"Generated.","b":true},{"text":" For each passage in your corpus, ask a model to write three questions it answers. Cheap, and it covers your whole corpus rather than only what people have searched for. Filter by checking that your current retriever ranks the passage highly for the generated question; if it does not, keep the pair, because that is exactly the case worth learning."}],[{"text":"Existing question-answer content.","b":true},{"text":" FAQs, support macros, documentation with headings phrased as questions."}]]},{"t":"p","runs":[{"text":"Five hundred pairs is enough to see a real change. Five thousand is comfortable."}]},{"t":"h2","id":"the-loss-and-why-batch-size-is-the-hyperparameter","runs":[{"text":"The loss, and why batch size is the hyperparameter"}]},{"t":"p","runs":[{"text":"The standard objective is multiple-negatives ranking loss. For each anchor in a batch, its own positive should score higher than "},{"text":"every other positive in the batch","i":true},{"text":", which serve as negatives for free."}]},{"t":"p","runs":[{"text":"The consequence is unusual and important: "},{"text":"the batch size is the number of negatives","b":true},{"text":". A batch of 16 asks the model to pick the right passage out of 16. A batch of 128 asks it to pick out of 128, which is a much harder task and produces a much better model."}]},{"t":"p","runs":[{"text":"So unlike almost everywhere else in this course, here you want the batch as large as you can fit. And when you cannot fit it, there is a technique for exactly this:"}]},{"t":"code","lang":"python","label":"Python","text":"from sentence_transformers.losses import CachedMultipleNegativesRankingLoss\nloss = CachedMultipleNegativesRankingLoss(model, mini_batch_size=16)"},{"t":"p","runs":[{"text":"Gradient caching computes embeddings in small chunks without gradients, then replays them, giving the gradient of an enormous batch with the memory of a small one. It makes batch sizes of 512 or more reachable on a single modest card."}]},{"t":"h2","id":"hard-negatives-and-the-trap-in-them","runs":[{"text":"Hard negatives, and the trap in them"}]},{"t":"p","runs":[{"text":"In-batch negatives are random, so most are trivially irrelevant. The model learns to separate refunds from shipping and never learns to separate a refund from an exchange."}]},{"t":"p","runs":[{"text":"Hard negatives fix this: for each query, retrieve the top results with your current model and keep the ones that are "},{"text":"not","i":true},{"text":" the gold passage."}]},{"t":"code","lang":"python","label":"Python","text":"from sentence_transformers.util import mine_hard_negatives\ntriplets = mine_hard_negatives(dataset, model, range_min=10, range_max=50,\n                               max_score=0.8, num_negatives=3)"},{"t":"p","runs":[{"text":"Those "},{"text":"range_min","code":true},{"text":" and "},{"text":"max_score","code":true},{"text":" arguments are the trap being handled. The very top results are frequently "},{"text":"also correct","i":true},{"text":" (your corpus has several passages about refunds) and training on a correct passage as a negative teaches the model to push apart two things that belong together. Mining from rank 10 to 50 rather than from rank 1, and capping similarity, avoids most of it. Read twenty mined negatives by hand before training on fifty thousand."}]},{"t":"h2","id":"matryoshka-if-you-will-truncate","runs":[{"text":"Matryoshka, if you will truncate"}]},{"t":"p","runs":[{"text":"Matryoshka loss trains the model so that the first 64, 128 and 256 dimensions of its output are each independently useful. You then store 768-dimensional vectors for reranking and search over 128-dimensional ones, cutting index memory by six times for a small quality cost. It is one extra wrapper around your loss and it is worth it on any index above a few million vectors."}]},{"t":"h2","id":"evaluate-on-retrieval-not-on-loss","runs":[{"text":"Evaluate on retrieval, not on loss"}]},{"t":"code","lang":"python","label":"Python","text":"from sentence_transformers.evaluation import InformationRetrievalEvaluator\nev = InformationRetrievalEvaluator(queries, corpus, relevant_docs)"},{"t":"p","runs":[{"text":"Recall at 5 and 10, NDCG at 10, on held-out queries against your "},{"text":"whole corpus","b":true},{"text":", not against a small sample of it. Retrieving the right passage from 200 candidates is a different and much easier task than retrieving it from 200,000, and a number from the first tells you nothing about the second."}]},{"t":"h2","id":"the-free-path-end-to-end","runs":[{"text":"The free path, end to end"}],"role":"free"},{"t":"p","runs":[{"text":"Base models: "},{"text":"all-MiniLM-L6-v2","code":true},{"text":" at 22M parameters for speed, "},{"text":"bge-base-en-v1.5","code":true},{"text":" or "},{"text":"e5-base-v2","code":true},{"text":" at around 110M for quality, and multilingual variants where you need them. All free, all on the Hub."}]},{"t":"p","runs":[{"text":"Training 5,000 pairs on a 110M model takes a few minutes on a free T4, and the 22M model trains on a laptop CPU. The resulting model is a few hundred megabytes, serves at thousands of embeddings per second, and runs anywhere."}]},{"t":"p","runs":[{"text":"There is no cheaper intervention in this field with a comparable effect on a retrieval system’s quality."}]}],"check":{"question":[{"text":"A team mines hard negatives by taking the top 3 retrieved passages that are not the labelled gold passage, and retrieval quality gets worse after training. Why?"}],"options":[{"text":[{"text":"Three negatives per query is too few, so the in-batch negatives dominate and the hard negatives have no effect."}],"correct":false,"tempting":[{"text":"Argues the hard negatives were too weak to matter, which cannot explain a decline relative to not using them at all."}]},{"text":[{"text":"Mining with the current model creates a feedback loop in which the model reinforces its own ranking errors."}],"correct":false,"tempting":[{"text":"Names a plausible-sounding self-reinforcement story, but mining from the current model is standard practice and works when false negatives are excluded."}]},{"text":[{"text":"Hard negatives require a triplet loss rather than multiple-negatives ranking loss, and mixing them produces an inconsistent objective."}],"correct":false,"tempting":[{"text":"Invents an incompatibility; multiple-negatives ranking loss accepts explicit negatives alongside in-batch ones by design."}]},{"text":[{"text":"The top-ranked non-gold passages are frequently also relevant, so training pushes apart passages that should be close together."}],"correct":true,"tempting":null}],"order":[0,2,1,3],"why":null},"prev":{"slug":"multi-gpu-without-tears","title":"Multi-GPU without tears","minutes":9},"next":{"slug":"training-a-reranker","title":"Training a reranker","minutes":8}}}