{"course":{"slug":"agents-and-automation","title":"Agents and Automation","url":"https://addaly.com/learn/agents-and-automation","level":"intermediate","track":"build"},"lesson":{"slug":"embeddings-in-practice","title":"What an embedding actually is","url":"https://addaly.com/learn/agents-and-automation/embeddings-in-practice","position":27,"of":72,"minutes":10,"answer":"An embedding is a list of numbers whose only meaning is distance to other numbers from the same model, so a similarity score is not comparable across models and changing model means re-encoding everything.","keyIdea":"An embedding is a list of numbers whose only meaning is distance to other numbers from the same model, so a similarity score is not comparable across models and changing model means re-encoding everything."},"body":{"format":"markdown","text":"## The object itself\n\nAn embedding model takes a piece of text and returns a fixed-length list of floating-point numbers: 384 of them for a small model, 1,536 or 3,072 for a large one. Nothing about any individual number means anything. The only thing that carries meaning is **distance**: text about similar things lands close together, text about different things lands apart.\n\nYou can see this on a laptop in about five minutes, with no GPU and no account:\n\n```python\nfrom sentence_transformers import SentenceTransformer\nm = SentenceTransformer(\"all-MiniLM-L6-v2\")     # 384 dims, ~80 MB\nv = m.encode([\"cancel my order\", \"I want a refund\", \"the roof is leaking\"])\nprint(v.shape)                                   # (3, 384)\n```\n\nTake cosine similarity of the pairs and the first two will score high and the third low. That is the entire idea, and everything built on embeddings (semantic search, clustering, recommendation, deduplication candidates, classification by nearest neighbour) is that idea with plumbing around it.\n\n## Cosine similarity, and the number that means nothing\n\nCosine similarity is the dot product of the two vectors divided by their lengths: 1.0 for identical direction, 0 for unrelated, negative for opposed. In practice, with most modern models, everything lands in a narrow band: a great many unrelated pairs score 0.3 to 0.5, and 0.8 can be either a strong match or a coincidence.\n\nThe important consequence: **a threshold you found on one model does not transfer to another**. The vectors live in a different space with a different scale, so 0.82 is not a level of similarity, it is a reading on an instrument you have not calibrated. Calibrate by sampling: take two hundred pairs from your own data, label them, and look at where the distributions of matching and non-matching pairs overlap. That overlap, not a number from a blog post, is your threshold.\n\nThe same fact explains a bigger operational point. If you change embedding model, every stored vector is worthless: you cannot compare old vectors with new ones, so you must re-encode the entire corpus. Store the model name and version alongside every vector, and treat a model change as a migration with a backfill, not a config edit.\n\n## Chunking is the decision that matters most\n\nThe unit you embed is the unit you retrieve. Embed a whole 40-page manual and you get one vector averaging everything it says, which is close to nothing. Embed single sentences and you retrieve fragments with no context: \"This is not permitted under clause 4\" without clause 4.\n\nA reasonable starting point is 200 to 500 words with a sentence or two of overlap, split on real boundaries (headings, paragraphs, list items) rather than at a fixed character count that cuts a table in half. Keep a pointer from every chunk to its parent document, and give the agent a tool to fetch the surrounding region. Retrieve small, read large.\n\n## Queries and passages are not the same kind of text\n\nA question (\"how do I cancel a subscription\") and the passage that answers it (\"Subscriptions may be terminated by written notice…\") do not look alike, so a model trained only to match similar sentences handles the pairing badly. Models trained for retrieval (the **E5** and **BGE** families, both free on Hugging Face) expect a prefix, typically `query: ` on the question and `passage: ` on the document.\n\nOmitting those prefixes does not raise an error. It quietly costs you accuracy, and it is one of the most common silent misconfigurations in retrieval systems.\n\n## You probably do not need a vector database\n\nFifty thousand chunks at 384 dimensions in 32-bit floats is 50,000 × 384 × 4 bytes, about 77 MB. That fits in memory on any machine, and searching it is one matrix multiplication:\n\n```python\nimport numpy as np\nscores = matrix @ query_vec        # (50000,) — a few milliseconds\ntop = np.argsort(-scores)[:10]\n```\n\nThat is the whole search engine. **sqlite-vec** adds persistence in a single file, **FAISS** adds approximate search when you reach millions, and **pgvector** puts it beside the rest of your data in PostgreSQL, which is usually the right answer for a business system because the filters and the permissions are already there. Reach for a dedicated vector service when you have measured that you need one.\n\n## What embeddings cannot do\n\nFour hard limits, each with a mechanism worth knowing.\n\n- **Negation.** \"Invoices that are not paid\" sits near \"invoices that are paid\", because a short negation word barely moves a vector summarising the topic. Use a filter on a field, not a sentence.\n- **Numbers and quantities.** \"Over £10,000\" is not distinguished from \"over £100\". Filter in SQL.\n- **Exact identity.** As in the deduplication lesson: near-identical strings are what embeddings are built to collapse.\n- **Recency and authority.** A vector has no idea which of two documents supersedes the other. That has to come from metadata you attach and rank on yourself.\n\nEach of these is fixed the same way: keep structured fields beside the vector, and let the vector do the one job it is good at."},"check":{"question":"A team switches from a 384-dimension embedding model to a 1,024-dimension one for better quality, re-encodes new documents as they arrive, and leaves existing vectors alone. Search quality collapses. Why?","options":[{"text":"The larger model produces longer vectors, so the index needs rebuilding for performance reasons.","correct":false,"tempting":"Dimension size affects speed and memory, but a slow index returns the same results, not wrong ones."},{"text":"The new model needs different chunk sizes, so the old chunks are now mismatched to it.","correct":false,"tempting":"Chunking preferences do shift between models, but that degrades results gradually rather than breaking comparison outright."},{"text":"The similarity threshold tuned for the old model is too low for the new one and needs raising.","correct":false,"tempting":"Thresholds genuinely do not transfer between models, but recalibrating one cannot repair scores computed across two different spaces."},{"text":"Vectors from two different models occupy unrelated spaces, so distances between an old vector and a new query are meaningless: the whole corpus must be re-encoded.","correct":true}]},"prev":{"slug":"retrieval-for-agents","title":"Grep first, embeddings later","url":"https://addaly.com/learn/agents-and-automation/retrieval-for-agents"},"next":{"slug":"writing-back-safely","title":"Writing into a system somebody depends on","url":"https://addaly.com/learn/agents-and-automation/writing-back-safely"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What an embedding actually is', in Agents and Automation. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/agents-and-automation/embeddings-in-practice (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What an embedding actually is","courseTitle":"Agents and Automation","module":{"index":3,"slug":"data-plumbing","title":"The data underneath: sources, documents, records"},"answer":null,"keyIdea":"An embedding is a list of numbers whose only meaning is distance to other numbers from the same model, so a similarity score is not comparable across models and changing model means re-encoding everything.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-object-itself","runs":[{"text":"The object itself"}]},{"t":"p","runs":[{"text":"An embedding model takes a piece of text and returns a fixed-length list of floating-point numbers: 384 of them for a small model, 1,536 or 3,072 for a large one. Nothing about any individual number means anything. The only thing that carries meaning is "},{"text":"distance","b":true},{"text":": text about similar things lands close together, text about different things lands apart."}]},{"t":"p","runs":[{"text":"You can see this on a laptop in about five minutes, with no GPU and no account:"}]},{"t":"code","lang":"python","label":"Python","text":"from sentence_transformers import SentenceTransformer\nm = SentenceTransformer(\"all-MiniLM-L6-v2\")     # 384 dims, ~80 MB\nv = m.encode([\"cancel my order\", \"I want a refund\", \"the roof is leaking\"])\nprint(v.shape)                                   # (3, 384)"},{"t":"p","runs":[{"text":"Take cosine similarity of the pairs and the first two will score high and the third low. That is the entire idea, and everything built on embeddings (semantic search, clustering, recommendation, deduplication candidates, classification by nearest neighbour) is that idea with plumbing around it."}]},{"t":"h2","id":"cosine-similarity-and-the-number-that-means-nothing","runs":[{"text":"Cosine similarity, and the number that means nothing"}]},{"t":"p","runs":[{"text":"Cosine similarity is the dot product of the two vectors divided by their lengths: 1.0 for identical direction, 0 for unrelated, negative for opposed. In practice, with most modern models, everything lands in a narrow band: a great many unrelated pairs score 0.3 to 0.5, and 0.8 can be either a strong match or a coincidence."}]},{"t":"p","runs":[{"text":"The important consequence: "},{"text":"a threshold you found on one model does not transfer to another","b":true},{"text":". The vectors live in a different space with a different scale, so 0.82 is not a level of similarity, it is a reading on an instrument you have not calibrated. Calibrate by sampling: take two hundred pairs from your own data, label them, and look at where the distributions of matching and non-matching pairs overlap. That overlap, not a number from a blog post, is your threshold."}]},{"t":"p","runs":[{"text":"The same fact explains a bigger operational point. If you change embedding model, every stored vector is worthless: you cannot compare old vectors with new ones, so you must re-encode the entire corpus. Store the model name and version alongside every vector, and treat a model change as a migration with a backfill, not a config edit."}]},{"t":"h2","id":"chunking-is-the-decision-that-matters-most","runs":[{"text":"Chunking is the decision that matters most"}]},{"t":"p","runs":[{"text":"The unit you embed is the unit you retrieve. Embed a whole 40-page manual and you get one vector averaging everything it says, which is close to nothing. Embed single sentences and you retrieve fragments with no context: “This is not permitted under clause 4” without clause 4."}]},{"t":"p","runs":[{"text":"A reasonable starting point is 200 to 500 words with a sentence or two of overlap, split on real boundaries (headings, paragraphs, list items) rather than at a fixed character count that cuts a table in half. Keep a pointer from every chunk to its parent document, and give the agent a tool to fetch the surrounding region. Retrieve small, read large."}]},{"t":"h2","id":"queries-and-passages-are-not-the-same-kind-of-text","runs":[{"text":"Queries and passages are not the same kind of text"}]},{"t":"p","runs":[{"text":"A question (“how do I cancel a subscription”) and the passage that answers it (“Subscriptions may be terminated by written notice…”) do not look alike, so a model trained only to match similar sentences handles the pairing badly. Models trained for retrieval (the "},{"text":"E5","b":true},{"text":" and "},{"text":"BGE","b":true},{"text":" families, both free on Hugging Face) expect a prefix, typically "},{"text":"query: ","code":true},{"text":" on the question and "},{"text":"passage: ","code":true},{"text":" on the document."}]},{"t":"p","runs":[{"text":"Omitting those prefixes does not raise an error. It quietly costs you accuracy, and it is one of the most common silent misconfigurations in retrieval systems."}]},{"t":"h2","id":"you-probably-do-not-need-a-vector-database","runs":[{"text":"You probably do not need a vector database"}]},{"t":"p","runs":[{"text":"Fifty thousand chunks at 384 dimensions in 32-bit floats is 50,000 × 384 × 4 bytes, about 77 MB. That fits in memory on any machine, and searching it is one matrix multiplication:"}]},{"t":"code","lang":"python","label":"Python","text":"import numpy as np\nscores = matrix @ query_vec        # (50000,) — a few milliseconds\ntop = np.argsort(-scores)[:10]"},{"t":"p","runs":[{"text":"That is the whole search engine. "},{"text":"sqlite-vec","b":true},{"text":" adds persistence in a single file, "},{"text":"FAISS","b":true},{"text":" adds approximate search when you reach millions, and "},{"text":"pgvector","b":true},{"text":" puts it beside the rest of your data in PostgreSQL, which is usually the right answer for a business system because the filters and the permissions are already there. Reach for a dedicated vector service when you have measured that you need one."}]},{"t":"h2","id":"what-embeddings-cannot-do","runs":[{"text":"What embeddings cannot do"}]},{"t":"p","runs":[{"text":"Four hard limits, each with a mechanism worth knowing."}]},{"t":"ul","items":[[{"text":"Negation.","b":true},{"text":" “Invoices that are not paid” sits near “invoices that are paid”, because a short negation word barely moves a vector summarising the topic. Use a filter on a field, not a sentence."}],[{"text":"Numbers and quantities.","b":true},{"text":" “Over £10,000” is not distinguished from “over £100”. Filter in SQL."}],[{"text":"Exact identity.","b":true},{"text":" As in the deduplication lesson: near-identical strings are what embeddings are built to collapse."}],[{"text":"Recency and authority.","b":true},{"text":" A vector has no idea which of two documents supersedes the other. That has to come from metadata you attach and rank on yourself."}]]},{"t":"p","runs":[{"text":"Each of these is fixed the same way: keep structured fields beside the vector, and let the vector do the one job it is good at."}]}],"check":{"question":[{"text":"A team switches from a 384-dimension embedding model to a 1,024-dimension one for better quality, re-encodes new documents as they arrive, and leaves existing vectors alone. Search quality collapses. Why?"}],"options":[{"text":[{"text":"The larger model produces longer vectors, so the index needs rebuilding for performance reasons."}],"correct":false,"tempting":[{"text":"Dimension size affects speed and memory, but a slow index returns the same results, not wrong ones."}]},{"text":[{"text":"The new model needs different chunk sizes, so the old chunks are now mismatched to it."}],"correct":false,"tempting":[{"text":"Chunking preferences do shift between models, but that degrades results gradually rather than breaking comparison outright."}]},{"text":[{"text":"The similarity threshold tuned for the old model is too low for the new one and needs raising."}],"correct":false,"tempting":[{"text":"Thresholds genuinely do not transfer between models, but recalibrating one cannot repair scores computed across two different spaces."}]},{"text":[{"text":"Vectors from two different models occupy unrelated spaces, so distances between an old vector and a new query are meaningless: the whole corpus must be re-encoded."}],"correct":true,"tempting":null}],"order":[0,1,2,3],"why":null},"prev":{"slug":"retrieval-for-agents","title":"Grep first, embeddings later","minutes":9},"next":{"slug":"writing-back-safely","title":"Writing into a system somebody depends on","minutes":10}}}