The library, and the four calls you need

Install and encode

pip install sentence-transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("BAAI/bge-small-en-v1.5")

vectors = model.encode(
    ["first document", "second document", "third document"],
    batch_size=64,
    normalize_embeddings=True,
    show_progress_bar=True,
)
print(vectors.shape)     # (3, 384)

Four arguments worth setting every time:

• batch_size: 32 to 128 on a CPU, larger on a GPU. Encoding a list is far faster than encoding in a loop, for the same reasons as module two.
• normalize_embeddings=True: scales every vector to unit length, which makes dot product and cosine similarity identical and lets you use the faster one. Set it and then be consistent; mixing normalised and unnormalised vectors in one index gives you nonsense.
• show_progress_bar: for anything over a few thousand items, so you know whether to make tea.
• convert_to_tensor=True when you want torch tensors on the GPU rather than NumPy arrays.

Similarity, in one call

sim = model.similarity(vectors[:1], vectors)
print(sim)      # tensor([[1.0000, 0.6421, 0.3117]])

model.similarity uses whatever the model was trained for (cosine for most, dot product for a few) which removes a decision you would otherwise get wrong. With normalised vectors you can also do it by hand:

import numpy as np
scores = vectors @ query_vec          # a plain matrix multiply
top = np.argsort(-scores)[:5]

One matrix multiply is a complete search engine over a hundred thousand documents, and it runs in milliseconds. Do not reach for a vector database until you have measured that this is too slow.

The prompt prefixes almost nobody reads about

Several of the best models were trained with an instruction prefix on the query but not on the documents. BGE, E5, GTE and Nomic families all do some version of this. Omit it and you lose several points of accuracy for no visible reason.

# E5 family
docs = model.encode([f"passage: {d}" for d in documents])
q = model.encode("query: where is my parcel")

# BGE family, via the library's own prompt support
q = model.encode("where is my parcel", prompt_name="query")

The model card says which. Read it before benchmarking anything: a large share of “this model scored worse than advertised” reports are this.

Matryoshka: shorter vectors on purpose

Some recent models are trained so that the first n dimensions are usable on their own. You can truncate 768 down to 256 and keep most of the quality:

model = SentenceTransformer("nomic-ai/nomic-embed-text-v1.5", trust_remote_code=True)
v = model.encode(texts, truncate_dim=256)

That is a third of the memory and roughly a third of the search time, for a small accuracy cost. On a million documents it is the difference between fitting in RAM and not. It only works for models trained this way: truncating an ordinary model’s vectors destroys them, because nothing made the early dimensions more important than the late ones.

Re-encoding is the cost that bites later

Every vector in your index was produced by one model at one revision. Change the model and every stored vector is worthless: you must re-encode the whole corpus. On a million chunks that is hours of compute and, if you are calling a hosted API, a real bill.

Two habits follow. Pick the model with the thirty-question test before encoding at scale rather than after. And keep the chunk text stored alongside the vectors, not only the vectors, so re-encoding is a local job rather than a re-extraction from source PDFs you may no longer have.

Saving and reloading vectors

import numpy as np
np.save("vectors.npy", vectors.astype("float16"))
vectors = np.load("vectors.npy").astype("float32")

Store at float16 and cast back for arithmetic: half the disk, no measurable retrieval loss. Store the document ids and texts alongside in a separate file, in the same order, and never sort one without the other: a vectors array whose order has drifted from its metadata is the single most annoying bug in this area, because every result is plausible and wrong.

Write the model name and revision into the file too. Vectors from two different models are not comparable and there is nothing in the array to tell you which produced it.

Do this now

Encode 1,000 sentences of your own, save them as float16, reload, and search with a matrix multiply. Time the search. It will be a few milliseconds, and that number is why most projects never need a vector database.