What a dot product measures

Multiply the matching slots, add them up

Two lists of the same length. Multiply slot 1 by slot 1, slot 2 by slot 2, and so on, then add every result into one number.

a = [2, 0, 3]
b = [1, 5, -1]
dot = 2*1 + 0*5 + 3*-1   # 2 + 0 - 3 = -1

One number out. That number is a score of how much the two lists agree.

Read the sign first. Positive means the two vectors mostly lean the same way. Zero means they are unrelated in the specific sense of being at right angles. Negative means they lean opposite. In the example above, a and b disagree slightly.

Where the number comes from

The dot product equals |a| × |b| × cos(θ): the length of the first, times the length of the second, times the cosine of the angle between them.

You do not need to work with that formula. You need one consequence of it: a dot product mixes two different things together. It is large when the vectors agree, and it is also large when the vectors are long. Length can drown out agreement.

Divide the dot product by both lengths and the lengths cancel out. What remains is cos(θ), a number from -1 to 1 that depends on angle alone. That is cosine similarity, and it is a dot product with the size effect removed.

def cosine(a, b):
    dot = sum(x*y for x, y in zip(a, b))
    na = sum(x*x for x in a) ** 0.5
    nb = sum(y*y for y in b) ** 0.5
    return dot / (na * nb)

Where models use it

Almost every number in a model that answers “how relevant is this to that” is a dot product.

Attention. Each token produces a query vector, each token produces a key vector. The attention score between two tokens is the dot product of one query and one key. A big score means “this token should look at that one”. When people say attention is expensive, this is why: for 4000 tokens, that is 16 million dot products, per head, per layer.

Retrieval. Your question becomes a vector, every document is already a vector, and the search is a dot product against each one. Vector databases exist to do that fast.

The final layer. A language model ends with a hidden state, one vector. To score the next token, it dot-products that state against one row per vocabulary item. 50,000 dot products, 50,000 scores, softmax over them, and you have the next-token distribution.

The honest limits

Cosine similarity measures whether two pieces of text get used in similar contexts. That is not the same as meaning the same thing, and people forget it constantly.

Antonyms score high. “Hot” and “cold” appear in near-identical sentences, so their vectors are close. “I approved the loan” and “I rejected the loan” are near neighbours. If you are building a retrieval system where the difference between approved and rejected matters, and in banking, health, or law it always matters, cosine similarity alone will hurt you, and it will do it quietly, with high confidence scores.

The second limit is the one from the formula. If you skip normalisation and rank by raw dot product, long documents win because long documents have long vectors. Your top result is then a 6000-word page that mentions everything, not the 200-word page that answers the question. This is one of the most common bugs in a first search system, and it never throws an error.