A matrix is a stack of questions

Stop looking at the grid

A matrix is drawn as a grid of numbers, and that drawing is why it feels like homework. The grid is storage. It is not what the thing is.

A matrix is a function. You feed it a vector, it gives you back a vector. That is the useful definition, and it is the one that makes neural networks make sense.

Each row asks one question

Here is the mechanism, and it is entirely built from the last lesson. To apply a matrix to a vector, take each row of the matrix and dot-product it with your vector. Each row produces one number. Stack those numbers and that is your output vector.

So a matrix with 4 rows turns any input into 4 numbers. Each row is a question being asked of the input, and its answer is one number.

# rows ask: "how much X?", "how much Y?", "X and Y together?"
M = [[1, 0],
     [0, 1],
     [1, 1]]
v = [3, 5]

out = [sum(r*x for r, x in zip(row, v)) for row in M]
# [3, 5, 8]

That reframes every linear layer in every model you will ever use. nn.Linear(768, 3072) is a matrix with 3072 rows and 768 columns. It takes a 768-number description and asks 3072 questions of it, each question learned during training. The output is 3072 answers.

The shape now reads as meaning, not bookkeeping. Rows out, columns in. If the columns do not match the length of your input, the questions are asking about slots your vector does not have, which is exactly what a shape mismatch error is telling you.

What questions can be learned

The answer is: anything that is a weighted sum of the input.

A matrix can rotate a set of points. It can stretch one direction and squash another. It can project 768 numbers down to 2 so you can plot them. It can copy an input into three separate outputs, which is precisely what a transformer does to build queries, keys and values from one hidden state.

What it cannot do is bend. Straight lines stay straight, parallel lines stay parallel, and the origin stays put. A matrix cannot decide "only respond if this value is above 30". That is a bend, and no grid of numbers will produce one.

Why that limit forces activation functions

Apply matrix A, then apply matrix B. Ask a question of the answers to some questions.

The result is the same as applying one single matrix, BA. Not similar to it. Identical to it, for every possible input. Multiplying matrices together is exactly the operation of composing them into one.

So a hundred stacked linear layers can do precisely as much as one linear layer. Not less, but not one bit more. All those parameters, all that compute, and the model can still only draw straight lines.

That is the entire reason ReLU exists. Between the layers, do something a matrix cannot do:

def relu(x):
    return max(0.0, x)

Negative becomes zero, positive passes through. It is barely an operation. But it is a bend, and it cannot be absorbed into the neighbouring matrices. Now B(relu(A(x))) is genuinely a new function that no single matrix can reproduce, and stacking starts buying you something. Depth in neural networks is not matrices piled up. It is matrices with bends wedged between them, and the bends are what makes the pile worth having.

A practical read

When you next see a model summary listing shapes — (768, 3072), (3072, 768), (768, 50257) — you can read the pipeline directly. Expand 768 features into 3072 questions. Bend. Compress the 3072 answers back into 768. That final (768, 50257) is one row per token in the vocabulary, each row asking "how much does this hidden state look like my token?"

No grids required.