Running a type checker: the bugs it finds that tests do not

What a test cannot see

A test runs one path through the code with one set of inputs. If the path where a function returns None was never exercised, the test does not know it exists. A type checker reads the code instead of running it and follows every path the annotations allow. That is a different kind of evidence, and it is cheap: no test data, no fixtures, one command.

Module 3 said hints are not enforced at runtime. This is where they get enforced.

Two checkers, one command

mypy is the original; pyright is Microsoft's, faster, and the engine inside VS Code's Pylance, so you may already be seeing its output as red underlines. Either works. Both are free.

pip install pyright        # or: pip install mypy
pyright src/               # or: mypy src/

Run it on the src/ folder from module 3's project layout. The first run on unannotated code says very little, because a function with no hints is treated as accepting and returning Any, and Any is compatible with everything. The checker becomes useful in proportion to the annotations you add, and the payoff is front-loaded: annotate the functions at the boundaries of your program and most of the value arrives.

The None problem

def find(items: list[dict], key: str) -> dict | None:
    for it in items:
        if it["key"] == key:
            return it
    return None

row = find(rows, "abc")
print(row["id"])

error: "__getitem__" method not defined on type "None"

The checker is saying: on one path this is None, and None["id"] will raise. You wrote dict | None yourself — that is the point. Writing the honest return type forces every caller to handle the missing case:

row = find(rows, "abc")
if row is None:
    raise KeyError("abc")
print(row["id"])

After the if, the checker knows row is a dict — this is called narrowing — and the error disappears. Optional[dict] from typing means the same as dict | None; the bar form needs Python 3.10 or later.

Most of what a type checker finds in ordinary code is this: a value that can be None used as though it cannot. It is also most of what crashes in production at 2 a.m.

Annotations that do work

from typing import Literal, TypedDict, Any

Role = Literal["system", "user", "assistant"]

class Message(TypedDict):
    role: Role
    content: str

def add(messages: list[Message], role: Role, content: str) -> None:
    messages.append({"role": role, "content": content})

add(msgs, "usre", "hi")     # error: "usre" is not a valid Role

A Literal turns a typo in a string constant into a checker error rather than a 400 from the API. A TypedDict describes the shape of a dict that has to stay a dict — the message format the SDKs want — so m["contnet"] is caught. Any is the escape hatch: it means "do not check this", and every Any is a place the checker goes blind. Use it at the edge where JSON arrives, then narrow to real types as fast as you can.

dict[str, Any] is honest for parsed JSON of unknown shape. list[str] beats list. A return type of None on a function that only has side effects catches the caller who assigns its result.

Protocol: an interface without inheritance

Module 7 built ChatProvider as a base class. The checker can describe the same interface structurally:

from typing import Protocol

class Completer(Protocol):
    def complete(self, messages: list[Message], max_tokens: int = 500) -> str: ...

def run(provider: Completer, text: str) -> str:
    return provider.complete([{"role": "user", "content": text}])

Any class with a matching complete method satisfies Completer, whether or not it inherits from anything. The fake in a test, the real Anthropic wrapper, a lambda-holding class — all pass, and a class whose complete returns a list instead of a string is flagged at the call site. This is duck typing with the checker watching, and it is usually the better choice when you control the callers but not the implementations.

What it will not catch

A type checker knows types, not values. price: float accepts -5.0. role: Role cannot tell you the system message was put last instead of first. A function annotated -> str that returns the wrong string is fine by the checker. Validation of values is pydantic's job; correctness of logic is the tests' job; the checker sits between them, catching the category of error where a value of the wrong kind reaches a place that cannot handle it.

It also stops at library boundaries that ship no types. requests has stubs (types-requests); some smaller packages have nothing, and calls into them come back as Any. # type: ignore on a line silences one error; use it with a comment saying why, and treat a growing count of them as a smell.

Making it stick

Add it to the checks you already run:

pyright src/ && pytest && python -m ruff check src/

and to the same CI job module 9 sets up. pyright --strict or mypy --strict turns on every check including "this function has no annotations"; start without strict, add annotations module by module, then turn it on for the modules that are done. Strict on day one produces a wall of errors that teaches nothing.

Try this now

Annotate find and its caller, run pyright, and read the error. Fix it with narrowing. Then change a role string to a typo and watch Literal catch it. Finally, write Completer as a Protocol and check that your FakeProvider from earlier satisfies it without inheriting.