Inputs designed to hurt

The gap between the inputs you imagine and the inputs that arrive

Generated code handles the inputs that resemble its training. Real systems receive inputs that resemble the world. The gap between those two sets is where a large share of production incidents live, and it is cheap to explore on purpose.

There are two levels of effort here. The corpus, which takes twenty minutes and everyone should do. And fuzzing, which takes an afternoon and is worth it for anything that parses.

The corpus, which costs twenty minutes

Keep a file of hostile inputs and run every parser, form handler and API endpoint against it. This one list has found more bugs in more codebases than any tool.

Emptiness and size. Empty string. A single space. A single character. Ten megabytes of one character. A list of zero items. A list of a hundred thousand items. Deeply nested JSON, forty levels down: this crashes recursive parsers and it is a denial-of-service vector.

Unicode. José in both normalisation forms, which compare unequal byte-for-byte and equal to a human. An emoji with a skin-tone modifier and a zero-width joiner, which is one grapheme and several code points and many bytes, so len() gives three different answers depending on what you asked. Right-to-left override characters, which make file‮gnp.exe display as something else entirely. The Turkish dotless ı, where lowercasing I gives different results by locale.

Numbers. Zero. Negative zero. The maximum and minimum integers for your type. 0.1 + 0.2. A number with fifteen significant digits arriving as a JSON float, which loses precision silently in JavaScript above 2^53. A quantity expressed as a string. A currency amount with three decimal places, because Kuwaiti dinars have them.

Time. The end of February in a leap year. The day a clock goes forward, which in some zones means 02:30 does not exist. The day it goes back, when 01:30 happens twice. A timestamp in a timezone with a 45-minute offset: Nepal is UTC+05:45 and it is a real place with real users. A date before 1970. A date in the year 10,000, which breaks every fixed-width parser.

Names and text. A name with an apostrophe, which is both a real name and an SQL injection test. A single-word name, common across much of Indonesia. A name longer than your column. A name in a script your font does not have. NULL as a surname, which is a real family name and breaks CSV pipelines constantly.

Write these into a fixtures file and parametrise your parser tests over it. Adding one line to that file after every incident is how it becomes valuable over time.

Fuzzing, for anything that parses

A fuzzer generates inputs automatically, watches which code paths they reach, and mutates the ones that reach new territory. It is coverage-guided search, and it finds crashes that no human would think to write.

# Python, with Atheris
import atheris, sys
with atheris.instrument_imports():
    import my_parser

def one(data):
    try:
        my_parser.parse(data)
    except my_parser.ParseError:
        pass          # expected — anything else is a finding

atheris.Setup(sys.argv, one)
atheris.Fuzz()

// Go, built in since 1.18
func FuzzParse(f *testing.F) {
    f.Add([]byte("a=1"))
    f.Fuzz(func(t *testing.T, b []byte) { Parse(b) })
}

All free: libFuzzer and AFL++ for C and C++, cargo fuzz for Rust, Atheris for Python, Jazzer for Java, Go’s built-in fuzzing. Run for ten minutes on your CSV parser, your config loader, your protocol decoder. Ten minutes is usually enough to find something.

The crashing input gets committed as a regression test. That is the whole workflow.

Where fuzzing does not reach

A fuzzer finds crashes, hangs and assertion failures. It does not find wrong answers, because it has no idea what the right answer is. A parser that silently returns the wrong value for a valid input will fuzz clean for ever.

So the layers compose rather than substitute: fuzzing for “does it fall over”, properties for “is it consistent”, differential runs for “does it match what we had”, and a human with a worked example for “is it right”. Each covers a different failure and no one of them is enough. A team that has all four has a verification capacity that can absorb cheap generation. A team that has a green test suite has a number.