The ambiguity audit

A procedure, not a talent

Senior engineers appear to have an instinct for the question nobody asked. It is not an instinct. It is a list, learned by being burned, and you can have the list without the burning.

Take any request and walk it against the categories below. It takes about ten minutes and it produces the questions that would otherwise be answered for you, silently, in whichever direction the priors point.

The worked example: ”Add search to the product catalogue.” A request that sounds complete and specifies almost nothing.

The categories, applied

Matching. Does searching for “blue shirt” require both words? Does it match “shirt, blue”? Does it match “shirts”? Is it case-sensitive? Are accents ignored, so that “cafe” finds “café”, and if so, in which direction? Does a search for “7” match “iPhone 7” and “7-pack”?

Ordering. What comes first when twenty things match? Relevance is not a specification; it is a placeholder for one. Newest? Most bought? Cheapest? A weighted combination somebody has to choose numbers for? And when two items score identically, what breaks the tie, because if nothing does, the order changes between requests and users notice.

Emptiness. Zero results: an empty page, suggestions, a relaxed search that drops a term? A one-character query: do you search on it? An empty query: everything, nothing, or the default listing?

Cardinality and size. How many results per page? What happens at result 10,000? Can somebody paginate to page 900, and if so does your database do a full scan to get there?

Visibility and permission. Do out-of-stock products appear? Discontinued ones? Products not yet published? Products a particular customer is not allowed to buy in their country? Search is the single most common place where an access-control rule is forgotten, because the query was written against the whole table.

Time. Is the index live or rebuilt nightly? If a merchant edits a price, when does search reflect it? “Eventually” is an answer, and it needs a number, because the support team will be asked.

Language. Which languages are the products in? Does a search in Hindi find a Hindi title? Does Devanagari text tokenise at all in the engine you were about to use? Does the search box work in a language that does not put spaces between words?

Failure. The search service is down. Show an error, fall back to a plain database LIKE query, or fall back to the category listing? This is a business decision with a revenue consequence and no technically correct answer.

Scale and cost. How many products? A thousand and a hundred million are different systems with different technology. What query rate?

Observability. Are queries logged? For how long? Search queries are among the most personally revealing data a system holds, and this decision belongs in the specification rather than in whatever the library does by default.

What the audit produced

Twenty-odd questions from a five-word request. In practice five or six of them turn out to matter and the rest have obvious answers, but you cannot tell which is which until you have asked, and the ones that matter are usually not the ones you expected.

The output is not a document. It is a short list of decisions, most of which take somebody ten seconds:

Match: all terms required, case- and accent-insensitive, no stemming in v1.

Order: relevance, ties broken by product id ascending so paging is stable.

Empty query: return the default category listing, not an error.

Permissions: only status = 'published' and in-stock; unpublished never appears, including for admins in v1.

Freshness: index updated within 60 seconds of a product edit.

Search down: fall back to ILIKE on title only, log it, show no error to the user.

Logging: queries retained 30 days, no user id attached.

Seven lines. Every one of them changes the code, six of them would otherwise have been guessed, and the tie-break line prevents a bug that is hard to diagnose later.

The two questions that catch the most

If you remember nothing else from this lesson, keep these two.

”What happens when there are none, one, and a great many?” Nearly every collection bug lives at one of those three points.

”What happens when the thing this depends on is unavailable?” Every distributed system question in miniature, and the answer is always a business trade rather than a technical one.

Where the audit goes

Into the ticket, or a file next to the code. Not into your head, and not into a chat window that will be closed.

The practical reason is that this list has three consumers. It tells whoever or whatever writes the code what to build. It tells the reviewer what to check against, which is the check that stopped working when internal consistency became free. And it becomes the test names, more or less directly. One artefact, three jobs, and they cannot disagree with each other because there is only one of them.