PDFs, scans and the layout problem

The first question: is there text in there?

Two documents can look identical on screen and be completely different objects. One is a PDF with a text layer: the characters are stored as characters, and extracting them is exact and free. The other is a photograph of a page wrapped in a PDF container, where the only content is pixels.

Find out before you design anything:

pdftotext -layout invoice.pdf - | head -40

Text comes out, and you are in the easy world. Nothing comes out, or a few stray words, and you need OCR. In a real supplier folder you will find both, plus hybrids where somebody scanned page two of a digital document, so the pipeline must decide per file rather than per source.

That -layout flag matters more than it looks. Without it, a two-column page interleaves the columns into nonsense. pdftotext ships with Poppler and is free on every platform; pypdf and pdfplumber do the same from Python, and pdfplumber additionally gives you the coordinates of every word, which is what you want if you ever need to prove where a number came from.

There are no tables in a PDF

This is the fact that surprises people. The format stores glyphs at positions. A table is something a human eye assembles from alignment and whitespace; nothing in the file says “row” or “column”. Every table extractor (pdfplumber, Camelot, Tabula) is inferring structure from geometry, and each infers differently on merged cells, wrapped text and rows without ruling lines.

So expect table extraction to be around 80 to 95 per cent right on real documents and build for that rather than treating a failure as a bug. Validate structurally: does the number of columns match the header, do the line items sum to the printed total. Arithmetic is the cheapest table validator ever invented and it catches nearly every misparse.

OCR, and where the accuracy actually comes from

Tesseract is free, offline, and good, on clean input. Typed black text scanned at 300 dpi is close to perfect. A phone photo of a crumpled thermal receipt under a desk lamp is a different task, and no engine handles it well.

The lever is preprocessing, not engine choice. Deskew, convert to greyscale, threshold, upscale small text. ImageMagick does all of it in one line and free:

convert receipt.jpg -colorspace Gray -deskew 40% -resize 200% -threshold 60% clean.png
tesseract clean.png - --psm 6

The --psm page-segmentation mode is worth a half hour of experimenting; mode 6 (“a single uniform block”) often beats the default on receipts and forms. Teams who spend a day on preprocessing typically get more improvement than teams who spend a week comparing OCR engines.

Vision models read documents now

Multimodal models handle layout-heavy pages (forms, statements, invoices with a shifting design) better than a rules-based pipeline, and they will describe a page structure that no extractor infers correctly.

Two honest limitations. They transcribe digits wrongly sometimes, and the wrong digit arrives inside a fluent, well-formed, entirely plausible answer, which is worse than an OCR engine returning garbage you can see is garbage. And unless you explicitly ask for it, there is no provenance: no coordinates, no quote, nothing tying the number to a place on the page.

So ask for the evidence. Request a verbatim quote of the surrounding line with every field, and then check mechanically that the quote appears in the OCR text of that page. Cross-check totals. Both are cheap and both convert a confident fabrication into a caught error.

Email is its own parsing problem

An email is a MIME tree with a plain-text part, an HTML part, attachments, and possibly an entirely different message inline. Two rules save most of the trouble.

Prefer the plain-text part, and strip the quoted history. Reply chains carry last month’s amounts, last month’s dates and last month’s requests. An extractor pointed at a full thread will confidently pull a figure from a message three replies down. The talon library does reply-stripping reasonably; a hand-written rule that cuts at the first line matching On .* wrote: or a run of > characters gets you most of the way.

Cut signature blocks before extraction. Otherwise every message yields the sender’s office address as the delivery address, over and over, and it will look correct.

Keep the trail

Store the original bytes, a hash, the extracted text, and for each extracted field the page number it came from. When somebody disputes a figure in six months, the ability to open page three and point at it is the difference between a two-minute answer and a lost argument.