Sending images, PDFs and audio, and what they cost

An image is tokens too

A vision model does not process a picture as a picture. The image is cut into patches, each patch is embedded, and the resulting vectors are placed in the same sequence as your text tokens. So an image has a token count, it consumes context window, and it appears on your bill.

The formula most providers use is close to width × height ÷ 750. Some practical values:

• 512 × 512: about 350 tokens
• 1092 × 1092: about 1,590 tokens, which is near the point of diminishing returns for most providers
• 2000 × 1500: about 4,000 tokens, and often downscaled by the provider anyway

A thousand-token image is roughly 750 words of text. Send eight product photos in one request and you have spent more on images than on your entire prompt.

The optimisation follows directly: resize before sending. If the answer depends on the overall layout of a page, 1024 pixels on the long edge is plenty. If it depends on small print, crop to the region rather than sending the whole page at high resolution: a 400 × 300 crop of the total line on an invoice costs 160 tokens and works better than a 3000-pixel scan, because you have removed the distractors as well as the cost.

PDFs are the hard case

There are three ways to get a PDF into a model, and they fail differently.

Extract the text layer. pdfplumber, PyMuPDF and pdftotext are free and fast. This works perfectly for PDFs generated by software and not at all for scans, which have no text layer. It also destroys most table structure: a two-column layout often comes out interleaved line by line, producing text that reads like nonsense and that a model will confidently summarise anyway.

Render pages to images and send them to a vision model. Preserves layout, handles scans, costs 1,500 to 2,000 tokens per page. A 40-page contract is 60,000 to 80,000 tokens per question, which is real money if you ask more than one.

Use a document-specific OCR pipeline: Tesseract is the free classic, docTR and Surya are stronger open-source options, and every cloud has a paid one. You get text with coordinates, which lets you rebuild tables and, crucially, cite a page and a position later.

The pattern that survives contact with reality is hybrid: extract the text layer, and if it comes back empty or nearly empty, fall back to rendering pages. Store both the text and the page image reference, so that when a user asks where a number came from you can show them.

Be honest about failure. Handwriting, faded thermal receipts, rotated scans, stamps over text and multi-column forms all break OCR and vision models alike, and the failure is usually silent: you get a plausible number that is wrong by a digit. For anything financial, extract with a model and then verify with arithmetic: do the line items sum to the stated total? A model that cannot add will happily report a total that does not match the rows above it, and a three-line check catches it.

Audio

Two routes.

Transcribe first. Whisper is open-weight and free; faster-whisper runs the small model on a CPU at several times real time, so an hour of audio transcribes in a few minutes on a laptop. You then have text, which is cheap, searchable, diffable and easy to evaluate. This is the right default for meetings, interviews, calls and voice notes.

Send audio to a model that accepts it natively. You keep tone, hesitation, overlapping speakers and emphasis, all of which a transcript discards. You pay considerably more and you lose the intermediate artefact you could have inspected.

Transcription has known, structured failure modes worth designing around: proper nouns and Indian names are frequently mangled, code-switching between languages mid-sentence confuses language detection, and (the dangerous one) Whisper-family models can hallucinate fluent text during silence, producing sentences nobody said. Voice-activity detection before transcription removes most of that.

Where the model must not be trusted

Across all three input types, the same rule holds and it is the one that separates a demo from a product. A model reading a document is doing recognition and inference at the same time, and it will not tell you which is which. Asked for an invoice date it cannot read, it will supply a well-formatted, plausible date rather than say the scan is illegible.

So build the exit ramp:

• Ask for a verbatim quote alongside every extracted value, then check the quote actually appears in the text you supplied. If it does not, the value is invented.
• Require an explicit null for anything unreadable, and say so in the prompt.
• Apply arithmetic and format checks: dates in range, totals that add up, tax at a legal rate.
• Route anything that fails to a human, and count how often that happens. That count is your real accuracy number.