Keeping up without drowning

The volume problem

More is published about this field every week than anybody can read, most of it is promotional, and a large share of what is presented as news is a company announcing a product. Attempting to keep up comprehensively is a full-time job that produces no benefit to your work.

The alternative is a filter, and the filter you already have is the decision record from earlier in the course: the list of conditions that would change your mind. Almost everything you read can be tested against it in three seconds. Does this bear on prompts 4 and 5? No. Move on.

A budget, and what to spend it on

One hour a month is enough. Spend it like this:

Twenty minutes on changelogs for the tools you actually use. Every major provider publishes one. This is the highest-value reading available and almost nobody does it, because it is dull. It turns most surprises into things you knew were coming.

Twenty minutes on one substantial piece, not five short ones. A technical report, a careful evaluation, a well-argued critique. Short posts are optimised for sharing rather than for being right.

Twenty minutes trying something. Reading about a capability tells you far less than ten minutes using it. If a new mode or a new model appears, run three of your twelve prompts through it. That is a measurement; a review is a report of somebody else’s.

What is worth following

Provider changelogs and status pages for what you use.

Primary sources. A model card, a technical report, the actual licence, the actual pricing page. A surprising proportion of confident writing about this field is a paraphrase of a paraphrase, and the original is usually shorter and clearer.

One or two people who are specific and admit error. The useful signal is somebody who says “I tested this and I was wrong last month”. Enthusiasm and permanent scepticism are both easy to produce and neither is informative.

Nothing that requires you to be angry. A large amount of coverage in this area is written to produce a reaction rather than to inform, in both directions. It is not neutral to consume, and it does not tell you anything you can act on.

What to ignore

• Announcements of things not yet available.
• Benchmark scores a point or two apart.
• “X is dead” of any kind.
• Screenshots of a remarkable output, which are one draw from a distribution and selected for being remarkable.
• Anything about a tool you do not use, unless it meets a condition on your list.
• Predictions about two years from now, from anybody.

Where to look, practically

Two habits cover most of it. Subscribe to the changelog or release notes of the two or three tools you depend on, by email or by feed, so they arrive rather than needing to be sought. And when something is claimed loudly enough that you keep encountering it, go to the primary source once (the technical report, the model card, the pricing page) rather than reading a fifth summary.

For anything you are considering buying, the most useful single source is usually the provider’s own documentation rather than its marketing, because the documentation has to be accurate for the product to work and the marketing does not.

The signal that is worth acting on

Four things are worth interrupting your month for:

1. A price change on something you pay for, in your currency.
2. A change in data terms for a tool you send work material to.
3. A deprecation notice for something you depend on.
4. A capability appearing at a much smaller size or lower price: a model you can run locally doing something that previously needed a frontier model. This is the change that most often shifts a real decision, and it happens quietly rather than with an announcement.

Everything else waits for the six-month review, where you will meet it with twelve prompts and a scoring sheet instead of an opinion. That is the whole strategy: read little, test what matters, and let the calendar entry do the work that anxiety would otherwise do.