A working setup with two models

Deciding once instead of every time

The arrangement that works is not one tool. It is two or three, with a rule that decides which to use so you are not making the choice every time. The rule that works is not about difficulty: it is about what kind of material you are holding.

Does this contain something that must not leave my machine?
    yes -> local model
    no  -> is this routine work, or a lot of it?
             yes -> hosted open model or the cheap tier
             no  -> the strong model

Material first, difficulty second. This matters because difficulty is a judgement you get wrong under time pressure, while “is there a client name in this” is a fact you can see.

Making it a habit rather than a decision

Put the local model somewhere immediate. A desktop app, or a front-end where it is one dropdown away from everything else. If reaching it takes three steps, you will not use it on the Tuesday when it matters.

Set the default to the cheap option. Whatever you use most, set it to open on the small fast model. Reaching for the strong one should be a deliberate act; sitting on it all day is how people spend a fortune and wait unnecessarily.

Keep a redaction habit rather than a redaction step. Writing “the supplier” instead of the company name costs nothing, produces the same answer, and removes the decision entirely for a large share of your work.

A concrete setup, end to end

For a self-employed person handling client material, on a 16 GB laptop:

1. Ollama with an 8B instruct model: installed once, used for anything with a name, a number or a contract in it.
2. A free front-end (Chatbox or Open WebUI) with the local model and two API keys configured, so all three are in one window.
3. A hosted open model on prepaid credit, for bulk work: tidying transcripts, classifying an inbox, first-pass translation.
4. One frontier model, on a subscription or on the API, for the hard tenth.
5. The twelve-prompt file in a folder next to it all, and a calendar entry for September.

Total ongoing cost: one subscription, or none, plus a few dollars. Setup: one evening. This is not an advanced configuration; it is what somebody who has thought about it once ends up with.

What to check every so often

Two things drift, and both are cheap to re-check at the six-month review.

The local model has probably improved. Small open models have been getting better at a rate that makes an eighteen-month-old choice worth revisiting. A model that failed your prompts 7 to 9 last year may pass them now at the same size.

Your rule may be too cautious or not cautious enough. If everything is going to the local model and you are dissatisfied with the answers, the rule is too tight. If nothing is, you have stopped noticing what is in your documents.

A worked example of the rule in use

A physiotherapist writing up notes. The clinical note itself, with the patient’s name and history, goes to the local model for tidying and structuring: that is material that must not leave the machine, and tidying prose is exactly what a small model does well.

The general question that came out of the session (what the current evidence says about a particular rehabilitation protocol) goes to the strong hosted model, because it needs knowledge the local model does not have and it contains nothing identifying. And the fifty referral letters that need reformatting go to the cheap hosted model in a batch overnight.

Three tools, one afternoon’s work, and the routing was decided by looking at what was in front of her rather than by thinking about model capability at all.

The point of the whole arrangement

It is not thrift, and it is not privacy purism. It is that no single tool is right for every kind of material you handle, and that the cost of having two or three ready is close to zero once they are set up.

The person with this arrangement is not choosing between privacy and capability, or between cost and quality. They have arranged things so that each piece of work goes to whichever tool suits it, and the deciding takes no thought because the rule was written down once. That is the whole ambition of this course applied to a single desk.