Shipping the adapter

Merge, or serve the adapter

Merged. Fold the adapter into the base and save full weights. The result loads anywhere, converts to GGUF, and needs no special support. It costs the full model size on disk and per deployment, and you cannot turn it off.

Unmerged. Keep the adapter as a separate 40 to 200 MB file. Serving stacks that support it can load one base model and many adapters, choosing per request.

That last capability is the one worth designing around. vLLM will hold a base model in memory and route each request to whichever adapter it names:

vllm serve Qwen/Qwen3-8B   --enable-lora   --max-loras 8 --max-lora-rank 32   --lora-modules support=/adapters/support-v3 triage=/adapters/triage-v1

{"model": "support", "messages": [...]}

One GPU, one copy of the base weights, eight tuned behaviours. For a multi-tenant product, or a team with several tasks, this is the difference between one machine and eight. The per-request cost of applying an adapter is small.

It also gives you the fix from the forgetting lesson: requests that need general behaviour can address the base model directly, so a tune that damaged general capability damages only the traffic that wanted the tune.

For laptop deployment you will merge and convert:

python convert_hf_to_gguf.py ./merged --outfile ft-f16.gguf --outtype f16
./llama-quantize ft-f16.gguf ft-Q4_K_M.gguf Q4_K_M

Verify the chat template survived the conversion, for the reasons the conversion lesson gave. This is the most common way a good fine-tune arrives broken.

Version everything together

A fine-tuned deployment has five moving parts, and all five must be pinned as one unit:

base model    Qwen/Qwen3-8B @ 9a3f21c
adapter       support-v3 (sha256 …)
prompt        prompts/support.qwen3-8b.md @ commit e41b0
sampler       temperature 0, top_p 1, repeat_penalty 1.0, max_tokens 400
eval          held-out set v4, task 0.86 / regression 0.94

Changing any one of them changes the behaviour. A team that upgrades the base model without re-tuning, or edits the prompt without re-running the evaluation, has shipped an untested system that looks like the tested one.

Keep this block in the repository next to the code, not in somebody’s notes.

The gate

Before any adapter is promoted, two numbers on two fixed sets:

• Task metric on the held-out set, compared against the base model with the best prompt, at identical settings.
• Regression score, compared against the recorded base answers.

Write both into the release. If the task gain does not exceed the noise on a set that size, do not ship it: an earlier lesson gave the arithmetic, and on forty examples a five-point gap is nothing.

Automate this as a script that takes an adapter path and prints two numbers. It takes an afternoon to write and it removes the entire class of argument about whether the new tune is better.

Rollback, and the copy that saves you

Keep the previous adapter and the ability to switch by configuration. Adapters are small enough that keeping every version you ever shipped costs almost nothing, and being able to revert in one deploy rather than one retrain is worth far more than the disk.

Keep the base model weights too, at the exact revision, on your own storage. The lesson on why labs release weights explained why: your fine-tune is meaningless without the base it was trained against, and a repository can be renamed, gated or withdrawn. An adapter without its base is an orphan.

The whole of this lesson reduces to one sentence: a fine-tune is a base revision plus an adapter plus a prompt plus a sampler plus two measurements, and shipping any of them separately is how a working system becomes a mystery.