The samplers most interfaces do not show you

The repetition problem, stated properly

A model that has just written a phrase finds that phrase more likely again, because it now appears in its own context. Left alone, this compounds: the model repeats a sentence, then a paragraph, then loops until it hits the token limit. Small models and quantised models do it more.

The classic remedy, repetition_penalty, divides the logits of tokens seen in the last N tokens. It works and it is blunt in a specific way worth understanding: it is token-blind. It penalises }, def, the, import, and every newline exactly as hard as it penalises the phrase you wanted to suppress. Above about 1.15 you can watch it break indentation, drop closing brackets and produce increasingly odd word choices, because the tokens that must repeat are being punished for repeating.

Two newer samplers fix this properly, and neither appears in most graphical interfaces.

DRY: penalise repeated sequences, not repeated tokens

DRY (“Do not Repeat Yourself”) looks for the longest suffix of the generated text that has occurred earlier, and penalises only the token that would extend that repetition. The penalty grows exponentially with the length of the repeated sequence.

The difference is decisive. A single } that has appeared two hundred times is untouched, because } alone is not a repeating sequence in context. A four-word phrase beginning to repeat for the third time is heavily penalised. You get repetition control without syntax damage, which repetition_penalty cannot offer at any setting.

{"dry_multiplier": 0.8, "dry_base": 1.75, "dry_allowed_length": 2}

dry_allowed_length is how long a repeat may be before penalty starts. Two is a good default; raise it for structured output where longer legitimate repeats occur.

XTC: remove the top choice sometimes

XTC (“Exclude Top Choices”) inverts the usual approach. Instead of truncating the tail, it occasionally removes the most likely tokens when several are already above a probability threshold, forcing a less predictable choice.

The reasoning: when a model is confident about several options, the top one is usually the blandest, most generic continuation. Removing it produces noticeably more varied text.

This is for creative writing and only for creative writing. On code, extraction or factual answers, deliberately discarding the model’s best token is precisely wrong. Two parameters, and both matter:

{"xtc_threshold": 0.1, "xtc_probability": 0.5}

Only tokens above the threshold are candidates for removal, and the removal happens with the given probability, so the top token still wins half the time.

Typical-p and Mirostat

Typical-p keeps tokens whose information content is close to the distribution’s expected information content: discarding both the boringly predictable and the wildly improbable. It produces text with steadier surprise, and it is worth trying when top-p output alternates between flat and unhinged.

Mirostat takes a different approach entirely: rather than fixing a truncation rule, it targets a perplexity level and adjusts as it generates, using feedback to hold the output at a chosen level of surprise. It largely replaces top-k and top-p when active. Some people find it excellent for long-form prose; it interacts confusingly with other samplers, so use it alone.

Order matters, and is not standard

These are applied in a sequence, and the sequence differs between backends. Penalties before or after temperature changes their strength. Truncation before or after temperature changes what survives. llama.cpp exposes --samplers to set the order explicitly:

--samplers "dry;top_k;typ_p;top_p;min_p;xtc;temperature"

This is why the same nominal settings can behave differently in two applications, and it is the answer when a configuration copied from a forum post does not reproduce.

Sane starting points

• Factual, extraction, classification, code: temperature 0. Nothing else matters, because there is no sampling to control. Reproducible, and it is the right default for anything a program consumes.
• General assistant: temperature 0.7, min_p 0.05, everything else off. Add dry_multiplier 0.8 if it loops.
• Creative prose: temperature 1.0, min_p 0.05, dry_multiplier 0.8, and xtc_threshold 0.1 with xtc_probability 0.5 if the writing feels generic.

Change one at a time, with the same prompt, and read the output. Sampler tuning is the part of this subject where people spend the most effort for the least measured benefit: get to a sane default, verify the template is right, and move on.