{"course":{"slug":"fine-tuning","title":"Fine-Tuning, and When Not To","url":"https://addaly.com/learn/fine-tuning","level":"advanced","track":"deep"},"lesson":{"slug":"classifiers-and-encoders","title":"When the answer is one of six labels","url":"https://addaly.com/learn/fine-tuning/classifiers-and-encoders","position":5,"of":84,"minutes":9,"answer":"When the output is one of a fixed set of labels, a 150M encoder fine-tuned on a thousand examples usually beats prompting a large decoder on accuracy.","keyIdea":"When the output is one of a fixed set of labels, a 150M encoder fine-tuned on a thousand examples usually beats prompting a large decoder on accuracy, latency and cost: at the price of needing labels and retraining to add a class."},"body":{"format":"markdown","text":"## You may not need a generative model at all\n\nA large share of what gets sent to a chat model is classification wearing a costume. Route this ticket to one of six queues. Is this review positive, negative or mixed. Does this sentence contain personal data. Is this transaction suspicious. The output space has six members, and you are paying a 70B decoder to write them out one token at a time.\n\nAn encoder model fine-tuned for classification does this better, and the gap is not small.\n\n## The arithmetic\n\nTake ticket routing, six queues, 2,000 labelled examples.\n\n- **Prompting a hosted frontier model.** Roughly 700 input tokens (instructions plus the ticket) and 5 output tokens. At typical 2026 pricing for a mid-tier hosted model that is on the order of $0.20 per thousand tickets, with a p95 latency near one second, and a rate limit that becomes your rate limit.\n- **Fine-tuned encoder.** ModernBERT-base is 149M parameters; DeBERTa-v3-base is 184M. Training on 2,000 examples takes about eight minutes on a free Colab or Kaggle T4. Inference runs at roughly 5 to 15 milliseconds per item on a CPU, batched far faster, with no per-call cost and no vendor.\n\nAccuracy usually favours the encoder too, once you have a thousand labels or more. The encoder sees the whole input bidirectionally and is optimised directly for the decision; the decoder is optimised to continue text and is being asked, indirectly, to make a decision.\n\n```python\nfrom transformers import AutoModelForSequenceClassification, AutoTokenizer\n\nmodel = AutoModelForSequenceClassification.from_pretrained(\n    \"answerdotai/ModernBERT-base\", num_labels=6)\n```\n\nThat `num_labels=6` replaces the pretraining head with a fresh linear layer of 768 × 6 weights. Everything below it is the pretrained encoder, and the whole thing is trained: full fine-tuning of a 149M model fits in 6 GB of GPU memory comfortably, so none of the LoRA arithmetic in this course is needed.\n\n## What you give up, precisely\n\nThis is a real trade, not a free win.\n\n- **You need labels.** A prompt works on day one with zero examples. An encoder needs several hundred before it is competitive and a thousand or two before it is clearly better. If a label costs a minute of somebody's attention, 2,000 labels is about 33 hours of work.\n- **No new classes without retraining.** Add a seventh queue and you retrain, cheap, but it is a step. A prompt takes a new class in one line.\n- **No explanation.** The classifier emits a label and a probability. If your users need a reason, you either add a second call to a generative model or you accept a score.\n- **Brittleness to drift.** An encoder trained on last year's tickets does not know this year's product names. It will not tell you so; it will just be wrong on the new ones. A frontier model degrades more gracefully because it reads the words.\n\n## The hybrid that most teams land on\n\nUse the big model to make the labels, and the small model to serve them. Run a frontier model over 3,000 historical tickets, have a person check a sample of 200 of those labels to get an agreement figure, then train the encoder on all 3,000. You have bought a labelled dataset for a few dollars and a serving model that costs nothing per call. Where the encoder's confidence is low, say the top probability is under 0.7, fall back to the expensive model for that item only. On real routing workloads that fallback fires on 5 to 15% of traffic.\n\nCheck your provider's terms before doing this. Several model licences restrict using outputs to train models, and what counts as a competing model is not always clear; that is the subject of a later lesson in this module.\n\n## The signal that you are in this situation\n\nYou are classifying, not generating, if any of these are true: the output is drawn from a fixed list; you evaluate with accuracy, precision or recall rather than with a human reading the text; the prompt contains the phrase \"answer with only one of\"; or you are post-processing the model's reply with a regex to extract the label. That regex is the clearest tell there is. It means you built a generative system and then spent effort hiding the generation.\n\nThe same logic covers extraction with a fixed schema (token classification, which is a per-token version of the same thing) and scoring (a regression head, one output instead of six). The family is called encoders, the library is `transformers`, and the whole path is free."},"check":{"question":"A team routes tickets with a hosted model and a regex that pulls the queue name out of the reply. They have 2,500 historical tickets already routed correctly by humans. Why does a fine-tuned 149M encoder tend to be more accurate here, not merely cheaper?","options":[{"text":"Smaller models overfit less, so the encoder generalises better to unseen tickets than a large model would.","correct":false,"tempting":"Inverts the usual relationship; small models are more prone to underfitting, and the encoder's advantage is not about overfitting at all."},{"text":"The encoder has read the company's tickets during pretraining, so it starts with knowledge the hosted model lacks.","correct":false,"tempting":"Invents a pretraining corpus; the encoder's pretraining is generic web and book text, exactly like the decoder's."},{"text":"Its head is trained directly on the six-way decision, while the decoder only decides indirectly.","correct":true},{"text":"Encoders are bidirectional and support longer inputs, so they see the whole ticket where a decoder truncates it.","correct":false,"tempting":"Half-true about bidirectionality but wrong about length: encoders typically have shorter context limits than modern decoders."}]},"prev":{"slug":"try-the-retriever-first","title":"Try the retriever first","url":"https://addaly.com/learn/fine-tuning/try-the-retriever-first"},"next":{"slug":"distillation-as-the-real-case","title":"Distillation, the case that usually survives","url":"https://addaly.com/learn/fine-tuning/distillation-as-the-real-case"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'When the answer is one of six labels', in Fine-Tuning, and When Not To. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/fine-tuning/classifiers-and-encoders (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"When the answer is one of six labels","courseTitle":"Fine-Tuning, and When Not To","module":{"index":1,"slug":"deciding-whether-to-fine-tune","title":"Deciding whether to fine-tune at all"},"answer":null,"keyIdea":"When the output is one of a fixed set of labels, a 150M encoder fine-tuned on a thousand examples usually beats prompting a large decoder on accuracy, latency and cost: at the price of needing labels and retraining to add a class.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"you-may-not-need-a-generative-model-at-all","runs":[{"text":"You may not need a generative model at all"}]},{"t":"p","runs":[{"text":"A large share of what gets sent to a chat model is classification wearing a costume. Route this ticket to one of six queues. Is this review positive, negative or mixed. Does this sentence contain personal data. Is this transaction suspicious. The output space has six members, and you are paying a 70B decoder to write them out one token at a time."}]},{"t":"p","runs":[{"text":"An encoder model fine-tuned for classification does this better, and the gap is not small."}]},{"t":"h2","id":"the-arithmetic","runs":[{"text":"The arithmetic"}]},{"t":"p","runs":[{"text":"Take ticket routing, six queues, 2,000 labelled examples."}]},{"t":"ul","items":[[{"text":"Prompting a hosted frontier model.","b":true},{"text":" Roughly 700 input tokens (instructions plus the ticket) and 5 output tokens. At typical 2026 pricing for a mid-tier hosted model that is on the order of $0.20 per thousand tickets, with a p95 latency near one second, and a rate limit that becomes your rate limit."}],[{"text":"Fine-tuned encoder.","b":true},{"text":" ModernBERT-base is 149M parameters; DeBERTa-v3-base is 184M. Training on 2,000 examples takes about eight minutes on a free Colab or Kaggle T4. Inference runs at roughly 5 to 15 milliseconds per item on a CPU, batched far faster, with no per-call cost and no vendor."}]]},{"t":"p","runs":[{"text":"Accuracy usually favours the encoder too, once you have a thousand labels or more. The encoder sees the whole input bidirectionally and is optimised directly for the decision; the decoder is optimised to continue text and is being asked, indirectly, to make a decision."}]},{"t":"code","lang":"python","label":"Python","text":"from transformers import AutoModelForSequenceClassification, AutoTokenizer\n\nmodel = AutoModelForSequenceClassification.from_pretrained(\n    \"answerdotai/ModernBERT-base\", num_labels=6)"},{"t":"p","runs":[{"text":"That "},{"text":"num_labels=6","code":true},{"text":" replaces the pretraining head with a fresh linear layer of 768 × 6 weights. Everything below it is the pretrained encoder, and the whole thing is trained: full fine-tuning of a 149M model fits in 6 GB of GPU memory comfortably, so none of the LoRA arithmetic in this course is needed."}]},{"t":"h2","id":"what-you-give-up-precisely","runs":[{"text":"What you give up, precisely"}]},{"t":"p","runs":[{"text":"This is a real trade, not a free win."}]},{"t":"ul","items":[[{"text":"You need labels.","b":true},{"text":" A prompt works on day one with zero examples. An encoder needs several hundred before it is competitive and a thousand or two before it is clearly better. If a label costs a minute of somebody’s attention, 2,000 labels is about 33 hours of work."}],[{"text":"No new classes without retraining.","b":true},{"text":" Add a seventh queue and you retrain, cheap, but it is a step. A prompt takes a new class in one line."}],[{"text":"No explanation.","b":true},{"text":" The classifier emits a label and a probability. If your users need a reason, you either add a second call to a generative model or you accept a score."}],[{"text":"Brittleness to drift.","b":true},{"text":" An encoder trained on last year’s tickets does not know this year’s product names. It will not tell you so; it will just be wrong on the new ones. A frontier model degrades more gracefully because it reads the words."}]]},{"t":"h2","id":"the-hybrid-that-most-teams-land-on","runs":[{"text":"The hybrid that most teams land on"}]},{"t":"p","runs":[{"text":"Use the big model to make the labels, and the small model to serve them. Run a frontier model over 3,000 historical tickets, have a person check a sample of 200 of those labels to get an agreement figure, then train the encoder on all 3,000. You have bought a labelled dataset for a few dollars and a serving model that costs nothing per call. Where the encoder’s confidence is low, say the top probability is under 0.7, fall back to the expensive model for that item only. On real routing workloads that fallback fires on 5 to 15% of traffic."}]},{"t":"p","runs":[{"text":"Check your provider’s terms before doing this. Several model licences restrict using outputs to train models, and what counts as a competing model is not always clear; that is the subject of a later lesson in this module."}]},{"t":"h2","id":"the-signal-that-you-are-in-this-situation","runs":[{"text":"The signal that you are in this situation"}]},{"t":"p","runs":[{"text":"You are classifying, not generating, if any of these are true: the output is drawn from a fixed list; you evaluate with accuracy, precision or recall rather than with a human reading the text; the prompt contains the phrase “answer with only one of”; or you are post-processing the model’s reply with a regex to extract the label. That regex is the clearest tell there is. It means you built a generative system and then spent effort hiding the generation."}]},{"t":"p","runs":[{"text":"The same logic covers extraction with a fixed schema (token classification, which is a per-token version of the same thing) and scoring (a regression head, one output instead of six). The family is called encoders, the library is "},{"text":"transformers","code":true},{"text":", and the whole path is free."}]}],"check":{"question":[{"text":"A team routes tickets with a hosted model and a regex that pulls the queue name out of the reply. They have 2,500 historical tickets already routed correctly by humans. Why does a fine-tuned 149M encoder tend to be more accurate here, not merely cheaper?"}],"options":[{"text":[{"text":"Smaller models overfit less, so the encoder generalises better to unseen tickets than a large model would."}],"correct":false,"tempting":[{"text":"Inverts the usual relationship; small models are more prone to underfitting, and the encoder’s advantage is not about overfitting at all."}]},{"text":[{"text":"The encoder has read the company’s tickets during pretraining, so it starts with knowledge the hosted model lacks."}],"correct":false,"tempting":[{"text":"Invents a pretraining corpus; the encoder’s pretraining is generic web and book text, exactly like the decoder’s."}]},{"text":[{"text":"Its head is trained directly on the six-way decision, while the decoder only decides indirectly."}],"correct":true,"tempting":null},{"text":[{"text":"Encoders are bidirectional and support longer inputs, so they see the whole ticket where a decoder truncates it."}],"correct":false,"tempting":[{"text":"Half-true about bidirectionality but wrong about length: encoders typically have shorter context limits than modern decoders."}]}],"order":[0,1,3,2],"why":null},"prev":{"slug":"try-the-retriever-first","title":"Try the retriever first","minutes":9},"next":{"slug":"distillation-as-the-real-case","title":"Distillation, the case that usually survives","minutes":10}}}