AutoModel gives you numbers, not answers

Three Auto classes, and only one of them is safe to guess

from transformers import AutoConfig, AutoTokenizer, AutoModel

name = "distilbert-base-uncased"
cfg = AutoConfig.from_pretrained(name)
tok = AutoTokenizer.from_pretrained(name)
model = AutoModel.from_pretrained(name)

Each Auto* class reads config.json, finds architectures, and instantiates the matching implementation. That is all the magic is: a lookup table from a string to a Python class, so you do not have to import DistilBertTokenizerFast by name.

AutoConfig and AutoTokenizer do what you expect. AutoModel does something that catches almost everyone.

The body without the head

out = model(**tok("the food was excellent", return_tensors="pt"))
print(out.last_hidden_state.shape)   # torch.Size([1, 6, 768])

No labels. No scores. A tensor of 6 positions × 768 numbers: one vector per token. AutoModel loads the body of the network and nothing else. It is the right class when you want representations to feed something of your own, and the wrong class when you want a prediction.

For a prediction you name the head:

from transformers import AutoModelForSequenceClassification

m = AutoModelForSequenceClassification.from_pretrained(
    "distilbert-base-uncased-finetuned-sst-2-english"
)

The common heads, and what each is for:

• AutoModelForSequenceClassification: one label for the whole input. Sentiment, intent, spam.
• AutoModelForTokenClassification: one label per token. Named entities, part of speech.
• AutoModelForQuestionAnswering: start and end positions of a span.
• AutoModelForCausalLM: next-token prediction. Every chat and completion model.
• AutoModelForSeq2SeqLM: encoder-decoder. Translation, older summarisers, T5.
• AutoModelForMaskedLM: fill in a masked token. BERT-style pretraining.

The warning that means your classifier is random

Load a head that the checkpoint does not contain and transformers tells you, in a message people read as boilerplate:

Some weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-uncased and are newly initialized: [‘classifier.weight’, ‘classifier.bias’]. You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

Those two tensors were filled with random numbers. The model will load, run, and return confident-looking scores that are pure noise. Nothing downstream will fail. You will get roughly 50% accuracy on a two-class problem and spend a day blaming the data.

The distinction is exactly this: distilbert-base-uncased is a base checkpoint with no classifier. distilbert-base-uncased-finetuned-sst-2-english has one. The warning is the only thing that tells them apart at load time, and it is worth treating as an error rather than a log line.

id2label, and why your labels say LABEL_0

When output comes back as LABEL_0 and LABEL_1, the uploader did not fill in the mapping:

print(m.config.id2label)   # {0: 'NEGATIVE', 1: 'POSITIVE'}

If it prints {0: 'LABEL_0', 1: 'LABEL_1'}, the card is the only place the real meaning exists, and sometimes it is only in a discussion thread. Getting the polarity backwards because you assumed index 1 meant positive is a common and completely silent error. Check, do not assume.

The other Auto classes you will meet

• AutoProcessor: the general wrapper for models that take more than text: it bundles a tokenizer with an image processor or a feature extractor. Vision-language models and Whisper use it.
• AutoImageProcessor: resizing and normalisation for vision models. Normalisation constants differ per model and using the wrong ones degrades accuracy without any error.
• AutoModelForVision2Seq, AutoModelForSpeechSeq2Seq: heads for captioning and transcription.

The rule that prevents most of this

Load the tokenizer and the model from the same repository identifier, always. A tokenizer from bert-base-uncased with weights from a fine-tune of a different vocabulary produces integers the model was never trained on. No exception is raised, because integers are integers. The output is fluent nonsense, which is the hardest kind of bug to spot.

When you must mix them, loading an adapter, say, check that len(tokenizer) equals model.config.vocab_size and stop if it does not.

Do this now

Load distilbert-base-uncased with AutoModelForSequenceClassification and read the warning in full. Then run it on ten sentences and look at the predictions. That is what a randomly initialised head looks like from the outside: confident, varied, and meaningless.