Illustrated portrait of Rafael Duarte, an AI tutor

AI tutor

Rafael Duarte AI

Fine-tuning: what it fixes, what it will not, and what it costs you.

Teaches in Português · English · Español

Ask RafaelHow Rafael explains

How Rafael explains

I argue for the cheaper fix first. A prompt changes in ten seconds and a fine-tune in a day, so in Fine-Tuning, and When Not To we sort thirty real failures before anything is trained. When training is the right answer, I walk through the memory arithmetic and the learning rate with you until the numbers make sense. I like a written decision record: what should improve, and what must not get worse. In Knowing If It Works I hold every tuned model to the same frozen baseline.

Suits you if a colleague has proposed fine-tuning and you want to know whether a prompt or a retriever would do first.

A sample exchange

  1. Can I fine-tune our product manual into the model so it knows our policies?

  2. Rafael Duarte AI

    Fine-tuning mostly teaches the shape of an answer: tone, length, structure, format. A fact that appears a handful of times in your data is a tiny signal against everything the base model already saw. So the tuned model tends to learn how a confident answer sounds and then invent the specifics. Put the manual in the context window through retrieval instead, where it can be cited and updated.1

    1. Fine-Tuning, and When Not To · Lesson 1.2 · What Fine-Tuning Changes, and What It Cannot Add

A sample, grounded in that lesson. Your own question gets its own answer.

Courses Rafael answers for

Ask RafaelSee all AI tutors