What Is LLM Fine-Tuning and How Does It Work

How Pretraining Creates the Foundation

Every large language model starts with pretraining, a process that trains the model on an enormous corpus of text, often spanning trillions of tokens drawn from web pages, books, code repositories, and academic papers. During pretraining, the model learns the statistical patterns of language at multiple levels: how words combine into phrases, how sentences form coherent paragraphs, how arguments are structured, and how reasoning works across domains. The result is a model with broad, general capability, one that can follow instructions, generate plausible text, answer questions about a wide range of topics, and perform basic reasoning.

But pretraining optimizes for a general objective, usually predicting the next token in a sequence. The model learns to generate text that is statistically likely given what came before, which is why its outputs are coherent and grammatically correct but often generic. It has not been explicitly taught to follow your company's style guide, to format answers in your preferred JSON schema, or to apply the clinical reasoning patterns used in radiology reports. Those specific behaviors are not well-represented in the pretraining data, or they are drowned out by the vastly larger volume of general text. Pretraining gives the model raw capability, fine-tuning directs that capability toward what you need.

What Happens During Fine-Tuning

Fine-tuning runs the same gradient descent optimization process used during pretraining, but on your curated dataset and with smaller updates. You feed the model examples of the inputs it will see in production alongside the outputs you want it to produce. The training process computes the difference between what the model currently outputs and what the training example says it should output, then adjusts the model's internal weights to reduce that difference. After thousands of these adjustments across your dataset, the model's default behavior shifts toward the patterns in your fine-tuning data.

The key insight is that fine-tuning does not erase what the model learned during pretraining. It modifies a relatively small set of weights in a way that biases the model toward your target behavior while preserving the general capability underneath. This is why a model fine-tuned on medical question-answering can still write Python code and answer history questions: the pretraining knowledge is still there, the fine-tuning just moved the model's default tendencies for medical inputs toward the patterns you provided.

The standard training pipeline involves splitting your data into training and validation sets, typically 80/20 or 90/10. The model trains on the training set for a specified number of epochs, with each epoch being one complete pass through the data. After each epoch, performance is measured on the validation set to detect overfitting, which occurs when the model memorizes the training examples rather than learning generalizable patterns. Most fine-tuning runs converge within 2 to 4 epochs, and training beyond that point typically degrades quality on new inputs.

Full Fine-Tuning vs Parameter-Efficient Methods

Full fine-tuning updates every weight in the model, which gives maximum flexibility but requires significant compute resources and carries the highest risk of catastrophic forgetting, where the model loses its general capabilities. For a 7B parameter model, full fine-tuning requires at least 28GB of GPU VRAM just to hold the model in FP16 precision, plus additional memory for the optimizer states and gradients, typically totaling 60 to 80GB. For a 70B model, this balloons to 560GB or more, requiring multiple high-end GPUs.

Parameter-efficient fine-tuning (PEFT) methods address this by updating only a small fraction of the model's weights. LoRA is the most popular: it freezes the original weights entirely and inserts small, trainable matrices into each transformer layer. These adapter matrices typically contain less than 1% of the total parameter count, so training them is fast, requires minimal GPU memory, and preserves the base model's general capability because its weights are never modified. At inference time, the adapter weights are merged with the base model, adding zero latency compared to a fully fine-tuned model.

QLoRA extends LoRA by quantizing the frozen base model to 4-bit precision, which cuts the memory footprint of the base model by roughly 75%. This enables fine-tuning a 70B model on a single 24GB consumer GPU, which would otherwise be impossible without a multi-GPU server. QLoRA trains the LoRA adapters in full precision while the base model is stored in 4-bit, achieving results that are close to full-precision LoRA at a fraction of the hardware cost.

Types of Fine-Tuning by Objective

The training objective, what the model is optimizing for during fine-tuning, determines what kind of behavior change you get. The three main objectives are supervised fine-tuning (SFT), instruction tuning, and preference optimization.

Supervised fine-tuning trains the model on input-output pairs where the output is the correct completion for a given input. If you are training a model to classify support tickets, each example is a ticket text paired with its correct category. If you are training a code generation model, each example is a function description paired with the correct implementation. SFT is the most straightforward form of fine-tuning and works well for tasks with clear, deterministic correct answers.

Instruction tuning is a variant of SFT where the training examples are formatted as instructions paired with responses that follow those instructions well. This teaches the model to be helpful and to follow directions, which is the key difference between a raw pretrained model (which just predicts the next token) and an assistant-style model (which tries to answer questions and follow instructions). Most of the models people interact with, like ChatGPT, Claude, and Llama-Chat, have been instruction-tuned as a post-pretraining step. The comparison of SFT and instruction tuning covers when to use each.

Preference optimization, most commonly implemented as DPO (Direct Preference Optimization), trains the model using pairs of responses where one is preferred over the other. Rather than showing the model a single correct output, you show it two outputs and indicate which one is better. This teaches the model to produce outputs that align with human preferences on subjective qualities like helpfulness, safety, and clarity. DPO is typically applied after an initial round of SFT and is what gives models their polished, helpful personality.

What Fine-Tuning Cannot Do

Fine-tuning changes the model's behavior but does not give it access to information at inference time. A model fine-tuned on your product documentation will memorize patterns from that documentation, but it cannot look up a specific product specification when asked. If the specification changes after training, the model still produces the old answer. For dynamic factual knowledge, retrieval-augmented generation or a memory layer is the correct solution, not fine-tuning.

Fine-tuning also cannot overcome fundamental model limitations. A 1B parameter model fine-tuned on graduate-level mathematics will not match a 70B model on complex proofs, because the smaller model lacks the capacity to represent the reasoning patterns required. Fine-tuning makes a model better within the bounds of its architecture, it does not turn a small model into a large one. For tasks that require sophisticated multi-step reasoning, starting with a more capable base model matters more than any amount of fine-tuning data.

Finally, fine-tuning produces a static snapshot. The model reflects the state of the training data at the time of training and does not update itself with new information. If your domain changes frequently, like stock prices, regulatory requirements, or product catalogs, you need either regular retraining cycles (expensive and slow) or a retrieval system that provides current information at runtime. Most production systems combine fine-tuning for behavior with retrieval for knowledge, getting the best of both approaches.