LLM Fine-Tuning
On This Page
What Fine-Tuning Actually Does
A pretrained language model like Llama 3, GPT-4, or Claude has been trained on a massive corpus of text, sometimes trillions of tokens, to learn general language patterns. It can follow instructions, answer questions, write code, and reason about problems because that pretraining gave it broad capability. But broad capability is not the same as specific competence. The pretrained model knows a little about everything and a lot about nothing in particular, which is why its answers to domain-specific questions tend to be surface-level and its outputs follow generic patterns rather than the exact format or tone your application needs.
Fine-tuning takes that broadly capable model and trains it further on a much smaller, carefully curated dataset that represents the task you care about. The training updates the model's internal weights, the numerical parameters that determine how it processes and generates text, to make it more likely to produce outputs that match the patterns in your fine-tuning data. If you train on thousands of examples of customer support conversations that follow your company's policies, the model learns to answer in that style by default. If you train on code reviews that follow your linting rules, it learns to generate code that passes your linter. The pretraining provides the foundation, and the fine-tuning shifts the model's behavior toward your specific use case.
What makes this powerful is that you do not need nearly as much data as the original pretraining required. A few thousand high-quality examples can meaningfully shift a model's behavior for a focused task, while pretraining consumed petabytes. You also do not need the same compute budget: fine-tuning a 7-billion parameter model with LoRA can cost under $5 on commodity GPU hardware in 2026, while pretraining the same model from scratch would cost millions. Fine-tuning is the practical path to customization because it starts from a capable base and makes targeted adjustments rather than rebuilding from nothing.
The result is a model that still has all the general capability it learned during pretraining but is biased toward the patterns you taught it during fine-tuning. It will still be able to answer general knowledge questions, but it will default to the style, terminology, and reasoning patterns present in your fine-tuning data. This is both the strength and the risk: a well-curated dataset produces a model that is measurably better at your task, while a poorly curated one can degrade general capability or embed biases you did not intend.
Why Fine-Tuning Matters
Fine-tuning solves problems that prompting alone cannot. The most important is behavioral consistency. A prompted model can be told to respond in a specific format or follow certain rules, but it drifts. Long conversations erode the influence of system instructions. Complex tasks with multiple steps see the model gradually forget constraints it was given at the top of the context. Rare edge cases produce outputs that ignore the prompt entirely because the model's pretrained behavior overpowers the instruction. Fine-tuning bakes the desired behavior into the weights themselves, so it becomes the model's default rather than a temporary override from a prompt. A model fine-tuned on thousands of examples of your desired output format will produce that format reliably without needing it spelled out in every prompt.
The second reason fine-tuning matters is latency and cost at scale. A complex prompt with detailed instructions, examples, and formatting rules consumes hundreds or thousands of tokens on every single API call. At scale, that is real money and real latency. A fine-tuned model does not need those instructions because the behavior is in its weights, so the prompt can be short: just the user's question and any relevant context. For a high-volume production system handling millions of requests per month, the token savings from shorter prompts can more than offset the one-time cost of fine-tuning.
The third reason is teaching the model things it genuinely does not know. Prompting and retrieval work when the model has the underlying capability and just needs the right information in its context window. But some tasks require pattern recognition that the model was never trained on: reading a proprietary log format, extracting structured data from a niche document type, or applying domain-specific reasoning that is not well-represented in the pretraining corpus. For these tasks, no amount of prompting helps because the model lacks the internal representations to handle the input. Fine-tuning on examples of that pattern builds the representations.
The fourth and most practically significant reason is that fine-tuning can make a smaller, cheaper model perform as well as a larger, more expensive one on a specific task. A 7B or 8B parameter model fine-tuned on your task can outperform GPT-4 at that task while costing a fraction per inference. This is the economics that makes fine-tuning attractive to production teams: you trade a one-time training cost for ongoing savings on every API call, with better quality on the specific task your application cares about.
Fine-Tuning Methods
The methods of fine-tuning fall into two broad categories: full fine-tuning, which updates every weight in the model, and parameter-efficient fine-tuning (PEFT), which updates only a small subset while leaving the rest frozen. Full fine-tuning is the most powerful but also the most expensive: training all weights of a 70B model requires multiple high-end GPUs with hundreds of gigabytes of VRAM and can cost thousands of dollars per run. It also carries the highest risk of catastrophic forgetting, where the model loses general capability as the fine-tuning data overwrites what it learned during pretraining.
Parameter-efficient methods were invented precisely to address these costs and risks. The most widely used is LoRA (Low-Rank Adaptation), which freezes the original model weights and inserts small trainable matrices into each attention layer. Instead of updating all 70 billion parameters, LoRA might train 10 to 50 million adapter parameters, less than 1% of the total. Because it leaves the base model untouched, catastrophic forgetting is far less likely, and the compute and memory requirements drop by an order of magnitude. A LoRA adapter for a 7B model fits in a few dozen megabytes, and training it requires a single consumer GPU. The page on how LoRA works covers the mechanics in detail.
QLoRA goes one step further by quantizing the frozen base model to 4-bit precision, which cuts the GPU memory needed to hold it by roughly 75%, while still training the LoRA adapters in full precision. This is what makes it possible to fine-tune a 70B model on a single 24GB GPU, something that would otherwise require a multi-GPU server. QLoRA has become the default choice for teams without dedicated ML infrastructure because it removes the hardware barrier entirely. The QLoRA guide walks through the full workflow on consumer hardware.
Beyond LoRA and QLoRA, other PEFT methods exist but see less production use. Prefix tuning prepends learnable tokens to the input and trains only those tokens. Adapters insert small trainable layers between the frozen layers of the model. (IA)3 rescales the model's activations with learned vectors. Each has trade-offs in performance, training speed, and ease of integration, but LoRA has won the ecosystem battle: every major training framework supports it, and most production fine-tuning uses LoRA or QLoRA. The comparison between supervised fine-tuning and instruction tuning covers the training objective side, while LoRA and QLoRA cover the weight-update strategy side.
The Data Problem
The quality of your fine-tuned model is determined almost entirely by the quality of your training data, and getting data right is where most fine-tuning projects either succeed or stall. A model trained on noisy, inconsistent, or poorly formatted examples will learn those qualities faithfully, because it has no way to know which patterns you intended and which were accidental. This is why data preparation typically consumes more project time than the training itself.
For supervised fine-tuning, the data takes the form of input-output pairs: given this input (a question, a document, a code snippet), the correct output is this. The quality bar is that every example should be one you would be happy to see the model produce in production. If a human expert would look at the output and say it is wrong, the model should not be trained on it. Depending on the task, you might need as few as 200 high-quality examples for a narrow task with clear patterns, or as many as 10,000 for a complex task with diverse inputs. The guide on preparing training data covers the full pipeline from collection through cleaning, formatting, and validation.
For preference-based training methods like DPO (Direct Preference Optimization), the data takes a different form: pairs of outputs where one is preferred over the other, given the same input. This teaches the model not just what a correct output looks like but which of two plausible outputs is better, which is how you teach nuanced qualities like helpfulness, safety, and tone. Preference data is harder to collect because it requires side-by-side comparisons, but it produces models that are better at the subjective qualities that make an AI system pleasant to use. The page on DPO alignment covers how this works.
The most common data failure is not volume but consistency. A dataset with conflicting examples, where similar inputs produce contradictory outputs, teaches the model to be uncertain and inconsistent. A dataset with a narrow distribution, where all examples follow one pattern, produces a model that fails on any variation. And a dataset with quality variance, where some examples are excellent and others are sloppy, produces a model that is unreliable because it learned both standards. The practical discipline is to audit every example, enforce formatting with automated checks, and split a held-out validation set before training starts so you can measure whether the model actually learned what you intended.
When to Fine-Tune
Fine-tuning is not always the right answer, and using it when a simpler approach would suffice wastes money and time. The decision depends on what exactly you are trying to accomplish and what alternatives you have tried.
Fine-tuning is the right choice when you need a model to produce a specific behavior consistently and you have verified that prompting alone is not reliable enough. If a detailed system prompt achieves 90% of what you need but fails on edge cases or drifts during long conversations, fine-tuning can push reliability to 99%. It is also the right choice when prompt length is a cost or latency problem: if you need 2,000 tokens of instructions and examples in every prompt to get the right behavior, fine-tuning those patterns into the model and cutting the prompt to 200 tokens saves money on every call.
Fine-tuning is also the right choice when the model genuinely does not know how to do something. If your task involves a proprietary format, a domain-specific reasoning pattern, or behavior that is not well-represented in the pretraining data, no prompt will teach the model something it has never seen. You need to show it examples. The page on when to fine-tune works through the decision framework with concrete criteria.
Fine-tuning is the wrong choice when the problem is information rather than behavior. If the model gives wrong answers because it does not have access to the right data, the solution is retrieval (RAG) or a memory layer that puts the right facts in context, not fine-tuning. Training the model on your documentation does not make it retrieve the right section at the right time, it makes it memorize the documentation and potentially hallucinate from it when the memorized content is slightly off. For factual grounding, retrieval is more flexible, cheaper to update, and auditable, while fine-tuning is static and opaque.
Fine-Tuning vs Memory and Retrieval
The choice between fine-tuning, retrieval, and memory is the most important architectural decision in an AI application, and it is not a one-or-the-other choice. Each solves a different problem, and most production systems use more than one.
Fine-tuning changes how the model behaves: its style, its format, its reasoning patterns, the skills it can perform. Once fine-tuned, these changes are baked into the weights and apply to every request without needing anything in the prompt. But fine-tuned knowledge is static, stuck at the state of the training data, and updating it requires running the training again.
Retrieval (RAG) provides dynamic factual knowledge by pulling relevant documents into the context window at request time. It solves the knowledge freshness problem because you update the knowledge base rather than retraining the model, and it provides citations because the source documents are visible. But retrieval does not change how the model reasons about what it retrieves, and it adds latency and cost to every call through the retrieval step.
Memory, as implemented in systems like Adaptive Recall, stores facts the system learns about individual users, tasks, and contexts across sessions and retrieves the relevant ones into context when they are needed. Memory solves the personalization and continuity problem: the system remembers that this particular user prefers Python over JavaScript, that they are working on a healthcare project, and that they corrected the system's answer last Thursday. Unlike fine-tuning, memory updates instantly and applies per-user rather than globally. Unlike RAG, memory captures information from interactions rather than from a static document corpus.
The decision framework is straightforward. If you need to change the model's behavior, style, or capabilities, fine-tune. If you need to provide factual knowledge that changes over time, use retrieval. If you need the system to learn and remember from individual interactions, use memory. A well-built production system often fine-tunes a base model for its domain, adds retrieval for its knowledge base, and adds memory for personalization and continuity, all in combination. The deeper comparison is covered in the existing guide on memory versus fine-tuning.
Cost and Compute
The cost of fine-tuning has dropped dramatically since 2023, and in 2026 it is accessible to individual developers, not just well-funded ML teams. The exact cost depends on the model size, the training method, the dataset size, and whether you use your own GPU or a cloud provider.
For a 7B to 8B parameter model using QLoRA on a single NVIDIA RTX 4090 (about $1,600 retail), training on 5,000 examples typically costs under $5 in electricity and takes 1 to 3 hours. Cloud alternatives like Lambda Labs, RunPod, and Vast.ai rent equivalent GPUs for $0.40 to $1.50 per hour, putting the cloud cost at $1 to $5 per run. This is the sweet spot for most projects: 7B to 8B fine-tuned models are fast enough for real-time inference, cheap to serve, and produce genuinely good results on focused tasks.
For larger models in the 70B range, QLoRA on a single 24GB GPU is technically possible but slow, and you typically want 48GB or more of VRAM (an A6000 or H100 rental at $2 to $6 per hour) for a practical workflow. A 70B QLoRA training run on 5,000 examples takes 6 to 12 hours and costs $12 to $72 in compute. Full fine-tuning of a 70B model requires multiple H100 GPUs and costs hundreds to low thousands per run, which is why PEFT methods dominate in practice.
API-based fine-tuning through providers like OpenAI, Google, and Together AI abstracts away the infrastructure entirely. OpenAI charges per training token, with GPT-4o mini fine-tuning at roughly $3 per million training tokens. A dataset of 5,000 examples averaging 500 tokens each (2.5 million training tokens) costs about $7.50 for a single training epoch, making it comparable to self-hosted costs but with no hardware management. The trade-off is that you do not own or control the fine-tuned model, cannot run it on your own infrastructure, and pay ongoing inference costs per token. The full cost breakdown is in how much fine-tuning costs in 2026.
Evaluating Results
A fine-tuned model that has not been rigorously evaluated is a liability, not an asset. The most common mistake is checking that it works on a few examples from the training set, where it will always look good, and shipping it without testing on inputs it has never seen. Proper evaluation requires a held-out test set that was set aside before training and never used during any part of the training process.
The evaluation should measure both task-specific quality and general capability retention. For task-specific quality, you define metrics that match your use case: accuracy for classification, BLEU or ROUGE for generation tasks, pass rate for code generation, or human preference ratings for conversational quality. These tell you whether the fine-tuning actually improved the model at what you trained it to do. For general capability, you run the fine-tuned model against a standard benchmark (MMLU, HumanEval, or a similar general-purpose eval) and compare it to the base model. If general scores dropped significantly, the fine-tuning caused catastrophic forgetting and you need to adjust your approach, either by using a smaller learning rate, fewer training steps, or a PEFT method that modifies fewer weights.
Beyond automated metrics, human evaluation is essential for subjective quality. Have domain experts rate the outputs of the fine-tuned model against the base model on real-world inputs, without knowing which is which. If the experts cannot consistently distinguish the fine-tuned model as better, the fine-tuning did not add enough value to justify the complexity it introduces to your system. The page on evaluating fine-tuned models walks through the full evaluation pipeline.
Common Mistakes
The first and most expensive mistake is fine-tuning when the problem is actually information, not behavior. If your model gives wrong answers about your product, the solution is almost always retrieval, not training. Fine-tuning on your documentation teaches the model to parrot that documentation from memory, which means it cannot distinguish between current information and stale information, it cannot cite its sources, and it cannot be updated without retraining. Retrieval solves all three of these problems more cheaply and more reliably.
The second mistake is training on too little data or data of uneven quality. A model trained on 50 examples will overfit, memorizing those examples verbatim rather than learning the underlying pattern. A model trained on a mix of excellent and mediocre examples will produce mediocre output, because it learned the average quality rather than the best quality. The minimum practical dataset for most tasks is 500 to 1,000 examples, and every example should meet the quality bar you expect from the deployed model.
The third mistake is training for too many epochs. An epoch is one full pass through the training data. Most fine-tuning jobs perform best at 2 to 4 epochs, and training for more than that risks overfitting to the training data and degrading on new inputs. The loss curve will often show this clearly: training loss continues to decrease while validation loss starts increasing, which is the textbook signal to stop. Many teams train for 10 or 20 epochs because they assume more is better, when in practice 3 epochs is usually the sweet spot.
The fourth mistake is not setting up evaluation before training. If you do not have a test set and metrics defined before you start, you have no way to know whether the fine-tuning helped, hurt, or made no difference. Evaluation is not optional and it is not something you do after deployment when users start complaining. It is a prerequisite for training, and the fact that your training loss went down is not evidence that the model is better, because low training loss correlates with memorization as much as with learning.
Fine-tuning changes a model's behavior, not its knowledge. Use it for style, format, and capability changes. Use retrieval for factual knowledge and memory for personalization. The most common mistakes are fine-tuning when retrieval would work, training on low-quality data, training for too many epochs, and skipping evaluation.