Fine-tuning language models: full fine-tuning, LoRA and other methods
Fine-tuning adapts a pretrained language model to a specific task by additional training on task-specific data. Full fine-tuning updates all model parameters, which is computationally expensive. LoRA and other parameter-efficient methods update a small subset of parameters, reducing cost while maintaining performance. This article explains these approaches and how to choose among them.
What is fine-tuning?
Fine-tuning adapts a previously trained model to a particular task or use case, as discussed in the IBM article on fine-tuning. It is a form of transfer learning, using the model's existing knowledge as a starting point for new tasks, and reduces the need for expensive computing power and labeled data when tailoring large models to specific needs.
The Google Machine Learning Crash Course explains that fine-tuning trains a foundation LLM on task-specific examples, often requiring only a few hundred or a few thousand examples. Because foundation models are already good at recognizing patterns, a small amount of extra training can adapt them to practical tasks such as classification or regression.
OpenAI's model optimization guide lists fine-tuning as one of its optimization techniques, alongside other methods such as direct preference optimization and reinforcement fine-tuning, indicating that fine-tuning is a standard tool for adapting deployed models to specific use cases.
Full fine-tuning and its costs
Full fine-tuning retrains all model parameters. As described in the IBM article on fine-tuning, training starts with random weight initialization and then iterates through forward passes and backpropagation, adjusting every weight and bias to reduce loss. This process is computationally intensive for models with billions of parameters.
The Google course notes that conventional fine-tuning adjusts every weight and bias in each backpropagation pass, which makes it computationally expensive even when the training set is small. A fine-tuned model has the same parameter count as the foundation model, so a ten-billion-parameter model remains a ten-billion-parameter model after fine-tuning.
Because full fine-tuning changes all weights, it requires substantial GPU memory and storage for the updated model. The scale of modern language models makes this approach prohibitive in many practical settings, which motivates the development of parameter-efficient alternatives.
LoRA: low-rank adaptation
Low-rank adaptation (LoRA) is a parameter-efficient technique that freezes the original model weights and adds lightweight low-rank matrices to each layer, as described in the IBM article on LoRA. During training, only these added matrices are updated, drastically reducing the number of trainable parameters and the GPU memory required.
The LoRA paper on arXiv shows that LoRA can reduce trainable parameters by 10,000 times and GPU memory requirements by 3 times compared to full fine-tuning of GPT-3 175B, while performing on par or better in model quality. IBM further notes that full fine-tuning of GPT-3 trains 175 billion parameters, whereas LoRA reduces this to roughly 18 million trainable parameters and cuts GPU memory by about two thirds.
LoRA builds on the idea that large models exhibit low-dimensional structure, so significant adaptations can be represented with fewer parameters. Once training completes, the small matrices can be merged into the original weights or kept as separate adapters for inference.
Other parameter-efficient methods and libraries
Quantization LoRA (QLoRA) is a variant that combines low-rank adaptation with a high-precision computing technique and a low-precision storage method, as mentioned in the IBM article on LoRA. This keeps the model size small while maintaining performance and accuracy.
The Hugging Face PEFT documentation introduces a library dedicated to parameter-efficient fine-tuning. PEFT methods fine-tune only a small number of extra parameters, significantly decreasing computational and storage costs while yielding performance comparable to fully fine-tuned models. The library integrates with Transformers, Diffusers, and Accelerate, making it easier to train and store large models on consumer hardware.
The Hugging Face LLM Course teaches practical fine-tuning using the Trainer API and custom training loops, covering best practices such as dataset preparation and distributed training. It also mentions distillation, which is a separate technique that trains a smaller student model from a larger teacher model's outputs, as elaborated in the Google course.
Choosing a method
The choice between full fine-tuning and parameter-efficient methods depends on available resources and the target deployment. Full fine-tuning provides maximum flexibility but requires significant compute and storage. LoRA and similar methods offer efficiency with minimal quality loss, making them suitable for adapting large models on limited hardware.
Distillation produces a smaller model with fewer parameters, which speeds up inference and reduces environmental impact, though with some loss in prediction quality. The Hugging Face PEFT library simplifies implementing LoRA and other parameter-efficient methods, while the LLM course provides a structured path for learning fine-tuning workflows.
Developers should consider the size of the base model, the amount of task-specific data, and the inference environment. For consumer hardware and rapid iteration, LoRA or QLoRA are practical choices. For deployment on constrained devices, distillation after fine-tuning can yield a compact model that retains most of the original's capability.
| Method | Parameters updated | Compute cost | Storage requirement | Typical use |
|---|---|---|---|---|
| Full fine-tuning | All parameters | High | Same as base model | Maximum performance on a specific task |
| LoRA | Low-rank matrices added to layers | Low | Small adapter added to base model | Efficient adaptation on consumer hardware |
| QLoRA | Low-rank matrices with quantization | Low | Very small adapter with quantized base | Adaptation on limited memory |
| Distillation | Trains a new smaller model | Medium | Smaller model replaces original | Deployment with reduced resources |
Frequently asked questions
What is the difference between full fine-tuning and LoRA?
Full fine-tuning updates all model parameters, while LoRA adds small trainable matrices and leaves original weights frozen, reducing compute and storage costs.
Can LoRA achieve the same quality as full fine-tuning?
According to the original LoRA paper, LoRA performs on par or better than full fine-tuning on several benchmark tasks despite having fewer trainable parameters.
What is the PEFT library?
PEFT is a Hugging Face library for parameter-efficient fine-tuning methods, including LoRA and others, integrated with Transformers and Diffusers to enable efficient training on consumer hardware.
How does distillation differ from fine-tuning?
Distillation trains a smaller student model from a larger teacher model's outputs, producing a smaller model for faster inference, while fine-tuning adapts the original model for a specific task.
What is QLoRA?
QLoRA combines low-rank adaptation with quantization to further reduce memory usage while maintaining performance, making it suitable for very limited hardware.