Fine-Tuning LLMs · Lesson 3
Efficient Fine-Tuning (PEFT)
Full fine-tuning updates every weight and needs a lot of GPU memory. Parameter-Efficient Fine-Tuning (PEFT) trains only a small number of new parameters.
LoRA freezes the base model and learns low-rank matrices added to selected layers. QLoRA does the same on a 4-bit quantised base model, making fine-tuning of 7–13B models possible on a single consumer or cloud GPU.
Adapters are small files that can be swapped per task on one shared base model. Libraries: Hugging Face PEFT, TRL, Unsloth, Axolotl.
Key points
- PEFT trains a small fraction of parameters
- LoRA adds low-rank update matrices
- QLoRA = LoRA on a 4-bit base
- Swappable adapters per task
from peft import LoraConfig, get_peft_model
config = LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05,
target_modules=["q_proj", "v_proj"], task_type="CAUSAL_LM")
model = get_peft_model(base_model, config)
model.print_trainable_parameters()