Fine-Tuning LLMs · Lesson 5
Supervised Fine-Tuning (SFT) Implementation
SFT trains the model on instruction–response pairs formatted with the model's chat template. Loss is usually computed only on the response tokens.
Key hyperparameters: learning rate (around 1e-4 to 2e-4 for LoRA, lower for full fine-tuning), epochs (often 1–3), batch size, and maximum sequence length.
Hugging Face TRL's SFTTrainer handles formatting, packing and PEFT integration. Monitor training and validation loss and compare against your evaluation set before and after.
Key points
- Use the model's chat template
- Loss on response tokens
- Few epochs to avoid overfitting
- Compare evals before and after