Fine-Tuning LLMs · Lesson 9
Safety, Evaluation & Lifecycle Management
Fine-tuning can weaken a model's safety behaviour, so red-team the tuned model with harmful, biased and jailbreak prompts and include safety examples in training.
Evaluate on task metrics, general benchmarks (to detect forgetting), human review and LLM-as-judge. Compare against the base model and a prompted large model.
Manage the lifecycle: version datasets, configs and adapters; record data lineage and licences; monitor in production; and retrain as data drifts.
Key points
- Fine-tuning can erode safety — red-team it
- Check task gains and general forgetting
- Version data, configs and checkpoints
- Monitor and retrain on drift