Technical
Low-Rank Adaptation(LoRA)
Definition
A parameter-efficient fine-tuning technique that trains small adapter layers instead of modifying all model weights.In-Depth Explanation
LoRA freezes the original model and trains small rank-decomposition matrices that modify behavior. This reduces training memory and compute by 10-100x while achieving comparable results to full fine-tuning. Multiple LoRA adapters can be swapped or combined for different tasks.
Real-World Example
A LoRA fine-tuned on legal documents adds legal expertise to a base model while using only a few hundred MB of additional weights.
5 views0 found helpful