Overview
LLaMA-Factory lowers the barrier to task-specific tuning of open-source LLMs. It bundles data prep, training, evaluation and deployment behind a simple command or WebUI.
Key Capabilities
- Supports SFT, reward-model, PPO, DPO, ORPO, KTO, etc.
- 16-bit, LoRA, QLoRA (2-8 bit) and GaLore/DoRA optimizers
- DeepSpeed, FSDP and NativeDDP distributed back-ends
- FlashAttention-2 & Unsloth acceleration
- Integrated dashboards: LlamaBoard, W&B, MLflow, SwanLab
