Real-time object detectors use heterogeneous operators and detection-specific components that break assumptions behind PEFT recipes borrowed from language models. YOLO-PEFT treats adapter placement as an auditable constraint-planning task: given a detector graph, a PEFT request and a resource budget, it assigns operator and semantic roles, evaluates operator-validity, detector-semantic, graph-interface and deployment predicates, records reason codes for excluded modules, and either emits a budgeted training plan or returns Refuse before training.
Key Findings
- Planner-selected RS-LoRA on VOC07+12 (trainval→test) reaches mAP50-95 of 0.7138 on YOLO11s and 0.7307 on YOLO12s, compared with 0.6428 and 0.6662 for Full-SFT — concrete gains in detection accuracy under the evaluated protocol.
- On a evaluated RT-DETR-L setup, multiple LoRA-family configurations triggered a predefined catastrophic threshold, enabling a calibrated Refuse-to-Full-SFT decision to avoid blind failure modes.
- A controlled YOLO11 audit shows LoRA reduces peak training memory by 43.9% while increasing wall-clock training time by ~1.72×, illustrating a memory/time trade-off common to PEFT in detectors.
- The planner records explicit reason codes for excluded modules and enforces operator/semantic predicates, replacing manual trial-and-error with inspectable, budgeted placement plans and preserving verified train-save-merge-export paths.
Who it's for and trade-offs
Great fit if you fine-tune real-time detectors under tight resource or deployment constraints and need auditable placement decisions rather than heuristic trial-and-error. It is valuable for engineering teams that require an explicit acceptance/refusal signal before spending compute on training. Look elsewhere if your detector architecture is outside the planner's validated coverage (the paper flags refusal on unseen architectures as an open validation problem) or if minimizing wall-clock training time is the top priority, since PEFT choices here increased training duration in exchange for memory savings.