Most agentic search setups struggle to transfer reasoning competence from strong proprietary teachers to smaller open models: hidden logits, tokenizer mismatches, and stylistic artifacts either block direct logit-matching or produce superficial imitation. The core insight of this paper is that a structured intermediate—an extractive, style-normalized JSON protocol—can carry the teacher’s reasoning plan and grounding facts without transferring verbose stylistic signals, enabling dense distillation signals alongside sparse RL rewards.
Key Findings
- Multi-Agent Protocol Distillation (MAPD) uses an offline multi-agent system to decompose queries, retrieve evidence, repair failed searches, and emit a JSON protocol containing task type, plan, and grounding facts — this protocol serves as a privileged dense target during student training, so the student learns procedure and grounding rather than surface style.
- Joint training (protocol-guided distillation + RL) consistently outperforms standard distillation and pure RL baselines: reported average success rates reach 39.4% on Qwen3-1.7B and 44.4% on Qwen3-4B, showing robustness across different proprietary teachers and student sizes.
- The protocol abstraction mitigates verbosity and style drift that arise from direct trajectory imitation, meaning students generalize reasoning behavior rather than mimic teacher phrasing; this makes the method more portable across teacher models with different styles.
Who it's for and trade-offs
Great fit if you need to transfer complex, retrieval-augmented multi-step reasoning from closed, high-capacity teacher models into smaller or open-source student policies while avoiding stylistic overfitting. It is especially useful when teacher logits are inaccessible or tokenizers differ. Look elsewhere if you require fully online distillation from a live teacher (MAPD relies on an offline MAS to produce protocols) or when computational budget forbids running the retrieval/repair pipeline needed to generate dense protocols.
Where it fits
MAPD sits between black-box teacher imitation and reward-only RL: it provides dense intermediate supervision that is model-agnostic (works with multiple proprietary teachers) and complements RL objectives to improve long-horizon, retrieval-heavy QA and agentic search tasks.
Method sketch
The pipeline converts multi-agent exploration traces into JSON protocols; during training a privileged branch consumes protocols to produce token distributions that supply a dense distillation loss, while the full student policy is optimized jointly with an outcome-based RL objective. Evaluations span seven QA benchmarks and cross-teacher settings to validate generalization and robustness.