Most small LLM agents fail to produce enough successful trajectories on their own to learn robust tool-use behaviors. The core insight of this work is that a large, capable teacher agent encodes reusable procedural knowledge in its successful interactions, and that those patterns can be packaged as hierarchical memories and injected into smaller students without additional training.
Key Findings
- Three complementary memory types: Workflow memory (task-level strategies), Subtask memory (intermediate-granularity behavioral examples), and Function memory (per-function calling conventions and common pitfalls). This split yields targeted guidance at different abstraction levels so students get both strategy and concrete examples.
- Training-free distillation with proactive (Workflow/Subtask at task start) and reactive (Function retrieved on tool-call errors) injection. This design reduces compute and engineering overhead compared to trajectory imitation or full-policy distillation while remaining broadly applicable.
- Empirical gains: across AppWorld, BFCL V3 and ToolSandbox, average accuracy improvements reported are +27.2, +11.2 and +3.4 percentage points respectively when using GPT-5-mini as teacher with 4B–8B student models. Subtask memory provides the largest single contribution to improvements.
- Analysis shows teacher efficacy depends on both teacher capability and student compatibility; smaller (4B) students see the largest relative benefit, while mismatched teacher-student pairs reduce gains.
Who it's for and tradeoffs
Great fit if you need to boost tool-use or multi-step agent performance for resource-constrained LLMs without retraining—especially in settings with a strong teacher agent available and standardized tool APIs. Look elsewhere if you require end-to-end policy learning from scratch, continuous online adaptation of student weights, or if your use case cannot be expressed as reusable subtask/function patterns. The approach trades off absolute optimality for low-cost, modular transferability and depends on the quality and compatibility of teacher trajectories.