Many recurring text-processing tasks are easy to describe but costly to solve by calling a large remote model for every input. The core insight of this work is to "compile" a natural-language specification into a small, reusable neural artifact: at compile time, high-capacity teacher models synthesize task-specific I/O examples, and those examples are used to finetune a compact LoRA-style adapter that specializes a frozen local interpreter. The result is a callable local function that runs without the teachers and can be stored, versioned, and composed like ordinary software.
Key Findings
- High-accuracy localization: On a hard subset of FuzzyBench where the prior fast PAW compiler produced no exact matches, Compile by Training reaches 83.6% semantic accuracy, improving correctness for many fuzzy text functions.
- Compile-time tradeoff: The approach raises accuracy by paying a higher compile-time cost—about a minute of finetuning per compile versus seconds for the fast compiler—while keeping per-call latency and dependency minimal because inference runs locally.
- Modular separation of roles: The pipeline separates synthesis (teacher LLMs) from execution (small frozen interpreter + adapter), enabling small deployable artifacts (PAW programs) that can be versioned and composed in production workflows.
- Practical demos: The authors demonstrate the approach in several deployed scenarios (multi-site website helper, language-directed 3D avatar, and a bidirectional English–Claudish translator), illustrating end-to-end usability beyond benchmarks.
Who it's for & tradeoffs
Great fit if you need repeatable, offline execution of fuzzy text functions with better-than-prompting accuracy and you can accept a non-negligible one-time compile cost. It is attractive for teams wanting to eliminate per-request remote LLM calls for latency, cost, or privacy reasons while keeping a compact local runtime.
Look elsewhere if you require strict formal guarantees or fully deterministic rule-based behavior, need zero compile latency, or cannot tolerate errors introduced by synthesized supervision: because the adapter is trained on teacher-generated examples, teacher mistakes can propagate unless validated. The method improves practical correctness but requires validation, testing, and governance when used in safety-sensitive pipelines.
Where it fits
This method sits between two extremes: lightweight prompt-based local inference (very low compile cost but limited accuracy) and always-calling large remote LLMs (high per-call cost and dependency). Compile by Training trades extra one-time compile cost to produce small, high-accuracy, versionable neural functions that run locally.