Compiles natural-language function specifications into compact, locally-executable neural programs (PAW) that run on a small frozen interpreter; a 4B compiler emits LoRA adapters for a 0.6B runtime to provide offline, low-memory fuzzy text functions.
Most day-to-day text processing tasks—intent routing, JSON repair, log triage, fuzzy search—are "fuzzy": they resist brittle rule systems but don't need a 30B model call per input. The paper's core insight is to turn a foundation model from a per-input problem solver into a tool builder: compile once into a small neural artifact and run that artifact locally and cheaply.
Treats hybrid layer selection as a budget-constrained subset optimization and introduces FlashMorph: a pipeline that equips each transformer layer with a linear-attention branch, jointly optimizes layerwise gates on synthetic long-context retrieval data, then discretizes, distills, and finetunes—achieving strong long-context recall using only 20M selection tokens.
Introduces a bounded-memory, typed-retrieval contract for long-horizon LLM agents and evaluates it in Slay the Spire 2 — assembling per-decision prompts from five typed slots rather than appending raw transcripts. Key outputs include ablationable memory layers, 298 labeled trajectories, and reproducible analysis scripts.
Predicts per-request MoE expert footprints from prefill activations and routes decode requests to workers that maximize expert-locality, lowering decode latency by combining offline K-means partitioning with online locality-band routing and a KV-block–coindexed signature cache.
