Recursive Language Models (RLMs) change where contextual reasoning happens: instead of forcing a single flat prompt into a model, RLMs let the model treat the context as a manipulable environment and spawn recursive sub-calls to examine or transform parts of that context. This makes it possible to work with effectively unbounded inputs (documents, codebases, chat histories) while keeping each LM call short and focused — at the cost of added orchestration and sandboxing complexity.
What Sets It Apart
- REPL-based inference model: the library exposes a REPL-like environment where the LM can read and mutate a context variable and launch sub-LM calls; this moves control flow into the inference layer rather than the client.
- Multi-environment sandboxing: supports non-isolated local execution and several isolated sandboxes (Docker, Modal, Prime, Daytona, e2b) so you can trade off convenience and security depending on the task.
- Backend-agnostic integration: works with API providers (OpenAI, Anthropic, router platforms) and local runtimes (vLLM), so experiments can run across hosted and on-prem setups.
- Run metadata & visualizer: optional trajectory logging captures full run configs and sub-call trees, and a simple visualizer helps inspect generated code, sub-calls, and the overall recursion graph for debugging and analysis.
Who it's for — trade-offs and caveats
Great fit if you want to prototype new inference paradigms that break the single-prompt assumption (e.g., long-document QA, multi-step code synthesis, programmatic reasoning) and need tooling to record and inspect recursion traces. It’s also useful for researchers reproducing the RLM paper experiments or integrating RLM-style control flow into research stacks.
Look elsewhere if you need a drop-in production LLM client for untrusted user input without strong isolation: the default non-isolated REPL runs in-process and can expose host resources; for production with untrusted inputs you must use the isolated sandboxes. This repo focuses on inference and orchestration rather than model training or large-scale serving optimizations; expect engineering work to harden, scale, and monitor RLM runs in production.
Where it fits
Positioned between single-call LLM clients and heavier orchestration frameworks: RLMs provide a lightweight programmatic layer for recursive decomposition and are complementary to RAG, tool-augmented agents, and workflow platforms — especially when the task benefits from explicit recursive inspection and sub-calls rather than a single large context.
