Most synthetic pipelines generate task text before realizing environments, which often loses procedural constraints and produces inconsistent artifacts. FACET flips that workflow: it reconstructs coherent scenarios from reusable agent skills, builds and repairs the actual execution environment first, then generates instruction, reference solution, and verifier against that same realized state. The core insight is that grounding all artifacts in a shared, executable container state drastically reduces mismatches between what a task asks for, what the solution assumes, and what the verifier checks.
Key Findings
-
Environment-first grounding: by building and exposing a realized container state before artifact generation, FACET ensures instructions and solutions refer to files, services, and schemas that actually exist — so tasks are less likely to be unsolvable due to missing dependencies.
-
Scenario reconstruction preserves intent: FACET recovers cross-skill dependencies, intermediate states, and workflow constraints from heterogeneous skill sources, keeping richer procedural information than naive multi-stage pipelines.
-
Execution-based validation and targeted repair: tasks are validated end-to-end (build/init, solution execution, verifier acceptance) and failures are fixed by repairing only the responsible component, preserving valid artifacts and avoiding unnecessary regeneration.
-
Empirical gains for terminal-agent training: FACET produces dense, verifiable terminal tasks and successful trajectories that improve fine-tuned models across scales on Terminal-Bench, highlighting the importance of environment-grounded construction for solution–verifier alignment.
Who It's For and Tradeoffs
Great fit if you need reliably executable, verifiable terminal tasks for training or evaluating CLI/terminal agents, especially when sourcing tasks from heterogeneous skill repositories. FACET is useful for dataset creators, research groups training terminal agents, and teams wanting reproducible verifier workflows.
Look elsewhere if your goal is lightweight textual task generation without executing or reproducing environments, or if you cannot afford the compute/storage overhead of building and validating many containerized environments — FACET prioritizes executable fidelity over cheap scale.
Where It Fits
FACET sits between purely textual task synthesis and fully recorded human terminal trajectories: it synthesizes structured, executable tasks with dense state checks and targeted repairs, making it suitable for building high-quality training corpora and benchmarks where verifier-solution alignment matters.
How It Works (brief)
- Reconstruct related skills into coherent scenarios capturing dependencies and intermediate states.
- Build and initialize a container environment; expose its realized state as the shared grounding interface.
- Generate instruction, reference solution, and verifier sequentially with read access to the same state.
- Execute the reference solution, run the verifier, and perform targeted repairs up to a bounded budget when mismatches occur.
This design trades generation-only scale for higher validity and alignment between task artifacts, which downstream models benefit from during fine-tuning.
