Recovers editable design files from raster images by growing an editable layer hierarchy via an agentic pipeline that selects and composes modality-specific tools. Introduces graceful verification (accept/prune/retry) to prevent error accumulation and presents the Figma Edit Replay Benchmark (909 files, 14,796 edits) to measure editability across layout, color, and text edits.
Generated instance set (seed 31415) for ASI‑Bench: includes four matched prompt variants, agent-visible inputs, reference artifacts, and instance metadata for 60 project-scale scientific research tasks across 11 domains; intended for evaluating autonomous research agents. Licensed Apache‑2.0.
Presents Metis, a prototype memory foundation model that embeds a persistent native memory state into the backbone so historical experience is compressed and accessed via memory attention. Key features: forward-only, gradient-free online memory updates; memory-specific mid-training objectives; and a dual text/code memory design.
Indexes chemistry literature as provenance-bearing atomic claims and provides a faceted taxonomy, evidence graph, and REST/SDK/MCP APIs so researchers and AI agents can retrieve verifiable, claim-level findings across papers; live index contains 2.4M claims from 147K papers.
Autonomously proposes, modifies, executes, and evaluates ML experiments to study recursive self-improvement in machine learning engineering. Implements an open stack (OpenMLE-Gym, -RL, -Evo) and post-trains Frontis-MA1 (35B) around four evolution operators (Draft, Improve, Debug, Crossover); releases model weights and the full codebase.
Designs and evaluates a foundation GUI agent that performs cross-platform GUI and CLI actions on real devices to complete long-horizon workflows. Emphasizes a unified action space, a large-scale real-device mobile runtime, an AutoResearch-style data flywheel, and online RL training across 10,000+ concurrent environments.
Evaluates vision-language model judges on computer-using agent (CUA) trajectories to measure verifier reliability. Provides OSReward-Hard and OSReward-Multi challenge sets, the OS-Shepherd-100K reasoning-annotated corpus, and trained OS-Shepherd reward models that match commercial judges at ~30–60× lower cost.
A 365-day, order-level simulation benchmark for evaluating long-term coherence of LLM agents in seller-side e-commerce. Grounded in 98,843 real product records and 26 interactive tools, it pairs prompt upstream supplier signals with delayed downstream order outcomes to stress planning, memory, and tool use over long horizons.
Treats agent self-improvement as natural selection over a population of harnesses (prompts, tools, skills, control flow), evolving a frozen-model agent by selecting harness edits that extend capability without regressing others. Uses a preserve-and-extend contract, lineage archive, and verifier-driven fitness (no gold solutions) to recombine complementary edits and transfer across benchmarks.
Evolves persistent, stateful environments to red-team tool-using AI agents — provides 10K+ validated scenarios across 50 domains and a feedback-driven attack policy (EMHA) to surface long‑horizon safety failures.
Reformulates long-horizon agent execution as explicit task-state management: a manager defines bounded subtasks, fresh-context executors run them, and read-only auditors verify outcomes. Shows large performance gains on WeaveBench, Terminal-Bench and OSWorld.
Provides 37,484 validated command-line tasks generated by recursive task synthesis, each paired with searchable metadata and a sanitized, runnable package (instructions, solution, verifier, and optional Dockerfile).