Provides an open release and system design for generalist computer-use agents (GCUAs), combining code execution, GUI actions, a real-device harness, and a cross-device SDK. Ships evaluation trajectories for multiple frontier models and shows large score and efficiency gains on OSWorld-v2.
Benchmark for evaluating agents' ability to deliver working AI systems in realistic customer environments: 95 end-to-end delivery tasks with scripted stakeholders and production-like systems, scored by improvement to the customer's business metric and penalized for stakeholder contact.