AIAny
Icon for item

ProgramDistill: From Interactive Web Apps to Verifiable Reference-Guided SWE Tasks

Extracts replayable behaviors from working web apps and converts them into verifiable, reference-guided software-engineering tasks to evaluate and diagnose coding agents; includes an automated pipeline that scales to thousands of tasks.

Introduction

Practical web development often forces engineers (or coding agents) to infer missing behavior from a working application rather than implement features from a spec. ProgramDistill tackles this gap by automatically factorizing interactive web apps into feature-level behaviors and turning those behaviors into replayable, reference-guided tasks that can verify agent implementations against a working reference.

Key Findings
  • Large-scale, automated task synthesis: a mine-craft-patch pipeline discovers 1,975 replay-verified behaviors across 26 applications and constructs 4,063 tasks without human intervention — meaning evaluation can scale beyond hand-curated feature lists.
  • Behavior-first, implementation-agnostic evaluation: tests target observed input-output interactions (replays) rather than source structure, so agents can use any architecture or language but are judged by whether reconstructed behaviors match the reference.
  • Difficulty and diagnosis at controlled granularity: full-application reconstruction yields wide variance (best agents hit ~49.2% and ~28.8% cumulative-workflow success), while partial restorations reveal steep drops in success as restoration depth increases (from 100%→64% and 96%→32%), exposing long-horizon and compositional failure modes.
  • Useful for benchmarking and curriculum design: the dataset’s feature factorization lets researchers measure performance per feature granularity and construct curricula that progressively increase restoration depth.
Who it's for and trade-offs

Great fit if you need a behavioral, reproducible benchmark for coding agents or want to study how models infer and reimplement features from working software — particularly for diagnosing compositional and long-horizon failures in web apps. Look elsewhere if you need language- or implementation-specific tests (ProgramDistill deliberately evaluates behavioral parity) or if your target domain is non-interactive systems where UI-driven replay is irrelevant.

Information

  • Websitearxiv.org
  • OrganizationsKAIST, Microsoft Research Montréal, Microsoft AI
  • AuthorsJeonghye Kim, Minseon Kim, Young Jin Kim, Matheus Pereira, Marc-Alexandre Côté, Alessandro Sordoni, Xingdi Yuan, Zhengyan Shi
  • Published date2026/09/16

More Items

Introduces physical rank consistency (PRC) and ActionPiece, a learned action tokenizer that preserves local physical relationships via joint supervision of representation learning and quantization, improving autoregressive vision-language-action policy performance across multiple benchmarks.

Estimates a model's probability of correctness by leveraging its past graded episodes: XConf recalls similar past cases and prompts the model to reflect and restate confidence, improving calibration and discrimination across reasoning, coding, multimodal QA, and agent tasks at much lower generation cost.

Transforms scientific code repositories into executable, agent-learnable environments that support task generation, execution, and scientific verification. Agent-guided repository transformation produces verified interaction trajectories used to train the PhAI-IDE model family. Intended for research on agent learning, scientific-code repair, and training RL/SFT models.