An agent can remember how to search a website and still repeat every click. The useful distinction here is between advice in memory and a callable action: encoding a workflow as a program makes it executable, testable, and reusable without retraining the underlying language model.
Key Findings
- On 812 WebArena tasks, the Claude 3.5 Sonnet agent reaches 40.4% success, compared with 32.7% for the static baseline and 36.3% for textual-skill AWM. The gains support changing how skills are represented and checked, rather than simply adding more memory.
- Average action counts are 5.0 for ASI, 5.6 for the static agent, and 5.9 for AWM. A skill call can contain several primitive actions, so these counts measure agent decisions rather than all underlying browser operations.
- A shopping-site ablation improves success from 32.6% with unverified text in memory to 39.0% with verified text, versus 40.1% for verified programs as actions. Verification explains much of the improvement; executable actions add further value.
How Skills Are Checked
The system induces Python functions from trajectories judged successful by an LLM. It rewrites the trajectory to call those functions, removes trailing primitive actions, then executes the prefix and lets the agent finish. Skills enter the library only if the resulting task is judged correct, a new skill was called, and its calls changed the environment. This is execution-based evidence, not a formal correctness proof.
Who Should Read It
A useful reference for researchers building web agents that face recurring workflows and can retain skills across tasks. The evidence comes from WebArena and selected extended or cross-site tasks with a particular model backbone; it does not establish universal transfer. Website-specific selectors and layouts can invalidate skills, and verification requires extra environment interactions.