AIAny
AI Agent Papers2026

SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks

Measures whether reusable skills improve agent task completion through paired tests. Across 87 tasks and 18 model–harness configurations, curated skills raise average pass rate by 16.6 percentage points; compact skill bundles work best.

Visit Website

Introduction

Adding more instructions can make an agent worse. The useful question is whether a reusable skill helps the same model solve the same task; paired evaluation separates that contribution from improvements caused by a stronger model or a different execution harness.

Key Findings
  • The June 2026 revision evaluates 87 tasks across eight domains and 18 model–harness configurations. Expert-curated skills increase average pass rate from 33.9% to 50.5%, a gain of 16.6 percentage points.
  • Gains vary from 4.1 to 25.7 percentage points across configurations. A skill library therefore needs evaluation with the agent stack that will actually use it.
  • Packages with at most three modules outperform larger bundles; compact procedural guidance can be more useful than exhaustive reference material.
  • Some smaller models with skills match larger models without them, making skill quality relevant to model selection as well as prompt design.
Who Should Read It

Useful for developers deciding how to package, test, and maintain agent expertise. Deterministic checks and controlled comparisons offer a practical evaluation pattern. The curated tasks do not establish that any community skill will help: validate your own workflows, and measure regressions alongside average gains.

Back

Information

  • Websitearxiv.org
  • OrganizationsBenchFlow, OSU, Amazon, UC Berkeley, UC Santa Cruz, UC Davis, Dartmouth, RLWRLD, Princeton University, Oxford University, Stanford University, USC, CMU, Foxconn, BU, Zenity, UNSW, UT Austin, MSU, Duke University, ByteDance, UT Dallas, UC San Diego, Columbia University, University of Rochester, Cornell Tech, Georgia Tech, Cornell University, NEU, UCLA, Snap Inc., Fanshawe College, University of Science and Technology of China, HKUST(GZ), Anyscale
  • AuthorsXiangyi Li, Yimin Liu, Wenbo Chen, Bingran You, Zonglin Di, Yifeng He, Shenghan Zheng, Kyoung Whan Choe, Jiankai Sun, Shuyi Wang …
  • Published date2026/02/13

Categories

Tags

More Items

AI Agent Papers2024

Agent Workflow Memory

Zora Zhiruo Wang, Jiayuan Mao +2Carnegie Mellon University, Massachusetts Institute of Technology

Extracts reusable workflows from past task trajectories to guide future web actions. Supports offline demonstrations and online experience, improving long-horizon navigation without manually written routines.

paperai-agentLLMai-workflowlong-horizon+1
AI Agent Papers2023

MemGPT: Towards LLMs as Operating Systems

Charles Packer, Sarah Wooders +5University of California, Berkeley

Lets language models move information between their context window and external storage. Memory editing, retrieval, and function chaining support conversations and document analysis beyond a single prompt's capacity.

paperLLMai-agentretrieval
AI Agent Papers2024

Executable Code Actions Elicit Better LLM Agents

Xingyao Wang, Yangyi Chen +5University of Illinois Urbana-Champaign, Apple

Lets language-model agents compose tools through executable Python and revise actions from interpreter feedback. Evaluates 17 models and releases 7,000 interaction trajectories for training agents that can debug their own code.

paperai-agentLLMpythonbenchmark