AI Dataset2026
Benchmark dataset for evaluating long-horizon coding agents and software-engineering tasks, containing English code and tabular metadata in Parquet format; small scale (<1K examples) for fast prototyping and evaluation.
AI Dataset2026
Provides agentic instruction‑tuning trajectories for software‑engineering tasks, formatted for supervised fine‑tuning and agent training. Contains multi‑file edits, tests, docs and structured agent traces (≈5,115 records, 1.9 GiB). Intended for commercial use; licensed CC‑BY 4.0 with additional permissive licenses.
AI Dataset2026
Provides per-decision training samples for RL-driven command-line LLM agents: each record pairs a task prompt plus terminal history with a teacher's next-action in Terminus-2 JSON. Around 31k verifier-passing samples from 630 ATCB tasks, formatted for NeMo Gym's terminus_judge and licensed CC-BY-4.0.
AI Dataset2026
Provides a curated benchmark of 170 real-world, multilingual code-refactoring instances to evaluate AI coding agents on large-scale, behavior-preserving, cross-file refactors. Each task includes rewritten issue descriptions and manually reviewed test suites to avoid over- and under-constraining evaluations.
AI Dataset2026
Provides a queryable dataset of 3,797,117 SKILL.md agent-skill files found on public GitHub, deduplicated by content hash and enriched with representative text, front matter, folder composition, repo metadata, and sampled commit history for research.
AI Agent Papers2026
Turns natural-language PLC requirements into verified, runnable IEC 61131-3 Structured Text by driving a closed loop of generation, compilation, deployment, and behavioral verification on a live OpenPLC runtime. The verification-gated harness forces inputs, traces execution, repairs failures, and renders ladder diagrams plus process simulation to raise dynamic runtime pass rates.
AI Agent Papers2026
Evaluates whether coding agents can modify real scientific software while preserving domain-specific scientific contracts. Contains 119 repository-level tasks across 98 GitHub projects and 20 scientific domains, measures reproducible edits in pinned Docker images, and analyzes recurring failure modes.
AI Agent Papers2026
Defines recursive self-improvement (RSI) with an evaluation metric (Headroom-Closed Index) and a five-level autonomy roadmap—from executing human-specified improvements to recursively improving improvement mechanisms—while analyzing scenarios, verification challenges, and governance risks.
AI Agent Papers2026
Extracts replayable behaviors from working web apps and converts them into verifiable, reference-guided software-engineering tasks to evaluate and diagnose coding agents; includes an automated pipeline that scales to thousands of tasks.
AI Agent Papers2026
Measures how individual harness components—planning, action space, and context management—affect coding agents' success, cost, and behavior. Uses a modular harness across 176 matched settings on SWE-Bench Verified and Terminal-Bench 2.1 to isolate component effects and surface model- and budget-dependent trade-offs.
AI Agent Papers2026
Provides reproducible, multi‑platform environments and execution‑grounded verification for hybrid computer‑use agents that interleave GUI exploration and code/CLI. Key features: a five‑platform harness, RecreationBench (250 tasks), reference‑grounded programmatic and visual assertions for automatic scoring.
Turns implemented functionality in open-source codebases into executable RL training environments by automatically generating behavioral specifications, execution-grounded tests, and post-rollout filtering; yields 5,545 verifiable coding tasks across 23 languages to train coding agents.