Practical, full-stack tutorial for building Retrieval-Augmented Generation (RAG) systems—covers data preprocessing, vector embedding and indexing, hybrid and multimodal retrieval, generation integration, evaluation and production-ready engineering. Includes hands-on projects and examples for developers with Python experience.
AI Agent2025
Evaluates and optimizes AI agents and language models in containerized environments, supporting large-scale parallel benchmarks and RL rollouts. Integrates with third‑party providers for thousands of parallel environments and serves as the official harness for Terminal‑Bench.
AI Agent2025
Open-source companion to a technical book that teaches how to design, evaluate and ship LLM-based AI agents — includes the full Chinese manuscript, community translations, chapter-aligned runnable example projects, and reproducible evaluation harnesses.
AI Agent2025
Automates multi-step web tasks by perceiving webpages as pixels and issuing low-level mouse, keyboard and scroll actions. A 7B-parameter multimodal agent trained on 145K synthetic trajectories (FaraGen), designed for on-device deployment and efficient task completion (~16 steps/task).
AI Dataset2025
Collects ~200,000 human responses to 20 visual/semantic association questions (e.g., Bouba–Kiki), with per-response image options and demographic metadata — useful for cross‑cultural perception and evaluation of multimodal systems, but not guaranteed as a rigorously controlled experimental sample.
Reduces object-driven shortcut learning in zero-shot compositional action recognition by enforcing temporal verb cues and regularizing against frequent object-verb co-occurrence priors. Proposes RCORE with Co-occurrence Prior Regularization (treats frequent co-occurrences as hard negatives) and Temporal Order Regularization. Evaluated on Sth-com and EK100-com with improved compositional generalization.
AI Dataset2026
Provides a physical reconstruction benchmark of OmniDocBench v1.5 by producing five real-world photographic variants (Scanning, Warping, Screen‑Photography, Illumination, Skew) for each of 1,355 pages, inheriting original ground-truth to enable controlled, scenario-wise evaluation of document parsing robustness.
AI Dataset2026
Provides 100 real-world, open-ended research tasks paired with expert-written rubrics (around 40 weighted criteria per task) to evaluate long-form, web-browsing research agents on factual accuracy, analysis depth, presentation, and citation quality.
AI Dataset2026
Benchmarks LLM agents on realistic legal work by packaging lawyer-style assignments with client materials and expert, per-deliverable rubrics. Includes an execution harness to run, score, and compare agents across a large, evolving task set spanning multiple practice areas.
AI Video Papers2026
Provides a diagnostic suite that audits video-understanding benchmarks to find samples solvable without visual or temporal input, filters those shortcuts, and produces a distilled video-native testbed that reveals major capability gaps in current Video-LLMs.
AI Infra2026
Provides an end-to-end platform to evaluate, observe, protect, and optimize LLM and AI agent deployments. Integrates OpenTelemetry tracing, 50+ evaluation metrics, agent simulations, an OpenAI‑compatible gateway, and guardrails; self‑hostable under Apache 2.0.
AI Dataset2026
A collection of biology-focused 'mystery' tasks for benchmarking model performance on biomedical reasoning, evidence synthesis, and problem solving; curated by Anthropic and hosted on Hugging Face, designed for granular evaluation of scientific decision-making.