Tag
Explore by tags
Showed that fine-tuning a GPT model on public GitHub code yields a capable program synthesizer, and introduced HumanEval — the docstring-to-function benchmark that still anchors code-generation evaluation. A production variant powers GitHub Copilot.
Orchestrates configurable deep-research agent workflows that combine LLMs, web search, and MCP tools to produce structured research reports and evaluation outputs. Supports LangGraph Studio, multiple model providers (OpenAI, Anthropic, local models), and Deep Research Bench evaluation for benchmarked comparisons.
A benchmark dataset for evaluating MLLM-driven interactive webpage code generation: provides prototyping screenshots, action.json interaction metadata, and example generation scripts across 127 webpages and 374 interactions to test dynamic UI-to-code capabilities.
Benchmark for evaluating OCR systems that convert PDFs and scans into Markdown and structured text: 1,403 PDFs and 7,010 unit tests covering text presence/absence, reading order, tables, and math formula accuracy. Diverse sources and ODC-BY-1.0 license for research use.
Autonomously executes diverse biomedical research tasks by combining LLM reasoning, retrieval-augmented planning, and code-based execution. Includes a web UI and Gradio demo, a curated Know‑How library, MCP integration, and a biology-tailored reasoning model (Biomni‑R0).
Argues AI has entered its 'second half': a working recipe (language pre-training priors + scale + reasoning) now generalizes RL across tasks, so the bottleneck shifts from inventing methods to defining problems and rethinking evaluation.
Practical, full-stack tutorial for building Retrieval-Augmented Generation (RAG) systems—covers data preprocessing, vector embedding and indexing, hybrid and multimodal retrieval, generation integration, evaluation and production-ready engineering. Includes hands-on projects and examples for developers with Python experience.
Open-source companion to a technical book that teaches how to design, evaluate and ship LLM-based AI agents — includes the full Chinese manuscript, community translations, chapter-aligned runnable example projects, and reproducible evaluation harnesses.
Automates multi-step web tasks by perceiving webpages as pixels and issuing low-level mouse, keyboard and scroll actions. A 7B-parameter multimodal agent trained on 145K synthetic trajectories (FaraGen), designed for on-device deployment and efficient task completion (~16 steps/task).
Collects ~200,000 human responses to 20 visual/semantic association questions (e.g., Bouba–Kiki), with per-response image options and demographic metadata — useful for cross‑cultural perception and evaluation of multimodal systems, but not guaranteed as a rigorously controlled experimental sample.
Reduces object-driven shortcut learning in zero-shot compositional action recognition by enforcing temporal verb cues and regularizing against frequent object-verb co-occurrence priors. Proposes RCORE with Co-occurrence Prior Regularization (treats frequent co-occurrences as hard negatives) and Temporal Order Regularization. Evaluated on Sth-com and EK100-com with improved compositional generalization.
Provides 100 real-world, open-ended research tasks paired with expert-written rubrics (around 40 weighted criteria per task) to evaluate long-form, web-browsing research agents on factual accuracy, analysis depth, presentation, and citation quality.