Enables agents to proactively discover multiple hidden problems in a user context and pair each with supporting evidence and concrete actions. Uses iterative discovery (batch rounds conditioned on prior finds) and reusable "thought templates" to expand coverage and ground claims.
Generates synthetic coding-agent session traces by pairing remotely hosted open agent models with local llama.cpp user models across real open-source codebases. Each trace records read/write/edit/bash actions and tool use; the dataset is a reproducible cartesian product (20×3×20×20 = 24,000 sessions) under an MIT license.
Benchmark for evaluating proactive LLM mediators in realistic, multi-domain conflict scenarios by constructing cases from real disputes, probing five socio-cognitive adaptation axes, and using a topic-localized evaluator that achieves 0.82 alignment with human experts.
Open-weight frontier LLM for agentic reasoning and long-context analysis (up to 1M tokens). Uses a LatentMoE + Mamba-2 hybrid with Multi-Token Prediction and NVFP4 efficiency (550B total / 55B active). Suited for multilingual agents, RAG, and heavy tool-use workloads.
Multilingual frontier LLM optimized for long-context reasoning and agentic workflows, combining a LatentMoE (Mamba-2 + MoE) hybrid architecture with Multi-Token Prediction and NVFP4 quantization; targeted for NVIDIA GPU deployments and governed by the OpenMDW-1.1 license.
Benchmark that measures an agent's ability to discriminate fine-grained relational structure in long-term memories. It embeds relation-controlled memory variants into realistic user–agent histories and tests downstream recovery and reasoning, highlighting where current memory systems fail.
Evaluates whether role-playing language agents follow a character's evolving psychological arc rather than a fixed persona, using ArcANE — an automatically constructed benchmark spanning 17 novels and 80 principal characters. Tests both in-text and out-of-text scenarios and compares context strategies and fine-tuned models.
End-to-end evaluation framework for conversational voice agents that runs bot-to-bot audio simulations and scores agents on task accuracy (EVA-A) and interaction experience (EVA-X). Includes per-scenario backend state, accent/noise perturbations, and 213 scenarios across airline, healthcare HR, and enterprise IT domains.
Dynamic interactive benchmark that tests whether LLM agents can adaptively plan and re-plan when world and user constraints are progressively revealed. Built on 307 household tasks with a multi-turn protocol that exposes hidden constraints only after plan violations, emphasizing iterative revision and constraint inference.
Provides a complete, lightly-processed export of AI Village's >1-year multi-agent data: per-agent computer sessions (with screenshots), turn-by-turn computer-use logs, group chats, agent memories, goals, and daily summaries for research into agentic behaviour, multi-agent dynamics, long-horizon memory, and AI safety. Access is manually reviewed.
Measures how coding agents explore repositories by asking them to return a ranked, line-level list of code regions relevant to an issue under a fixed line budget. Covers 848 issues across 203 repos and 10 languages; evaluates coverage, ranking, and context-efficiency to isolate exploration quality.
Decouples perception and reasoning for hours-long videos by streaming inputs into a three-tier Hierarchical Graph Memory and using an agentic Observation–Reason–Action retrieval loop; reduces reasoning context to ~2% of full video while improving benchmark accuracy.