Workflow-aware benchmark for autonomous medical-AI research that splits agent execution into five stages (Plan, Setup, Validate, Inference, Submit) and evaluates long-horizon runs across segmentation, image enhancement, VQA, report generation, and lesion detection with stage-level scoring.
A benchmark for evaluating web-browsing agents in Korean contexts, composed of 400 tasks (300 manually verified by native speakers). Includes a human-verified split and an adversarial synthetic split to probe failure modes; reveals large performance gaps for both frontier and Korean models.
Native multimodal model for image/text/video→text tasks with million‑token context support. Uses a sparse-attention operator to cut long‑context compute and latency, and targets agentic, coding, and long-horizon conversational workloads.
Agentic LLM for long-horizon, environment-driven workflows: decomposes goals, generates and executes code/tool calls, evaluates outputs, and iterates. The Pro variant emphasizes coding and terminal execution and is published for use with sglang and multi-node H100 deployment.
Generates synthetic coding-agent session traces by pairing remotely hosted open agent models with local llama.cpp user models across real open-source codebases. Each trace records read/write/edit/bash actions and tool use; the dataset is a reproducible cartesian product (20×3×20×20 = 24,000 sessions) under an MIT license.
Represents episodic memory as a Cue–Tag–Content graph and integrates LLM reasoning into active retrieval so agents iteratively reconstruct and prune evidence paths for long-horizon questions. Reports up to 23% gains on LoCoMo / LongMemEval while reducing token and runtime costs.
Benchmark for evaluating proactive LLM mediators in realistic, multi-domain conflict scenarios by constructing cases from real disputes, probing five socio-cognitive adaptation axes, and using a topic-localized evaluator that achieves 0.82 alignment with human experts.
Open-weight frontier LLM for agentic reasoning and long-context analysis (up to 1M tokens). Uses a LatentMoE + Mamba-2 hybrid with Multi-Token Prediction and NVFP4 efficiency (550B total / 55B active). Suited for multilingual agents, RAG, and heavy tool-use workloads.
Multilingual frontier LLM optimized for long-context reasoning and agentic workflows, combining a LatentMoE (Mamba-2 + MoE) hybrid architecture with Multi-Token Prediction and NVFP4 quantization; targeted for NVIDIA GPU deployments and governed by the OpenMDW-1.1 license.
Benchmark that measures an agent's ability to discriminate fine-grained relational structure in long-term memories. It embeds relation-controlled memory variants into realistic user–agent histories and tests downstream recovery and reasoning, highlighting where current memory systems fail.
Evaluates whether role-playing language agents follow a character's evolving psychological arc rather than a fixed persona, using ArcANE — an automatically constructed benchmark spanning 17 novels and 80 principal characters. Tests both in-text and out-of-text scenarios and compares context strategies and fine-tuned models.
Provides compact, agentic text-generation for long-horizon, tool-enabled workflows — trading some peak capability for lower latency and easier on-prem deployment. Key features: adaptive/coherent thinking traces, function-calling support, and sglang/docker-ready serving.