Analyzes how single-domain RL fine-tuning on LLMs induces cross-domain interference and shows this damage concentrates in a low-dimensional shared conflict subspace; proposes a local perturbation theory and short domain "refresh" procedures that selectively recover earlier domains with minimal collateral loss.
A benchmark for evaluating web-browsing agents in Korean contexts, composed of 400 tasks (300 manually verified by native speakers). Includes a human-verified split and an adversarial synthetic split to probe failure modes; reveals large performance gaps for both frontier and Korean models.
Enables agents to proactively discover multiple hidden problems in a user context and pair each with supporting evidence and concrete actions. Uses iterative discovery (batch rounds conditioned on prior finds) and reusable "thought templates" to expand coverage and ground claims.
Provides ~1M synthetic Salvadoran‑Spanish personas (148k records, ~300M tokens) grounded in 2024 census distributions for demographics, occupations and locations; intended for training/evaluating localized LLMs and synthetic-data workflows. CC BY 4.0, adults only.
Around 80K short audio clips paired with transcripts in JSON, organized for easy loading with the Hugging Face datasets ecosystem—designed for short-form speech tasks (ASR, TTS, fine-tuning) and quick prototyping with common Python data tools.
Represents episodic memory as a Cue–Tag–Content graph and integrates LLM reasoning into active retrieval so agents iteratively reconstruct and prune evidence paths for long-horizon questions. Reports up to 23% gains on LoCoMo / LongMemEval while reducing token and runtime costs.
Benchmark for evaluating proactive LLM mediators in realistic, multi-domain conflict scenarios by constructing cases from real disputes, probing five socio-cognitive adaptation axes, and using a topic-localized evaluator that achieves 0.82 alignment with human experts.
Provides 600,000 synthetic Vietnamese persona texts (100,000 records, 6 personas per record) aligned to Vietnam's 2024 census and surveys for training and evaluating NLP / text-generation models; includes 21 demographic and persona fields, CC BY 4.0, single train split.
Benchmark that measures an agent's ability to discriminate fine-grained relational structure in long-term memories. It embeds relation-controlled memory variants into realistic user–agent histories and tests downstream recovery and reasoning, highlighting where current memory systems fail.
Evaluates whether role-playing language agents follow a character's evolving psychological arc rather than a fixed persona, using ArcANE — an automatically constructed benchmark spanning 17 novels and 80 principal characters. Tests both in-text and out-of-text scenarios and compares context strategies and fine-tuned models.
End-to-end evaluation framework for conversational voice agents that runs bot-to-bot audio simulations and scores agents on task accuracy (EVA-A) and interaction experience (EVA-X). Includes per-scenario backend state, accent/noise perturbations, and 213 scenarios across airline, healthcare HR, and enterprise IT domains.
Dynamic interactive benchmark that tests whether LLM agents can adaptively plan and re-plan when world and user constraints are progressively revealed. Built on 307 household tasks with a multi-turn protocol that exposes hidden constraints only after plan violations, emphasizing iterative revision and constraint inference.