Measures how well LLMs and agent-driven workflows prepare supervised training data end-to-end by jointly benchmarking data construction and data-quality evaluation across six domains, using a downstream-grounded protocol and new metrics.
AI Dataset2026
Provides 462 unrestricted long-form chain-of-thought reasoning traces distilled from the full Mythos V2 model (≈104.7M characters); intended for long-context evaluation, trace analysis and process-level supervision. License unknown—verify before reuse.
AI Dataset2026
Provides the gated, official OSWorld 2.0 Python task class files (task_*.py) required to run the benchmark; distributed via a Hugging Face gated dataset to reduce benchmark leakage. Download requires accepting gated access on Hugging Face.
AI Agent Papers2026
Represents episodic memory as a Cue–Tag–Content graph and integrates LLM reasoning into active retrieval so agents iteratively reconstruct and prune evidence paths for long-horizon questions. Reports up to 23% gains on LoCoMo / LongMemEval while reducing token and runtime costs.
AI Agent Papers2026
Benchmark for long-horizon computer-use agents that must orchestrate GUI, CLI, and code operations within single trajectories across 114 real-world tasks. Evaluated on a real Ubuntu desktop and paired with a trajectory-aware judge that inspects deliverables, artifacts, and action traces—revealing a top PassRate of ~41.2%.
Generates outcome-specific, dialectical rationales with an LLM and derives continuous, calibrated risk scores for irregularly sampled medical time series—mitigating risk polarization. Reports +3.3% average AUPRC and 81% reduction in calibration error across three benchmarks; code released.
AI Agent Papers2026
Turns raw datasets into verifiable multimodal news features via a multi-agent newsroom pipeline. Key innovations: (1) an Inspector that links each claim to data/code/external references for re-execution and audit; (2) multimodal asset generation (interactive maps, audio, visuals) tailored to the story.
Shifts branching and credit assignment in agentic RL from coarse units to fine-grained decision points in generated sequences. Uses a Branching Score combining token uncertainty and policy-induced likelihood gains plus procedure-level advantage scaling; improves performance across 13 benchmarks while keeping efficient tool calls.
AI Agent Papers2026
Orchestrates teams of sub-agents across text, image, audio and video by modality-aware task decomposition, online sub-agent specialization, and parallel execution; introduces DA-GRPO to train Orchestra-o1-8B and reports a ~10.3% accuracy improvement on the OmniGAIA benchmark.
AI Video Papers2026
Proposes chunk-level multimodal retrieval and chunk-adaptive reranking for retrieval-augmented generation on long egocentric videos; introduces V-RAGBench to decouple retrieval vs. generation evaluation and CARVE to run parallel retrievers and select per-chunk configurations.
AI Dataset2026
Benchmark for evaluating multimodal LLM safety in Korean cultural contexts — includes KSAFE-MM-G which localizes global safety queries into Korean scenarios and KSAFE-MM-C which targets culture-specific visual-textual vulnerabilities. Provides curated image–text pairs and jailbreak-style prompts to reveal both unsafe behaviors and over-refusal.
AI Dataset2026
Provides kanji-level evaluation data for Japanese TTS: disambiguated sentence contexts targeting 4,378 kanji-reading pairs (2,136 Jōyō kanji) with 13,095 native-speaker–verified sentences and katakana-marked ground-truth readings for kanji-level error metrics.