Evaluates proactive, multimodal agents on 400 bilingual real‑world tasks across five capability axes (Skill Usage, Exploration, Long‑Context Reasoning, Multimodal Understanding, Cross‑Platform Coordination) using live Docker‑based, stepwise closed‑loop evaluation to separate base model skills from framework design.
Uses large-scale text-to-video generative pretraining to create GenCeption, a feed-forward perception model that performs diverse vision tasks from text instructions—depth, surface normals, camera pose, referring segmentation, and 3D keypoints—often matching or surpassing specialized models while requiring far less task-specific data.
Explores unsupervised visual pretraining on visually rich documents to improve language-model intelligence; shows visual-pretrained models outperform text-only counterparts on the same corpora. Key aspects: direct use of images/layouts (no OCR-only pipeline), scalable across backbones and benchmarks.
Reconstructs 4D dynamic human scenes from sparse, low-overlap multi-camera captures by decoupling background synthesis and human modeling. Synthesizes hundreds of camera-controlled background views with a video diffusion model, initializes deformable Gaussian humans via cross-view identity and triangulated keypoints, then applies motion-adaptive recursive enhancement to reduce artifacts.
Proposes Riemannian Isometric Policy Optimization (RIPO) to fix exploration collapse in PPO-style RL for LLMs by aligning policy updates with the policy manifold's Riemannian geometry, improving exploration–exploitation balance and optimization stability across competition benchmarks.
Provides a deliberative Agent OS layer for robots that handles scene-conditioned planning, context-isolated skill execution, multi-stage verification, persistent multi-modal graph memory, and edge–cloud collaboration. Introduces EmbodiedWorldBench (16 scenes, 200+ tasks) and a failure-driven self-evolution loop; shows improved task success and strong memory benchmark scores.
Unifies high-level visual-language reasoning and low-level control for visual navigation by decoupling cognition and control: a slow vision-language reasoner produces pixel goals with explicit chain-of-thought, and a fast action expert converts those anchors into continuous waypoints for robust urban and indoor navigation.
Uses pretrained multimodal LLMs as zero-shot, training-free reward models for text-to-image RL by scoring how well the original text prompt can be recovered from a generated image via image-conditioned prompt log-likelihood; includes a Self-SpectraReward closed-loop variant.
Acquires repository knowledge via a targeted QA loop before generating patches, decoupling knowledge acquisition from repair. A Questioner and Answerer produce evidence-grounded QA pairs that a Resolver uses to generate fixes; improves Pass@1 on SWE-bench Verified with modest overhead.
Builds structured knowledge graphs for retrieval-augmented generation via a multi-step GraphRAG pipeline that separates extraction from consolidation. Key features include typed two-stage extraction, DBSCAN-backed deduplication, LLM summarization, Leiden community detection, and a compact 7B extractor model (Meno-Lite-0.1).
Comprehensive benchmark and automated evaluation framework for keyframe-conditioned video generation—decomposes keyframe execution into six metrics and assesses overall video quality with evidence-grounded MLLM judgments and specialized perception models.
Specialized LLM for clinical workflows trained via a human-gated self-evolution loop to improve patient consultation, multimodal clinical reasoning, interactive diagnosis, and EHR tool use. Iteratively refines targeted synthetic and curated data based on benchmark failures to raise specific capabilities without broad regressions.