AI Dataset2026
Zeren Jiang, Yushi Lan +9Visual Geometry Group, University of Oxford, Nanyang Technological University +1
Provides multiview synthetic RGB video clips with per-frame depth, instance masks, dense long-range 3D point tracks, camera poses, and SMPL‑X human pose/shape labels for 4D reconstruction, tracking, and geometry-aware novel-view synthesis. Includes ~4.7K clips (1.4M frames) and is licensed for AI training.
AI Dataset2026
Provides ~50M multimodal annotations organized for unified training across structured visual understanding, segmentation, dense geometric prediction, and multi-view reconstruction — released as task-specific JSONL files that reference original image assets rather than redistributing raw images.
AI Model2026
Depth-conditioned LoRA for Krea‑2 that extracts a depth map from any input image and generates new images preserving the original 3D structure and composition while changing content and style. Single 862MB LoRA, works with Krea‑2‑Raw and Krea‑2‑Turbo.
Trains a single diffusion model that unifies 3D scene reconstruction and generative modeling by operating directly in pixel/rendered-image space. Supervises diffusion on rendered views and adds a geometry-perception loss from a pretrained 3D foundation model, reducing latent information loss and improving 3D fidelity.
AI Dataset2026
Provides structured egocentric manipulation signals from smartphone videos: MANO 3D hand reconstructions, metric camera trajectories, and fine-grained atomic action segments (full release ≈2,000 hours planned). Supplies aligned hands.npz, camera_traj.npz, undistorted intrinsics and segment annotations for embodied-learning pipelines.
Expresses diverse computer-vision tasks as instruction-driven text, image, or mixed generation from a single unified multimodal model, producing outputs for detection, segmentation, depth, pose, OCR and more. Trained on a converted SenseNova‑Vision instruction–response corpus and requires no task-specific prediction heads.
AI Dataset2026
Provides 1,800+ hours of synchronized egocentric multi-view recordings with 3D hand reconstructions, wide‑FOV depth, and hierarchical task/subtask annotations for embodied AI and robot learning. Includes six fisheye views, hand meshes, and per-episode temporal labels across 44k+ episodes.
AI Agent Papers2026
Generates large-scale, explorable 3D open-world scenes from open-ended text prompts, producing editable instance-level assets and a consistent global terrain. Uses agentic planning to convert text into region/terrain/asset specifications and a coarse-to-fine pipeline for terrain construction, mesh reconstruction, and render-based refinement.
AI Video Papers2026
Predicts future video frames conditioned on an observed frame, a language instruction, and a sequence of end-effector poses and gripper states for robot manipulation. Uses per-arm SE(3) geometric encoding (PRoPE-style), a lightweight depth branch, SAM3 masks with a frozen V-JEPA teacher, and distribution-matching distillation for efficient, consistent action-conditioned rollouts.
AI Agent Papers2026
Yansong Ning, Jingwen Ye +6Affiliation: AI Thrust, HKUST(GZ), TEG AIPD, Tencentyning092connect.hkust-gz.edu.cn, [email protected]{jingwenye,wadewdzhang}@tencent.com
Evaluates and trains multimodal agents to construct interactive 3D open worlds from user queries — provides a large benchmark of assets, seed worlds, and reverse-synthesized queries plus a sandbox RL gym for tool-driven editing and rubric-based verification. Reports that frontier MLLMs perform under 60% and that RL fine-tuning improves precise 3D editing.
Performs causal, bounded‑memory streaming 3D reconstruction by caching KV features from only the preceding 11 frames, predicting a per‑frame point map and adjacent relative pose, and composing these local predictions into a global trajectory; includes a lightweight rotation refiner and composition‑aware loss to limit drift.
Converts posed indoor RGB(-D) video into editable, simulation-ready 3D scene graphs by parsing multi-view evidence into per-object bundles, generating complete object assets from that evidence, and placing them with GizmoAct, a VLM policy that refines 9-DoF poses through closed-loop GUI actions.