HiPHI addresses a recurring bottleneck for embodied AI: large-scale data with both high-fidelity physical states and grounded object interactions. Rather than trading precision for coverage, HiPHI was collected to span a wide motion-space while preserving sub-millimeter optical tracking and synchronized mesh-level object state, making the dataset directly executable for humanoid policy learning and motion priors.
What Sets It Apart
- Scale + precision: 617.5 hours (≈200.1M frames) captured at 90 Hz with sub-millimeter optical tracking — substantially larger than typical lab MoCap corpora while keeping high spatial fidelity.
- Interaction grounding: 245.7 hours of human–object interaction with synchronized object trajectories and shared OBJ meshes (40 distinct real objects), so object state (pose/mesh) is part of the motion record rather than a separate annotation.
- Semantic indexing: Motion clips are organized by Frame–LU units (214 Frame–LU labels across 22 frames) and paired with natural-language descriptions to support semantic retrieval and systematic motion-space design.
- Practical export: Standardized BVH body representation (55 joints), performer metadata (132 performers), and benchmark protocol oriented to motion-space coverage, interaction consistency, and robot learning tasks.
Who it's for and trade-offs
Great fit if you are training or evaluating humanoid controllers, motion priors, imitation/retargeting systems, or object-aware loco-manipulation policies and need synchronized object state at high temporal and spatial fidelity. It’s also suited for research on motion-space coverage, motion retrieval using Frame‑LU labels, and benchmarking embodied learning.
Look elsewhere if you need unrestricted commercial licensing (HiPHI uses a ModalityNet Open Research License v1.0 for non-commercial research) or if your use case only requires Internet-scale, low-fidelity video where approximate 2D/3D poses suffice. The dataset’s size (hundreds of hours, ~215 GB on Hugging Face) and specialized format (BVH + object meshes/trajectories) mean nontrivial storage and preprocessing effort for some pipelines.
Practical notes
- Typical contents: 371.8 h body-only motion + 245.7 h HOI; 132 performers; 90 Hz capture; ~200.1M frames; BVH body format; package-local object trajectories + OBJ meshes.
- Hosted on Hugging Face and accompanied by a project page and benchmark suite; license targets academic/non-commercial research.
This framing highlights why HiPHI matters now: it fills the gap between small, high-fidelity lab datasets and large but ungrounded internet video corpora by delivering both scale and physical grounding for embodied AI research.
