Robot demonstrations often specify the goal but leave the steps between actions implicit. FineART makes those intermediate steps explicit across entire episodes, turning long-horizon manipulation into a language-supervised learning problem rather than a collection of isolated action sequences.
What Sets It Apart
- All 40,543 episodes have temporally segmented subtask labels, totaling 533,913 annotations across 151 tasks. This complete coverage lets researchers study step-by-step supervision without relying on a small, selectively annotated subset.
- Independent human annotators wrote the labels without VLM assistance, and a separate team audited them. The annotations distinguish objects with concrete details such as color, helping models learn grounded instructions rather than generic action templates.
- Four camera views accompany joint and gripper states and actions. The combination supports research on hierarchical policies, where a high-level component selects the next language-described subtask and a low-level policy executes it.
Who It’s For—and the Trade-offs
Great fit for researchers training or evaluating long-horizon, bimanual manipulation policies that need dense language supervision. Look elsewhere for single-arm coverage, broad robot-platform diversity, or balanced representation of rare skills: the data comes from one Trossen ALOHA-style setup, and everyday pick-and-place and sorting make up about 77% of recorded hours. Its 1,718 hours of four-camera footage also make storage and data loading a practical consideration.
