Desktop CAD workflows combine long-horizon visual state changes with precise low-level GUI actions, yet public multimodal datasets rarely preserve full task context. This collection supplies 1,021.64 hours and 597 self-contained workflows across 10 CAD/BIM/visualization applications, enabling behavior cloning and agent evaluation on real engineering tasks.
What Sets It Apart
- End-to-end workflow packaging: each item bundles the problem statement, input/reference assets, a gold output, evaluation rubrics, a narrated screen recording, and synchronized mouse/keyboard events — so models can be trained and assessed against concrete deliverables, not just frame-level labels.
- Broad software coverage and scale: 10 applications (AutoCAD, SOLIDWORKS, CATIA, Siemens NX, SketchUp, Revit variants, STAAD.Pro, V-Ray, D5 Render) and 1,021.64 hours — so agents can learn cross-application GUI behaviors and domain-specific operations.
- Frame-action alignment and derived formats: frame_events, events, narration and parquet indices enable precise temporal grounding and efficient dataset loading — so research on action grounding, long-horizon planning, and imitation learning is more reproducible.
Who it's for and trade-offs
Great fit if you build or evaluate desktop software agents, behavior-cloning models for CAD UI actions, multimodal models that link visual states with symbolic/temporal actions, or benchmarks requiring gold outputs and rubrics. Look elsewhere if you need annotated semantic object labels inside 3D models (mesh/voxel ground truth) or a clear open-source license—this dataset's license is not specified on the card and should be checked before redistribution.
Where it fits
Practically, this dataset complements large-scale trimmed tutorial corpora by preserving task intent and deliverables; pair it with synthetic CAD render datasets for geometry supervision or with smaller parametric CAD datasets when you need labeled 3D geometry rather than GUI action traces.
