AIAny
Icon for item

Thinking with Visual Primitives

Provides the dataset and accompanying technical report for a DeepSeek project that interleaves spatial markers (points and boxes) into multimodal LLM reasoning. Includes a public subset of data and benchmarks under an MIT license; model weights are not included.

Introduction

Why this matters — modern multimodal LLMs often fail on dense spatial and structural reasoning because natural language alone can't unambiguously reference complex layouts. This release bundles the technical report and a public subset of data used to test a "point-to-reason" paradigm where minimal visual primitives (points and bounding boxes) are interleaved into the model's reasoning trajectory to reduce the Reference Gap.

What Sets It Apart
  • Point-as-thought: Visual primitives are treated as atomic reasoning tokens, so the model can anchor abstract language to precise coordinates rather than relying solely on verbal descriptions — this reduces ambiguous references in tasks like counting, topological navigation, and spatial relation inference.
  • Token efficiency engineering: The work describes compressing visual token caches (KV compression per 4 tokens) so experiments target lower image-token budgets without discarding spatial fidelity — meaning experiments can scale with less compute for image-heavy reasoning.
  • Benchmarks and transparency: The dataset card accompanies a technical report (released 2026-04-30) and a curated public subset used for in-house benchmarks, which helps reproduce key evaluation trends even though full model weights remain private.
Who it's for

Great fit if you are a researcher engineering multimodal reasoning pipelines or building evaluation suites that require precise spatial grounding (e.g., counting, maze/topology tasks, grounded QA). The public subset is useful for benchmarking and method prototyping.

Look elsewhere if your goal is an off-the-shelf foundation model or ready-to-run weights: the release focuses on data, methodology, and a technical report; full model weights and some internal benchmarks are planned but not included in the public release.

Where it fits

This dataset and report sit between purely linguistic reasoning corpora and full vision-model weight releases: it's most valuable when paired with an adaptable multimodal LLM or used to design token-level grounding strategies for spatial reasoning.

Information

  • Websitehuggingface.co
  • AuthorsNodeLinker, DeepSeek AI
  • Published date2026/04/30

Categories

More Items

Hugging Face

A 16 GB, 507-file PhD‑level cybersecurity knowledge base for training and evaluating security-focused LLMs and automation. Covers offensive/defensive/forensics/cloud/iot and AI-security across 30+ domains with real-world labs and framework mappings.

Hugging Face

Structured dataset for training and evaluating LLM agentic behavior: function-calling conversations, JSON-mode structured outputs, and extraction samples for teaching models to generate tool calls and strict structured responses. Includes single-turn and multi-turn scenarios across several configs.

Hugging Face

A multi-task English NLU benchmark for evaluating models across nine tasks (acceptability, sentiment, paraphrase, similarity, and various NLI setups), with a diagnostic evaluation set and an online leaderboard to compare generalization and transfer learning.