VideoJev reframes video understanding as a sequence of structured decisions a downstream agent can use directly, rather than free-form text generation. The core insight is that replacing autoregressive answer decoding with a shared head that scores temporal intervals and textual options yields decisions (event spans or choices) that are both semantically rich and computationally efficient—so the same visual encoding can serve many subsequent queries without recomputation.
Key Findings
- Decision-oriented readout: Adapting pretrained multimodal LLMs to produce interval and option scores unifies temporal grounding and multiple-choice video QA under a single decision interface, avoiding the need for natural-language generation when callers need typed answers.
- Reuse of visual work: Cached inference for incremental queries and joint inference for batches of queries amortize vision costs; this yields measured speedups of ~2.8–3.1× for temporal grounding and over 100× for multi-query video QA workloads.
- Empirical gains: On three temporal-grounding benchmarks, VideoJev-4B and VideoJev-35B-A3B improved mean mIoU over their backbones by 6.7 and 10.8 points respectively, while retaining competitive QA accuracy in a Thinking-mode evaluation.
Who it's for and tradeoffs
Great fit if you build interactive video agents, multimodal workflows, or systems that ask many constrained questions about the same visual history and need calibrated, typed answers rather than narrative explanations. Look elsewhere if your task requires open-ended video narration, dense pixel-level outputs (e.g., segmentation masks), or generative explanations for end users—VideoJev prioritizes decision efficiency and reusability over free-form text generation. The design also depends on strong pretrained MLLMs and an engineered decision head; integration requires adapting model readouts and serving strategies (caching, joint batching).