Video Understanding: From Geometry and Semantics to Unified Models

• URL: http://arxiv.org/abs/2603.17840v1
• Date: Wed, 18 Mar 2026 15:31:47 GMT
• Title: Video Understanding: From Geometry and Semantics to Unified Models
• Abstract summary: Video understanding aims to enable models to perceive, interact with dynamic visual world.<n>In contrast to image understanding, video understanding inherently requires modeling dynamics and evolving visual context.
• Score: 60.523344000396975
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video understanding aims to enable models to perceive, reason about, and interact with the dynamic visual world. In contrast to image understanding, video understanding inherently requires modeling temporal dynamics and evolving visual context, placing stronger demands on spatiotemporal reasoning and making it a foundational problem in computer vision. In this survey, we present a structured overview of video understanding by organizing the literature into three complementary perspectives: low-level video geometry understanding, high-level semantic understanding, and unified video understanding models. We further highlight a broader shift from isolated, task-specific pipelines toward unified modeling paradigms that can be adapted to diverse downstream objectives, enabling a more systematic view of recent progress. By consolidating these perspectives, this survey provides a coherent map of the evolving video understanding landscape, summarizes key modeling trends and design principles, and outlines open challenges toward building robust, scalable, and unified video foundation models.

Related papers

• Feed-Forward 3D Scene Modeling: A Problem-Driven Perspective [91.23306722968509]
  Generalizable feed-forward 3D reconstruction has witnessed rapid development in recent years.<n>Existing feed-forward approaches share similar high-level architectural patterns.<n>We propose a novel taxonomy centered on model design strategies that are agnostic to the output format.
  arXiv  Detail & Related papers  (2026-04-15T16:07:18Z)
• Video Understanding: Through A Temporal Lens [5.153774021264937]
  This thesis explores the central question of how to leverage temporal relations among video elements to advance video understanding.<n>The work presents a five-fold contribution: (1) an automatic annotation framework that utilizes large vision-language models and a noise-robust contrastive learning objective with a subtractive angular margin; (2) a parameter-efficient fine-tuning strategy using "recurrent adapters" to capture temporal dynamics in low-data regimes; (3) the integration of State Space Layers for efficient long-form video modeling; and (4) a novel contrastive learning framework designed to explicitly model fine-grained relations between motions and video moments.
  arXiv  Detail & Related papers  (2026-01-31T12:01:09Z)
• From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models [66.36007274540113]
  Multimodal Large Language Models (MLLMs) strive to achieve a profound, human-like understanding of and interaction with the physical world.<n>They often exhibit a shallow and incoherent integration when acquiring information (Perception) and conducting reasoning (Cognition)<n>This survey introduces a novel and unified analytical framework: From Perception to Cognition"
  arXiv  Detail & Related papers  (2025-09-29T18:25:40Z)
• Video Understanding by Design: How Datasets Shape Architectures and Insights [47.846604113207206]
  Video understanding has advanced rapidly, fueled by increasingly complex datasets and powerful architectures.<n>This survey is the first to adopt a dataset-driven perspective, showing how motion complexity, temporal span, hierarchical composition, and multimodal richness impose inductive biases that models should encode.
  arXiv  Detail & Related papers  (2025-09-11T05:06:30Z)
• Spatial Understanding from Videos: Structured Prompts Meet Simulation Data [89.77871049500546]
  We present a unified framework for enhancing 3D spatial reasoning in pre-trained vision-language models without modifying their architecture.<n>This framework combines SpatialMind, a structured prompting strategy that decomposes complex scenes and questions into interpretable reasoning steps, with ScanForgeQA, a scalable question-answering dataset built from diverse 3D simulation scenes.
  arXiv  Detail & Related papers  (2025-06-04T07:36:33Z)
• Foundational Models Defining a New Era in Vision: A Survey and Outlook [151.49434496615427]
  Vision systems to see and reason about the compositional nature of visual scenes are fundamental to understanding our world. The models learned to bridge the gap between such modalities coupled with large-scale training data facilitate contextual reasoning, generalization, and prompt capabilities at test time. The output of such models can be modified through human-provided prompts without retraining, e.g., segmenting a particular object by providing a bounding box, having interactive dialogues by asking questions about an image or video scene or manipulating the robot's behavior through language instructions.
  arXiv  Detail & Related papers  (2023-07-25T17:59:18Z)
• Learning Structured Representations of Visual Scenes [1.6244541005112747]
  We study how machines can describe the content of the individual image or video with visual relationships as the structured representations. Specifically, we explore how structured representations of visual scenes can be effectively constructed and learned in both the static-image and video settings.
  arXiv  Detail & Related papers  (2022-07-09T05:40:08Z)
• Unified Graph Structured Models for Video Understanding [93.72081456202672]
  We propose a message passing graph neural network that explicitly models relational-temporal relations. We show how our method is able to more effectively model relationships between relevant entities in the scene.
  arXiv  Detail & Related papers  (2021-03-29T14:37:35Z)
• Video Action Understanding [6.04698204230499]
  This tutorial introduces and systematizes fundamental topics, basic concepts, and notable examples in supervised video action understanding. Specifically, we clarify a taxonomy of action problems, catalog and highlight video datasets, describe common video data preparation methods, and formalize domain-specific metrics to baseline proposed solutions.
  arXiv  Detail & Related papers  (2020-10-13T19:29:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.