Looking Beyond the Obvious: A Survey on Abstract Concept Recognition for Video Understanding

• URL: http://arxiv.org/abs/2508.20765v1
• Date: Thu, 28 Aug 2025 13:19:49 GMT
• Title: Looking Beyond the Obvious: A Survey on Abstract Concept Recognition for Video Understanding
• Authors: Gowreesh Mago, Pascal Mettes, Stevan Rudinac,
• Abstract summary: Abstract concept recognition forms a crucial open challenge in video understanding.<n>Recent advances in foundation models make for an ideal setting to address abstract concept understanding in videos.<n>We study different tasks and datasets used to understand abstract concepts in video content.
• Score: 22.83154031059576
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The automatic understanding of video content is advancing rapidly. Empowered by deeper neural networks and large datasets, machines are increasingly capable of understanding what is concretely visible in video frames, whether it be objects, actions, events, or scenes. In comparison, humans retain a unique ability to also look beyond concrete entities and recognize abstract concepts like justice, freedom, and togetherness. Abstract concept recognition forms a crucial open challenge in video understanding, where reasoning on multiple semantic levels based on contextual information is key. In this paper, we argue that the recent advances in foundation models make for an ideal setting to address abstract concept understanding in videos. Automated understanding of high-level abstract concepts is imperative as it enables models to be more aligned with human reasoning and values. In this survey, we study different tasks and datasets used to understand abstract concepts in video content. We observe that, periodically and over a long period, researchers have attempted to solve these tasks, making the best use of the tools available at their disposal. We advocate that drawing on decades of community experience will help us shed light on this important open grand challenge and avoid ``re-inventing the wheel'' as we start revisiting it in the era of multi-modal foundation models.

Related papers

• Thinker: A vision-language foundation model for embodied intelligence [9.661713829767605]
  We propose Thinker, a large vision-language foundation model for embodied intelligence.<n>We construct a large-scale dataset tailored for robotic perception and reasoning.<n>We introduce a simple yet effective approach that substantially enhances the model's capacity for video comprehension.
  arXiv  Detail & Related papers  (2026-01-29T02:52:08Z)
• Infinite Video Understanding [50.78256932424239]
  We argue that framing Infinite Video Understanding as a blue-sky research objective provides a vital north star for the multimedia.<n>We outline the core challenges and key research directions towards achieving this transformative capability.
  arXiv  Detail & Related papers  (2025-07-11T23:07:04Z)
• Motion-Grounded Video Reasoning: Understanding and Perceiving Motion at Pixel Level [63.18855743293851]
  Motion-Grounded Video Reasoning is a new motion understanding task that requires visual answers (video segmentation masks) according to the input question.<n>This task extends existing grounding work on explicit action/motion grounding to a more general format by enabling implicit reasoning via questions.<n>We introduce a novel baseline model named Motion-Grounded Video Reasoning Assistant (MORA)
  arXiv  Detail & Related papers  (2024-11-15T03:45:09Z)
• Exploring Explainability in Video Action Recognition [5.7782784592048575]
  Video Action Recognition and Image Classification are foundational tasks in computer vision. Video-TCAV aims to quantify the importance of specific concepts in the decision-making process of Video Action Recognition models. We propose a machine-assisted approach to generate spatial andtemporal concepts relevant to Video Action Recognition for testing Video-TCAV.
  arXiv  Detail & Related papers  (2024-04-13T19:34:14Z)
• Knowledge-enhanced Multi-perspective Video Representation Learning for Scene Recognition [33.800842679024164]
  We address the problem of video scene recognition, whose goal is to learn a high-level video representation to classify scenes in videos. Most existing works identify scenes for videos only from visual or textual information in a temporal perspective. We propose a novel two-stream framework to model video representations from multiple perspectives.
  arXiv  Detail & Related papers  (2024-01-09T04:37:10Z)
• How Would The Viewer Feel? Estimating Wellbeing From Video Scenarios [73.24092762346095]
  We introduce two large-scale datasets with over 60,000 videos annotated for emotional response and subjective wellbeing. The Video Cognitive Empathy dataset contains annotations for distributions of fine-grained emotional responses, allowing models to gain a detailed understanding of affective states. The Video to Valence dataset contains annotations of relative pleasantness between videos, which enables predicting a continuous spectrum of wellbeing.
  arXiv  Detail & Related papers  (2022-10-18T17:58:25Z)
• Deep Learning to See: Towards New Foundations of Computer Vision [88.69805848302266]
  This book criticizes the supposed scientific progress in the field of computer vision. It proposes the investigation of vision within the framework of information-based laws of nature.
  arXiv  Detail & Related papers  (2022-06-30T15:20:36Z)
• Automatic Concept Extraction for Concept Bottleneck-based Video Classification [58.11884357803544]
  We present an automatic Concept Discovery and Extraction module that rigorously composes a necessary and sufficient set of concept abstractions for concept-based video classification. Our method elicits inherent complex concept abstractions in natural language to generalize concept-bottleneck methods to complex tasks.
  arXiv  Detail & Related papers  (2022-06-21T06:22:35Z)
• Object-Centric Representation Learning for Video Question Answering [27.979053252431306]
  Video answering (Video QA) presents a powerful testbed for human-like intelligent behaviors. The task demands new capabilities to integrate processing, language understanding, binding abstract concepts to concrete visual artifacts. We propose a new query-guided representation framework to turn a video into a relational graph of objects.
  arXiv  Detail & Related papers  (2021-04-12T02:37:20Z)
• Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense [142.53911271465344]
  We argue that the next generation of AI must embrace "dark" humanlike common sense for solving novel tasks. We identify functionality, physics, intent, causality, and utility (FPICU) as the five core domains of cognitive AI with humanlike common sense.
  arXiv  Detail & Related papers  (2020-04-20T04:07:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.