Deconstruct Complexity (DeComplex): A Novel Perspective on Tackling Dense Action Detection

• URL: http://arxiv.org/abs/2501.18509v1
• Date: Thu, 30 Jan 2025 17:20:42 GMT
• Title: Deconstruct Complexity (DeComplex): A Novel Perspective on Tackling Dense Action Detection
• Authors: Faegheh Sardari, Armin Mustafa, Philip J. B. Jackson, Adrian Hilton,
• Abstract summary: We introduce a novel perspective inspired by how humans tackle complex tasks by breaking them into manageable sub-tasks. Instead of relying on a single network to address the entire problem, we propose decomposing the problem into detecting key concepts present in action classes. Our experiments demonstrate the superiority of our approach over state-of-the-art methods.
• Score: 23.100602876056165
• License:
• Abstract: Dense action detection involves detecting multiple co-occurring actions in an untrimmed video while action classes are often ambiguous and represent overlapping concepts. To address this challenge task, we introduce a novel perspective inspired by how humans tackle complex tasks by breaking them into manageable sub-tasks. Instead of relying on a single network to address the entire problem, as in current approaches, we propose decomposing the problem into detecting key concepts present in action classes, specifically, detecting dense static concepts and detecting dense dynamic concepts, and assigning them to distinct, specialized networks. Furthermore, simultaneous actions in a video often exhibit interrelationships, and exploiting these relationships can improve performance. However, we argue that current networks fail to effectively learn these relationships due to their reliance on binary cross-entropy optimization, which treats each class independently. To address this limitation, we propose providing explicit supervision on co-occurring concepts during network optimization through a novel language-guided contrastive learning loss. Our extensive experiments demonstrate the superiority of our approach over state-of-the-art methods, achieving substantial relative improvements of 23.4% and 2.5% mAP on the challenging benchmark datasets, Charades and MultiTHUMOS.

Related papers

• Low-rank Prompt Interaction for Continual Vision-Language Retrieval [47.323830129786145]
  We propose the Low-rank Prompt Interaction to address the problem of multi-modal understanding. Considering that the training parameters scale to the number of layers and tasks, we propose low-rank interaction-augmented decomposition. We also adopt hierarchical low-rank contrastive learning to ensure robustness training.
  arXiv  Detail & Related papers  (2025-01-24T10:00:47Z)
• An Effective-Efficient Approach for Dense Multi-Label Action Detection [23.100602876056165]
  It is necessary to simultaneously learn (i) temporal dependencies and (ii) co-occurrence action relationships. Recent approaches model temporal information by extracting multi-scale features through hierarchical transformer-based networks. We argue that combining this with multiple sub-sampling processes in hierarchical designs can lead to further loss of positional information.
  arXiv  Detail & Related papers  (2024-06-10T11:33:34Z)
• Learning Transferable Adversarial Robust Representations via Multi-view Consistency [57.73073964318167]
  We propose a novel meta-adversarial multi-view representation learning framework with dual encoders. We demonstrate the effectiveness of our framework on few-shot learning tasks from unseen domains.
  arXiv  Detail & Related papers  (2022-10-19T11:48:01Z)
• Learning Dynamics and Generalization in Reinforcement Learning [59.530058000689884]
  We show theoretically that temporal difference learning encourages agents to fit non-smooth components of the value function early in training. We show that neural networks trained using temporal difference algorithms on dense reward tasks exhibit weaker generalization between states than randomly networks and gradient networks trained with policy methods.
  arXiv  Detail & Related papers  (2022-06-05T08:49:16Z)
• Weakly-Supervised Spatio-Temporal Anomaly Detection in Surveillance Video [128.41392860714635]
  We introduce Weakly-Supervised Snoma-Temporally Detection (WSSTAD) in surveillance video. WSSTAD aims to localize a-temporal tube (i.e. sequence of bounding boxes at consecutive times) that encloses abnormal event. We propose a dual-branch network which takes as input proposals with multi-granularities in both spatial-temporal domains.
  arXiv  Detail & Related papers  (2021-08-09T06:11:14Z)
• Cross-modal Consensus Network for Weakly Supervised Temporal Action Localization [74.34699679568818]
  Weakly supervised temporal action localization (WS-TAL) is a challenging task that aims to localize action instances in the given video with video-level categorical supervision. We propose a cross-modal consensus network (CO2-Net) to tackle this problem.
  arXiv  Detail & Related papers  (2021-07-27T04:21:01Z)
• Learning to Relate Depth and Semantics for Unsupervised Domain Adaptation [87.1188556802942]
  We present an approach for encoding visual task relationships to improve model performance in an Unsupervised Domain Adaptation (UDA) setting. We propose a novel Cross-Task Relation Layer (CTRL), which encodes task dependencies between the semantic and depth predictions. Furthermore, we propose an Iterative Self-Learning (ISL) training scheme, which exploits semantic pseudo-labels to provide extra supervision on the target domain.
  arXiv  Detail & Related papers  (2021-05-17T13:42:09Z)
• Multi-object Tracking with a Hierarchical Single-branch Network [31.680667324595557]
  We propose an online multi-object tracking framework based on a hierarchical single-branch network. Our novel iHOIM loss function unifies the objectives of the two sub-tasks and encourages better detection performance. Experimental results on MOT16 and MOT20 datasets show that we can achieve state-of-the-art tracking performance.
  arXiv  Detail & Related papers  (2021-01-06T12:14:58Z)
• Cascaded Human-Object Interaction Recognition [175.60439054047043]
  We introduce a cascade architecture for a multi-stage, coarse-to-fine HOI understanding. At each stage, an instance localization network progressively refines HOI proposals and feeds them into an interaction recognition network. With our carefully-designed human-centric relation features, these two modules work collaboratively towards effective interaction understanding.
  arXiv  Detail & Related papers  (2020-03-09T17:05:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.