Spatial-Temporal Perception with Causal Inference for Naturalistic Driving Action Recognition

• URL: http://arxiv.org/abs/2503.04078v1
• Date: Thu, 06 Mar 2025 04:28:11 GMT
• Title: Spatial-Temporal Perception with Causal Inference for Naturalistic Driving Action Recognition
• Authors: Qing Chang, Wei Dai, Zhihao Shuai, Limin Yu, Yutao Yue,
• Abstract summary: Naturalistic driving action recognition is essential for vehicle cabin monitoring systems.<n>Previous approaches have struggled with practical implementation due to their limited ability to observe subtle behavioral differences.<n>We propose a novel Spatial-Temporal Perception architecture that emphasizes both temporal information and spatial relationships.
• Score: 6.115044825582411
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Naturalistic driving action recognition is essential for vehicle cabin monitoring systems. However, the complexity of real-world backgrounds presents significant challenges for this task, and previous approaches have struggled with practical implementation due to their limited ability to observe subtle behavioral differences and effectively learn inter-frame features from video. In this paper, we propose a novel Spatial-Temporal Perception (STP) architecture that emphasizes both temporal information and spatial relationships between key objects, incorporating a causal decoder to perform behavior recognition and temporal action localization. Without requiring multimodal input, STP directly extracts temporal and spatial distance features from RGB video clips. Subsequently, these dual features are jointly encoded by maximizing the expected likelihood across all possible permutations of the factorization order. By integrating temporal and spatial features at different scales, STP can perceive subtle behavioral changes in challenging scenarios. Additionally, we introduce a causal-aware module to explore relationships between video frame features, significantly enhancing detection efficiency and performance. We validate the effectiveness of our approach using two publicly available driver distraction detection benchmarks. The results demonstrate that our framework achieves state-of-the-art performance.

Related papers

• Event-Based Tracking Any Point with Motion-Augmented Temporal Consistency [58.719310295870024]
  This paper presents an event-based framework for tracking any point.<n>It tackles the challenges posed by spatial sparsity and motion sensitivity in events.<n>It achieves 150% faster processing with competitive model parameters.
  arXiv  Detail & Related papers  (2024-12-02T09:13:29Z)
• DeepInteraction++: Multi-Modality Interaction for Autonomous Driving [80.8837864849534]
  We introduce a novel modality interaction strategy that allows individual per-modality representations to be learned and maintained throughout. DeepInteraction++ is a multi-modal interaction framework characterized by a multi-modal representational interaction encoder and a multi-modal predictive interaction decoder. Experiments demonstrate the superior performance of the proposed framework on both 3D object detection and end-to-end autonomous driving tasks.
  arXiv  Detail & Related papers  (2024-08-09T14:04:21Z)
• Harnessing Temporal Causality for Advanced Temporal Action Detection [53.654457142657236]
  We introduce CausalTAD, which combines causal attention and causal Mamba to achieve state-of-the-art performance on benchmarks. We ranked 1st in the Action Recognition, Action Detection, and Audio-Based Interaction Detection tracks at the EPIC-Kitchens Challenge 2024, and 1st in the Moment Queries track at the Ego4D Challenge 2024.
  arXiv  Detail & Related papers  (2024-07-25T06:03:02Z)
• Fine-grained Dynamic Network for Generic Event Boundary Detection [9.17191007695011]
  We propose a novel dynamic pipeline for generic event boundaries named DyBDet. By introducing a multi-exit network architecture, DyBDet automatically learns the allocation to different video snippets. Experiments on the challenging Kinetics-GEBD and TAPOS datasets demonstrate that adopting the dynamic strategy significantly benefits GEBD tasks.
  arXiv  Detail & Related papers  (2024-07-05T06:02:46Z)
• Intrinsic Dynamics-Driven Generalizable Scene Representations for Vision-Oriented Decision-Making Applications [0.21051221444478305]
  How to improve the ability of scene representation is a key issue in vision-oriented decision-making applications. We propose an intrinsic dynamics-driven representation learning method with sequence models in visual reinforcement learning.
  arXiv  Detail & Related papers  (2024-05-30T06:31:03Z)
• Generalizing Event-Based Motion Deblurring in Real-World Scenarios [62.995994797897424]
  Event-based motion deblurring has shown promising results by exploiting low-latency events. We propose a scale-aware network that allows flexible input spatial scales and enables learning from different temporal scales of motion blur. A two-stage self-supervised learning scheme is then developed to fit real-world data distribution.
  arXiv  Detail & Related papers  (2023-08-11T04:27:29Z)
• Two-stream Multi-level Dynamic Point Transformer for Two-person Interaction Recognition [45.0131792009999]
  We propose a point cloud-based network named Two-stream Multi-level Dynamic Point Transformer for two-person interaction recognition. Our model addresses the challenge of recognizing two-person interactions by incorporating local-region spatial information, appearance information, and motion information. Our network outperforms state-of-the-art approaches in most standard evaluation settings.
  arXiv  Detail & Related papers  (2023-07-22T03:51:32Z)
• Spatiotemporal Multi-scale Bilateral Motion Network for Gait Recognition [3.1240043488226967]
  In this paper, motivated by optical flow, the bilateral motion-oriented features are proposed. We develop a set of multi-scale temporal representations that force the motion context to be richly described at various levels of temporal resolution.
  arXiv  Detail & Related papers  (2022-09-26T01:36:22Z)
• Learning Appearance-motion Normality for Video Anomaly Detection [11.658792932975652]
  We propose spatial-temporal memories augmented two-stream auto-encoder framework. It learns the appearance normality and motion normality independently and explores the correlations via adversarial learning. Our framework outperforms the state-of-the-art methods, achieving AUCs of 98.1% and 89.8% on UCSD Ped2 and CUHK Avenue datasets.
  arXiv  Detail & Related papers  (2022-07-27T08:30:19Z)
• Confidence-guided Adaptive Gate and Dual Differential Enhancement for Video Salient Object Detection [47.68968739917077]
  Video salient object detection (VSOD) aims to locate and segment the most attractive object by exploiting both spatial cues and temporal cues hidden in video sequences. We propose a new framework to adaptively capture available information from spatial and temporal cues, which contains Confidence-guided Adaptive Gate (CAG) modules and Dual Differential Enhancement (DDE) modules.
  arXiv  Detail & Related papers  (2021-05-14T08:49:37Z)
• Unsupervised Feature Learning for Event Data: Direct vs Inverse Problem Formulation [53.850686395708905]
  Event-based cameras record an asynchronous stream of per-pixel brightness changes. In this paper, we focus on single-layer architectures for representation learning from event data. We show improvements of up to 9 % in the recognition accuracy compared to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-09-23T10:40:03Z)
• A Spatial-Temporal Attentive Network with Spatial Continuity for Trajectory Prediction [74.00750936752418]
  We propose a novel model named spatial-temporal attentive network with spatial continuity (STAN-SC) First, spatial-temporal attention mechanism is presented to explore the most useful and important information. Second, we conduct a joint feature sequence based on the sequence and instant state information to make the generative trajectories keep spatial continuity.
  arXiv  Detail & Related papers  (2020-03-13T04:35:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.