ACIT: Attention-Guided Cross-Modal Interaction Transformer for Pedestrian Crossing Intention Prediction

• URL: http://arxiv.org/abs/2511.20020v1
• Date: Tue, 25 Nov 2025 07:41:11 GMT
• Title: ACIT: Attention-Guided Cross-Modal Interaction Transformer for Pedestrian Crossing Intention Prediction
• Abstract summary: This paper proposes an attention-guided cross-modal interaction Transformer (ACIT) for pedestrian crossing intention prediction.<n>ACIT leverages six visual and motion modalities, which are grouped into three interaction pairs.<n>ACIT outperforms state-of-the-art methods, achieving accuracy rates of 70% and 89% on the JAADbeh and JAADall datasets.
• Score: 3.878105750489656
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Predicting pedestrian crossing intention is crucial for autonomous vehicles to prevent pedestrian-related collisions. However, effectively extracting and integrating complementary cues from different types of data remains one of the major challenges. This paper proposes an attention-guided cross-modal interaction Transformer (ACIT) for pedestrian crossing intention prediction. ACIT leverages six visual and motion modalities, which are grouped into three interaction pairs: (1) Global semantic map and global optical flow, (2) Local RGB image and local optical flow, and (3) Ego-vehicle speed and pedestrian's bounding box. Within each visual interaction pair, a dual-path attention mechanism enhances salient regions within the primary modality through intra-modal self-attention and facilitates deep interactions with the auxiliary modality (i.e., optical flow) via optical flow-guided attention. Within the motion interaction pair, cross-modal attention is employed to model the cross-modal dynamics, enabling the effective extraction of complementary motion features. Beyond pairwise interactions, a multi-modal feature fusion module further facilitates cross-modal interactions at each time step. Furthermore, a Transformer-based temporal feature aggregation module is introduced to capture sequential dependencies. Experimental results demonstrate that ACIT outperforms state-of-the-art methods, achieving accuracy rates of 70% and 89% on the JAADbeh and JAADall datasets, respectively. Extensive ablation studies are further conducted to investigate the contribution of different modules of ACIT.

Related papers

• Adaptive Cross-Modal Fusion with Sparse Attention for Pedestrian Crossing Intention Prediction [4.8915598762204135]
  ADAPT (Adaptive Domain-Aware Pedestrian Crossing Transformer) is a multimodal framework for predicting pedestrian crossing intention.<n> ADAPT processes four aligned visual modalities, including RGB images, local depth maps, global semantic maps, and global depth maps.<n>Experiments on the JAAD and PIE benchmark datasets demonstrate that ADAPT consistently outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-07-14T03:05:09Z)
• Dual-Interaction-Aware Cooperative Control Strategy for Alleviating Mixed Traffic Congestion [36.780597614300504]
  Connected and Automated Vehicles (CAVs) are expected to significantly reduce traffic congestion through cooperative strategies.<n>Uncertainty and diversity in the behaviors of Human-Adaptive Human-Driven Vehicles (HDVs) in mixed traffic environments present major challenges for CAV cooperation.<n>This paper proposes a Dual-Interaction-Aware Cooperative Control (DIACC) strategy that enhances both local and global interaction perception.
  arXiv  Detail & Related papers  (2026-03-04T09:00:49Z)
• Residual Cross-Modal Fusion Networks for Audio-Visual Navigation [17.19858148800535]
  We propose a Cross-Modal Residual Fusion Network, which introduces residual interactions between audio and visual streams to achieve complementary modeling and fine-grained alignment.<n> Experiments on the Replica and Matterport3D datasets demonstrate that CRFN significantly outperforms state-of-the-art fusion baselines and achieves stronger cross-domain generalization.
  arXiv  Detail & Related papers  (2026-01-11T12:11:36Z)
• Pedestrian Crossing Intention Prediction Using Multimodal Fusion Network [3.878105750489656]
  Pedestrian crossing intention prediction is essential for the deployment of autonomous vehicles (AVs) in urban environments.<n>This paper proposes a multimodal fusion network that leverages seven modality features from both visual and motion branches.<n>Experiments on the JAAD dataset validate the effectiveness of the proposed network, achieving superior performance compared to the baseline methods.
  arXiv  Detail & Related papers  (2025-11-25T07:18:12Z)
• BIDA: A Bi-level Interaction Decision-making Algorithm for Autonomous Vehicles in Dynamic Traffic Scenarios [5.193590097161461]
  We design a bi-level interaction decision-making algorithm (BIDA) that integrates interactive Monte Carlo tree search (MCTS) with deep reinforcement learning (DRL)<n>Specifically, we adopt three types of DRL algorithms to construct a reliable value network and policy network, which guide the online deduction process of interactive MCTS.<n> Experimental evaluations demonstrate that our BIDA not only enhances interactive deduction and reduces computational costs, but also outperforms other latest benchmarks.
  arXiv  Detail & Related papers  (2025-06-19T19:03:40Z)
• Wi-CBR: Salient-aware Adaptive WiFi Sensing for Cross-domain Behavior Recognition [8.028748052177146]
  We propose a novel Salient-aware Adaptive WiFi Sensing for Cross-domain Behavior Recognition (Wi-CBR)<n>Specifically, we propose a novel Salient-aware Adaptive WiFi Sensing for Cross-domain Behavior Recognition (Wi-CBR)
  arXiv  Detail & Related papers  (2025-06-13T09:38:57Z)
• Unicorn: A Universal and Collaborative Reinforcement Learning Approach Towards Generalizable Network-Wide Traffic Signal Control [13.106167353085878]
  Adaptive traffic signal control (ATSC) is crucial in reducing congestion, maximizing throughput, and improving mobility in rapidly growing urban areas.<n>Recent advancements in parameter-sharing multi-agent reinforcement learning (MARL) have greatly enhanced the scalable and adaptive optimization of complex, dynamic flows in large-scale homogeneous networks.<n>We present Unicorn, a universal and collaborative MARL framework designed for efficient and adaptable network-wide ATSC.
  arXiv  Detail & Related papers  (2025-03-14T15:13:42Z)
• DeepInteraction++: Multi-Modality Interaction for Autonomous Driving [80.8837864849534]
  We introduce a novel modality interaction strategy that allows individual per-modality representations to be learned and maintained throughout.<n>DeepInteraction++ is a multi-modal interaction framework characterized by a multi-modal representational interaction encoder and a multi-modal predictive interaction decoder.<n>Experiments demonstrate the superior performance of the proposed framework on both 3D object detection and end-to-end autonomous driving tasks.
  arXiv  Detail & Related papers  (2024-08-09T14:04:21Z)
• ProIn: Learning to Predict Trajectory Based on Progressive Interactions for Autonomous Driving [11.887346755144485]
  A progressive interaction network is proposed to enable the agent's feature to progressively focus on relevant maps. The network progressively encodes the complex influence of map constraints into the agent's feature through graph convolutions. Experiments have validated the superiority of progressive interactions to the existing one-stage interaction.
  arXiv  Detail & Related papers  (2024-03-25T02:38:34Z)
• Interactive Autonomous Navigation with Internal State Inference and Interactivity Estimation [58.21683603243387]
  We propose three auxiliary tasks with relational-temporal reasoning and integrate them into the standard Deep Learning framework. These auxiliary tasks provide additional supervision signals to infer the behavior patterns other interactive agents. Our approach achieves robust and state-of-the-art performance in terms of standard evaluation metrics.
  arXiv  Detail & Related papers  (2023-11-27T18:57:42Z)
• Interaction Transformer for Human Reaction Generation [61.22481606720487]
  We propose a novel interaction Transformer (InterFormer) consisting of a Transformer network with both temporal and spatial attentions. Our method is general and can be used to generate more complex and long-term interactions.
  arXiv  Detail & Related papers  (2022-07-04T19:30:41Z)
• Group Gated Fusion on Attention-based Bidirectional Alignment for Multimodal Emotion Recognition [63.07844685982738]
  This paper presents a new model named as Gated Bidirectional Alignment Network (GBAN), which consists of an attention-based bidirectional alignment network over LSTM hidden states. We empirically show that the attention-aligned representations outperform the last-hidden-states of LSTM significantly. The proposed GBAN model outperforms existing state-of-the-art multimodal approaches on the IEMOCAP dataset.
  arXiv  Detail & Related papers  (2022-01-17T09:46:59Z)
• Cross-modal Consensus Network for Weakly Supervised Temporal Action Localization [74.34699679568818]
  Weakly supervised temporal action localization (WS-TAL) is a challenging task that aims to localize action instances in the given video with video-level categorical supervision. We propose a cross-modal consensus network (CO2-Net) to tackle this problem.
  arXiv  Detail & Related papers  (2021-07-27T04:21:01Z)
• Cascaded Human-Object Interaction Recognition [175.60439054047043]
  We introduce a cascade architecture for a multi-stage, coarse-to-fine HOI understanding. At each stage, an instance localization network progressively refines HOI proposals and feeds them into an interaction recognition network. With our carefully-designed human-centric relation features, these two modules work collaboratively towards effective interaction understanding.
  arXiv  Detail & Related papers  (2020-03-09T17:05:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.