A Two-stage Transformer Framework for Temporal Localization of Distracted Driver Behaviors

• URL: http://arxiv.org/abs/2603.21048v1
• Date: Sun, 22 Mar 2026 04:09:47 GMT
• Title: A Two-stage Transformer Framework for Temporal Localization of Distracted Driver Behaviors
• Abstract summary: We develop and evaluate a temporal action localization framework tailored for driver monitoring scenarios.<n>Our approach follows a two-stage pipeline that combines VideoMAE-based feature extraction with an Augmented Self-Mask Attention (AMA) detector.<n> Experimental results reveal a distinct trade-off between model capacity and efficiency.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The identification of hazardous driving behaviors from in-cabin video streams is essential for enhancing road safety and supporting the detection of traffic violations and unsafe driver actions. However, current temporal action localization techniques often struggle to balance accuracy with computational efficiency. In this work, we develop and evaluate a temporal action localization framework tailored for driver monitoring scenarios, particularly suitable for periodic inspection settings such as transportation safety checkpoints or fleet management assessment systems. Our approach follows a two-stage pipeline that combines VideoMAE-based feature extraction with an Augmented Self-Mask Attention (AMA) detector, enhanced by a Spatial Pyramid Pooling-Fast (SPPF) module to capture multi-scale temporal features. Experimental results reveal a distinct trade-off between model capacity and efficiency. At the feature extraction stage, the ViT-Giant backbone delivers higher representations with 88.09% Top-1 test accuracy, while the ViT-based variant proves to be a practical alternative, achieving 82.55% accuracy with significantly lower computational fine-tuning costs (101.85 GFLOPs/segment compared to 1584.06 GFLOPs/segment for Giant). In the downstream localization task, the integration of SPPF consistently improves performance across all configurations. Notably, the ViT-Giant + SPPF model achieves a peak mAP of 92.67%, while the lightweight ViT-based configuration maintains robust results.

Related papers

• Knowledge-Distilled End-to-End Reinforcement Learning for Smooth 6-DOF Thrust Control and Rapid Adaptation to Ocean Currents in Remotely Operated Vehicles [16.40999985739862]
  This paper proposes the thrust rapid current adaptation policy optimization (TSRCA-PPO) method which learns a near-optimal strategy by a twostage distillation learning framework.<n> Simulation results demonstrate that the proposed TSRCA-PPO method consistently outperforms the conventional cascaded P-PID controller.
  arXiv  Detail & Related papers  (2026-08-09T09:23:15Z)
• FlowCTS: On-policy Continuous Trajectory Supervision of Flow Models [65.32587991232592]
  We propose Flow Continuous Trajectory Supervision (FlowCTS)<n>FlowCTS matches subsequent student and reference trajectories from the same student-visited state.<n>Under a multi-reference setup, single-state FlowCTS-OPD outperforms vanilla KL-based OPD with faster convergence.
  arXiv  Detail & Related papers  (2026-07-27T15:03:22Z)
• TubeLite: Lightweight Multi-Actor Spatio-Temporal Action Detection [74.20082571588867]
  TubeLite is a lightweight framework for temporal-temporal action detection in videos.<n>It focuses on stable tube construction and boundary-aware temporal modeling.<n>It achieves strong video-level localization performance despite its compact design.
  arXiv  Detail & Related papers  (2026-07-06T05:12:30Z)
• Attention-Augmented YOLOv8 with Ghost Convolution for Real-Time Vehicle Detection in Intelligent Transportation Systems [11.507914098454421]
  This paper presents an enhanced YOLOv8n-based model that integrates the Ghost Module, Convolutional Block Attention Module, and Deformable Convolutional Networks v2.<n>The proposed model achieves an 8.97% improvement over the baseline YOLOv8n, along with 96.2% precision, 93.7% recall, and a 94.93% F1-score.
  arXiv  Detail & Related papers  (2026-04-22T14:09:27Z)
• ToaSt: Token Channel Selection and Structured Pruning for Efficient ViT [14.21482208417138]
  Vision Transformers (ViTs) have achieved remarkable success across various vision tasks, yet their deployment is often hindered by prohibitive computational costs.<n>We propose ToaSt, a decoupled framework applying specialized strategies to distinct ViT components.
  arXiv  Detail & Related papers  (2026-02-17T16:52:13Z)
• Spatially-Aware Adaptive Trajectory Optimization with Controller-Guided Feedback for Autonomous Racing [74.83272587893508]
  We present a framework for autonomous raceline optimization that combines NURBS-based trajectory representation, CMA-ES global trajectory optimization, and controller-guided spatial feedback.<n>In simulation, our approach achieves a 17.38% lap time reduction compared to a controller parametrized with maximum static acceleration.<n>On real hardware, tested with different tire compounds ranging from high to low friction, we obtain a 7.60% lap time improvement without explicitly parametrizing friction.
  arXiv  Detail & Related papers  (2026-02-17T15:10:44Z)
• HPTune: Hierarchical Proactive Tuning for Collision-Free Model Predictive Control [55.015741360766945]
  We propose a hierarchical proactive tuning (HPTune) framework that combines both a fast-level tuning and a slow-level tuning.<n>We integrate HPTune with the Doppler LiDAR that provides obstacle velocities apart from position-only measurements for enhanced motion predictions.<n>It is found that HPTune enables situation-tailored motion planning by formulating a safe, agile collision avoidance strategy.
  arXiv  Detail & Related papers  (2026-01-29T07:15:39Z)
• Efficient On-Board Processing of Oblique UAV Video for Rapid Flood Extent Mapping [7.460695517551536]
  Temporal Token Reuse (TTR) is an adaptive inference framework capable of accelerating video segmentation on embedded devices.<n>We show that TTR achieves a 30% reduction in inference latency with negligible degradation in segmentation accuracy ( 0.5% mIoU)<n>These findings confirm that TTR effectively shifts the operational frontier, enabling high-fidelity, real-time oblique video understanding for time-critical remote sensing missions.
  arXiv  Detail & Related papers  (2026-01-16T13:41:56Z)
• RMT-PPAD: Real-time Multi-task Learning for Panoptic Perception in Autonomous Driving [18.945598464194607]
  RMT-PPAD is a real-time, transformer-based multi-task model.<n>It jointly performs object detection, drivable area segmentation, and lane line segmentation.<n>The results show that RMT-PPAD consistently delivers stable performance.
  arXiv  Detail & Related papers  (2025-08-02T16:34:24Z)
• A Self-Supervised Transformer for Unusable Shared Bike Detection [1.3948892744961128]
  This paper proposes a novel Self-Supervised Transformer (SSTransformer) framework for automatically detecting unusable shared bikes.<n>The model incorporates a self-supervised pre-training strategy to enhance its feature extraction capabilities, followed by fine-tuning for efficient status recognition.<n>In experiments on a real-world dataset of 10,730 bikes, SSTransformer significantly outperforms traditional machine learning, ensemble learning, and deep learning baselines.
  arXiv  Detail & Related papers  (2025-05-02T00:20:38Z)
• Fast-COS: A Fast One-Stage Object Detector Based on Reparameterized Attention Vision Transformer for Autonomous Driving [3.617580194719686]
  This paper introduces Fast-COS, a novel single-stage object detection framework crafted specifically for driving scenes.<n> RAViT achieves 81.4% Top-1 accuracy on the ImageNet-1K dataset.<n>It surpasses leading models in efficiency, delivering up to 75.9% faster GPU inference and 1.38 higher throughput on edge devices.
  arXiv  Detail & Related papers  (2025-02-11T09:54:09Z)
• MATE: Motion-Augmented Temporal Consistency for Event-based Point Tracking [58.719310295870024]
  This paper presents an event-based framework for tracking any point.<n>To resolve ambiguities caused by event sparsity, a motion-guidance module incorporates kinematic vectors into the local matching process.<n>The method improves the $Survival_50$ metric by 17.9% over event-only tracking of any point baseline.
  arXiv  Detail & Related papers  (2024-12-02T09:13:29Z)
• StreamYOLO: Real-time Object Detection for Streaming Perception [84.2559631820007]
  We endow the models with the capacity of predicting the future, significantly improving the results for streaming perception. We consider multiple velocities driving scene and propose Velocity-awared streaming AP (VsAP) to jointly evaluate the accuracy. Our simple method achieves the state-of-the-art performance on Argoverse-HD dataset and improves the sAP and VsAP by 4.7% and 8.2% respectively.
  arXiv  Detail & Related papers  (2022-07-21T12:03:02Z)
• Finding Action Tubes with a Sparse-to-Dense Framework [62.60742627484788]
  We propose a framework that generates action tube proposals from video streams with a single forward pass in a sparse-to-dense manner. We evaluate the efficacy of our model on the UCF101-24, JHMDB-21 and UCFSports benchmark datasets.
  arXiv  Detail & Related papers  (2020-08-30T15:38:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.