DiGIT: Multi-Dilated Gated Encoder and Central-Adjacent Region Integrated Decoder for Temporal Action Detection Transformer

• URL: http://arxiv.org/abs/2505.05711v1
• Date: Fri, 09 May 2025 01:17:30 GMT
• Title: DiGIT: Multi-Dilated Gated Encoder and Central-Adjacent Region Integrated Decoder for Temporal Action Detection Transformer
• Authors: Ho-Joong Kim, Yearang Lee, Jung-Ho Hong, Seong-Whan Lee,
• Abstract summary: Key limitation in query-based detectors for temporal action detection arises from direct adaptation of originally designed architectures for object detection.<n>We propose a multi-dilated gated encoder and central-adjacent region integrated decoder for temporal action detection transformer (DiGIT)<n>Our approach replaces the existing encoder that consists of multi-scale deformable attention and feedforward network with our multi-dilated gated encoder.
• Score: 25.180317527112372
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we examine a key limitation in query-based detectors for temporal action detection (TAD), which arises from their direct adaptation of originally designed architectures for object detection. Despite the effectiveness of the existing models, they struggle to fully address the unique challenges of TAD, such as the redundancy in multi-scale features and the limited ability to capture sufficient temporal context. To address these issues, we propose a multi-dilated gated encoder and central-adjacent region integrated decoder for temporal action detection transformer (DiGIT). Our approach replaces the existing encoder that consists of multi-scale deformable attention and feedforward network with our multi-dilated gated encoder. Our proposed encoder reduces the redundant information caused by multi-level features while maintaining the ability to capture fine-grained and long-range temporal information. Furthermore, we introduce a central-adjacent region integrated decoder that leverages a more comprehensive sampling strategy for deformable cross-attention to capture the essential information. Extensive experiments demonstrate that DiGIT achieves state-of-the-art performance on THUMOS14, ActivityNet v1.3, and HACS-Segment. Code is available at: https://github.com/Dotori-HJ/DiGIT

Related papers

• AuxDet: Auxiliary Metadata Matters for Omni-Domain Infrared Small Target Detection [58.67129770371016]
  We propose a novel IRSTD framework that reimagines the IRSTD paradigm by incorporating textual metadata for scene-aware optimization.<n>AuxDet consistently outperforms state-of-the-art methods, validating the critical role of auxiliary information in improving robustness and accuracy.
  arXiv  Detail & Related papers  (2025-05-21T07:02:05Z)
• Col-OLHTR: A Novel Framework for Multimodal Online Handwritten Text Recognition [82.88856416080331]
  Online Handwritten Text Recognition (OLHTR) has gained considerable attention for its diverse range of applications.<n>Current approaches usually treat OLHTR as a sequence recognition task, employing either a single trajectory or image encoder, or multi-stream encoders.<n>We propose a Collaborative learning-based OLHTR framework, called Col-OLHTR, that learns multimodal features during training while maintaining a single-stream inference process.
  arXiv  Detail & Related papers  (2025-02-10T02:12:24Z)
• PCA-Featured Transformer for Jamming Detection in 5G UAV Networks [0.5999777817331317]
  Unmanned Aerial Vehicles (UAVs) face significant security risks from jamming attacks, which can compromise network functionality.<n>Traditional detection methods often fall short when confronting AI-powered jamming that dynamically modifies its behavior.<n>We introduce a novel U-shaped transformer architecture to refine feature representations for improved wireless security.
  arXiv  Detail & Related papers  (2024-12-19T16:13:04Z)
• EMDFNet: Efficient Multi-scale and Diverse Feature Network for Traffic Sign Detection [11.525603303355268]
  The detection of small objects, particularly traffic signs, is a critical subtask within object detection and autonomous driving. Motivated by these challenges, we propose a novel object detection network named Efficient Multi-scale and Diverse Feature Network (EMDFNet) EMDFNet integrates an Augmented Shortcut Module and an Efficient Hybrid to address the aforementioned issues simultaneously.
  arXiv  Detail & Related papers  (2024-08-26T11:26:27Z)
• LW-DETR: A Transformer Replacement to YOLO for Real-Time Detection [63.780355815743135]
  We present a light-weight detection transformer, LW-DETR, which outperforms YOLOs for real-time object detection. The architecture is a simple stack of a ViT encoder, a projector, and a shallow DETR decoder.
  arXiv  Detail & Related papers  (2024-06-05T17:07:24Z)
• Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inference [95.42299246592756]
  We study the UNet encoder and empirically analyze the encoder features. We find that encoder features change minimally, whereas the decoder features exhibit substantial variations across different time-steps. We validate our approach on other tasks: text-to-video, personalized generation and reference-guided generation.
  arXiv  Detail & Related papers  (2023-12-15T08:46:43Z)
• A Spatial-Temporal Deformable Attention based Framework for Breast Lesion Detection in Videos [107.96514633713034]
  We propose a spatial-temporal deformable attention based framework, named STNet. Our STNet introduces a spatial-temporal deformable attention module to perform local spatial-temporal feature fusion. Experiments on the public breast lesion ultrasound video dataset show that our STNet obtains a state-of-the-art detection performance.
  arXiv  Detail & Related papers  (2023-09-09T07:00:10Z)
• Spatial-Temporal Graph Enhanced DETR Towards Multi-Frame 3D Object Detection [54.041049052843604]
  We present STEMD, a novel end-to-end framework that enhances the DETR-like paradigm for multi-frame 3D object detection. First, to model the inter-object spatial interaction and complex temporal dependencies, we introduce the spatial-temporal graph attention network. Finally, it poses a challenge for the network to distinguish between the positive query and other highly similar queries that are not the best match.
  arXiv  Detail & Related papers  (2023-07-01T13:53:14Z)
• MED-VT++: Unifying Multimodal Learning with a Multiscale Encoder-Decoder Video Transformer [12.544216587327387]
  We present an end-to-end trainable unified multiscale encoder-decoder transformer that is focused on dense prediction tasks in video. The presented Multiscale-Decoder Video (MED-VT) uses multiscale representation throughout and employs an optional input beyond video. We present a transductive learning scheme through many-to-many label propagation to provide temporally consistent video predictions.
  arXiv  Detail & Related papers  (2023-04-12T15:50:19Z)
• Pedestrian Spatio-Temporal Information Fusion For Video Anomaly Detection [1.5736899098702974]
  An anomaly detection method is proposed to integrate the information of pedestrians. Anomaly detection is realized according to the difference between the output frame and the true value. The experimental results on the CUHK Avenue and ShanghaiTech datasets show that the proposed method is superior to the current mainstream video anomaly detection methods.
  arXiv  Detail & Related papers  (2022-11-18T06:41:02Z)
• Multimodal Graph Learning for Deepfake Detection [10.077496841634135]
  Existing deepfake detectors face several challenges in achieving robustness and generalization. We propose a novel framework, namely Multimodal Graph Learning (MGL), that leverages information from multiple modalities. Our proposed method aims to effectively identify and utilize distinguishing features for deepfake detection.
  arXiv  Detail & Related papers  (2022-09-12T17:17:49Z)
• CvT-ASSD: Convolutional vision-Transformer Based Attentive Single Shot MultiBox Detector [15.656374849760734]
  We present a novel object detection architecture, named Convolutional vision Transformer Based Attentive Single Shot MultiBox Detector (CvT-ASSD) Our model CvT-ASSD can leads to good system efficiency and performance while being pretrained on large-scale detection datasets such as PASCAL VOC and MS COCO.
  arXiv  Detail & Related papers  (2021-10-24T06:45:33Z)
• Temporal-Channel Transformer for 3D Lidar-Based Video Object Detection in Autonomous Driving [121.44554957537613]
  We propose a new transformer, called Temporal-Channel Transformer, to model the spatial-temporal domain and channel domain relationships for video object detecting from Lidar data. Specifically, the temporal-channel encoder of the transformer is designed to encode the information of different channels and frames. We achieve the state-of-the-art performance in grid voxel-based 3D object detection on the nuScenes benchmark.
  arXiv  Detail & Related papers  (2020-11-27T09:35:39Z)
• Suppress and Balance: A Simple Gated Network for Salient Object Detection [89.88222217065858]
  We propose a simple gated network (GateNet) to solve both issues at once. With the help of multilevel gate units, the valuable context information from the encoder can be optimally transmitted to the decoder. In addition, we adopt the atrous spatial pyramid pooling based on the proposed "Fold" operation (Fold-ASPP) to accurately localize salient objects of various scales.
  arXiv  Detail & Related papers  (2020-07-16T02:00:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.