Cross-Enhancement Transformer for Action Segmentation

• URL: http://arxiv.org/abs/2205.09445v1
• Date: Thu, 19 May 2022 10:06:30 GMT
• Title: Cross-Enhancement Transformer for Action Segmentation
• Authors: Jiahui Wang, Zhenyou Wang, Shanna Zhuang, Hui Wang
• Abstract summary: A novel encoder-decoder structure is proposed in this paper, called Cross-Enhancement Transformer. Our approach can be effective learning of temporal structure representation with interactive self-attention mechanism. In addition, a new loss function is proposed to enhance the training process that penalizes over-segmentation errors.
• Score: 5.752561578852787
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Temporal convolutions have been the paradigm of choice in action segmentation, which enhances long-term receptive fields by increasing convolution layers. However, high layers cause the loss of local information necessary for frame recognition. To solve the above problem, a novel encoder-decoder structure is proposed in this paper, called Cross-Enhancement Transformer. Our approach can be effective learning of temporal structure representation with interactive self-attention mechanism. Concatenated each layer convolutional feature maps in encoder with a set of features in decoder produced via self-attention. Therefore, local and global information are used in a series of frame actions simultaneously. In addition, a new loss function is proposed to enhance the training process that penalizes over-segmentation errors. Experiments show that our framework performs state-of-the-art on three challenging datasets: 50Salads, Georgia Tech Egocentric Activities and the Breakfast dataset.

Related papers

• Mutual Information-driven Triple Interaction Network for Efficient Image Dehazing [54.168567276280505]
  We propose a novel Mutual Information-driven Triple interaction Network (MITNet) for image dehazing. The first stage, named amplitude-guided haze removal, aims to recover the amplitude spectrum of the hazy images for haze removal. The second stage, named phase-guided structure refined, devotes to learning the transformation and refinement of the phase spectrum.
  arXiv  Detail & Related papers  (2023-08-14T08:23:58Z)
• Generative-Contrastive Learning for Self-Supervised Latent Representations of 3D Shapes from Multi-Modal Euclidean Input [44.10761155817833]
  We propose a combined generative and contrastive neural architecture for learning latent representations of 3D shapes. The architecture uses two encoder branches for voxel grids and multi-view images from the same underlying shape.
  arXiv  Detail & Related papers  (2023-01-11T18:14:24Z)
• Feedback Chain Network For Hippocampus Segmentation [59.74305660815117]
  We propose a novel hierarchical feedback chain network for the hippocampus segmentation task. The proposed approach achieves state-of-the-art performance on three publicly available datasets.
  arXiv  Detail & Related papers  (2022-11-15T04:32:10Z)
• Defect Transformer: An Efficient Hybrid Transformer Architecture for Surface Defect Detection [2.0999222360659604]
  We propose an efficient hybrid transformer architecture, termed Defect Transformer (DefT), for surface defect detection. DefT incorporates CNN and transformer into a unified model to capture local and non-local relationships collaboratively. Experiments on three datasets demonstrate the superiority and efficiency of our method compared with other CNN- and transformer-based networks.
  arXiv  Detail & Related papers  (2022-07-17T23:37:48Z)
• MISSU: 3D Medical Image Segmentation via Self-distilling TransUNet [55.16833099336073]
  We propose to self-distill a Transformer-based UNet for medical image segmentation. It simultaneously learns global semantic information and local spatial-detailed features. Our MISSU achieves the best performance over previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-06-02T07:38:53Z)
• TC-Net: Triple Context Network for Automated Stroke Lesion Segmentation [0.5482532589225552]
  We propose a new network, Triple Context Network (TC-Net), with the capture of spatial contextual information as the core. Our network is evaluated on the open dataset ATLAS, achieving the highest score of 0.594, Hausdorff distance of 27.005 mm, and average symmetry surface distance of 7.137 mm.
  arXiv  Detail & Related papers  (2022-02-28T11:12:16Z)
• Unsupervised Motion Representation Learning with Capsule Autoencoders [54.81628825371412]
  Motion Capsule Autoencoder (MCAE) models motion in a two-level hierarchy. MCAE is evaluated on a novel Trajectory20 motion dataset and various real-world skeleton-based human action datasets.
  arXiv  Detail & Related papers  (2021-10-01T16:52:03Z)
• CrossTransformers: spatially-aware few-shot transfer [92.33252608837947]
  Given new tasks with very little data, modern vision systems degrade remarkably quickly. We show how the neural network representations which underpin modern vision systems are subject to supervision collapse. We propose self-supervised learning to encourage general-purpose features that transfer better.
  arXiv  Detail & Related papers  (2020-07-22T15:37:08Z)
• Beyond Single Stage Encoder-Decoder Networks: Deep Decoders for Semantic Image Segmentation [56.44853893149365]
  Single encoder-decoder methodologies for semantic segmentation are reaching their peak in terms of segmentation quality and efficiency per number of layers. We propose a new architecture based on a decoder which uses a set of shallow networks for capturing more information content. In order to further improve the architecture we introduce a weight function which aims to re-balance classes to increase the attention of the networks to under-represented objects.
  arXiv  Detail & Related papers  (2020-07-19T18:44:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.