Image Segmentation with transformers: An Overview, Challenges and Future

• URL: http://arxiv.org/abs/2501.09372v1
• Date: Thu, 16 Jan 2025 08:34:39 GMT
• Title: Image Segmentation with transformers: An Overview, Challenges and Future
• Authors: Deepjyoti Chetia, Debasish Dutta, Sanjib Kr Kalita,
• Abstract summary: This paper explores the shortcomings of CNN-based models and the shift towards transformer architectures. The paper discusses current challenges in transformer-based segmentation and outlines promising future trends.
• Score: 0.0
• License:
• Abstract: Image segmentation, a key task in computer vision, has traditionally relied on convolutional neural networks (CNNs), yet these models struggle with capturing complex spatial dependencies, objects with varying scales, need for manually crafted architecture components and contextual information. This paper explores the shortcomings of CNN-based models and the shift towards transformer architectures -to overcome those limitations. This work reviews state-of-the-art transformer-based segmentation models, addressing segmentation-specific challenges and their solutions. The paper discusses current challenges in transformer-based segmentation and outlines promising future trends, such as lightweight architectures and enhanced data efficiency. This survey serves as a guide for understanding the impact of transformers in advancing segmentation capabilities and overcoming the limitations of traditional models.

Related papers

• Inverting Visual Representations with Detection Transformers [0.8124699127636158]
  We apply the approach of training inverse models to reconstruct input images from intermediate layers within a Detection Transformer. We demonstrate critical properties of Detection Transformers, including contextual shape robustness, inter-layer correlation, and preservation to color perturbations.
  arXiv  Detail & Related papers  (2024-12-09T14:43:06Z)
• A Review of Transformer-Based Models for Computer Vision Tasks: Capturing Global Context and Spatial Relationships [0.5639904484784127]
  Transformer-based models have transformed the landscape of natural language processing (NLP) These models are renowned for their ability to capture long-range dependencies and contextual information. We discuss potential research directions and applications of transformer-based models in computer vision.
  arXiv  Detail & Related papers  (2024-08-27T16:22:18Z)
• Introduction to Transformers: an NLP Perspective [59.0241868728732]
  We introduce basic concepts of Transformers and present key techniques that form the recent advances of these models. This includes a description of the standard Transformer architecture, a series of model refinements, and common applications.
  arXiv  Detail & Related papers  (2023-11-29T13:51:04Z)
• Understanding Video Transformers for Segmentation: A Survey of Application and Interpretability [10.180033230324561]
  Recently, approaches in this research area shifted from concentrating on ConvNet-based to transformer-based models. Various interpretability approaches have appeared for transformer models and video temporal dynamics.
  arXiv  Detail & Related papers  (2023-10-18T19:58:25Z)
• Rich CNN-Transformer Feature Aggregation Networks for Super-Resolution [50.10987776141901]
  Recent vision transformers along with self-attention have achieved promising results on various computer vision tasks. We introduce an effective hybrid architecture for super-resolution (SR) tasks, which leverages local features from CNNs and long-range dependencies captured by transformers. Our proposed method achieves state-of-the-art SR results on numerous benchmark datasets.
  arXiv  Detail & Related papers  (2022-03-15T06:52:25Z)
• Visformer: The Vision-friendly Transformer [105.52122194322592]
  We propose a new architecture named Visformer, which is abbreviated from the Vision-friendly Transformer' With the same computational complexity, Visformer outperforms both the Transformer-based and convolution-based models in terms of ImageNet classification accuracy.
  arXiv  Detail & Related papers  (2021-04-26T13:13:03Z)
• TransVG: End-to-End Visual Grounding with Transformers [102.11922622103613]
  We present a transformer-based framework for visual grounding, namely TransVG, to address the task of grounding a language query to an image. We show that the complex fusion modules can be replaced by a simple stack of transformer encoder layers with higher performance.
  arXiv  Detail & Related papers  (2021-04-17T13:35:24Z)
• Understanding Robustness of Transformers for Image Classification [34.51672491103555]
  Vision Transformer (ViT) has surpassed ResNets for image classification. Details of the Transformer architecture lead one to wonder whether these networks are as robust. We find that ViT models are at least as robust as the ResNet counterparts on a broad range of perturbations.
  arXiv  Detail & Related papers  (2021-03-26T16:47:55Z)
• Transformers Solve the Limited Receptive Field for Monocular Depth Prediction [82.90445525977904]
  We propose TransDepth, an architecture which benefits from both convolutional neural networks and transformers. This is the first paper which applies transformers into pixel-wise prediction problems involving continuous labels.
  arXiv  Detail & Related papers  (2021-03-22T18:00:13Z)
• Transformers in Vision: A Survey [101.07348618962111]
  Transformers enable modeling long dependencies between input sequence elements and support parallel processing of sequence. Transformers require minimal inductive biases for their design and are naturally suited as set-functions. This survey aims to provide a comprehensive overview of the Transformer models in the computer vision discipline.
  arXiv  Detail & Related papers  (2021-01-04T18:57:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.