RTNet: Relation Transformer Network for Diabetic Retinopathy Multi-lesion Segmentation

• URL: http://arxiv.org/abs/2201.11037v1
• Date: Wed, 26 Jan 2022 16:19:04 GMT
• Title: RTNet: Relation Transformer Network for Diabetic Retinopathy Multi-lesion Segmentation
• Authors: Shiqi Huang, Jianan Li, Yuze Xiao, Ning Shen and Tingfa Xu
• Abstract summary: We find that certain lesions are closed to specific vessels and present relative patterns to each other. A self-attention transformer exploits global dependencies among lesion features, while a cross-attention transformer allows interactions between lesion and vessel features. By integrating the above blocks of dual-branches, our network segments the four kinds of lesions simultaneously.
• Score: 10.643730843316948
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automatic diabetic retinopathy (DR) lesions segmentation makes great sense of assisting ophthalmologists in diagnosis. Although many researches have been conducted on this task, most prior works paid too much attention to the designs of networks instead of considering the pathological association for lesions. Through investigating the pathogenic causes of DR lesions in advance, we found that certain lesions are closed to specific vessels and present relative patterns to each other. Motivated by the observation, we propose a relation transformer block (RTB) to incorporate attention mechanisms at two main levels: a self-attention transformer exploits global dependencies among lesion features, while a cross-attention transformer allows interactions between lesion and vessel features by integrating valuable vascular information to alleviate ambiguity in lesion detection caused by complex fundus structures. In addition, to capture the small lesion patterns first, we propose a global transformer block (GTB) which preserves detailed information in deep network. By integrating the above blocks of dual-branches, our network segments the four kinds of lesions simultaneously. Comprehensive experiments on IDRiD and DDR datasets well demonstrate the superiority of our approach, which achieves competitive performance compared to state-of-the-arts.

Related papers

• Wavelet-based Global-Local Interaction Network with Cross-Attention for Multi-View Diabetic Retinopathy Detection [11.977013678444273]
  We propose a novel method to overcome the challenges of difficult lesion information learning and inadequate multi-view fusion. Specifically, we introduce a two-branch network to obtain both local lesion features and their global dependencies. We present a cross-view fusion module to improve multi-view fusion and reduce redundancy.
  arXiv  Detail & Related papers  (2025-03-25T03:44:57Z)
• LesionDiffusion: Towards Text-controlled General Lesion Synthesis [1.6029418399561406]
  We propose LesionDiffusion, a text-controllable lesion synthesis framework for 3D CT imaging. Our model provides greater control over lesion attributes and supports a wider variety of lesion types. We introduce a dataset of 1,505 annotated CT scans with paired lesion masks and structured reports, covering 14 lesion types across 8 organs.
  arXiv  Detail & Related papers  (2025-03-02T05:36:04Z)
• RotCAtt-TransUNet++: Novel Deep Neural Network for Sophisticated Cardiac Segmentation [0.0]
  We present RotCAtt-TransUNet++, a novel architecture tailored for robust segmentation of complex cardiac structures. Our approach emphasizes modeling global contexts by aggregating multiscale features with nested skip connections in the encoder. Experimental results demonstrate that our proposed model outperforms existing SOTA approaches across four cardiac datasets and one abdominal dataset.
  arXiv  Detail & Related papers  (2024-09-09T02:18:50Z)
• IAFI-FCOS: Intra- and across-layer feature interaction FCOS model for lesion detection of CT images [5.198119863305256]
  Multi-scale feature fusion mechanism of most traditional detectors are unable to transmit detail information without loss. We propose a novel intra- and across-layer feature interaction FCOS model (IAFI-FCOS) with a multi-scale feature fusion mechanism ICAF-FPN. Our approach has been extensively experimented on both the private pancreatic lesion dataset and the public DeepLesion dataset.
  arXiv  Detail & Related papers  (2024-09-01T10:58:48Z)
• Lesion-aware network for diabetic retinopathy diagnosis [28.228110579446227]
  We propose a CNN-based diabetic retinopathy (DR) diagnosis network with attention mechanism involved, termed lesion-aware network. The proposed LANet is constructed by embedding the LAM and FPM into the CNN decoders for DR-related information utilization. Our method outperforms the mainstream methods with an area under curve of 0.967 in DR screening, and increases the overall average precision by 7.6%, 2.1%, and 1.2% in lesion segmentation on three datasets.
  arXiv  Detail & Related papers  (2024-08-14T03:06:04Z)
• Modality Exchange Network for Retinogeniculate Visual Pathway Segmentation [5.726588626363204]
  We propose a novel Modality Exchange Network (ME-Net) that effectively utilizes multi-modal magnetic resonance (MR) imaging information to enhance RGVP segmentation. Specifically, we design a channel and spatially mixed attention module to exchange modality information between T1-weighted and fractional anisotropy MR images. Experimental results demonstrate that our method outperforms existing state-of-the-art approaches in terms of RGVP segmentation performance.
  arXiv  Detail & Related papers  (2024-01-03T11:41:57Z)
• Scale-aware Super-resolution Network with Dual Affinity Learning for Lesion Segmentation from Medical Images [50.76668288066681]
  We present a scale-aware super-resolution network to adaptively segment lesions of various sizes from low-resolution medical images. Our proposed network achieved consistent improvements compared to other state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-30T14:25:55Z)
• A Global and Patch-wise Contrastive Loss for Accurate Automated Exudate Detection [12.669734891001667]
  Diabetic retinopathy (DR) is a leading global cause of blindness. Early detection of hard exudates plays a crucial role in identifying DR, which aids in treating diabetes and preventing vision loss. We present a novel supervised contrastive learning framework to optimize hard exudate segmentation.
  arXiv  Detail & Related papers  (2023-02-22T17:39:00Z)
• Reliable Joint Segmentation of Retinal Edema Lesions in OCT Images [55.83984261827332]
  In this paper, we propose a novel reliable multi-scale wavelet-enhanced transformer network. We develop a novel segmentation backbone that integrates a wavelet-enhanced feature extractor network and a multi-scale transformer module. Our proposed method achieves better segmentation accuracy with a high degree of reliability as compared to other state-of-the-art segmentation approaches.
  arXiv  Detail & Related papers  (2022-12-01T07:32:56Z)
• Factored Attention and Embedding for Unstructured-view Topic-related Ultrasound Report Generation [70.7778938191405]
  We propose a novel factored attention and embedding model (termed FAE-Gen) for the unstructured-view topic-related ultrasound report generation. The proposed FAE-Gen mainly consists of two modules, i.e., view-guided factored attention and topic-oriented factored embedding, which capture the homogeneous and heterogeneous morphological characteristic across different views.
  arXiv  Detail & Related papers  (2022-03-12T15:24:03Z)
• External Attention Assisted Multi-Phase Splenic Vascular Injury Segmentation with Limited Data [72.99534552950138]
  The spleen is one of the most commonly injured solid organs in blunt abdominal trauma. accurate segmentation of splenic vascular injury is challenging for the following reasons.
  arXiv  Detail & Related papers  (2022-01-04T02:35:56Z)
• Weakly-Supervised Cross-Domain Adaptation for Endoscopic Lesions Segmentation [79.58311369297635]
  We propose a new weakly-supervised lesions transfer framework, which can explore transferable domain-invariant knowledge across different datasets. A Wasserstein quantified transferability framework is developed to highlight widerange transferable contextual dependencies. A novel self-supervised pseudo label generator is designed to equally provide confident pseudo pixel labels for both hard-to-transfer and easy-to-transfer target samples.
  arXiv  Detail & Related papers  (2020-12-08T02:26:03Z)
• What Can Be Transferred: Unsupervised Domain Adaptation for Endoscopic Lesions Segmentation [51.7837386041158]
  We develop a new unsupervised semantic transfer model including two complementary modules for endoscopic lesions segmentation. Specifically, T_D focuses on where to translate transferable visual information of medical lesions via residual transferability-aware bottleneck. T_F highlights how to augment transferable semantic features of various lesions and automatically ignore untransferable representations.
  arXiv  Detail & Related papers  (2020-04-24T00:57:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.