$\ell_1$DecNet+: A new architecture framework by $\ell_1$ decomposition and iteration unfolding for sparse feature segmentation

• URL: http://arxiv.org/abs/2203.02690v2
• Date: Sun, 16 Jun 2024 02:15:56 GMT
• Title: $\ell_1$DecNet+: A new architecture framework by $\ell_1$ decomposition and iteration unfolding for sparse feature segmentation
• Authors: Yumeng Ren, Yiming Gao, Chunlin Wu, Xue-cheng Tai,
• Abstract summary: $ell_$DecNet is an unfolded network derived from a variational decomposition model incorporating $ell_$ related sparse regularization. We develop $ell_$DecNet+, a learnable architecture framework consisting of our $ell_$DecNet and a segmentation module which operates over extracted sparse features. We evaluate the effectiveness of $ell_$DecNet+ on two commonly encountered sparse segmentation tasks: retinal vessel segmentation in medical image processing and pavement crack detection in industrial abnormality identification.
• Score: 4.150107303000611
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: $\ell_1$ based sparse regularization plays a central role in compressive sensing and image processing. In this paper, we propose $\ell_1$DecNet, as an unfolded network derived from a variational decomposition model incorporating $\ell_1$ related sparse regularization and solved by scaled alternating direction method of multipliers (ADMM). $\ell_1$DecNet effectively decomposes an input image into a sparse feature and a learned dense feature, and thus helps the subsequent sparse feature related operations. Based on this, we develop $\ell_1$DecNet+, a learnable architecture framework consisting of our $\ell_1$DecNet and a segmentation module which operates over extracted sparse features instead of original images. This architecture combines well the benefits of mathematical modeling and data-driven approaches. To our best knowledge, this is the first study to incorporate mathematical image prior into feature extraction in segmentation network structures. Moreover, our $\ell_1$DecNet+ framework can be easily extended to 3D case. We evaluate the effectiveness of $\ell_1$DecNet+ on two commonly encountered sparse segmentation tasks: retinal vessel segmentation in medical image processing and pavement crack detection in industrial abnormality identification. Experimental results on different datasets demonstrate that, our $\ell_1$DecNet+ architecture with various lightweight segmentation modules can achieve equal or better performance than their enlarged versions respectively. This leads to especially practical advantages on resource-limited devices.

Related papers

• A Novel Structure-Agnostic Multi-Objective Approach for Weight-Sharing Compression in Deep Neural Networks [0.24578723416255746]
  We propose a multi-objective evolutionary algorithm (MOEA) based compression framework independent of neural network architecture, dimension, task, and dataset. We use uniformly sized bins to quantize network weights into a single codebook for efficient weight representation. The experimental results show that we can reduce the neural network memory by $13.72 sim14.98 times$ on CIFAR-10, $11.61 sim 12.99times$ on CIFAR-100, and $7.44 sim 8.58times$ on ImageNet.
  arXiv  Detail & Related papers  (2025-01-06T15:51:29Z)
• SPPNet: A Single-Point Prompt Network for Nuclei Image Segmentation [6.149725843029721]
  Single-point prompt network is proposed for nuclei image segmentation. We replace the original image encoder with a lightweight vision transformer. The proposed model is evaluated on the MoNuSeg-2018 dataset.
  arXiv  Detail & Related papers  (2023-08-23T16:13:58Z)
• Efficient Scale-Invariant Generator with Column-Row Entangled Pixel Synthesis [3.222802562733787]
  We propose a new generative model that is both efficient and scale-equivariant without using any spatial convolutions or coarse-to-fine design. Experiments on various datasets, including FFHQ, LSUN-Church, MetFaces, and Flickr-Scenery, confirm CREPS' ability to synthesize scale-consistent and alias-free images.
  arXiv  Detail & Related papers  (2023-03-24T17:12:38Z)
• Simplifying and Understanding State Space Models with Diagonal Linear RNNs [56.33053691749856]
  This work disposes of the discretization step, and proposes a model based on vanilla Diagonal Linear RNNs. We empirically show that, despite being conceptually much simpler, $mathrmDLR$ is as performant as previously-proposed SSMs. We also characterize the expressivity of SSMs and attention-based models via a suite of $13$ synthetic sequence-to-sequence tasks.
  arXiv  Detail & Related papers  (2022-12-01T18:53:06Z)
• Pushing the Efficiency Limit Using Structured Sparse Convolutions [82.31130122200578]
  We propose Structured Sparse Convolution (SSC), which leverages the inherent structure in images to reduce the parameters in the convolutional filter. We show that SSC is a generalization of commonly used layers (depthwise, groupwise and pointwise convolution) in efficient architectures'' Architectures based on SSC achieve state-of-the-art performance compared to baselines on CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet classification benchmarks.
  arXiv  Detail & Related papers  (2022-10-23T18:37:22Z)
• R2U++: A Multiscale Recurrent Residual U-Net with Dense Skip Connections for Medical Image Segmentation [0.5735035463793008]
  We propose a new U-Net based medical image segmentation architecture R2U++. In the proposed architecture, the plain convolutional backbone is replaced by a deeper recurrent residual convolution block. The semantic gap between encoder and decoder is reduced by dense skip pathways.
  arXiv  Detail & Related papers  (2022-06-03T19:42:44Z)
• Deep ensembles in bioimage segmentation [74.01883650587321]
  In this work, we propose an ensemble of convolutional neural networks (CNNs) In ensemble methods, many different models are trained and then used for classification, the ensemble aggregates the outputs of the single classifiers. The proposed ensemble is implemented by combining different backbone networks using the DeepLabV3+ and HarDNet environment.
  arXiv  Detail & Related papers  (2021-12-24T05:54:21Z)
• Dep-$L_0$: Improving $L_0$-based Network Sparsification via Dependency Modeling [6.081082481356211]
  Training deep neural networks with an $L_0$ regularization is one of the prominent approaches for network pruning or sparsification. We show that this method performs inconsistently on large-scale learning tasks, such as ResNet50 on ImageNet. We propose a dependency modeling of binary gates, which can be modeled effectively as a multi-layer perceptron.
  arXiv  Detail & Related papers  (2021-06-30T19:33:35Z)
• Deep ensembles based on Stochastic Activation Selection for Polyp Segmentation [82.61182037130406]
  This work deals with medical image segmentation and in particular with accurate polyp detection and segmentation during colonoscopy examinations. Basic architecture in image segmentation consists of an encoder and a decoder. We compare some variant of the DeepLab architecture obtained by varying the decoder backbone.
  arXiv  Detail & Related papers  (2021-04-02T02:07:37Z)
• Improving Robustness and Generality of NLP Models Using Disentangled Representations [62.08794500431367]
  Supervised neural networks first map an input $x$ to a single representation $z$, and then map $z$ to the output label $y$. We present methods to improve robustness and generality of NLP models from the standpoint of disentangled representation learning. We show that models trained with the proposed criteria provide better robustness and domain adaptation ability in a wide range of supervised learning tasks.
  arXiv  Detail & Related papers  (2020-09-21T02:48:46Z)
• Structured Convolutions for Efficient Neural Network Design [65.36569572213027]
  We tackle model efficiency by exploiting redundancy in the textitimplicit structure of the building blocks of convolutional neural networks. We show how this decomposition can be applied to 2D and 3D kernels as well as the fully-connected layers.
  arXiv  Detail & Related papers  (2020-08-06T04:38:38Z)
• U$^2$-Net: Going Deeper with Nested U-Structure for Salient Object Detection [6.071985784990975]
  We design a simple yet powerful deep network architecture, U$2$-Net, for salient object detection (SOD) The architecture of our U$2$-Net is a two-level nested U-structure. We instantiate two models of the proposed architecture, U$2$-Net (176.3 MB, 30 FPS on GTX 1080Ti GPU) and U$2$-Net$dagger$ (4.7 MB, 40 FPS)
  arXiv  Detail & Related papers  (2020-05-18T18:08:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.