MRF-UNets: Searching UNet with Markov Random Fields

• URL: http://arxiv.org/abs/2207.06168v1
• Date: Wed, 13 Jul 2022 13:04:18 GMT
• Title: MRF-UNets: Searching UNet with Markov Random Fields
• Authors: Zifu Wang, Matthew B. Blaschko
• Abstract summary: We propose MRF-NAS that extends and improves the recent Adaptive and Optimal Network Width Search (AOWS) method. We find an architecture, MRF-UNet, that shows several interesting characteristics. Experiments show that our MRF-UNets significantly outperform several benchmarks on three aerial image datasets and two medical image datasets.
• Score: 25.607512500358723
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: UNet [27] is widely used in semantic segmentation due to its simplicity and effectiveness. However, its manually-designed architecture is applied to a large number of problem settings, either with no architecture optimizations, or with manual tuning, which is time consuming and can be sub-optimal. In this work, firstly, we propose Markov Random Field Neural Architecture Search (MRF-NAS) that extends and improves the recent Adaptive and Optimal Network Width Search (AOWS) method [4] with (i) a more general MRF framework (ii) diverse M-best loopy inference (iii) differentiable parameter learning. This provides the necessary NAS framework to efficiently explore network architectures that induce loopy inference graphs, including loops that arise from skip connections. With UNet as the backbone, we find an architecture, MRF-UNet, that shows several interesting characteristics. Secondly, through the lens of these characteristics, we identify the sub-optimality of the original UNet architecture and further improve our results with MRF-UNetV2. Experiments show that our MRF-UNets significantly outperform several benchmarks on three aerial image datasets and two medical image datasets while maintaining low computational costs. The code is available at: https://github.com/zifuwanggg/MRF-UNets.

Related papers

• Multi-conditioned Graph Diffusion for Neural Architecture Search [8.290336491323796]
  We present a graph diffusion-based NAS approach that uses discrete conditional graph diffusion processes to generate high-performing neural network architectures. We show promising results on six standard benchmarks, yielding novel and unique architectures at a fast speed.
  arXiv  Detail & Related papers  (2024-03-09T21:45:31Z)
• MF-NeRF: Memory Efficient NeRF with Mixed-Feature Hash Table [62.164549651134465]
  We propose MF-NeRF, a memory-efficient NeRF framework that employs a Mixed-Feature hash table to improve memory efficiency and reduce training time while maintaining reconstruction quality. Our experiments with state-of-the-art Instant-NGP, TensoRF, and DVGO, indicate our MF-NeRF could achieve the fastest training time on the same GPU hardware with similar or even higher reconstruction quality.
  arXiv  Detail & Related papers  (2023-04-25T05:44:50Z)
• HKNAS: Classification of Hyperspectral Imagery Based on Hyper Kernel Neural Architecture Search [104.45426861115972]
  We propose to directly generate structural parameters by utilizing the specifically designed hyper kernels. We obtain three kinds of networks to separately conduct pixel-level or image-level classifications with 1-D or 3-D convolutions. A series of experiments on six public datasets demonstrate that the proposed methods achieve state-of-the-art results.
  arXiv  Detail & Related papers  (2023-04-23T17:27:40Z)
• Neural Networks with A La Carte Selection of Activation Functions [0.0]
  Activation functions (AFs) are pivotal to the success (or failure) of a neural network. We combine a slew of known AFs into successful architectures, proposing three methods to do so beneficially. We show that all methods often produce significantly better results for 25 classification problems when compared with a standard network composed of ReLU hidden units and a softmax output unit.
  arXiv  Detail & Related papers  (2022-06-24T09:09:39Z)
• iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients [75.41173109807735]
  Differentiable ARchiTecture Search (DARTS) has recently become the mainstream of neural architecture search (NAS) We tackle the hypergradient computation in DARTS based on the implicit function theorem. We show that the architecture optimisation with the proposed method, named iDARTS, is expected to converge to a stationary point.
  arXiv  Detail & Related papers  (2021-06-21T00:44:11Z)
• Trilevel Neural Architecture Search for Efficient Single Image Super-Resolution [127.92235484598811]
  This paper proposes a trilevel neural architecture search (NAS) method for efficient single image super-resolution (SR) For modeling the discrete search space, we apply a new continuous relaxation on the discrete search spaces to build a hierarchical mixture of network-path, cell-operations, and kernel-width. An efficient search algorithm is proposed to perform optimization in a hierarchical supernet manner.
  arXiv  Detail & Related papers  (2021-01-17T12:19:49Z)
• Smooth Variational Graph Embeddings for Efficient Neural Architecture Search [41.62970837629573]
  We propose a two-sided variational graph autoencoder, which allows to smoothly encode and accurately reconstruct neural architectures from various search spaces. We evaluate the proposed approach on neural architectures defined by the ENAS approach, the NAS-Bench-101 and the NAS-Bench-201 search spaces.
  arXiv  Detail & Related papers  (2020-10-09T17:05:41Z)
• DrNAS: Dirichlet Neural Architecture Search [88.56953713817545]
  We treat the continuously relaxed architecture mixing weight as random variables, modeled by Dirichlet distribution. With recently developed pathwise derivatives, the Dirichlet parameters can be easily optimized with gradient-based generalization. To alleviate the large memory consumption of differentiable NAS, we propose a simple yet effective progressive learning scheme.
  arXiv  Detail & Related papers  (2020-06-18T08:23:02Z)
• When Residual Learning Meets Dense Aggregation: Rethinking the Aggregation of Deep Neural Networks [57.0502745301132]
  We propose Micro-Dense Nets, a novel architecture with global residual learning and local micro-dense aggregations. Our micro-dense block can be integrated with neural architecture search based models to boost their performance.
  arXiv  Detail & Related papers  (2020-04-19T08:34:52Z)
• Bayesian Neural Architecture Search using A Training-Free Performance Metric [7.775212462771685]
  Recurrent neural networks (RNNs) are a powerful approach for time series prediction. This paper proposes to tackle the architecture optimization problem with a variant of the Bayesian Optimization (BO) algorithm. Also, we propose three fixed-length encoding schemes to cope with the variable-length architecture representation.
  arXiv  Detail & Related papers  (2020-01-29T08:42:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.