Comprehensive Examination of Unrolled Networks for Solving Linear Inverse Problems

• URL: http://arxiv.org/abs/2501.04608v2
• Date: Fri, 10 Jan 2025 03:08:11 GMT
• Title: Comprehensive Examination of Unrolled Networks for Solving Linear Inverse Problems
• Authors: Eric Chen, Xi Chen, Arian Maleki, Shirin Jalali,
• Abstract summary: Unrolled networks have become prevalent in various computer vision and imaging tasks.<n>This paper aims to unify some ideas and methodologies used in unrolled networks to reduce the number of design choices a user has to make.
• Score: 18.311622347325724
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Unrolled networks have become prevalent in various computer vision and imaging tasks. Although they have demonstrated remarkable efficacy in solving specific computer vision and computational imaging tasks, their adaptation to other applications presents considerable challenges. This is primarily due to the multitude of design decisions that practitioners working on new applications must navigate, each potentially affecting the network's overall performance. These decisions include selecting the optimization algorithm, defining the loss function, and determining the number of convolutional layers, among others. Compounding the issue, evaluating each design choice requires time-consuming simulations to train, fine-tune the neural network, and optimize for its performance. As a result, the process of exploring multiple options and identifying the optimal configuration becomes time-consuming and computationally demanding. The main objectives of this paper are (1) to unify some ideas and methodologies used in unrolled networks to reduce the number of design choices a user has to make, and (2) to report a comprehensive ablation study to discuss the impact of each of the choices involved in designing unrolled networks and present practical recommendations based on our findings. We anticipate that this study will help scientists and engineers design unrolled networks for their applications and diagnose problems within their networks efficiently.

Related papers

• Slicing for AI: An Online Learning Framework for Network Slicing Supporting AI Services [5.80147190706865]
  6G networks will embrace a new realm of AI-driven services that requires innovative network slicing strategies. This paper proposes an online learning framework to optimize the allocation of computational and communication resources to AI services.
  arXiv  Detail & Related papers  (2024-10-20T14:38:54Z)
• Switchable Decision: Dynamic Neural Generation Networks [98.61113699324429]
  We propose a switchable decision to accelerate inference by dynamically assigning resources for each data instance. Our method benefits from less cost during inference while keeping the same accuracy.
  arXiv  Detail & Related papers  (2024-05-07T17:44:54Z)
• Computation-efficient Deep Learning for Computer Vision: A Survey [121.84121397440337]
  Deep learning models have reached or even exceeded human-level performance in a range of visual perception tasks. Deep learning models usually demand significant computational resources, leading to impractical power consumption, latency, or carbon emissions in real-world scenarios. New research focus is computationally efficient deep learning, which strives to achieve satisfactory performance while minimizing the computational cost during inference.
  arXiv  Detail & Related papers  (2023-08-27T03:55:28Z)
• Graph Reinforcement Learning for Network Control via Bi-Level Optimization [37.00510744883984]
  We argue that data-driven strategies can automate this process and learn efficient algorithms without compromising optimality. We present network control problems through the lens of reinforcement learning and propose a graph network-based framework to handle a broad class of problems.
  arXiv  Detail & Related papers  (2023-05-16T03:20:22Z)
• Multiobjective Evolutionary Pruning of Deep Neural Networks with Transfer Learning for improving their Performance and Robustness [15.29595828816055]
  This work proposes MO-EvoPruneDeepTL, a multi-objective evolutionary pruning algorithm. We use Transfer Learning to adapt the last layers of Deep Neural Networks, by replacing them with sparse layers evolved by a genetic algorithm. Experiments show that our proposal achieves promising results in all the objectives, and direct relation are presented.
  arXiv  Detail & Related papers  (2023-02-20T19:33:38Z)
• Analysis and Design of Quadratic Neural Networks for Regression, Classification, and Lyapunov Control of Dynamical Systems [0.0]
  This paper addresses the analysis and design of quadratic neural networks. Networks offer several advantages, the most important of which are the fact that the architecture is a by-product of the design and is not determined a-priori. Several examples will show the effectiveness of quadratic neural networks in applications.
  arXiv  Detail & Related papers  (2022-07-26T18:10:05Z)
• Offline Contextual Bandits for Wireless Network Optimization [107.24086150482843]
  In this paper, we investigate how to learn policies that can automatically adjust the configuration parameters of every cell in the network in response to the changes in the user demand. Our solution combines existent methods for offline learning and adapts them in a principled way to overcome crucial challenges arising in this context.
  arXiv  Detail & Related papers  (2021-11-11T11:31:20Z)
• Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions [80.78077900288868]
  We decompose the design methodology of early-exit networks to its key components and survey the recent advances in each one of them. We position early-exiting against other efficient inference solutions and provide our insights on the current challenges and most promising future directions for research in the field.
  arXiv  Detail & Related papers  (2021-06-09T12:33:02Z)
• Apply Artificial Neural Network to Solving Manpower Scheduling Problem [15.848399017432262]
  This paper proposes a new model combined with deep learning to solve the multi-shift manpower scheduling problem. We will use the neural network training method based on the time series to solve long-term and long-period scheduling tasks. Our research shows that neural networks and deep learning strategies have the potential to solve similar problems effectively.
  arXiv  Detail & Related papers  (2021-05-07T23:54:00Z)
• An adaptive artificial neural network-based generative design method for layout designs [17.377351418260577]
  An adaptive artificial neural network-based generative design approach is proposed and developed. A novel adaptive learning and optimization strategy is proposed, which allows the design space to be effectively explored. The performance of the proposed design method is demonstrated on two heat source layout design problems.
  arXiv  Detail & Related papers  (2021-01-29T05:32:17Z)
• Spiking Neural Networks Hardware Implementations and Challenges: a Survey [53.429871539789445]
  Spiking Neural Networks are cognitive algorithms mimicking neuron and synapse operational principles. We present the state of the art of hardware implementations of spiking neural networks. We discuss the strategies employed to leverage the characteristics of these event-driven algorithms at the hardware level.
  arXiv  Detail & Related papers  (2020-05-04T13:24:00Z)
• Fitting the Search Space of Weight-sharing NAS with Graph Convolutional Networks [100.14670789581811]
  We train a graph convolutional network to fit the performance of sampled sub-networks. With this strategy, we achieve a higher rank correlation coefficient in the selected set of candidates.
  arXiv  Detail & Related papers  (2020-04-17T19:12:39Z)
• Binary Neural Networks: A Survey [126.67799882857656]
  The binary neural network serves as a promising technique for deploying deep models on resource-limited devices. The binarization inevitably causes severe information loss, and even worse, its discontinuity brings difficulty to the optimization of the deep network. We present a survey of these algorithms, mainly categorized into the native solutions directly conducting binarization, and the optimized ones using techniques like minimizing the quantization error, improving the network loss function, and reducing the gradient error.
  arXiv  Detail & Related papers  (2020-03-31T16:47:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.