Deep learning for inverse problems with unknown operator

• URL: http://arxiv.org/abs/2108.02744v1
• Date: Thu, 5 Aug 2021 17:21:12 GMT
• Title: Deep learning for inverse problems with unknown operator
• Authors: Miguel del Alamo
• Abstract summary: In inverse problems where the forward operator $T$ is unknown, we have access to training data consisting of functions $f_i$ and their noisy images $Tf_i$. We propose a new method that requires minimal assumptions on the data, and prove reconstruction rates that depend on the number of training points and the noise level.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We consider ill-posed inverse problems where the forward operator $T$ is unknown, and instead we have access to training data consisting of functions $f_i$ and their noisy images $Tf_i$. This is a practically relevant and challenging problem which current methods are able to solve only under strong assumptions on the training set. Here we propose a new method that requires minimal assumptions on the data, and prove reconstruction rates that depend on the number of training points and the noise level. We show that, in the regime of "many" training data, the method is minimax optimal. The proposed method employs a type of convolutional neural networks (U-nets) and empirical risk minimization in order to "fit" the unknown operator. In a nutshell, our approach is based on two ideas: the first is to relate U-nets to multiscale decompositions such as wavelets, thereby linking them to the existing theory, and the second is to use the hierarchical structure of U-nets and the low number of parameters of convolutional neural nets to prove entropy bounds that are practically useful. A significant difference with the existing works on neural networks in nonparametric statistics is that we use them to approximate operators and not functions, which we argue is mathematically more natural and technically more convenient.

Related papers

• Just How Flexible are Neural Networks in Practice? [89.80474583606242]
  It is widely believed that a neural network can fit a training set containing at least as many samples as it has parameters. In practice, however, we only find solutions via our training procedure, including the gradient and regularizers, limiting flexibility.
  arXiv  Detail & Related papers  (2024-06-17T12:24:45Z)
• Robust Capped lp-Norm Support Vector Ordinal Regression [85.84718111830752]
  Ordinal regression is a specialized supervised problem where the labels show an inherent order. Support Vector Ordinal Regression, as an outstanding ordinal regression model, is widely used in many ordinal regression tasks. We introduce a new model, Capped $ell_p$-Norm Support Vector Ordinal Regression(CSVOR), that is robust to outliers.
  arXiv  Detail & Related papers  (2024-04-25T13:56:05Z)
• Efficient Verification-Based Face Identification [50.616875565173274]
  We study the problem of performing face verification with an efficient neural model $f$. Our model leads to a substantially small $f$ requiring only 23k parameters and 5M floating point operations (FLOPS) We use six face verification datasets to demonstrate that our method is on par or better than state-of-the-art models.
  arXiv  Detail & Related papers  (2023-12-20T18:08:02Z)
• Inverse Problems with Learned Forward Operators [2.162017337541015]
  This chapter reviews reconstruction methods in inverse problems with learned forward operators that follow two different paradigms. The framework of regularisation by projection is then used to find a reconstruction. A common theme emerges: both methods require, or at least benefit from, training data not only for the forward operator, but also for its adjoint.
  arXiv  Detail & Related papers  (2023-11-21T11:15:14Z)
• In-Context Operator Learning with Data Prompts for Differential Equation Problems [12.61842281581773]
  This paper introduces a new neural-network-based approach, namely In-Context Operator Networks (ICON) ICON simultaneously learn operators from the prompted data and apply it to new questions during the inference stage, without any weight update. Our numerical results show the neural network's capability as a few-shot operator learner for a diversified type of differential equation problems.
  arXiv  Detail & Related papers  (2023-04-17T05:22:26Z)
• Improving Representational Continuity via Continued Pretraining [76.29171039601948]
  Transfer learning community (LP-FT) outperforms naive training and other continual learning methods. LP-FT also reduces forgetting in a real world satellite remote sensing dataset (FMoW) variant of LP-FT gets state-of-the-art accuracies on an NLP continual learning benchmark.
  arXiv  Detail & Related papers  (2023-02-26T10:39:38Z)
• Unsupervised Training for Neural TSP Solver [2.9398911304923447]
  We introduce a novel unsupervised learning approach for solving the travelling salesman problem. We use a relaxation of an integer linear program for TSP to construct a loss function that does not require correct instance labels. Our approach is also more stable and easier to train than reinforcement learning.
  arXiv  Detail & Related papers  (2022-07-27T17:33:29Z)
• From Kernel Methods to Neural Networks: A Unifying Variational Formulation [25.6264886382888]
  We present a unifying regularization functional that depends on an operator and on a generic Radon-domain norm. Our framework offers guarantees of universal approximation for a broad family of regularization operators or, equivalently, for a wide variety of shallow neural networks.
  arXiv  Detail & Related papers  (2022-06-29T13:13:53Z)
• Transfer Learning via Test-Time Neural Networks Aggregation [11.42582922543676]
  It has been demonstrated that deep neural networks outperform traditional machine learning. Deep networks lack generalisability, that is, they will not perform as good as in a new (testing) set drawn from a different distribution.
  arXiv  Detail & Related papers  (2022-06-27T15:46:05Z)
• Towards an Understanding of Benign Overfitting in Neural Networks [104.2956323934544]
  Modern machine learning models often employ a huge number of parameters and are typically optimized to have zero training loss. We examine how these benign overfitting phenomena occur in a two-layer neural network setting. We show that it is possible for the two-layer ReLU network interpolator to achieve a near minimax-optimal learning rate.
  arXiv  Detail & Related papers  (2021-06-06T19:08:53Z)
• RNN Training along Locally Optimal Trajectories via Frank-Wolfe Algorithm [50.76576946099215]
  We propose a novel and efficient training method for RNNs by iteratively seeking a local minima on the loss surface within a small region. We develop a novel RNN training method that, surprisingly, even with the additional cost, the overall training cost is empirically observed to be lower than back-propagation.
  arXiv  Detail & Related papers  (2020-10-12T01:59:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.