Twin Neural Network Regression

• URL: http://arxiv.org/abs/2012.14873v1
• Date: Tue, 29 Dec 2020 17:52:31 GMT
• Title: Twin Neural Network Regression
• Authors: Sebastian J. Wetzel, Kevin Ryczko, Roger G. Melko, Isaac Tamblyn
• Abstract summary: We introduce twin neural network (TNN) regression. This method predicts differences between the target values of two different data points rather than the targets themselves. We show that TNNs are able to compete or yield more accurate predictions for different data sets, compared to other state-of-the-art methods.
• Score: 0.802904964931021
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We introduce twin neural network (TNN) regression. This method predicts differences between the target values of two different data points rather than the targets themselves. The solution of a traditional regression problem is then obtained by averaging over an ensemble of all predicted differences between the targets of an unseen data point and all training data points. Whereas ensembles are normally costly to produce, TNN regression intrinsically creates an ensemble of predictions of twice the size of the training set while only training a single neural network. Since ensembles have been shown to be more accurate than single models this property naturally transfers to TNN regression. We show that TNNs are able to compete or yield more accurate predictions for different data sets, compared to other state-of-the-art methods. Furthermore, TNN regression is constrained by self-consistency conditions. We find that the violation of these conditions provides an estimate for the prediction uncertainty.

Related papers

• Statistical Properties of Deep Neural Networks with Dependent Data [0.0]
  This paper establishes statistical properties of deep neural network (DNN) estimators under dependent data. The framework provided also offers potential for research into other DNN architectures and time-series applications.
  arXiv  Detail & Related papers  (2024-10-14T21:46:57Z)
• Twin Neural Network Improved k-Nearest Neighbor Regression [0.0]
  Twin neural network regression is trained to predict differences between regression targets rather than the targets themselves. A solution to the original regression problem can be obtained by ensembling predicted differences between the targets of an unknown data point and multiple known anchor data points.
  arXiv  Detail & Related papers  (2023-10-01T13:20:49Z)
• Boosted Dynamic Neural Networks [53.559833501288146]
  A typical EDNN has multiple prediction heads at different layers of the network backbone. To optimize the model, these prediction heads together with the network backbone are trained on every batch of training data. Treating training and testing inputs differently at the two phases will cause the mismatch between training and testing data distributions. We formulate an EDNN as an additive model inspired by gradient boosting, and propose multiple training techniques to optimize the model effectively.
  arXiv  Detail & Related papers  (2022-11-30T04:23:12Z)
• Deep Regression Ensembles [0.0]
  We introduce a methodology for designing and training deep neural networks (DNN) that we call "Deep Regression Ensembles" (DRE) It bridges the gap between DNN and two-layer neural networks trained with random feature regression. Our experiments show that a single DRE architecture is at par with or exceeds state-of-the-art DNN in many data sets.
  arXiv  Detail & Related papers  (2022-03-10T15:13:46Z)
• Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training data [52.771780951404565]
  Shift-Robust GNN (SR-GNN) is designed to account for distributional differences between biased training data and the graph's true inference distribution. We show that SR-GNN outperforms other GNN baselines by accuracy, eliminating at least (40%) of the negative effects introduced by biased training data.
  arXiv  Detail & Related papers  (2021-08-02T18:00:38Z)
• Twin Neural Network Regression is a Semi-Supervised Regression Algorithm [0.90238471756546]
  Twin neural network regression (TNNR) is a semi-supervised regression algorithm. TNNR is trained to predict differences between the target values of two different data points rather than the targets themselves.
  arXiv  Detail & Related papers  (2021-06-11T02:10:52Z)
• Towards an Understanding of Benign Overfitting in Neural Networks [104.2956323934544]
  Modern machine learning models often employ a huge number of parameters and are typically optimized to have zero training loss. We examine how these benign overfitting phenomena occur in a two-layer neural network setting. We show that it is possible for the two-layer ReLU network interpolator to achieve a near minimax-optimal learning rate.
  arXiv  Detail & Related papers  (2021-06-06T19:08:53Z)
• Optimization Variance: Exploring Generalization Properties of DNNs [83.78477167211315]
  The test error of a deep neural network (DNN) often demonstrates double descent. We propose a novel metric, optimization variance (OV), to measure the diversity of model updates.
  arXiv  Detail & Related papers  (2021-06-03T09:34:17Z)
• Improving Uncertainty Calibration via Prior Augmented Data [56.88185136509654]
  Neural networks have proven successful at learning from complex data distributions by acting as universal function approximators. They are often overconfident in their predictions, which leads to inaccurate and miscalibrated probabilistic predictions. We propose a solution by seeking out regions of feature space where the model is unjustifiably overconfident, and conditionally raising the entropy of those predictions towards that of the prior distribution of the labels.
  arXiv  Detail & Related papers  (2021-02-22T07:02:37Z)
• Frequentist Uncertainty in Recurrent Neural Networks via Blockwise Influence Functions [121.10450359856242]
  Recurrent neural networks (RNNs) are instrumental in modelling sequential and time-series data. Existing approaches for uncertainty quantification in RNNs are based predominantly on Bayesian methods. We develop a frequentist alternative that: (a) does not interfere with model training or compromise its accuracy, (b) applies to any RNN architecture, and (c) provides theoretical coverage guarantees on the estimated uncertainty intervals.
  arXiv  Detail & Related papers  (2020-06-20T22:45:32Z)
• One Versus all for deep Neural Network Incertitude (OVNNI) quantification [12.734278426543332]
  We propose a new technique to quantify the epistemic uncertainty of data easily. This method consists in mixing the predictions of an ensemble of DNNs trained to classify One class vs All the other classes (OVA) with predictions from a standard DNN trained to perform All vs All (AVA) classification.
  arXiv  Detail & Related papers  (2020-06-01T14:06:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.