Deep and interpretable regression models for ordinal outcomes

• URL: http://arxiv.org/abs/2010.08376v4
• Date: Tue, 20 Apr 2021 15:09:52 GMT
• Title: Deep and interpretable regression models for ordinal outcomes
• Authors: Lucas Kook, Lisa Herzog, Torsten Hothorn, Oliver D\"urr, Beate Sick
• Abstract summary: Outcomes with a natural order commonly occur in prediction tasks and often the available input data are a mixture of complex data. We present ordinal neural network transformation models (ONTRAMs) which unite Deep Learning (DL) with classical ordinal regression approaches. The performance of the most flexible ONTRAM is by definition equivalent to a standard multi-class DL model trained with cross-entropy while being faster in training when facing ordinal outcomes.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Outcomes with a natural order commonly occur in prediction tasks and often the available input data are a mixture of complex data like images and tabular predictors. Deep Learning (DL) models are state-of-the-art for image classification tasks but frequently treat ordinal outcomes as unordered and lack interpretability. In contrast, classical ordinal regression models consider the outcome's order and yield interpretable predictor effects but are limited to tabular data. We present ordinal neural network transformation models (ONTRAMs), which unite DL with classical ordinal regression approaches. ONTRAMs are a special case of transformation models and trade off flexibility and interpretability by additively decomposing the transformation function into terms for image and tabular data using jointly trained neural networks. The performance of the most flexible ONTRAM is by definition equivalent to a standard multi-class DL model trained with cross-entropy while being faster in training when facing ordinal outcomes. Lastly, we discuss how to interpret model components for both tabular and image data on two publicly available datasets.

Related papers

• Mutual Exclusivity Training and Primitive Augmentation to Induce Compositionality [84.94877848357896]
  Recent datasets expose the lack of the systematic generalization ability in standard sequence-to-sequence models. We analyze this behavior of seq2seq models and identify two contributing factors: a lack of mutual exclusivity bias and the tendency to memorize whole examples. We show substantial empirical improvements using standard sequence-to-sequence models on two widely-used compositionality datasets.
  arXiv  Detail & Related papers  (2022-11-28T17:36:41Z)
• Learning the Evolutionary and Multi-scale Graph Structure for Multivariate Time Series Forecasting [50.901984244738806]
  We show how to model the evolutionary and multi-scale interactions of time series. In particular, we first provide a hierarchical graph structure cooperated with the dilated convolution to capture the scale-specific correlations. A unified neural network is provided to integrate the components above to get the final prediction.
  arXiv  Detail & Related papers  (2022-06-28T08:11:12Z)
• On the Generalization and Adaption Performance of Causal Models [99.64022680811281]
  Differentiable causal discovery has proposed to factorize the data generating process into a set of modules. We study the generalization and adaption performance of such modular neural causal models. Our analysis shows that the modular neural causal models outperform other models on both zero and few-shot adaptation in low data regimes.
  arXiv  Detail & Related papers  (2022-06-09T17:12:32Z)
• Performance and Interpretability Comparisons of Supervised Machine Learning Algorithms: An Empirical Study [3.7881729884531805]
  The paper is organized in a findings-based manner, with each section providing general conclusions. Overall, XGB and FFNNs were competitive, with FFNNs showing better performance in smooth models. RF did not perform well in general, confirming the findings in the literature.
  arXiv  Detail & Related papers  (2022-04-27T12:04:33Z)
• Ordinal-ResLogit: Interpretable Deep Residual Neural Networks for Ordered Choices [6.982614422666432]
  We develop a fully interpretable deep learning-based ordinal regression model. Formulations for market share, substitution patterns, and elasticities are derived. Our results show that Ordinal-ResLogit outperforms the traditional ordinal regression model for both datasets.
  arXiv  Detail & Related papers  (2022-04-20T02:14:28Z)
• Continuously Generalized Ordinal Regression for Linear and Deep Models [41.03778663275373]
  Ordinal regression is a classification task where classes have an order and prediction error increases the further the predicted class is from the true class. We propose a new approach for modeling ordinal data that allows class-specific hyperplane slopes. Our method significantly outperforms the standard ordinal logistic model over a thorough set of ordinal regression benchmark datasets.
  arXiv  Detail & Related papers  (2022-02-14T19:49:05Z)
• Discovering Invariant Rationales for Graph Neural Networks [104.61908788639052]
  Intrinsic interpretability of graph neural networks (GNNs) is to find a small subset of the input graph's features. We propose a new strategy of discovering invariant rationale (DIR) to construct intrinsically interpretable GNNs.
  arXiv  Detail & Related papers  (2022-01-30T16:43:40Z)
• X-model: Improving Data Efficiency in Deep Learning with A Minimax Model [78.55482897452417]
  We aim at improving data efficiency for both classification and regression setups in deep learning. To take the power of both worlds, we propose a novel X-model. X-model plays a minimax game between the feature extractor and task-specific heads.
  arXiv  Detail & Related papers  (2021-10-09T13:56:48Z)
• Structured Reordering for Modeling Latent Alignments in Sequence Transduction [86.94309120789396]
  We present an efficient dynamic programming algorithm performing exact marginal inference of separable permutations. The resulting seq2seq model exhibits better systematic generalization than standard models on synthetic problems and NLP tasks.
  arXiv  Detail & Related papers  (2021-06-06T21:53:54Z)
• Learning a binary search with a recurrent neural network. A novel approach to ordinal regression analysis [0.0]
  This article investigates the application of sequence-to-sequence learning methods provided by the deep learning framework in ordinal regression. A method for visualizing the model's explanatory variables according to the ordinal target variable is proposed. The method is compared to traditional ordinal regression methods on a number of benchmark dataset, and is shown to have comparable or significantly better predictive power.
  arXiv  Detail & Related papers  (2021-01-07T16:16:43Z)
• Deep transformation models: Tackling complex regression problems with neural network based transformation models [0.0]
  We present a deep transformation model for probabilistic regression. It estimates the whole conditional probability distribution, which is the most thorough way to capture uncertainty about the outcome. Our method works for complex input data, which we demonstrate by employing a CNN architecture on image data.
  arXiv  Detail & Related papers  (2020-04-01T14:23:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.