Interpretable Neural Networks for Panel Data Analysis in Economics

• URL: http://arxiv.org/abs/2010.05311v3
• Date: Sun, 29 Nov 2020 14:57:43 GMT
• Title: Interpretable Neural Networks for Panel Data Analysis in Economics
• Authors: Yucheng Yang, Zhong Zheng, Weinan E
• Abstract summary: We propose a class of interpretable neural network models that can achieve both high prediction accuracy and interpretability. We apply the model to predicting individual's monthly employment status using high-dimensional administrative data. We achieve an accuracy of 94.5% in the test set, which is comparable to the best performed conventional machine learning methods.
• Score: 10.57079240576682
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The lack of interpretability and transparency are preventing economists from using advanced tools like neural networks in their empirical research. In this paper, we propose a class of interpretable neural network models that can achieve both high prediction accuracy and interpretability. The model can be written as a simple function of a regularized number of interpretable features, which are outcomes of interpretable functions encoded in the neural network. Researchers can design different forms of interpretable functions based on the nature of their tasks. In particular, we encode a class of interpretable functions named persistent change filters in the neural network to study time series cross-sectional data. We apply the model to predicting individual's monthly employment status using high-dimensional administrative data. We achieve an accuracy of 94.5% in the test set, which is comparable to the best performed conventional machine learning methods. Furthermore, the interpretability of the model allows us to understand the mechanism that underlies the prediction: an individual's employment status is closely related to whether she pays different types of insurances. Our work is a useful step towards overcoming the black-box problem of neural networks, and provide a new tool for economists to study administrative and proprietary big data.

Related papers

• Coding schemes in neural networks learning classification tasks [52.22978725954347]
  We investigate fully-connected, wide neural networks learning classification tasks. We show that the networks acquire strong, data-dependent features. Surprisingly, the nature of the internal representations depends crucially on the neuronal nonlinearity.
  arXiv  Detail & Related papers  (2024-06-24T14:50:05Z)
• From Neurons to Neutrons: A Case Study in Interpretability [5.242869847419834]
  We argue that high-dimensional neural networks can learn low-dimensional representations of their training data that are useful beyond simply making good predictions. This indicates that such approaches to interpretability can be useful for deriving a new understanding of a problem from models trained to solve it.
  arXiv  Detail & Related papers  (2024-05-27T17:59:35Z)
• The Contextual Lasso: Sparse Linear Models via Deep Neural Networks [5.607237982617641]
  We develop a new statistical estimator that fits a sparse linear model to the explanatory features such that the sparsity pattern and coefficients vary as a function of the contextual features. An extensive suite of experiments on real and synthetic data suggests that the learned models, which remain highly transparent, can be sparser than the regular lasso.
  arXiv  Detail & Related papers  (2023-02-02T05:00:29Z)
• Contrastive Brain Network Learning via Hierarchical Signed Graph Pooling Model [64.29487107585665]
  Graph representation learning techniques on brain functional networks can facilitate the discovery of novel biomarkers for clinical phenotypes and neurodegenerative diseases. Here, we propose an interpretable hierarchical signed graph representation learning model to extract graph-level representations from brain functional networks. In order to further improve the model performance, we also propose a new strategy to augment functional brain network data for contrastive learning.
  arXiv  Detail & Related papers  (2022-07-14T20:03:52Z)
• A Theoretical Analysis on Feature Learning in Neural Networks: Emergence from Inputs and Advantage over Fixed Features [18.321479102352875]
  An important characteristic of neural networks is their ability to learn representations of the input data with effective features for prediction. We consider learning problems motivated by practical data, where the labels are determined by a set of class relevant patterns and the inputs are generated from these. We prove that neural networks trained by gradient descent can succeed on these problems.
  arXiv  Detail & Related papers  (2022-06-03T17:49:38Z)
• Convolutional Motif Kernel Networks [1.104960878651584]
  We show that our model is able to robustly learn on small datasets and reaches state-of-the-art performance on relevant healthcare prediction tasks. Our proposed method can be utilized on DNA and protein sequences.
  arXiv  Detail & Related papers  (2021-11-03T15:06:09Z)
• Dynamic Inference with Neural Interpreters [72.90231306252007]
  We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules. inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
  arXiv  Detail & Related papers  (2021-10-12T23:22:45Z)
• FF-NSL: Feed-Forward Neural-Symbolic Learner [70.978007919101]
  This paper introduces a neural-symbolic learning framework, called Feed-Forward Neural-Symbolic Learner (FF-NSL) FF-NSL integrates state-of-the-art ILP systems based on the Answer Set semantics, with neural networks, in order to learn interpretable hypotheses from labelled unstructured data.
  arXiv  Detail & Related papers  (2021-06-24T15:38:34Z)
• Generative Counterfactuals for Neural Networks via Attribute-Informed Perturbation [51.29486247405601]
  We design a framework to generate counterfactuals for raw data instances with the proposed Attribute-Informed Perturbation (AIP) By utilizing generative models conditioned with different attributes, counterfactuals with desired labels can be obtained effectively and efficiently. Experimental results on real-world texts and images demonstrate the effectiveness, sample quality as well as efficiency of our designed framework.
  arXiv  Detail & Related papers  (2021-01-18T08:37:13Z)
• Consistent feature selection for neural networks via Adaptive Group Lasso [3.42658286826597]
  We propose and establish a theoretical guarantee for the use of the adaptive group for selecting important features of neural networks. Specifically, we show that our feature selection method is consistent for single-output feed-forward neural networks with one hidden layer and hyperbolic tangent activation function.
  arXiv  Detail & Related papers  (2020-05-30T18:50:56Z)
• Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision [57.14468881854616]
  We propose an auxiliary training objective that improves the generalization capabilities of neural networks. We use pairs of minimally-different examples with different labels, a.k.a counterfactual or contrasting examples, which provide a signal indicative of the underlying causal structure of the task. Models trained with this technique demonstrate improved performance on out-of-distribution test sets.
  arXiv  Detail & Related papers  (2020-04-20T02:47:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.