Combining Discrete Choice Models and Neural Networks through Embeddings: Formulation, Interpretability and Performance

• URL: http://arxiv.org/abs/2109.12042v1
• Date: Fri, 24 Sep 2021 15:55:31 GMT
• Title: Combining Discrete Choice Models and Neural Networks through Embeddings: Formulation, Interpretability and Performance
• Authors: Ioanna Arkoudi, Carlos Lima Azevedo, Francisco C. Pereira
• Abstract summary: This study proposes a novel approach that combines theory and data-driven choice models using Artificial Neural Networks (ANNs) In particular, we use continuous vector representations, called embeddings, for encoding categorical or discrete explanatory variables. Our models deliver state-of-the-art predictive performance, outperforming existing ANN-based models while drastically reducing the number of required network parameters.
• Score: 10.57079240576682
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This study proposes a novel approach that combines theory and data-driven choice models using Artificial Neural Networks (ANNs). In particular, we use continuous vector representations, called embeddings, for encoding categorical or discrete explanatory variables with a special focus on interpretability and model transparency. Although embedding representations within the logit framework have been conceptualized by Camara (2019), their dimensions do not have an absolute definitive meaning, hence offering limited behavioral insights. The novelty of our work lies in enforcing interpretability to the embedding vectors by formally associating each of their dimensions to a choice alternative. Thus, our approach brings benefits much beyond a simple parsimonious representation improvement over dummy encoding, as it provides behaviorally meaningful outputs that can be used in travel demand analysis and policy decisions. Additionally, in contrast to previously suggested ANN-based Discrete Choice Models (DCMs) that either sacrifice interpretability for performance or are only partially interpretable, our models preserve interpretability of the utility coefficients for all the input variables despite being based on ANN principles. The proposed models were tested on two real world datasets and evaluated against benchmark and baseline models that use dummy-encoding. The results of the experiments indicate that our models deliver state-of-the-art predictive performance, outperforming existing ANN-based models while drastically reducing the number of required network parameters.

Related papers

• Interpret the Internal States of Recommendation Model with Sparse Autoencoder [26.021277330699963]
  RecSAE is an automatic, generalizable probing method for interpreting the internal states of Recommendation models. We train an autoencoder with sparsity constraints to reconstruct internal activations of recommendation models. We automated the construction of concept dictionaries based on the relationship between latent activations and input item sequences.
  arXiv  Detail & Related papers  (2024-11-09T08:22:31Z)
• Meaning Representations from Trajectories in Autoregressive Models [106.63181745054571]
  We propose to extract meaning representations from autoregressive language models by considering the distribution of all possible trajectories extending an input text. This strategy is prompt-free, does not require fine-tuning, and is applicable to any pre-trained autoregressive model. We empirically show that the representations obtained from large models align well with human annotations, outperform other zero-shot and prompt-free methods on semantic similarity tasks, and can be used to solve more complex entailment and containment tasks that standard embeddings cannot handle.
  arXiv  Detail & Related papers  (2023-10-23T04:35:58Z)
• Discrete-Choice Model with Generalized Additive Utility Network [0.0]
  Multinomial logit models (MNLs) with linear utility functions have been used in practice because they are ease to use and interpretable. We developed utility functions with a novel neural-network architecture based on generalized additive models. Our models were comparable to ASU-DNN in accuracy and exhibited improved interpretability compared to previous models.
  arXiv  Detail & Related papers  (2023-09-29T04:40:01Z)
• ProtoVAE: Prototypical Networks for Unsupervised Disentanglement [1.6114012813668934]
  We introduce a novel deep generative VAE-based model, ProtoVAE, that leverages a deep metric learning Prototypical network trained using self-supervision. Our model is completely unsupervised and requires no priori knowledge of the dataset, including the number of factors. We evaluate our proposed model on the benchmark dSprites, 3DShapes, and MPI3D disentanglement datasets.
  arXiv  Detail & Related papers  (2023-05-16T01:29:26Z)
• Neural Additive Models for Location Scale and Shape: A Framework for Interpretable Neural Regression Beyond the Mean [1.0923877073891446]
  Deep neural networks (DNNs) have proven to be highly effective in a variety of tasks. Despite this success, the inner workings of DNNs are often not transparent. This lack of interpretability has led to increased research on inherently interpretable neural networks.
  arXiv  Detail & Related papers  (2023-01-27T17:06:13Z)
• Fair Interpretable Representation Learning with Correction Vectors [60.0806628713968]
  We propose a new framework for fair representation learning that is centered around the learning of "correction vectors" We show experimentally that several fair representation learning models constrained in such a way do not exhibit losses in ranking or classification performance.
  arXiv  Detail & Related papers  (2022-02-07T11:19:23Z)
• It's FLAN time! Summing feature-wise latent representations for interpretability [0.0]
  We propose a novel class of structurally-constrained neural networks, which we call FLANs (Feature-wise Latent Additive Networks) FLANs process each input feature separately, computing for each of them a representation in a common latent space. These feature-wise latent representations are then simply summed, and the aggregated representation is used for prediction.
  arXiv  Detail & Related papers  (2021-06-18T12:19:33Z)
• Generative Counterfactuals for Neural Networks via Attribute-Informed Perturbation [51.29486247405601]
  We design a framework to generate counterfactuals for raw data instances with the proposed Attribute-Informed Perturbation (AIP) By utilizing generative models conditioned with different attributes, counterfactuals with desired labels can be obtained effectively and efficiently. Experimental results on real-world texts and images demonstrate the effectiveness, sample quality as well as efficiency of our designed framework.
  arXiv  Detail & Related papers  (2021-01-18T08:37:13Z)
• Explaining and Improving Model Behavior with k Nearest Neighbor Representations [107.24850861390196]
  We propose using k nearest neighbor representations to identify training examples responsible for a model's predictions. We show that kNN representations are effective at uncovering learned spurious associations. Our results indicate that the kNN approach makes the finetuned model more robust to adversarial inputs.
  arXiv  Detail & Related papers  (2020-10-18T16:55:25Z)
• Interpreting Graph Neural Networks for NLP With Differentiable Edge Masking [63.49779304362376]
  Graph neural networks (GNNs) have become a popular approach to integrating structural inductive biases into NLP models. We introduce a post-hoc method for interpreting the predictions of GNNs which identifies unnecessary edges. We show that we can drop a large proportion of edges without deteriorating the performance of the model.
  arXiv  Detail & Related papers  (2020-10-01T17:51:19Z)
• Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study [81.11161697133095]
  We take the NER task as a testbed to analyze the generalization behavior of existing models from different perspectives. Experiments with in-depth analyses diagnose the bottleneck of existing neural NER models. As a by-product of this paper, we have open-sourced a project that involves a comprehensive summary of recent NER papers.
  arXiv  Detail & Related papers  (2020-01-12T04:33:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.