The Missing Link: Allocation Performance in Causal Machine Learning

• URL: http://arxiv.org/abs/2407.10779v1
• Date: Mon, 15 Jul 2024 14:57:40 GMT
• Title: The Missing Link: Allocation Performance in Causal Machine Learning
• Authors: Unai Fischer-Abaigar, Christoph Kern, Frauke Kreuter,
• Abstract summary: We show how the performance of a single CATE model can vary significantly across different decision-making scenarios. We highlight the differential influence of challenges such as distribution shifts on predictions and allocations.
• Score: 7.093692674858259
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automated decision-making (ADM) systems are being deployed across a diverse range of critical problem areas such as social welfare and healthcare. Recent work highlights the importance of causal ML models in ADM systems, but implementing them in complex social environments poses significant challenges. Research on how these challenges impact the performance in specific downstream decision-making tasks is limited. Addressing this gap, we make use of a comprehensive real-world dataset of jobseekers to illustrate how the performance of a single CATE model can vary significantly across different decision-making scenarios and highlight the differential influence of challenges such as distribution shifts on predictions and allocations.

Related papers

• Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making [85.24399869971236]
  We aim to evaluate Large Language Models (LLMs) for embodied decision making. Existing evaluations tend to rely solely on a final success rate. We propose a generalized interface (Embodied Agent Interface) that supports the formalization of various types of tasks.
  arXiv  Detail & Related papers  (2024-10-09T17:59:00Z)
• Most Influential Subset Selection: Challenges, Promises, and Beyond [9.479235005673683]
  We study the Most Influential Subset Selection (MISS) problem, which aims to identify a subset of training samples with the greatest collective influence. We conduct a comprehensive analysis of the prevailing approaches in MISS, elucidating their strengths and weaknesses. We demonstrate that an adaptive version of theses which applies them iteratively, can effectively capture the interactions among samples.
  arXiv  Detail & Related papers  (2024-09-25T20:00:23Z)
• Meta Reasoning for Large Language Models [58.87183757029041]
  We introduce Meta-Reasoning Prompting (MRP), a novel and efficient system prompting method for large language models (LLMs) MRP guides LLMs to dynamically select and apply different reasoning methods based on the specific requirements of each task. We evaluate the effectiveness of MRP through comprehensive benchmarks.
  arXiv  Detail & Related papers  (2024-06-17T16:14:11Z)
• MARS: Benchmarking the Metaphysical Reasoning Abilities of Language Models with a Multi-task Evaluation Dataset [50.36095192314595]
  Large Language Models (LLMs) function as conscious agents with generalizable reasoning capabilities. This ability remains underexplored due to the complexity of modeling infinite possible changes in an event. We introduce the first-ever benchmark, MARS, comprising three tasks corresponding to each step.
  arXiv  Detail & Related papers  (2024-06-04T08:35:04Z)
• On Task Performance and Model Calibration with Supervised and Self-Ensembled In-Context Learning [71.44986275228747]
  In-context learning (ICL) has become an efficient approach propelled by the recent advancements in large language models (LLMs) However, both paradigms are prone to suffer from the critical problem of overconfidence (i.e., miscalibration)
  arXiv  Detail & Related papers  (2023-12-21T11:55:10Z)
• Some challenges of calibrating differentiable agent-based models [0.0]
  Agent-based models (ABMs) are promising approach to modelling and reasoning about complex systems. Their application in practice is impeded by their complexity, discrete nature, and the difficulty of performing parameter inference and optimisation tasks.
  arXiv  Detail & Related papers  (2023-07-03T15:07:10Z)
• Causal Fairness Analysis [68.12191782657437]
  We introduce a framework for understanding, modeling, and possibly solving issues of fairness in decision-making settings. The main insight of our approach will be to link the quantification of the disparities present on the observed data with the underlying, and often unobserved, collection of causal mechanisms. Our effort culminates in the Fairness Map, which is the first systematic attempt to organize and explain the relationship between different criteria found in the literature.
  arXiv  Detail & Related papers  (2022-07-23T01:06:34Z)
• Context matters for fairness -- a case study on the effect of spatial distribution shifts [10.351739012146378]
  We present a case study on the newly released American Census datasets. We show how remarkably can spatial distribution shifts affect predictive- and fairness-related performance of a model. Our study suggests that robustness to distribution shifts is necessary before deploying a model to another context.
  arXiv  Detail & Related papers  (2022-06-23T01:09:46Z)
• Seeing Differently, Acting Similarly: Imitation Learning with Heterogeneous Observations [126.78199124026398]
  In many real-world imitation learning tasks, the demonstrator and the learner have to act in different but full observation spaces. In this work, we model the above learning problem as Heterogeneous Observations Learning (HOIL) We propose the Importance Weighting with REjection (IWRE) algorithm based on the techniques of importance-weighting, learning with rejection, and active querying to solve the key challenge of occupancy measure matching.
  arXiv  Detail & Related papers  (2021-06-17T05:44:04Z)
• RDMSim: An Exemplar for Evaluation and Comparison of Decision-Making Techniques for Self-Adaptation [1.846852980615761]
  RDMSim enables researchers to evaluate and compare decision-making techniques for self-adaptation. The focus of the exemplar is on the domain problem related to Remote Data Mirroring.
  arXiv  Detail & Related papers  (2021-05-05T11:03:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.