Meta-Learning With Hierarchical Models Based on Similarity of Causal Mechanisms

• URL: http://arxiv.org/abs/2310.12595v2
• Date: Thu, 15 Feb 2024 10:52:33 GMT
• Title: Meta-Learning With Hierarchical Models Based on Similarity of Causal Mechanisms
• Authors: Sophie Wharrie, Samuel Kaski
• Abstract summary: This work is motivated by personalised medicine, where a patient is a task and complex diseases are heterogeneous across patients in cause and progression. We introduce to meta-learning, formulated as Bayesian hierarchical modelling, a proxy measure of similarity of the causal mechanisms of tasks. We show that such pooling improves predictions in three health-related case studies.
• Score: 23.842687721181107
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work the goal is to generalise to new data in a non-iid setting where datasets from related tasks are observed, each generated by a different causal mechanism, and the test dataset comes from the same task distribution. This setup is motivated by personalised medicine, where a patient is a task and complex diseases are heterogeneous across patients in cause and progression. The difficulty is that there usually is not enough data in one task to identify the causal mechanism, and unless the mechanisms are the same, pooling data across tasks, which meta-learning does one way or the other, may lead to worse predictors when the test setting may be uncontrollably different. In this paper we introduce to meta-learning, formulated as Bayesian hierarchical modelling, a proxy measure of similarity of the causal mechanisms of tasks, by learning a suitable embedding of the tasks from the whole data set. This embedding is used as auxiliary data for assessing which tasks should be pooled in the hierarchical model. We show that such pooling improves predictions in three health-related case studies, and by sensitivity analyses on simulated data that the method aids generalisability by utilising interventional data to identify tasks with similar causal mechanisms for pooling, even in limited data settings.

Related papers

• Continual Multiple Instance Learning for Hematologic Disease Diagnosis [38.13262557169157]
  We propose the first continual learning method tailored specifically to multiple instance learning (MIL)<n>Our method is rehearsal-based over a selection of single instances from various bags.<n>We show that our method considerably outperforms state-of-the-art methods, providing the first continual learning approach for MIL.
  arXiv  Detail & Related papers  (2025-08-06T12:03:25Z)
• Approaching Metaheuristic Deep Learning Combos for Automated Data Mining [0.5419570023862531]
  This work proposes a means of combining meta-heuristic methods with conventional classifiers and neural networks in order to perform automated data mining. Experiments on the MNIST dataset for handwritten digit recognition were performed. It was empirically observed that using a ground truth labeled dataset's validation accuracy is inadequate for correcting labels of other previously unseen data instances.
  arXiv  Detail & Related papers  (2024-10-16T10:28:22Z)
• MoMA: Momentum Contrastive Learning with Multi-head Attention-based Knowledge Distillation for Histopathology Image Analysis [5.396167537615578]
  A lack of quality data is a common issue when it comes to a specific task in computational pathology. We propose to exploit knowledge distillation, i.e., utilize the existing model to learn a new, target model. We employ a student-teacher framework to learn a target model from a pre-trained, teacher model without direct access to source data.
  arXiv  Detail & Related papers  (2023-08-31T08:54:59Z)
• iSCAN: Identifying Causal Mechanism Shifts among Nonlinear Additive Noise Models [48.33685559041322]
  This paper focuses on identifying the causal mechanism shifts in two or more related datasets over the same set of variables. Code implementing the proposed method is open-source and publicly available at https://github.com/kevinsbello/iSCAN.
  arXiv  Detail & Related papers  (2023-06-30T01:48:11Z)
• A Meta-Learning Approach to Population-Based Modelling of Structures [0.0]
  A major problem of machine-learning approaches in structural dynamics is the frequent lack of structural data. Inspired by the recently-emerging field of population-based structural health monitoring, this work attempts to create models that are able to transfer knowledge within populations of structures. The models trained using meta-learning approaches, are able to outperform conventional machine learning methods regarding inference about structures of the population.
  arXiv  Detail & Related papers  (2023-02-15T23:01:59Z)
• Data-IQ: Characterizing subgroups with heterogeneous outcomes in tabular data [81.43750358586072]
  We propose Data-IQ, a framework to systematically stratify examples into subgroups with respect to their outcomes. We experimentally demonstrate the benefits of Data-IQ on four real-world medical datasets.
  arXiv  Detail & Related papers  (2022-10-24T08:57:55Z)
• Regressing Relative Fine-Grained Change for Sub-Groups in Unreliable Heterogeneous Data Through Deep Multi-Task Metric Learning [0.5999777817331317]
  We investigate how techniques in multi-task metric learning can be applied for theregression of fine-grained change in real data. The techniques investigated are specifically tailored for handling heterogeneous data sources.
  arXiv  Detail & Related papers  (2022-08-11T12:57:11Z)
• GenHPF: General Healthcare Predictive Framework with Multi-task Multi-source Learning [9.406539794019581]
  General Healthcare Predictive Framework (GenHPF) is applicable to any EHR with minimal preprocessing for multiple prediction tasks. Our framework significantly outperforms baseline models that utilize domain knowledge in multi-source learning.
  arXiv  Detail & Related papers  (2022-07-20T12:46:26Z)
• Identifying the Context Shift between Test Benchmarks and Production Data [1.2259552039796024]
  There exists a performance gap between machine learning models' accuracy on dataset benchmarks and real-world production data. We outline two methods for identifying changes in context that lead to distribution shifts and model prediction errors. We present two case-studies to highlight the implicit assumptions underlying applied machine learning models that tend to lead to errors.
  arXiv  Detail & Related papers  (2022-07-03T14:54:54Z)
• Continual Learning with Bayesian Model based on a Fixed Pre-trained Feature Extractor [55.9023096444383]
  Current deep learning models are characterised by catastrophic forgetting of old knowledge when learning new classes. Inspired by the process of learning new knowledge in human brains, we propose a Bayesian generative model for continual learning.
  arXiv  Detail & Related papers  (2022-04-28T08:41:51Z)
• BERT WEAVER: Using WEight AVERaging to enable lifelong learning for transformer-based models in biomedical semantic search engines [49.75878234192369]
  We present WEAVER, a simple, yet efficient post-processing method that infuses old knowledge into the new model. We show that applying WEAVER in a sequential manner results in similar word embedding distributions as doing a combined training on all data at once.
  arXiv  Detail & Related papers  (2022-02-21T10:34:41Z)
• MIRACLE: Causally-Aware Imputation via Learning Missing Data Mechanisms [82.90843777097606]
  We propose a causally-aware imputation algorithm (MIRACLE) for missing data. MIRACLE iteratively refines the imputation of a baseline by simultaneously modeling the missingness generating mechanism. We conduct extensive experiments on synthetic and a variety of publicly available datasets to show that MIRACLE is able to consistently improve imputation.
  arXiv  Detail & Related papers  (2021-11-04T22:38:18Z)
• MIMO: Mutual Integration of Patient Journey and Medical Ontology for Healthcare Representation Learning [49.57261599776167]
  We propose an end-to-end robust Transformer-based solution, Mutual Integration of patient journey and Medical Ontology (MIMO) for healthcare representation learning and predictive analytics.
  arXiv  Detail & Related papers  (2021-07-20T07:04:52Z)
• Pre-training transformer-based framework on large-scale pediatric claims data for downstream population-specific tasks [3.1580072841682734]
  This study presents the Claim Pre-Training (Claim-PT) framework, a generic pre-training model that first trains on the entire pediatric claims dataset. The effective knowledge transfer is completed through the task-aware fine-tuning stage. We conducted experiments on a real-world claims dataset with more than one million patient records.
  arXiv  Detail & Related papers  (2021-06-24T15:25:41Z)
• Adversarial Sample Enhanced Domain Adaptation: A Case Study on Predictive Modeling with Electronic Health Records [57.75125067744978]
  We propose a data augmentation method to facilitate domain adaptation. adversarially generated samples are used during domain adaptation. Results confirm the effectiveness of our method and the generality on different tasks.
  arXiv  Detail & Related papers  (2021-01-13T03:20:20Z)
• Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction [55.94378672172967]
  We focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients. We introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks. Our new model is built upon a carefully designed meta-learner, called Prototypical Network, that is a simple yet effective meta learning machine for few-shot image classification.
  arXiv  Detail & Related papers  (2020-09-02T02:50:30Z)
• Estimating Structural Target Functions using Machine Learning and Influence Functions [103.47897241856603]
  We propose a new framework for statistical machine learning of target functions arising as identifiable functionals from statistical models. This framework is problem- and model-agnostic and can be used to estimate a broad variety of target parameters of interest in applied statistics. We put particular focus on so-called coarsening at random/doubly robust problems with partially unobserved information.
  arXiv  Detail & Related papers  (2020-08-14T16:48:29Z)
• Causal Feature Selection for Algorithmic Fairness [61.767399505764736]
  We consider fairness in the integration component of data management. We propose an approach to identify a sub-collection of features that ensure the fairness of the dataset.
  arXiv  Detail & Related papers  (2020-06-10T20:20:10Z)
• Ensemble Transfer Learning for the Prediction of Anti-Cancer Drug Response [49.86828302591469]
  In this paper, we apply transfer learning to the prediction of anti-cancer drug response. We apply the classic transfer learning framework that trains a prediction model on the source dataset and refines it on the target dataset. The ensemble transfer learning pipeline is implemented using LightGBM and two deep neural network (DNN) models with different architectures.
  arXiv  Detail & Related papers  (2020-05-13T20:29:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.