AdapTable: Test-Time Adaptation for Tabular Data via Shift-Aware Uncertainty Calibrator and Label Distribution Handler

• URL: http://arxiv.org/abs/2407.10784v3
• Date: Mon, 26 Aug 2024 04:58:15 GMT
• Title: AdapTable: Test-Time Adaptation for Tabular Data via Shift-Aware Uncertainty Calibrator and Label Distribution Handler
• Authors: Changhun Kim, Taewon Kim, Seungyeon Woo, June Yong Yang, Eunho Yang,
• Abstract summary: We propose AdapTable, a framework for adapting machine learning models to target data without accessing source data. AdapTable operates in two stages: 1) calibrating model predictions using a shift-aware uncertainty calibrator, and 2) adjusting these predictions to match the target label distribution with a label distribution handler. Our results demonstrate AdapTable's ability to handle various real-world distribution shifts, achieving up to a 16% improvement on the dataset.
• Score: 29.395855812763617
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In real-world scenarios, tabular data often suffer from distribution shifts that threaten the performance of machine learning models. Despite its prevalence and importance, handling distribution shifts in the tabular domain remains underexplored due to the inherent challenges within the tabular data itself. In this sense, test-time adaptation (TTA) offers a promising solution by adapting models to target data without accessing source data, crucial for privacy-sensitive tabular domains. However, existing TTA methods either 1) overlook the nature of tabular distribution shifts, often involving label distribution shifts, or 2) impose architectural constraints on the model, leading to a lack of applicability. To this end, we propose AdapTable, a novel TTA framework for tabular data. AdapTable operates in two stages: 1) calibrating model predictions using a shift-aware uncertainty calibrator, and 2) adjusting these predictions to match the target label distribution with a label distribution handler. We validate the effectiveness of AdapTable through theoretical analysis and extensive experiments on various distribution shift scenarios. Our results demonstrate AdapTable's ability to handle various real-world distribution shifts, achieving up to a 16% improvement on the HELOC dataset.

Related papers

• Label Distribution Shift-Aware Prediction Refinement for Test-Time Adaptation [12.231303059518229]
  We introduce label Distribution shift-Aware prediction Refinement for Test-time adaptation (DART) DART is a novel TTA method that refines predictions by focusing on class-wise confusion patterns. Our method exhibits 5-18% gains in accuracy under label distribution shifts on CIFAR-10C, without any performance degradation when there is no label distribution shift.
  arXiv  Detail & Related papers  (2024-11-20T05:58:52Z)
• Diffusion-nested Auto-Regressive Synthesis of Heterogeneous Tabular Data [56.48119008663155]
  This paper proposes a Diffusion-nested Autoregressive model (TabDAR) to address these issues. We conduct extensive experiments on ten datasets with distinct properties, and the proposed TabDAR outperforms previous state-of-the-art methods by 18% to 45% on eight metrics across three distinct aspects.
  arXiv  Detail & Related papers  (2024-10-28T20:49:26Z)
• Adapting Conformal Prediction to Distribution Shifts Without Labels [16.478151550456804]
  Conformal prediction (CP) enables machine learning models to output prediction sets with guaranteed coverage rate. Our goal is to improve the quality of CP-generated prediction sets using only unlabeled data from the test domain. This is achieved by two new methods called ECP and EACP, that adjust the score function in CP according to the base model's uncertainty on the unlabeled test data.
  arXiv  Detail & Related papers  (2024-06-03T15:16:02Z)
• Channel-Selective Normalization for Label-Shift Robust Test-Time Adaptation [16.657929958093824]
  Test-time adaptation is an approach to adjust models to a new data distribution during inference. Test-time batch normalization is a simple and popular method that achieved compelling performance on domain shift benchmarks. We propose to tackle this challenge by only selectively adapting channels in a deep network, minimizing drastic adaptation that is sensitive to label shifts.
  arXiv  Detail & Related papers  (2024-02-07T15:41:01Z)
• Benchmarking Distribution Shift in Tabular Data with TableShift [32.071534049494076]
  TableShift is a distribution shift benchmark for tabular data. It covers domains including finance, education, public policy, healthcare, and civic participation. We conduct a large-scale study comparing several state-of-the-art data models alongside robust learning and domain generalization methods.
  arXiv  Detail & Related papers  (2023-12-10T18:19:07Z)
• Training-Free Generalization on Heterogeneous Tabular Data via Meta-Representation [67.30538142519067]
  We propose Tabular data Pre-Training via Meta-representation (TabPTM) A deep neural network is then trained to associate these meta-representations with dataset-specific classification confidences. Experiments validate that TabPTM achieves promising performance in new datasets, even under few-shot scenarios.
  arXiv  Detail & Related papers  (2023-10-31T18:03:54Z)
• Adaptive Test-Time Personalization for Federated Learning [51.25437606915392]
  We introduce a novel setting called test-time personalized federated learning (TTPFL) In TTPFL, clients locally adapt a global model in an unsupervised way without relying on any labeled data during test-time. We propose a novel algorithm called ATP to adaptively learn the adaptation rates for each module in the model from distribution shifts among source domains.
  arXiv  Detail & Related papers  (2023-10-28T20:42:47Z)
• Generalized Robust Test-Time Adaptation in Continuous Dynamic Scenarios [18.527640606971563]
  Test-time adaptation (TTA) adapts pre-trained models to test distributions during the inference phase exclusively employing unlabeled test data streams. We propose a Generalized Robust Test-Time Adaptation (GRoTTA) method to effectively address the difficult problem.
  arXiv  Detail & Related papers  (2023-10-07T07:13:49Z)
• Towards Stable Test-Time Adaptation in Dynamic Wild World [60.98073673220025]
  Test-time adaptation (TTA) has shown to be effective at tackling distribution shifts between training and testing data by adapting a given model on test samples. Online model updating of TTA may be unstable and this is often a key obstacle preventing existing TTA methods from being deployed in the real world.
  arXiv  Detail & Related papers  (2023-02-24T02:03:41Z)
• CAFA: Class-Aware Feature Alignment for Test-Time Adaptation [50.26963784271912]
  Test-time adaptation (TTA) aims to address this challenge by adapting a model to unlabeled data at test time. We propose a simple yet effective feature alignment loss, termed as Class-Aware Feature Alignment (CAFA), which simultaneously encourages a model to learn target representations in a class-discriminative manner.
  arXiv  Detail & Related papers  (2022-06-01T03:02:07Z)
• Certifying Model Accuracy under Distribution Shifts [151.67113334248464]
  We present provable robustness guarantees on the accuracy of a model under bounded Wasserstein shifts of the data distribution. We show that a simple procedure that randomizes the input of the model within a transformation space is provably robust to distributional shifts under the transformation.
  arXiv  Detail & Related papers  (2022-01-28T22:03:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.