A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processing

• URL: http://arxiv.org/abs/2210.04675v1
• Date: Mon, 10 Oct 2022 13:26:40 GMT
• Title: A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processing
• Authors: Sophie Henning, William H. Beluch, Alexander Fraser, Annemarie Friedrich
• Abstract summary: We provide guidance for NLP researchers and practitioners dealing with imbalanced data. We first discuss various types of controlled and real-world class imbalance. We organize the methods by whether they are based on sampling, data augmentation, choice of loss function, staged learning, or model design.
• Score: 68.37496795076203
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many natural language processing (NLP) tasks are naturally imbalanced, as some target categories occur much more frequently than others in the real world. In such scenarios, current NLP models still tend to perform poorly on less frequent classes. Addressing class imbalance in NLP is an active research topic, yet, finding a good approach for a particular task and imbalance scenario is difficult. With this survey, the first overview on class imbalance in deep-learning based NLP, we provide guidance for NLP researchers and practitioners dealing with imbalanced data. We first discuss various types of controlled and real-world class imbalance. Our survey then covers approaches that have been explicitly proposed for class-imbalanced NLP tasks or, originating in the computer vision community, have been evaluated on them. We organize the methods by whether they are based on sampling, data augmentation, choice of loss function, staged learning, or model design. Finally, we discuss open problems such as dealing with multi-label scenarios, and propose systematic benchmarking and reporting in order to move forward on this problem as a community.

Related papers

• Uncertainty Aware Learning for Language Model Alignment [97.36361196793929]
  We propose uncertainty-aware learning (UAL) to improve the model alignment of different task scenarios. We implement UAL in a simple fashion -- adaptively setting the label smoothing value of training according to the uncertainty of individual samples. Experiments on widely used benchmarks demonstrate that our UAL significantly and consistently outperforms standard supervised fine-tuning.
  arXiv  Detail & Related papers  (2024-06-07T11:37:45Z)
• NLPBench: Evaluating Large Language Models on Solving NLP Problems [41.01588131136101]
  Large language models (LLMs) have shown promise in enhancing the capabilities of natural language processing (NLP) We present a unique benchmarking dataset, NLPBench, comprising 378 college-level NLP questions spanning various NLP topics sourced from Yale University's prior final exams. Our evaluation, centered on LLMs such as GPT-3.5/4, PaLM-2, and LLAMA-2, incorporates advanced prompting strategies like the chain-of-thought (CoT) and tree-of-thought (ToT)
  arXiv  Detail & Related papers  (2023-09-27T13:02:06Z)
• Uncertainty in Natural Language Processing: Sources, Quantification, and Applications [56.130945359053776]
  We provide a comprehensive review of uncertainty-relevant works in the NLP field. We first categorize the sources of uncertainty in natural language into three types, including input, system, and output. We discuss the challenges of uncertainty estimation in NLP and discuss potential future directions.
  arXiv  Detail & Related papers  (2023-06-05T06:46:53Z)
• Active Learning Principles for In-Context Learning with Large Language Models [65.09970281795769]
  This paper investigates how Active Learning algorithms can serve as effective demonstration selection methods for in-context learning. We show that in-context example selection through AL prioritizes high-quality examples that exhibit low uncertainty and bear similarity to the test examples.
  arXiv  Detail & Related papers  (2023-05-23T17:16:04Z)
• Adaptive Ranking-based Sample Selection for Weakly Supervised Class-imbalanced Text Classification [4.151073288078749]
  We propose Adaptive Ranking-based Sample Selection (ARS2) to alleviate the data imbalance issue in the weak supervision (WS) paradigm. ARS2 calculates a probabilistic margin score based on the output of the current model to measure and rank the cleanliness of each data point. Experiments show that ARS2 outperformed the state-of-the-art imbalanced learning and WS methods, leading to a 2%-57.8% improvement on their F1-score.
  arXiv  Detail & Related papers  (2022-10-06T17:49:22Z)
• Class-Imbalanced Complementary-Label Learning via Weighted Loss [8.934943507699131]
  Complementary-label learning (CLL) is widely used in weakly supervised classification. It faces a significant challenge in real-world datasets when confronted with class-imbalanced training samples. We propose a novel problem setting that enables learning from class-imbalanced complementary labels for multi-class classification.
  arXiv  Detail & Related papers  (2022-09-28T16:02:42Z)
• CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learning [55.733193075728096]
  Modern deep neural networks can easily overfit to biased training data containing corrupted labels or class imbalance. Sample re-weighting methods are popularly used to alleviate this data bias issue. We propose a meta-model capable of adaptively learning an explicit weighting scheme directly from data.
  arXiv  Detail & Related papers  (2022-02-11T13:49:51Z)
• Fairness-aware Class Imbalanced Learning [57.45784950421179]
  We evaluate long-tail learning methods for tweet sentiment and occupation classification. We extend a margin-loss based approach with methods to enforce fairness.
  arXiv  Detail & Related papers  (2021-09-21T22:16:30Z)
• Low-Resource Adaptation of Neural NLP Models [0.30458514384586405]
  This thesis investigates methods for dealing with low-resource scenarios in information extraction and natural language understanding. We develop and adapt neural NLP models to explore a number of research questions concerning NLP tasks with minimal or no training data.
  arXiv  Detail & Related papers  (2020-11-09T12:13:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.