Diversity Over Quantity: A Lesson From Few Shot Relation Classification

• URL: http://arxiv.org/abs/2412.05434v1
• Date: Fri, 06 Dec 2024 21:41:01 GMT
• Title: Diversity Over Quantity: A Lesson From Few Shot Relation Classification
• Authors: Amir DN Cohen, Shauli Ravfogel, Shaltiel Shmidman, Yoav Goldberg,
• Abstract summary: We show that training on a diverse set of relations significantly enhances a model's ability to generalize to unseen relations. We introduce REBEL-FS, a new FSRC benchmark that incorporates an order of magnitude more relation types than existing datasets.
• Score: 62.66895901654023
• License:
• Abstract: In few-shot relation classification (FSRC), models must generalize to novel relations with only a few labeled examples. While much of the recent progress in NLP has focused on scaling data size, we argue that diversity in relation types is more crucial for FSRC performance. In this work, we demonstrate that training on a diverse set of relations significantly enhances a model's ability to generalize to unseen relations, even when the overall dataset size remains fixed. We introduce REBEL-FS, a new FSRC benchmark that incorporates an order of magnitude more relation types than existing datasets. Through systematic experiments, we show that increasing the diversity of relation types in the training data leads to consistent gains in performance across various few-shot learning scenarios, including high-negative settings. Our findings challenge the common assumption that more data alone leads to better performance and suggest that targeted data curation focused on diversity can substantially reduce the need for large-scale datasets in FSRC.

Related papers

• Exploring the Efficacy of Meta-Learning: Unveiling Superior Data Diversity Utilization of MAML Over Pre-training [1.3980986259786223]
  We show that dataset diversity can impact the performance of vision models. Our study shows positive correlations between test set accuracy and data diversity. These findings support our hypothesis and demonstrate a promising way for a deeper exploration of how formal data diversity influences model performance.
  arXiv  Detail & Related papers  (2025-01-15T00:56:59Z)
• $\textbf{Only-IF}$:Revealing the Decisive Effect of Instruction Diversity on Generalization [1.6958018695660049]
  We show that $textbfonly emerges$ when training data is diversified enough across semantic domains. We extend our analysis to real-world scenarios, including fine-tuning of $textit$textbfspecialist$$ and $textit$textbfgeneralist$$ models.
  arXiv  Detail & Related papers  (2024-10-07T03:15:11Z)
• Large Margin Prototypical Network for Few-shot Relation Classification with Fine-grained Features [30.11073476165794]
  Relation classification (RC) plays a pivotal role in both natural language understanding and knowledge graph completion. Conventional approaches on RC, regardless of feature engineering or deep learning based, can obtain promising performance on categorizing common types of relation. In this paper, we consider few-shot learning is of great practical significance to RC and thus improve a modern framework of metric learning for few-shot RC.
  arXiv  Detail & Related papers  (2024-09-06T03:28:38Z)
• Balancing Similarity and Complementarity for Federated Learning [91.65503655796603]
  Federated Learning (FL) is increasingly important in mobile and IoT systems. One key challenge in FL is managing statistical heterogeneity, such as non-i.i.d. data. We introduce a novel framework, textttFedSaC, which balances similarity and complementarity in FL cooperation.
  arXiv  Detail & Related papers  (2024-05-16T08:16:19Z)
• RelVAE: Generative Pretraining for few-shot Visual Relationship Detection [2.2230760534775915]
  We present the first pretraining method for few-shot predicate classification that does not require any annotated relations. We construct few-shot training splits and show quantitative experiments on VG200 and VRD datasets.
  arXiv  Detail & Related papers  (2023-11-27T19:08:08Z)
• Leveraging Diffusion Disentangled Representations to Mitigate Shortcuts in Underspecified Visual Tasks [92.32670915472099]
  We propose an ensemble diversification framework exploiting the generation of synthetic counterfactuals using Diffusion Probabilistic Models (DPMs) We show that diffusion-guided diversification can lead models to avert attention from shortcut cues, achieving ensemble diversity performance comparable to previous methods requiring additional data collection.
  arXiv  Detail & Related papers  (2023-10-03T17:37:52Z)
• Detail Reinforcement Diffusion Model: Augmentation Fine-Grained Visual Categorization in Few-Shot Conditions [11.121652649243119]
  Diffusion models have been widely adopted in data augmentation due to their outstanding diversity in data generation. We propose a novel approach termed the detail reinforcement diffusion model(DRDM) It leverages the rich knowledge of large models for fine-grained data augmentation and comprises two key components including discriminative semantic recombination (DSR) and spatial knowledge reference(SKR)
  arXiv  Detail & Related papers  (2023-09-15T01:28:59Z)
• Evaluating and Incentivizing Diverse Data Contributions in Collaborative Learning [89.21177894013225]
  For a federated learning model to perform well, it is crucial to have a diverse and representative dataset. We show that the statistical criterion used to quantify the diversity of the data, as well as the choice of the federated learning algorithm used, has a significant effect on the resulting equilibrium. We leverage this to design simple optimal federated learning mechanisms that encourage data collectors to contribute data representative of the global population.
  arXiv  Detail & Related papers  (2023-06-08T23:38:25Z)
• Towards Realistic Low-resource Relation Extraction: A Benchmark with Empirical Baseline Study [51.33182775762785]
  This paper presents an empirical study to build relation extraction systems in low-resource settings. We investigate three schemes to evaluate the performance in low-resource settings: (i) different types of prompt-based methods with few-shot labeled data; (ii) diverse balancing methods to address the long-tailed distribution issue; and (iii) data augmentation technologies and self-training to generate more labeled in-domain data.
  arXiv  Detail & Related papers  (2022-10-19T15:46:37Z)
• Relation-Guided Representation Learning [53.60351496449232]
  We propose a new representation learning method that explicitly models and leverages sample relations. Our framework well preserves the relations between samples. By seeking to embed samples into subspace, we show that our method can address the large-scale and out-of-sample problem.
  arXiv  Detail & Related papers  (2020-07-11T10:57:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.