Concept Matching for Low-Resource Classification
- URL: http://arxiv.org/abs/2006.00937v1
- Date: Mon, 1 Jun 2020 13:34:01 GMT
- Title: Concept Matching for Low-Resource Classification
- Authors: Federico Errica, Ludovic Denoyer, Bora Edizel, Fabio Petroni, Vassilis
Plachouras, Fabrizio Silvestri, Sebastian Riedel
- Abstract summary: We propose a model to tackle classification tasks in the presence of very little training data.
We approximate the notion of exact match with a theoretically sound mechanism that computes a probability of matching in the input space.
- Score: 36.871182660669746
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: We propose a model to tackle classification tasks in the presence of very
little training data. To this aim, we approximate the notion of exact match
with a theoretically sound mechanism that computes a probability of matching in
the input space. Importantly, the model learns to focus on elements of the
input that are relevant for the task at hand; by leveraging highlighted
portions of the training data, an error boosting technique guides the learning
process. In practice, it increases the error associated with relevant parts of
the input by a given factor. Remarkable results on text classification tasks
confirm the benefits of the proposed approach in both balanced and unbalanced
cases, thus being of practical use when labeling new examples is expensive. In
addition, by inspecting its weights, it is often possible to gather insights on
what the model has learned.
Related papers
- Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple
Logits Retargeting Approach [102.0769560460338]
We develop a simple logits approach (LORT) without the requirement of prior knowledge of the number of samples per class.
Our method achieves state-of-the-art performance on various imbalanced datasets, including CIFAR100-LT, ImageNet-LT, and iNaturalist 2018.
arXiv Detail & Related papers (2024-03-01T03:27:08Z) - Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice.
We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
arXiv Detail & Related papers (2023-12-27T09:03:43Z) - Surprisal Driven $k$-NN for Robust and Interpretable Nonparametric
Learning [1.4293924404819704]
We shed new light on the traditional nearest neighbors algorithm from the perspective of information theory.
We propose a robust and interpretable framework for tasks such as classification, regression, density estimation, and anomaly detection using a single model.
Our work showcases the architecture's versatility by achieving state-of-the-art results in classification and anomaly detection.
arXiv Detail & Related papers (2023-11-17T00:35:38Z) - Training Ensembles with Inliers and Outliers for Semi-supervised Active
Learning [16.204251285425478]
In this work, we present an approach that leverages three highly synergistic components.
Joint training with inliers and outliers, semi-supervised learning through pseudo-labeling, and model ensembling are used.
Remarkably, despite its simplicity, our proposed approach outperforms all other methods in terms of performance.
arXiv Detail & Related papers (2023-07-07T17:50:07Z) - Generalization Properties of Retrieval-based Models [50.35325326050263]
Retrieval-based machine learning methods have enjoyed success on a wide range of problems.
Despite growing literature showcasing the promise of these models, the theoretical underpinning for such models remains underexplored.
We present a formal treatment of retrieval-based models to characterize their generalization ability.
arXiv Detail & Related papers (2022-10-06T00:33:01Z) - CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep
Learning [55.733193075728096]
Modern deep neural networks can easily overfit to biased training data containing corrupted labels or class imbalance.
Sample re-weighting methods are popularly used to alleviate this data bias issue.
We propose a meta-model capable of adaptively learning an explicit weighting scheme directly from data.
arXiv Detail & Related papers (2022-02-11T13:49:51Z) - You Only Need End-to-End Training for Long-Tailed Recognition [8.789819609485225]
Cross-entropy loss tends to produce highly correlated features on imbalanced data.
We propose two novel modules, Block-based Relatively Balanced Batch Sampler (B3RS) and Batch Embedded Training (BET)
Experimental results on the long-tailed classification benchmarks, CIFAR-LT and ImageNet-LT, demonstrate the effectiveness of our method.
arXiv Detail & Related papers (2021-12-11T11:44:09Z) - Contrastive Learning for Fair Representations [50.95604482330149]
Trained classification models can unintentionally lead to biased representations and predictions.
Existing debiasing methods for classification models, such as adversarial training, are often expensive to train and difficult to optimise.
We propose a method for mitigating bias by incorporating contrastive learning, in which instances sharing the same class label are encouraged to have similar representations.
arXiv Detail & Related papers (2021-09-22T10:47:51Z) - Combining Feature and Instance Attribution to Detect Artifacts [62.63504976810927]
We propose methods to facilitate identification of training data artifacts.
We show that this proposed training-feature attribution approach can be used to uncover artifacts in training data.
We execute a small user study to evaluate whether these methods are useful to NLP researchers in practice.
arXiv Detail & Related papers (2021-07-01T09:26:13Z) - Adaptive Prototypical Networks with Label Words and Joint Representation
Learning for Few-Shot Relation Classification [17.237331828747006]
This work focuses on few-shot relation classification (FSRC)
We propose an adaptive mixture mechanism to add label words to the representation of the class prototype.
Experiments have been conducted on FewRel under different few-shot (FS) settings.
arXiv Detail & Related papers (2021-01-10T11:25:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.