LTGC: Long-tail Recognition via Leveraging LLMs-driven Generated Content

• URL: http://arxiv.org/abs/2403.05854v4
• Date: Sun, 26 May 2024 04:22:24 GMT
• Title: LTGC: Long-tail Recognition via Leveraging LLMs-driven Generated Content
• Authors: Qihao Zhao, Yalun Dai, Hao Li, Wei Hu, Fan Zhang, Jun Liu,
• Abstract summary: Long-tail recognition is challenging because it requires the model to learn good representations from tail categories and address imbalances across all categories. We propose a novel generative and fine-tuning framework, LTGC, to handle long-tail recognition via leveraging generated content.
• Score: 17.022005679738733
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long-tail recognition is challenging because it requires the model to learn good representations from tail categories and address imbalances across all categories. In this paper, we propose a novel generative and fine-tuning framework, LTGC, to handle long-tail recognition via leveraging generated content. Firstly, inspired by the rich implicit knowledge in large-scale models (e.g., large language models, LLMs), LTGC leverages the power of these models to parse and reason over the original tail data to produce diverse tail-class content. We then propose several novel designs for LTGC to ensure the quality of the generated data and to efficiently fine-tune the model using both the generated and original data. The visualization demonstrates the effectiveness of the generation module in LTGC, which produces accurate and diverse tail data. Additionally, the experimental results demonstrate that our LTGC outperforms existing state-of-the-art methods on popular long-tailed benchmarks.

Related papers

• WarriorCoder: Learning from Expert Battles to Augment Code Large Language Models [67.15146980023621]
  We propose WarriorCoder, a novel paradigm learns from expert battles to address limitations of current approaches. We create an arena where leading expert code LLMs challenge each other, with evaluations conducted by impartial judges. This competitive framework generates novel training data from scratch, leveraging the strengths of all participants.
  arXiv  Detail & Related papers  (2024-12-23T08:47:42Z)
• Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild [84.57103623507082]
  This paper introduces Model-GLUE, a holistic Large Language Models scaling guideline. We benchmark existing scaling techniques, especially selective merging, and variants of mixture. We then formulate an optimal strategy for the selection and aggregation of a heterogeneous model zoo. Our methodology involves the clustering of mergeable models and optimal merging strategy selection, and the integration of clusters.
  arXiv  Detail & Related papers  (2024-10-07T15:55:55Z)
• HDL-GPT: High-Quality HDL is All You Need [0.0]
  This paper presents Hardware Description Language Generative Pre-trained Transformers (HDL-GPT) HDL-GPT is a novel approach that leverages the vast repository of open-source High Definition Language (HDL) codes to train superior quality large code models. We demonstrate improvements of 50% to 200% over SOTA HDL models on current benchmarks in tasks ranging from HDL circuit explanations, code generation, formal and simulation testbench creation, triaging bugs, and fixing them.
  arXiv  Detail & Related papers  (2024-07-25T22:48:08Z)
• Generative Active Learning for Long-tailed Instance Segmentation [55.66158205855948]
  We propose BSGAL, a new algorithm that estimates the contribution of generated data based on cache gradient. Experiments show that BSGAL outperforms the baseline approach and effectually improves the performance of long-tailed segmentation.
  arXiv  Detail & Related papers  (2024-06-04T15:57:43Z)
• GOLD: Generalized Knowledge Distillation via Out-of-Distribution-Guided Language Data Generation [21.56082253577229]
  Gold is a task-agnostic data generation and knowledge distillation framework. It employs an iterative out-of-distribution-guided feedback mechanism for the LLM. An energy-based OOD evaluation approach is also introduced to deal with noisy generated data.
  arXiv  Detail & Related papers  (2024-03-28T18:08:22Z)
• DiffuLT: How to Make Diffusion Model Useful for Long-tail Recognition [25.842677223769943]
  This paper proposes a new pipeline for long-tail (LT) recognition. Instead of re-weighting or re-sampling, we utilize the long-tailed dataset itself to generate a balanced proxy. Specifically, a randomly diffusion model, trained exclusively on the long-tailed dataset, is employed to synthesize new samples for underrepresented classes.
  arXiv  Detail & Related papers  (2024-03-08T09:19:29Z)
• Augmenting Interpretable Models with LLMs during Training [73.40079895413861]
  We propose Augmented Interpretable Models (Aug-imodels) to build efficient and interpretable models. Aug-imodels use LLMs during fitting but not during inference, allowing complete transparency. We explore two instantiations of Aug-imodels in natural-language processing: (i) Aug-GAM, which augments a generalized additive model with decoupled embeddings from an LLM and (ii) Aug-Tree, which augments a decision tree with LLM feature expansions.
  arXiv  Detail & Related papers  (2022-09-23T18:36:01Z)
• Improving GANs for Long-Tailed Data through Group Spectral Regularization [51.58250647277375]
  We propose a novel group Spectral Regularizer (gSR) that prevents the spectral explosion alleviating mode collapse. We find that gSR effectively combines with existing augmentation and regularization techniques, leading to state-of-the-art image generation performance on long-tailed data.
  arXiv  Detail & Related papers  (2022-08-21T17:51:05Z)
• Teacher Guided Training: An Efficient Framework for Knowledge Transfer [86.6784627427194]
  We propose the teacher-guided training (TGT) framework for training a high-quality compact model. TGT exploits the fact that the teacher has acquired a good representation of the underlying data domain. We find that TGT can improve accuracy on several image classification benchmarks and a range of text classification and retrieval tasks.
  arXiv  Detail & Related papers  (2022-08-14T10:33:58Z)
• Exploring Long Tail Visual Relationship Recognition with Large Vocabulary [40.51076584921913]
  We make the first large-scale study concerning the task of Long-Tail Visual Relationship Recognition (LTVRR) LTVRR aims at improving the learning of structured visual relationships that come from the long-tail. We introduce two LTVRR-related benchmarks, dubbed VG8K-LT and GQA-LT, built upon the widely used Visual Genome and GQA datasets.
  arXiv  Detail & Related papers  (2020-03-25T19:03:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.