Distance Sampling-based Paraphraser Leveraging ChatGPT for Text Data Manipulation

• URL: http://arxiv.org/abs/2405.00367v1
• Date: Wed, 1 May 2024 07:44:28 GMT
• Title: Distance Sampling-based Paraphraser Leveraging ChatGPT for Text Data Manipulation
• Authors: Yoori Oh, Yoseob Han, Kyogu Lee,
• Abstract summary: We propose a novel approach to tackle the data imbalance problem in audio-language retrieval task. A distance sampling-based paraphraser leveraging ChatGPT generates a controllable distribution of manipulated text data. The proposed approach is shown to significantly enhance performance in audio-text retrieval, outperforming conventional text augmentation techniques.
• Score: 15.765495448426904
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: There has been growing interest in audio-language retrieval research, where the objective is to establish the correlation between audio and text modalities. However, most audio-text paired datasets often lack rich expression of the text data compared to the audio samples. One of the significant challenges facing audio-text datasets is the presence of similar or identical captions despite different audio samples. Therefore, under many-to-one mapping conditions, audio-text datasets lead to poor performance of retrieval tasks. In this paper, we propose a novel approach to tackle the data imbalance problem in audio-language retrieval task. To overcome the limitation, we introduce a method that employs a distance sampling-based paraphraser leveraging ChatGPT, utilizing distance function to generate a controllable distribution of manipulated text data. For a set of sentences with the same context, the distance is used to calculate a degree of manipulation for any two sentences, and ChatGPT's few-shot prompting is performed using a text cluster with a similar distance defined by the Jaccard similarity. Therefore, ChatGPT, when applied to few-shot prompting with text clusters, can adjust the diversity of the manipulated text based on the distance. The proposed approach is shown to significantly enhance performance in audio-text retrieval, outperforming conventional text augmentation techniques.

Related papers

• Learning Robust Named Entity Recognizers From Noisy Data With Retrieval Augmentation [67.89838237013078]
  Named entity recognition (NER) models often struggle with noisy inputs. We propose a more realistic setting in which only noisy text and its NER labels are available. We employ a multi-view training framework that improves robust NER without retrieving text during inference.
  arXiv  Detail & Related papers  (2024-07-26T07:30:41Z)
• Bridging Language Gaps in Audio-Text Retrieval [28.829775980536574]
  We propose a language enhancement (LE) using a multilingual text encoder (SONAR) to encode the text data with language-specific information. We optimize the audio encoder through the application of consistent ensemble distillation (CED), enhancing support for variable-length audio-text retrieval. Our methodology excels in English audio-text retrieval, demonstrating state-of-the-art (SOTA) performance on commonly used datasets such as AudioCaps and Clotho.
  arXiv  Detail & Related papers  (2024-06-11T07:12:12Z)
• Spotting AI's Touch: Identifying LLM-Paraphrased Spans in Text [61.22649031769564]
  We propose a novel framework, paraphrased text span detection (PTD) PTD aims to identify paraphrased text spans within a text. We construct a dedicated dataset, PASTED, for paraphrased text span detection.
  arXiv  Detail & Related papers  (2024-05-21T11:22:27Z)
• Augmenting text for spoken language understanding with Large Language Models [13.240782495441275]
  We show how to use transcript-semantic parse data (unpaired text) without corresponding speech. Experiments show that unpaired text from existing and new domains improves performance by 2% and 30% in absolute Exact Match (EM) respectively. We propose to prompt Large Language Models (LLMs) to generate unpaired text for existing and new domains.
  arXiv  Detail & Related papers  (2023-09-17T22:25:34Z)
• Parameter Efficient Audio Captioning With Faithful Guidance Using Audio-text Shared Latent Representation [0.9285295512807729]
  We propose a data augmentation technique for generating hallucinated audio captions and show that similarity based on an audio-text shared latent space is suitable for detecting hallucination. We then propose a parameter efficient inference time faithful decoding algorithm that enables smaller audio captioning models with performance equivalent to larger models trained with more data.
  arXiv  Detail & Related papers  (2023-09-06T19:42:52Z)
• Text-Only Domain Adaptation for End-to-End Speech Recognition through Down-Sampling Acoustic Representation [67.98338382984556]
  Mapping two modalities, speech and text, into a shared representation space, is a research topic of using text-only data to improve end-to-end automatic speech recognition (ASR) performance in new domains. In this paper, we proposed novel representations match strategy through down-sampling acoustic representation to align with text modality. Our ASR model can learn unified representations from both modalities better, allowing for domain adaptation using text-only data of the target domain.
  arXiv  Detail & Related papers  (2023-09-04T08:52:59Z)
• AugGPT: Leveraging ChatGPT for Text Data Augmentation [59.76140039943385]
  We propose a text data augmentation approach based on ChatGPT (named AugGPT) AugGPT rephrases each sentence in the training samples into multiple conceptually similar but semantically different samples. Experiment results on few-shot learning text classification tasks show the superior performance of the proposed AugGPT approach.
  arXiv  Detail & Related papers  (2023-02-25T06:58:16Z)
• Audio-text Retrieval in Context [24.38055340045366]
  In this work, we investigate several audio features as well as sequence aggregation methods for better audio-text alignment. We build our contextual audio-text retrieval system using pre-trained audio features and a descriptor-based aggregation method. With our proposed system, a significant improvement has been achieved on bidirectional audio-text retrieval, on all metrics including recall, median and mean rank.
  arXiv  Detail & Related papers  (2022-03-25T13:41:17Z)
• Zero-Shot Text-to-Speech for Text-Based Insertion in Audio Narration [62.75234183218897]
  We propose a one-stage context-aware framework to generate natural and coherent target speech without any training data of the speaker. We generate the mel-spectrogram of the edited speech with a transformer-based decoder. It outperforms a recent zero-shot TTS engine by a large margin.
  arXiv  Detail & Related papers  (2021-09-12T04:17:53Z)
• Continuous speech separation: dataset and analysis [52.10378896407332]
  In natural conversations, a speech signal is continuous, containing both overlapped and overlap-free components. This paper describes a dataset and protocols for evaluating continuous speech separation algorithms.
  arXiv  Detail & Related papers  (2020-01-30T18:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.