Active Learning with Task Adaptation Pre-training for Speech Emotion Recognition

• URL: http://arxiv.org/abs/2405.00307v1
• Date: Wed, 1 May 2024 04:05:29 GMT
• Title: Active Learning with Task Adaptation Pre-training for Speech Emotion Recognition
• Authors: Dongyuan Li, Ying Zhang, Yusong Wang, Funakoshi Kataro, Manabu Okumura,
• Abstract summary: Speech emotion recognition (SER) has garnered increasing attention due to its wide range of applications. We propose an active learning (AL)-based fine-tuning framework for SER, called textscAfter. Our proposed method improves accuracy by 8.45% and reduces time consumption by 79%.
• Score: 17.59356583727259
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Speech emotion recognition (SER) has garnered increasing attention due to its wide range of applications in various fields, including human-machine interaction, virtual assistants, and mental health assistance. However, existing SER methods often overlook the information gap between the pre-training speech recognition task and the downstream SER task, resulting in sub-optimal performance. Moreover, current methods require much time for fine-tuning on each specific speech dataset, such as IEMOCAP, which limits their effectiveness in real-world scenarios with large-scale noisy data. To address these issues, we propose an active learning (AL)-based fine-tuning framework for SER, called \textsc{After}, that leverages task adaptation pre-training (TAPT) and AL methods to enhance performance and efficiency. Specifically, we first use TAPT to minimize the information gap between the pre-training speech recognition task and the downstream speech emotion recognition task. Then, AL methods are employed to iteratively select a subset of the most informative and diverse samples for fine-tuning, thereby reducing time consumption. Experiments demonstrate that our proposed method \textsc{After}, using only 20\% of samples, improves accuracy by 8.45\% and reduces time consumption by 79\%. The additional extension of \textsc{After} and ablation studies further confirm its effectiveness and applicability to various real-world scenarios. Our source code is available on Github for reproducibility. (https://github.com/Clearloveyuan/AFTER).

Related papers

• Predictive Speech Recognition and End-of-Utterance Detection Towards Spoken Dialog Systems [55.99999020778169]
  We study a function that can predict the forthcoming words and estimate the time remaining until the end of an utterance. We develop a cross-attention-based algorithm that incorporates both acoustic and linguistic information. Results demonstrate the proposed model's ability to predict upcoming words and estimate future EOU events up to 300ms prior to the actual EOU.
  arXiv  Detail & Related papers  (2024-09-30T06:29:58Z)
• Efficient Cross-Task Prompt Tuning for Few-Shot Conversational Emotion Recognition [6.988000604392974]
  Emotion Recognition in Conversation (ERC) has been widely studied due to its importance in developing emotion-aware empathetic machines. We propose a derivative-free optimization method called Cross-Task Prompt Tuning (CTPT) for few-shot conversational emotion recognition.
  arXiv  Detail & Related papers  (2023-10-23T06:46:03Z)
• Active Learning Based Fine-Tuning Framework for Speech Emotion Recognition [20.28850074164053]
  Speech emotion recognition (SER) has drawn increasing attention for its applications in human-machine interaction. Existing SER methods ignore the information gap between the pre-training speech recognition task and the downstream SER task, leading to sub-optimal performance. We propose an active learning (AL) based Fine-Tuning framework for SER that leverages task adaptation pre-training (TAPT) and AL methods to enhance performance and efficiency.
  arXiv  Detail & Related papers  (2023-09-30T07:23:29Z)
• TEMPERA: Test-Time Prompting via Reinforcement Learning [57.48657629588436]
  We propose Test-time Prompt Editing using Reinforcement learning (TEMPERA) In contrast to prior prompt generation methods, TEMPERA can efficiently leverage prior knowledge. Our method achieves 5.33x on average improvement in sample efficiency when compared to the traditional fine-tuning methods.
  arXiv  Detail & Related papers  (2022-11-21T22:38:20Z)
• Instance-wise Prompt Tuning for Pretrained Language Models [72.74916121511662]
  Instance-wise Prompt Tuning (IPT) is the first prompt learning paradigm that injects knowledge from the input data instances to the prompts. IPT significantly outperforms task-based prompt learning methods, and achieves comparable performance to conventional finetuning with only 0.5% - 1.5% of tuned parameters.
  arXiv  Detail & Related papers  (2022-06-04T10:08:50Z)
• Self-Supervised Speech Representation Learning: A Review [105.1545308184483]
  Self-supervised representation learning methods promise a single universal model that would benefit a wide variety of tasks and domains. Speech representation learning is experiencing similar progress in three main categories: generative, contrastive, and predictive methods. This review presents approaches for self-supervised speech representation learning and their connection to other research areas.
  arXiv  Detail & Related papers  (2022-05-21T16:52:57Z)
• Pretext Tasks selection for multitask self-supervised speech representation learning [23.39079406674442]
  This paper introduces a method to select a group of pretext tasks among a set of candidates. Experiments conducted on speaker recognition and automatic speech recognition validate our approach.
  arXiv  Detail & Related papers  (2021-07-01T16:36:29Z)
• Weighted Training for Cross-Task Learning [71.94908559469475]
  We introduce Target-Aware Weighted Training (TAWT), a weighted training algorithm for cross-task learning. We show that TAWT is easy to implement, is computationally efficient, requires little hyper parameter tuning, and enjoys non-asymptotic learning-theoretic guarantees. As a byproduct, the proposed representation-based task distance allows one to reason in a theoretically principled way about several critical aspects of cross-task learning.
  arXiv  Detail & Related papers  (2021-05-28T20:27:02Z)
• Two-stage Textual Knowledge Distillation for End-to-End Spoken Language Understanding [18.275646344620387]
  This work proposes a two-stage textual knowledge distillation method that matches utterance-level representations and predicted logits of two modalities during pre-training and fine-tuning. We push the state-of-the-art on the Fluent Speech Commands, achieving 99.7% test accuracy in the full dataset setting and 99.5% in the 10% subset setting.
  arXiv  Detail & Related papers  (2020-10-25T12:36:05Z)
• A Transfer Learning Method for Speech Emotion Recognition from Automatic Speech Recognition [0.0]
  We show a transfer learning method in speech emotion recognition based on a Time-Delay Neural Network architecture. We achieve the highest significantly higher accuracy when compared to state-of-the-art, using five-fold cross validation.
  arXiv  Detail & Related papers  (2020-08-06T20:37:22Z)
• Pre-training Text Representations as Meta Learning [113.3361289756749]
  We introduce a learning algorithm which directly optimize model's ability to learn text representations for effective learning of downstream tasks. We show that there is an intrinsic connection between multi-task pre-training and model-agnostic meta-learning with a sequence of meta-train steps.
  arXiv  Detail & Related papers  (2020-04-12T09:05:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.