CNN-based Spoken Term Detection and Localization without Dynamic Programming

• URL: http://arxiv.org/abs/2103.05468v1
• Date: Sun, 7 Mar 2021 14:50:58 GMT
• Title: CNN-based Spoken Term Detection and Localization without Dynamic Programming
• Authors: Tzeviya Sylvia Fuchs, Yael Segal and Joseph Keshet
• Abstract summary: The proposed algorithm infers whether a term was uttered within a given speech signal or not by predicting the word embeddings of various parts of the speech signal. The algorithm simultaneously predicts all possible locations of the target term and does not need dynamic programming for optimal search.
• Score: 16.322420712725716
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we propose a spoken term detection algorithm for simultaneous prediction and localization of in-vocabulary and out-of-vocabulary terms within an audio segment. The proposed algorithm infers whether a term was uttered within a given speech signal or not by predicting the word embeddings of various parts of the speech signal and comparing them to the word embedding of the desired term. The algorithm utilizes an existing embedding space for this task and does not need to train a task-specific embedding space. At inference the algorithm simultaneously predicts all possible locations of the target term and does not need dynamic programming for optimal search. We evaluate our system on several spoken term detection tasks on read speech corpora.

Related papers

• Predictive Speech Recognition and End-of-Utterance Detection Towards Spoken Dialog Systems [55.99999020778169]
  We study a function that can predict the forthcoming words and estimate the time remaining until the end of an utterance. We develop a cross-attention-based algorithm that incorporates both acoustic and linguistic information. Results demonstrate the proposed model's ability to predict upcoming words and estimate future EOU events up to 300ms prior to the actual EOU.
  arXiv  Detail & Related papers  (2024-09-30T06:29:58Z)
• Curriculum Learning for Goal-Oriented Semantic Communications with a Common Language [60.85719227557608]
  A holistic goal-oriented semantic communication framework is proposed to enable a speaker and a listener to cooperatively execute a set of sequential tasks. A common language based on a hierarchical belief set is proposed to enable semantic communications between speaker and listener. An optimization problem is defined to determine the perfect and abstract description of the events.
  arXiv  Detail & Related papers  (2022-04-21T22:36:06Z)
• Speaker Embedding-aware Neural Diarization: a Novel Framework for Overlapped Speech Diarization in the Meeting Scenario [51.5031673695118]
  We reformulate overlapped speech diarization as a single-label prediction problem. We propose the speaker embedding-aware neural diarization (SEND) system.
  arXiv  Detail & Related papers  (2022-03-18T06:40:39Z)
• On the Difficulty of Segmenting Words with Attention [32.97060026226872]
  We show, however, that even on monolingual data this approach is brittle. In experiments with different input types, data sizes, and segmentation algorithms, only models trained to predict phones from words succeed in the task.
  arXiv  Detail & Related papers  (2021-09-21T11:37:08Z)
• Direct speech-to-speech translation with discrete units [64.19830539866072]
  We present a direct speech-to-speech translation (S2ST) model that translates speech from one language to speech in another language without relying on intermediate text generation. We propose to predict the self-supervised discrete representations learned from an unlabeled speech corpus instead. When target text transcripts are available, we design a multitask learning framework with joint speech and text training that enables the model to generate dual mode output (speech and text) simultaneously in the same inference pass.
  arXiv  Detail & Related papers  (2021-07-12T17:40:43Z)
• A Novel Word Sense Disambiguation Approach Using WordNet Knowledge Graph [0.0]
  This paper presents a knowledge-based word sense disambiguation algorithm, namely Sequential Contextual Similarity Matrix multiplication (SCSMM) The SCSMM algorithm combines semantic similarity, knowledge, and document context to respectively exploit the merits of local context. The proposed algorithm outperformed all other algorithms when disambiguating nouns on the combined gold standard datasets.
  arXiv  Detail & Related papers  (2021-01-08T06:47:32Z)
• Multi-view Temporal Alignment for Non-parallel Articulatory-to-Acoustic Speech Synthesis [59.623780036359655]
  Articulatory-to-acoustic (A2A) synthesis refers to the generation of audible speech from captured movement of the speech articulators. This technique has numerous applications, such as restoring oral communication to people who cannot longer speak due to illness or injury. We propose a solution to this problem based on the theory of multi-view learning.
  arXiv  Detail & Related papers  (2020-12-30T15:09:02Z)
• Unsupervised Pattern Discovery from Thematic Speech Archives Based on Multilingual Bottleneck Features [41.951988293049205]
  We propose a two-stage approach, which comprises unsupervised acoustic modeling and decoding, followed by pattern mining in acoustic unit sequences. The proposed system is able to effectively extract topic-related words and phrases from the lecture recordings on MIT OpenCourseWare.
  arXiv  Detail & Related papers  (2020-11-03T20:06:48Z)
• Acoustic Word Embedding System for Code-Switching Query-by-example Spoken Term Detection [17.54377669932433]
  We propose a deep convolutional neural network-based acoustic word embedding system on code-switching query by example spoken term detection. We combine audio data in two languages for training instead of only using one single language.
  arXiv  Detail & Related papers  (2020-05-24T15:27:56Z)
• Continuous speech separation: dataset and analysis [52.10378896407332]
  In natural conversations, a speech signal is continuous, containing both overlapped and overlap-free components. This paper describes a dataset and protocols for evaluating continuous speech separation algorithms.
  arXiv  Detail & Related papers  (2020-01-30T18:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.