TorchDIVA: An Extensible Computational Model of Speech Production built on an Open-Source Machine Learning Library

• URL: http://arxiv.org/abs/2210.09334v1
• Date: Mon, 17 Oct 2022 18:00:52 GMT
• Title: TorchDIVA: An Extensible Computational Model of Speech Production built on an Open-Source Machine Learning Library
• Abstract summary: The DIVA model is a computational model of speech motor control that combines a simulation of the brain regions responsible for speech production with a model of the human vocal tract. The model is currently implemented in Matlab Simulink; however, this is less than ideal as most of the development in speech technology research is done in Python. We present TorchDIVA, a full rebuild of DIVA in Python using PyTorch tensors.
• Score: 19.81775668191837
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The DIVA model is a computational model of speech motor control that combines a simulation of the brain regions responsible for speech production with a model of the human vocal tract. The model is currently implemented in Matlab Simulink; however, this is less than ideal as most of the development in speech technology research is done in Python. This means there is a wealth of machine learning tools which are freely available in the Python ecosystem that cannot be easily integrated with DIVA. We present TorchDIVA, a full rebuild of DIVA in Python using PyTorch tensors. DIVA source code was directly translated from Matlab to Python, and built-in Simulink signal blocks were implemented from scratch. After implementation, the accuracy of each module was evaluated via systematic block-by-block validation. The TorchDIVA model is shown to produce outputs that closely match those of the original DIVA model, with a negligible difference between the two. We additionally present an example of the extensibility of TorchDIVA as a research platform. Speech quality enhancement in TorchDIVA is achieved through an integration with an existing PyTorch generative vocoder called DiffWave. A modified DiffWave mel-spectrum upsampler was trained on human speech waveforms and conditioned on the TorchDIVA speech production. The results indicate improved speech quality metrics in the DiffWave-enhanced output as compared to the baseline. This enhancement would have been difficult or impossible to accomplish in the original Matlab implementation. This proof-of-concept demonstrates the value TorchDIVA will bring to the research community. Researchers can download the new implementation at: https://github.com/skinahan/DIVA_PyTorch

Related papers

• pyALDIC: A Python Implementation of Augmented Lagrangian Digital Image Correlation with a GUI, Adaptive Meshing, and Mask-Aware Subset Splitting [7.421737610767278]
  pyALDIC is an open-source Python implementation of augmented Lagrangian digital image correlation (AL-DIC) for full-field displacement and strain measurement.<n>The software combines a graphical user interface with a scriptable Python API and supports adaptive quadtree meshing, mask-aware subset splitting near cracks and holes, and selectable Local DIC and AL-DIC solver modes.
  arXiv  Detail & Related papers  (2026-07-23T18:31:06Z)
• TYrPPG: Uncomplicated and Enhanced Learning Capability rPPG for Remote Heart Rate Estimation [51.56484100374058]
  This paper introduces an innovative video understanding block (GVB) designed for efficient RGB videos.<n>Based on the Mam structure, this block integrates 2D-CNN and 3D-CNN to enhance video understanding for analysis.<n>Experiments show that our TYr can achieve state-of-the-art performance in commonly used datasets.
  arXiv  Detail & Related papers  (2025-11-08T03:46:58Z)
• ByzFL: Research Framework for Robust Federated Learning [11.23722364748134]
  We present ByzFL, an open-source library for developing and robust benchmarking learning (FL) algorithms.<n>ByzFL provides a unified framework that includes implementations of state-of-the-art robust aggregators.<n>The library enables systematic experimentation via a single-based configuration file and includes built-in utilities for result visualization.
  arXiv  Detail & Related papers  (2025-05-30T17:08:15Z)
• Running Conventional Automatic Speech Recognition on Memristor Hardware: A Simulated Approach [18.47703842449581]
  We show how an ML system with millions of parameters would behave on memristor hardware.<n>We limit the relative degradation in word error rate to 25% when using a 3-bit weight precision to execute linear operations.
  arXiv  Detail & Related papers  (2025-05-30T15:42:41Z)
• Exploring Green AI for Audio Deepfake Detection [21.17957700009653]
  State-of-the-art audio deepfake detectors leveraging deep neural networks exhibit impressive recognition performance. Deep NLP models produce around 626k lbs of COtextsubscript2 which is equivalent to five times of average US car emission at its lifetime. This study presents a novel framework for audio deepfake detection that can be seamlessly trained using standard CPU resources.
  arXiv  Detail & Related papers  (2024-03-21T10:54:21Z)
• pyvene: A Library for Understanding and Improving PyTorch Models via Interventions [79.72930339711478]
  $textbfpyvene$ is an open-source library that supports customizable interventions on a range of different PyTorch modules. We show how $textbfpyvene$ provides a unified framework for performing interventions on neural models and sharing the intervened upon models with others.
  arXiv  Detail & Related papers  (2024-03-12T16:46:54Z)
• Pythae: Unifying Generative Autoencoders in Python -- A Benchmarking Use Case [0.0]
  We present Pythae, a versatile open-source Python library providing straightforward, reproducible and reliable use of generative autoencoder models. We present and compare 19 generative autoencoder models representative of some of the main improvements on downstream tasks.
  arXiv  Detail & Related papers  (2022-06-16T17:11:41Z)
• Design and Implementation of a Quantum Kernel for Natural Language Processing [0.8702432681310401]
  This thesis leverage the DisCoCat model to design a quantum-based kernel function that can be used by a support vector machine (SVM) for NLP tasks. Two similarity measures were studied: (i) the transition amplitude approach and (ii) the SWAP test. The explicit model from previous work was used to train word embeddings and achieved a testing accuracy of $93.09 pm 0.01$%.
  arXiv  Detail & Related papers  (2022-05-13T00:45:46Z)
• Self-supervised Learning with Random-projection Quantizer for Speech Recognition [51.24368930992091]
  We present a simple and effective self-supervised learning approach for speech recognition. The approach learns a model to predict masked speech signals, in the form of discrete labels. It achieves similar word-error-rates as previous work using self-supervised learning with non-streaming models.
  arXiv  Detail & Related papers  (2022-02-03T21:29:04Z)
• PyHHMM: A Python Library for Heterogeneous Hidden Markov Models [63.01207205641885]
  PyHHMM is an object-oriented Python implementation of Heterogeneous-Hidden Markov Models (HHMMs) PyHHMM emphasizes features not supported in similar available frameworks: a heterogeneous observation model, missing data inference, different model order selection criterias, and semi-supervised training. PyHHMM relies on the numpy, scipy, scikit-learn, and seaborn Python packages, and is distributed under the Apache-2.0 License.
  arXiv  Detail & Related papers  (2022-01-12T07:32:36Z)
• pymdp: A Python library for active inference in discrete state spaces [52.85819390191516]
  pymdp is an open-source package for simulating active inference in Python. We provide the first open-source package for simulating active inference with POMDPs.
  arXiv  Detail & Related papers  (2022-01-11T12:18:44Z)
• Streaming end-to-end multi-talker speech recognition [34.76106500736099]
  We propose the Streaming Unmixing and Recognition Transducer (SURT) for end-to-end multi-talker speech recognition. Our model employs the Recurrent Neural Network Transducer (RNN-T) as the backbone that can meet various latency constraints. Based on experiments on the publicly available LibriSpeechMix dataset, we show that HEAT can achieve better accuracy compared with PIT.
  arXiv  Detail & Related papers  (2020-11-26T06:28:04Z)
• TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning [68.8204255655161]
  We present TorchIO, an open-source Python library to enable efficient loading, preprocessing, augmentation and patch-based sampling of medical images for deep learning. TorchIO follows the style of PyTorch and integrates standard medical image processing libraries to efficiently process images during training of neural networks. It includes a command-line interface which allows users to apply transforms to image files without using Python.
  arXiv  Detail & Related papers  (2020-03-09T13:36:16Z)
• DFTpy: An efficient and object-oriented platform for orbital-free DFT simulations [55.41644538483948]
  In this work, we present DFTpy, an open source software implementing OFDFT written entirely in Python 3. We showcase the electronic structure of a million-atom system of aluminum metal which was computed on a single CPU. DFTpy is released under the MIT license.
  arXiv  Detail & Related papers  (2020-02-07T19:07:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.