Lend me an Ear: Speech Enhancement Using a Robotic Arm with a Microphone Array

• URL: http://arxiv.org/abs/2602.17818v1
• Date: Thu, 19 Feb 2026 20:35:43 GMT
• Title: Lend me an Ear: Speech Enhancement Using a Robotic Arm with a Microphone Array
• Abstract summary: Speech enhancement performance degrades significantly in noisy environments.<n>Existing speech enhancement solutions rely on advanced digital signal processing techniques, deep learning methods, or complex software optimization techniques.<n>This paper introduces a novel enhancement strategy by dynamically modifying the geometry of a microphone array to adapt to changing acoustic conditions.
• Score: 3.7370186160766443
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Speech enhancement performance degrades significantly in noisy environments, limiting the deployment of speech-controlled technologies in industrial settings, such as manufacturing plants. Existing speech enhancement solutions primarly rely on advanced digital signal processing techniques, deep learning methods, or complex software optimization techniques. This paper introduces a novel enhancement strategy that incorporates a physical optimization stage by dynamically modifying the geometry of a microphone array to adapt to changing acoustic conditions. A sixteen-microphone array is mounted on a robotic arm manipulator with seven degrees of freedom, with microphones divided into four groups of four, including one group positioned near the end-effector. The system reconfigures the array by adjusting the manipulator joint angles to place the end-effector microphones closer to the target speaker, thereby improving the reference signal quality. This proposed method integrates sound source localization techniques, computer vision, inverse kinematics, minimum variance distortionless response beamformer and time-frequency masking using a deep neural network. Experimental results demonstrate that this approach outperforms other traditional recording configruations, achieving higher scale-invariant signal-to-distortion ratio and lower word error rate accross multiple input signal-to-noise ratio conditions.

Related papers

• A Synergistic Framework of Nonlinear Acoustic Computing and Reinforcement Learning for Real-World Human-Robot Interaction [15.759904937490832]
  This paper introduces a novel framework integrating nonlinear acoustic computing and reinforcement learning to enhance human-robot interaction under complex noise and reverberation.<n>The proposed system demonstrates broad application prospects in AI hardware, robot, machine audition, artificial audition, and brain-machine interfaces.
  arXiv  Detail & Related papers  (2025-05-04T06:03:12Z)
• Deep Active Speech Cancellation with Mamba-Masking Network [62.73250985838971]
  We present a novel deep learning network for Active Speech Cancellation (ASC)<n>The proposed Mamba-Masking architecture introduces a masking mechanism that directly interacts with the encoded reference signal.<n> Experimental results demonstrate substantial performance gains, achieving up to 7.2dB improvement in ANC scenarios and 6.2dB in ASC.
  arXiv  Detail & Related papers  (2025-02-03T09:22:26Z)
• End-to-end multi-channel speaker extraction and binaural speech synthesis [26.373624846079686]
  Speech clarity and spatial audio immersion are two most critical factors in enhancing remote conferencing experiences.<n>We introduce an end-to-end deep learning framework that has the capacity of mapping multi-channel noisy and reverberant signals to clean and spatialized speech directly.<n>In this framework, a novel magnitude-weighted interaural level difference loss function is proposed that aims to improve the accuracy of spatial rendering.
  arXiv  Detail & Related papers  (2024-10-08T06:55:35Z)
• A Physics-Informed Neural Network-Based Approach for the Spatial Upsampling of Spherical Microphone Arrays [40.98027720342511]
  We present a method for spatially upsampling spherical microphone arrays with a limited number of capsules. Our approach exploits a physics-informed neural network with Rowdy activation functions, leveraging physical constraints to provide high-order microphone array signals.
  arXiv  Detail & Related papers  (2024-07-26T13:35:06Z)
• Multi-Microphone Speech Emotion Recognition using the Hierarchical Token-semantic Audio Transformer Architecture [11.063156506583562]
  We propose processing multi-microphone signals to address these challenges and improve emotion classification accuracy. We adopt a state-of-the-art transformer model, the HTS-AT, to handle multi-channel audio inputs. Our multi-microphone model achieves superior performance compared to single-channel baselines when tested on real-world reverberant environments.
  arXiv  Detail & Related papers  (2024-06-05T13:50:59Z)
• Guided Speech Enhancement Network [17.27704800294671]
  Multi-microphone speech enhancement problem is often decomposed into two decoupled steps: a beamformer that provides spatial filtering and a single-channel speech enhancement model. We propose a speech enhancement solution that takes both the raw microphone and beamformer outputs as the input for an ML model. We name the ML module in our solution as GSENet, short for Guided Speech Enhancement Network.
  arXiv  Detail & Related papers  (2023-03-13T21:48:20Z)
• High Fidelity Neural Audio Compression [92.4812002532009]
  We introduce a state-of-the-art real-time, high-fidelity, audio leveraging neural networks. It consists in a streaming encoder-decoder architecture with quantized latent space trained in an end-to-end fashion. We simplify and speed-up the training by using a single multiscale spectrogram adversary.
  arXiv  Detail & Related papers  (2022-10-24T17:52:02Z)
• Direction-Aware Adaptive Online Neural Speech Enhancement with an Augmented Reality Headset in Real Noisy Conversational Environments [21.493664174262737]
  This paper describes the practical response- and performance-aware development of online speech enhancement for an augmented reality (AR) headset. It helps a user understand conversations made in real noisy echoic environments (e.g., cocktail party) The method is used with a blind dereverberation method called weighted prediction error (WPE) for transcribing the noisy reverberant speech of a speaker.
  arXiv  Detail & Related papers  (2022-07-15T05:14:27Z)
• End-to-End Binaural Speech Synthesis [71.1869877389535]
  We present an end-to-end speech synthesis system that combines a low-bitrate audio system with a powerful decoder. We demonstrate the capability of the adversarial loss in capturing environment effects needed to create an authentic auditory scene.
  arXiv  Detail & Related papers  (2022-07-08T05:18:36Z)
• Multi-Discriminator Sobolev Defense-GAN Against Adversarial Attacks for End-to-End Speech Systems [78.5097679815944]
  This paper introduces a defense approach against end-to-end adversarial attacks developed for cutting-edge speech-to-text systems. First, we represent speech signals with 2D spectrograms using the short-time Fourier transform. Second, we iteratively find a safe vector using a spectrogram subspace projection operation. Third, we synthesize a spectrogram with such a safe vector using a novel GAN architecture trained with Sobolev integral probability metric.
  arXiv  Detail & Related papers  (2021-03-15T01:11:13Z)
• Hierarchical Timbre-Painting and Articulation Generation [92.59388372914265]
  We present a fast and high-fidelity method for music generation, based on specified f0 and loudness. The synthesized audio mimics the timbre and articulation of a target instrument.
  arXiv  Detail & Related papers  (2020-08-30T05:27:39Z)
• Temporal-Spatial Neural Filter: Direction Informed End-to-End Multi-channel Target Speech Separation [66.46123655365113]
  Target speech separation refers to extracting the target speaker's speech from mixed signals. Two main challenges are the complex acoustic environment and the real-time processing requirement. We propose a temporal-spatial neural filter, which directly estimates the target speech waveform from multi-speaker mixture.
  arXiv  Detail & Related papers  (2020-01-02T11:12:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.