CNN Models for Microphone Array Covariance Matrix Upsampling and Acoustic Imaging

• URL: http://arxiv.org/abs/2607.01295v1
• Date: Wed, 01 Jul 2026 13:25:30 GMT
• Title: CNN Models for Microphone Array Covariance Matrix Upsampling and Acoustic Imaging
• Abstract summary: Upsampling virtually a tetrahedral 4-microphone array to a spherical 32-microphone array employs deep learning techniques.<n>Five neural network architectures are investigated for covariance upsampling for acoustic imaging using the real-world STARSS23 dataset.<n>Results show that all models outperform a random-guess baseline, which yields an RMSE of 0.548, with the best-performing architecture achieving an RMSE of 0.432.
• Score: 25.465068576945953
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Acoustic imaging visualization is a core methodology in acoustics, enabling spatial analysis of sound sources and acoustic scenes. However, limited sensor availability in practical systems motivate approaches that enhance spatial resolution without increasing the hardware complexity. In this paper, we focus on upsampling virtually a tetrahedral 4-microphone array to a spherical 32-microphone array by estimating the covariance matrices of the channels employing deep learning techniques. Five neural network architectures are investigated for covariance upsampling for acoustic imaging using the real-world STARSS23 dataset. These models are developed to estimate a 32-microphone, time-frequency covariance matrix from a 4-microphone input covariance representation. The proposed architectures are based on 2D convolutional layers to capture the underlying spatial-spectral structure of covariance matrices, and are further enhanced with frequency dynamic convolution to model their frequency-dependent properties. The proposed architectures are evaluated in terms of root mean square error (RMSE) and using delay-and-sum beamforming acoustic imaging. Quantitative results show that all models outperform a random-guess baseline, which yields an RMSE of 0.548, with the best-performing architecture achieving an RMSE of 0.432. We analyze qualitatively the performance of the proposed models through beamforming heatmap visualizations derived from the 4-channel input covariance, the 32-channel ground truth, and the predicted 32-channel covariance matrices. These results demonstrate that covariance upsampling significantly enhances the effective performance of the 4-channel microphone array, producing sound maps that closely resemble those obtained with the 32-channel array.

Related papers

• BenthicDINO: Physics-Informed Self-Distillation for View-Invariant Side-Scan Sonar Representations [0.0]
  We introduce a physics-informed self-distillation framework built upon the DINOv3 architecture utilizing a ConvNeXt-v2-Tiny backbone to maximize data efficiency.<n>We propose a dense, hierarchical feature fusion strategy across all four network stages to preserve fine-grained sediment details alongside deep semantic abstractions.
  arXiv  Detail & Related papers  (2026-08-24T13:11:22Z)
• GrainGS: Gradient-Decoupled Gaussian Splatting for Efficient Dynamic Novel View Synthesis [94.20462261657525]
  GrainGS is a dynamic framework that combines a hierarchical anchor scaffold with per-Gaussian deformation.<n>It achieves high reconstruction quality, real-time novel view synthesis, and compact storage.
  arXiv  Detail & Related papers  (2026-07-23T15:52:35Z)
• PrismAudio: Decomposed Chain-of-Thoughts and Multi-dimensional Rewards for Video-to-Audio Generation [57.864929968616586]
  Video-to-Audio (V2A) generation requires balancing four critical perceptual dimensions.<n>We introduce PrismAudio, the first framework to integrate Reinforcement Learning into V2A generation with specialized Chain-of-Thought (CoT) planning.
  arXiv  Detail & Related papers  (2025-11-24T07:11:12Z)
• SCas4D: Structural Cascaded Optimization for Boosting Persistent 4D Novel View Synthesis [53.10680153186481]
  We propose SCas4D, a cascaded optimization framework that leverages structural patterns in 3D Gaussian Splatting for dynamic scenes.<n>By progressively refining deformations from coarse part-level to fine point-level, SCas4D achieves convergence within 100 iterations per time frame.<n>The approach also demonstrates effectiveness in self-supervised articulated object segmentation, novel view synthesis, and dense point tracking tasks.
  arXiv  Detail & Related papers  (2025-10-08T06:39:33Z)
• InJecteD: Analyzing Trajectories and Drift Dynamics in Denoising Diffusion Probabilistic Models for 2D Point Cloud Generation [48.55037712252843]
  InJecteD is a framework for interpreting Denoising Diffusion Probabilistic Models (DDPMs)<n>We apply this framework to three datasets from the Datasaurus Dozen bullseye, dino, and circle.<n>Our approach quantifies trajectory properties, including displacement, velocity, clustering, and drift field dynamics.
  arXiv  Detail & Related papers  (2025-09-09T14:53:19Z)
• Ultra-High-Resolution Image Synthesis: Data, Method and Evaluation [21.46605047406198]
  Aesthetic-4K dataset is curated for comprehensive research on ultra-high-resolution image synthesis.<n>Diffusion-4K is an innovative framework for the direct generation of ultra-high-resolution images.
  arXiv  Detail & Related papers  (2025-06-02T05:19:40Z)
• Benchmarking 3D multi-coil NC-PDNet MRI reconstruction [2.5195203844628558]
  Non-Cartesian Primal-Dual Network (NC-PDNet) trained on compressed data with varying input channel numbers achieves an average PSNR of 42.98 dB for 1 mm isotropic 32 channel whole-brain 3D reconstruction. With an inference time of 4.95sec and a GPU memory usage of 5.49 GB, our approach demonstrates significant potential for clinical research application.
  arXiv  Detail & Related papers  (2024-11-08T09:14:36Z)
• Joint Beam Search Integrating CTC, Attention, and Transducer Decoders [53.297697898510194]
  We propose a joint modeling scheme where four decoders share the same encoder.<n>The 4D model is trained jointly, which will bring model regularization and maximize the model robustness.<n>In addition, we propose three novel joint beam search algorithms by combining three decoders.
  arXiv  Detail & Related papers  (2024-06-05T05:18:20Z)
• Probabilistic-based Feature Embedding of 4-D Light Fields for Compressive Imaging and Denoising [62.347491141163225]
  4-D light field (LF) poses great challenges in achieving efficient and effective feature embedding. We propose a probabilistic-based feature embedding (PFE), which learns a feature embedding architecture by assembling various low-dimensional convolution patterns. Our experiments demonstrate the significant superiority of our methods on both real-world and synthetic 4-D LF images.
  arXiv  Detail & Related papers  (2023-06-15T03:46:40Z)
• Adversarial Audio Synthesis with Complex-valued Polynomial Networks [60.231877895663956]
  Time-frequency (TF) representations in audio have been increasingly modeled real-valued networks. We introduce complex-valued networks called APOLLO, that integrate such complex-valued representations in a natural way. APOLLO results in $17.5%$ improvement over adversarial methods and $8.2%$ over the state-of-the-art diffusion models on SC09 in audio generation.
  arXiv  Detail & Related papers  (2022-06-14T12:58:59Z)
• XCycles Backprojection Acoustic Super-Resolution [4.022090911982323]
  This work proposes a novel backprojection model architecture for the acoustic image super-resolution problem. It uses the iterative correction procedure in each cycle to reconstruct the residual error correction for the encoded features in both low- and high-resolution space. It also contributed to a drastically reduced sub-sampling error produced during the data acquisition.
  arXiv  Detail & Related papers  (2021-05-19T13:43:15Z)
• An ensemble classifier for vibration-based quality monitoring [0.0]
  This paper develops a novel ensemble classifier based on the Dempster-Shafer theory of evidence. The effectiveness of the proposed framework is validated by its application to 15 UCI and KEEL machine learning datasets.
  arXiv  Detail & Related papers  (2020-07-17T07:23:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.