Uncertainty-Aware Crossmodal Fusion for Classification of Animal Behavior

• URL: http://arxiv.org/abs/2608.02104v1
• Date: Mon, 03 Aug 2026 12:05:28 GMT
• Title: Uncertainty-Aware Crossmodal Fusion for Classification of Animal Behavior
• Abstract summary: We propose Uncertainty-Aware Fusion (UAF), a dual-stream framework that estimates Gaussian uncertainty for each representation and fuses them via uncertainty weighting.<n>UAF achieves 59.4% accuracy / 39.7% macro F1 on the 17-class SoundWel pig vocalization benchmark and 73.1% accuracy / 71.5% macro F1 on the 3-class DogBark dataset, outperforming static-concatenation fusion by 15.7% and 20.4% relative macro F1, respectively.
• Score: 1.6178468864923607
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Artificial intelligence offers substantial potential for acoustic monitoring of animals, from welfare assessment in precision livestock farming to wildlife conservation and ecological research, where vocalizations can indicate health, stress, and social states earlier and at lower cost than manual observation. However, recordings in these settings are obtained under uncontrolled conditions, including environmental noise, reverberation, overlapping calls, and sensors that degrade without notice. As a consequence, automated classification of animal vocalizations remains challenging, and the two dominant acoustic representations show complementary limitations: raw waveforms preserve temporal microstructure but degrade under clipping and reverberation, while log-Mel spectrograms capture harmonic organization but lose phase information and are sensitive to broadband noise. To address these challenges, we propose Uncertainty-Aware Fusion (UAF), a dual-stream framework that estimates Gaussian uncertainty for each representation and fuses them via uncertainty weighting. This mechanism assigns greater weight to the more confident representation with no reliability labels required. In a cross-species, identity-based evaluation excluding all individuals seen during training, UAF (mean pooling) achieves 59.4\% accuracy / 39.7\% macro F1 on the 17-class SoundWel pig vocalization benchmark and 73.1\% accuracy / 71.5\% macro F1 on the 3-class DogBark dataset, outperforming static-concatenation fusion by 15.7\% and 20.4\% relative macro F1, respectively. Ablations over four temporal aggregation strategies show that uncertainty fusion, rather than the temporal characteristics of animal calls, is the primary driver of the performance gain.

Related papers

• MAST: Label-Efficient, Robust, and Generalizable Sound Detection for Biodiversity Monitoring via Masked Audio Pretraining and Self-Training [7.983925207958734]
  Time-frequency annotation of animal vocalizations is expensive, site-specific, and difficult to sustain at scale.<n>We present a label-efficient sound detection framework that combines masked audio pretraining with a lightweight detector on mel spectrograms.<n>We evaluate the performance on two ecologically distinct domains: tropical rainforest soundscapes (Indonesia) and bird vocalizations in Mediterranean habitats (Spain)
  arXiv  Detail & Related papers  (2026-09-14T08:41:23Z)
• Attention-Guided Reliability Scaling for Contrastive Decoding in Robust Audio-Visual Speech Recognition [50.15288513086986]
  Contrastive decoding (CD) originally introduced to stabilize large language model (LLM)-based audio-visual speech recognition (AVSR) systems.<n>We propose reliability-aware scaling of CD for AVSR.
  arXiv  Detail & Related papers  (2026-08-26T08:25:15Z)
• Dynamic Spectral Denoising with Global-Context Attention for Multi-Behavior Recommendation [58.33732779555114]
  We argue that the key bottleneck is a representation-level failure caused by two coupled heterogeneities.<n>We propose Dynamic Spectral Denoising with Global-Context Attention for Multi-Behavior Recommendation (SpectraMB)<n>SpectraMB achieves the best results in most evaluation settings and exhibits improved robustness under noisy interactions.
  arXiv  Detail & Related papers  (2026-06-01T15:58:13Z)
• Multi-Object Tracking Consistently Improves Wildlife Inference [4.13115026589334]
  This study exploits the temporal nature of camera-trap data to augment inferred predictions from a wildlife classification model.<n>We adopt several standard Multi-Object Tracking (MOT) models to link detections across consecutive frames.<n>The fused probability score produces a single consensus class label estimate that overrides misclassifications caused by noise.
  arXiv  Detail & Related papers  (2026-05-15T22:13:25Z)
• Synthetic Data for Veterinary EHR De-identification: Benefits, Limits, and Safety Trade-offs Under Fixed Compute [0.0]
  This study evaluates whether large language model (LLM)-generated synthetic narratives improve de-identification safety.<n>We generated 10,382 synthetic notes using a privacy-preserving "template-only" regime.<n>We conclude that synthetic augmentation is effective for expanding exposure but is complementary, not substitutive, for safety-critical veterinary de-identification.
  arXiv  Detail & Related papers  (2026-01-13T19:35:25Z)
• Multi-Representation Attention Framework for Underwater Bioacoustic Denoising and Recognition [0.924965746838578]
  We introduce a multi-step, attention-guided framework that first segments spectrograms to generate soft masks of biologically relevant energy.<n>Image and mask embeddings are integrated via mid-level fusion, enabling the model to focus on salient spectrogram regions.<n>Using real-world recordings from the Saguenay St. Lawrence Marine Park Research Station in Canada, we demonstrate that segmentation-driven attention and mid-level fusion improve signal discrimination.
  arXiv  Detail & Related papers  (2025-10-29T22:49:15Z)
• Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Models [57.474294329887236]
  Diffusion large language models (dLLMs) generate text through iterative denoising.<n>Current decoding strategies discard rich intermediate predictions in favor of the final output.<n>We introduce two complementary methods that exploit temporal consistency.
  arXiv  Detail & Related papers  (2025-08-12T17:59:57Z)
• FM2S: Towards Spatially-Correlated Noise Modeling in Zero-Shot Fluorescence Microscopy Image Denoising [33.383511185170214]
  Fluorescence Micrograph to Self (FM2S) is a zero-shot denoiser that achieves efficient Fluorescence Micrograph to Self (FM2S) denoising through three key innovations.<n>Experiments across FMI datasets demonstrate FM2S's superiority: It outperforms CVF-SID by 1.4dB PSNR on average while requiring 0.1% parameters of AP-BSN.
  arXiv  Detail & Related papers  (2024-12-13T10:45:25Z)
• Advanced Framework for Animal Sound Classification With Features Optimization [35.2832738406242]
  We propose an automated classification framework applicable to general animal sound classification. Our approach consistently outperforms baseline methods by over 25% in precision, recall, and accuracy.
  arXiv  Detail & Related papers  (2024-07-03T18:33:47Z)
• WhaleNet: a Novel Deep Learning Architecture for Marine Mammals Vocalizations on Watkins Marine Mammal Sound Database [49.1574468325115]
  We introduce textbfWhaleNet (Wavelet Highly Adaptive Learning Ensemble Network), a sophisticated deep ensemble architecture for the classification of marine mammal vocalizations. We achieve an improvement in classification accuracy by $8-10%$ over existing architectures, corresponding to a classification accuracy of $97.61%$.
  arXiv  Detail & Related papers  (2024-02-20T11:36:23Z)
• A noise-robust acoustic method for recognizing foraging activities of grazing cattle [35.21388806827219]
  We present the operating principle and generalization capability of an acoustic method called Noise-Robust Foraging Activity Recognizer (NRFAR) In noiseless conditions, NRFAR reached an average balanced accuracy of 86.4%, outperforming two previous acoustic methods by more than 7.5%. NRFAR has been shown to be effective in harsh free-ranging environments and could be used as a reliable solution to improve pasture management and monitor the health and welfare of dairy cows.
  arXiv  Detail & Related papers  (2023-04-28T13:06:14Z)
• S3: Supervised Self-supervised Learning under Label Noise [53.02249460567745]
  In this paper we address the problem of classification in the presence of label noise. In the heart of our method is a sample selection mechanism that relies on the consistency between the annotated label of a sample and the distribution of the labels in its neighborhood in the feature space. Our method significantly surpasses previous methods on both CIFARCIFAR100 with artificial noise and real-world noisy datasets such as WebVision and ANIMAL-10N.
  arXiv  Detail & Related papers  (2021-11-22T15:49:20Z)
• Discriminative Singular Spectrum Classifier with Applications on Bioacoustic Signal Recognition [67.4171845020675]
  We present a bioacoustic signal classifier equipped with a discriminative mechanism to extract useful features for analysis and classification efficiently. Unlike current bioacoustic recognition methods, which are task-oriented, the proposed model relies on transforming the input signals into vector subspaces. The validity of the proposed method is verified using three challenging bioacoustic datasets containing anuran, bee, and mosquito species.
  arXiv  Detail & Related papers  (2021-03-18T11:01:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.