Audio-Visual Class-Incremental Learning for Fish Feeding intensity Assessment in Aquaculture

• URL: http://arxiv.org/abs/2504.15171v1
• Date: Mon, 21 Apr 2025 15:24:34 GMT
• Title: Audio-Visual Class-Incremental Learning for Fish Feeding intensity Assessment in Aquaculture
• Authors: Meng Cui, Xianghu Yue, Xinyuan Qian, Jinzheng Zhao, Haohe Liu, Xubo Liu, Daoliang Li, Wenwu Wang,
• Abstract summary: Fish Feeding Intensity Assessment (FFIA) is crucial in industrial aquaculture management.<n>Recent multi-modal approaches have shown promise in improving FFIA robustness and efficiency.<n>We first introduce AV-CIL-FFIA, a new dataset comprising 81,932 labelled audio-visual clips capturing feeding intensities across six different fish species in real aquaculture environments.<n>Then, we pioneer audio-visual class incremental learning (CIL) for FFIA and demonstrate through benchmarking on AV-CIL-FFIA that it significantly outperforms single-modality methods.
• Score: 29.42598968673262
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fish Feeding Intensity Assessment (FFIA) is crucial in industrial aquaculture management. Recent multi-modal approaches have shown promise in improving FFIA robustness and efficiency. However, these methods face significant challenges when adapting to new fish species or environments due to catastrophic forgetting and the lack of suitable datasets. To address these limitations, we first introduce AV-CIL-FFIA, a new dataset comprising 81,932 labelled audio-visual clips capturing feeding intensities across six different fish species in real aquaculture environments. Then, we pioneer audio-visual class incremental learning (CIL) for FFIA and demonstrate through benchmarking on AV-CIL-FFIA that it significantly outperforms single-modality methods. Existing CIL methods rely heavily on historical data. Exemplar-based approaches store raw samples, creating storage challenges, while exemplar-free methods avoid data storage but struggle to distinguish subtle feeding intensity variations across different fish species. To overcome these limitations, we introduce HAIL-FFIA, a novel audio-visual class-incremental learning framework that bridges this gap with a prototype-based approach that achieves exemplar-free efficiency while preserving essential knowledge through compact feature representations. Specifically, HAIL-FFIA employs hierarchical representation learning with a dual-path knowledge preservation mechanism that separates general intensity knowledge from fish-specific characteristics. Additionally, it features a dynamic modality balancing system that adaptively adjusts the importance of audio versus visual information based on feeding behaviour stages. Experimental results show that HAIL-FFIA is superior to SOTA methods on AV-CIL-FFIA, achieving higher accuracy with lower storage needs while effectively mitigating catastrophic forgetting in incremental fish species learning.

Related papers

• EKPC: Elastic Knowledge Preservation and Compensation for Class-Incremental Learning [53.88000987041739]
  Class-Incremental Learning (CIL) aims to enable AI models to continuously learn from sequentially arriving data of different classes over time.<n>We propose the Elastic Knowledge Preservation and Compensation (EKPC) method, integrating Importance-aware importance Regularization (IPR) and Trainable Semantic Drift Compensation (TSDC) for CIL.
  arXiv  Detail & Related papers  (2025-06-14T05:19:58Z)
• Ustnlp16 at SemEval-2025 Task 9: Improving Model Performance through Imbalance Handling and Focal Loss [38.70308073598037]
  classification tasks often suffer from severe class imbalances, short and unstructured text, and overlapping semantic categories.<n>We present our system for SemEval- 2025 Task 9: Food Hazard Detection, which ad- dresses these issues by applying data augmenta- tion techniques to improve classification perfor- mance.
  arXiv  Detail & Related papers  (2025-04-24T16:35:44Z)
• FisherMask: Enhancing Neural Network Labeling Efficiency in Image Classification Using Fisher Information [2.762397703396293]
  FisherMask is a Fisher information-based active learning (AL) approach that identifies key network parameters by masking them. Our experiments demonstrate that FisherMask significantly outperforms state-of-the-art methods on diverse datasets.
  arXiv  Detail & Related papers  (2024-11-08T18:10:46Z)
• EXACFS -- A CIL Method to mitigate Catastrophic Forgetting [3.2562590728737058]
  This paper introduces EXponentially Averaged Class-wise Feature Significance (EXACFS) to mitigate this issue in the class incremental learning setting.<n>Experiments on CIFAR-100 and ImageNet-100 demonstrate EXACFS's superior performance in preserving stability while acquiring plasticity.
  arXiv  Detail & Related papers  (2024-10-31T09:11:56Z)
• Learning from Neighbors: Category Extrapolation for Long-Tail Learning [62.30734737735273]
  We offer a novel perspective on long-tail learning, inspired by an observation: datasets with finer granularity tend to be less affected by data imbalance.<n>We introduce open-set auxiliary classes that are visually similar to existing ones, aiming to enhance representation learning for both head and tail classes.<n>To prevent the overwhelming presence of auxiliary classes from disrupting training, we introduce a neighbor-silencing loss.
  arXiv  Detail & Related papers  (2024-10-21T13:06:21Z)
• Enhancing Training Data Attribution for Large Language Models with Fitting Error Consideration [74.09687562334682]
  We introduce a novel training data attribution method called Debias and Denoise Attribution (DDA) Our method significantly outperforms existing approaches, achieving an averaged AUC of 91.64%. DDA exhibits strong generality and scalability across various sources and different-scale models like LLaMA2, QWEN2, and Mistral.
  arXiv  Detail & Related papers  (2024-10-02T07:14:26Z)
• Enabling Quartile-based Estimated-Mean Gradient Aggregation As Baseline for Federated Image Classifications [5.5099914877576985]
  Federated Learning (FL) has revolutionized how we train deep neural networks by enabling decentralized collaboration while safeguarding sensitive data and improving model performance. This paper introduces an innovative solution named Estimated Mean Aggregation (EMA) that not only addresses these challenges but also provides a fundamental reference point as a $mathsfbaseline$ for advanced aggregation techniques in FL systems.
  arXiv  Detail & Related papers  (2023-09-21T17:17:28Z)
• Uncertainty Estimation by Fisher Information-based Evidential Deep Learning [61.94125052118442]
  Uncertainty estimation is a key factor that makes deep learning reliable in practical applications. We propose a novel method, Fisher Information-based Evidential Deep Learning ($mathcalI$-EDL) In particular, we introduce Fisher Information Matrix (FIM) to measure the informativeness of evidence carried by each sample, according to which we can dynamically reweight the objective loss terms to make the network more focused on the representation learning of uncertain classes.
  arXiv  Detail & Related papers  (2023-03-03T16:12:59Z)
• Disentangled Representation Learning for RF Fingerprint Extraction under Unknown Channel Statistics [77.13542705329328]
  We propose a framework of disentangled representation learning(DRL) that first learns to factor the input signals into a device-relevant component and a device-irrelevant component via adversarial learning. The implicit data augmentation in the proposed framework imposes a regularization on the RFF extractor to avoid the possible overfitting of device-irrelevant channel statistics. Experiments validate that the proposed approach, referred to as DR-RFF, outperforms conventional methods in terms of generalizability to unknown complicated propagation environments.
  arXiv  Detail & Related papers  (2022-08-04T15:46:48Z)
• Scale-Equivalent Distillation for Semi-Supervised Object Detection [57.59525453301374]
  Recent Semi-Supervised Object Detection (SS-OD) methods are mainly based on self-training, generating hard pseudo-labels by a teacher model on unlabeled data as supervisory signals. We analyze the challenges these methods meet with the empirical experiment results. We introduce a novel approach, Scale-Equivalent Distillation (SED), which is a simple yet effective end-to-end knowledge distillation framework robust to large object size variance and class imbalance.
  arXiv  Detail & Related papers  (2022-03-23T07:33:37Z)
• Impact of Channel Variation on One-Class Learning for Spoof Detection [5.549602650463701]
  Spoofing detection increases the reliability of the ASV system but degrades significantly due to channel variation. "Which data-feeding strategy is optimal for MCT?" is not known in the case of spoof detection. This study highlights the relevance of the deemed-of-low-importance process of data-feeding and mini-batching to raise awareness of the need to refine it for better performance.
  arXiv  Detail & Related papers  (2021-09-30T07:56:16Z)
• A deep neural network for multi-species fish detection using multiple acoustic cameras [0.0]
  We present a novel approach that takes advantage of both CNN (Convolutional Neural Network) and classical CV (Computer Vision) techniques. The pipeline pre-treats the acoustic images to extract 2 features, in order to localise the signals and improve the detection performances. The YOLOv3-based model was trained with data of fish from multiple species recorded by the two common acoustic cameras.
  arXiv  Detail & Related papers  (2021-09-22T11:47:24Z)
• Adversarial Feature Hallucination Networks for Few-Shot Learning [84.31660118264514]
  Adversarial Feature Hallucination Networks (AFHN) is based on conditional Wasserstein Generative Adversarial networks (cWGAN) Two novel regularizers are incorporated into AFHN to encourage discriminability and diversity of the synthesized features.
  arXiv  Detail & Related papers  (2020-03-30T02:43:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.