Learning A Robust RGB-Thermal Detector for Extreme Modality Imbalance

• URL: http://arxiv.org/abs/2505.22154v1
• Date: Wed, 28 May 2025 09:18:55 GMT
• Title: Learning A Robust RGB-Thermal Detector for Extreme Modality Imbalance
• Abstract summary: RGB-Thermal (RGB-T) object detection utilizes thermal infrared (TIR) images to complement RGB data, improving robustness in challenging conditions.<n>Traditional RGB-T detectors assume balanced training data, where both modalities contribute equally.<n>This paper addresses these challenges by proposing a novel base-and-auxiliary detector architecture.
• Score: 10.189143024910598
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: RGB-Thermal (RGB-T) object detection utilizes thermal infrared (TIR) images to complement RGB data, improving robustness in challenging conditions. Traditional RGB-T detectors assume balanced training data, where both modalities contribute equally. However, in real-world scenarios, modality degradation-due to environmental factors or technical issues-can lead to extreme modality imbalance, causing out-of-distribution (OOD) issues during testing and disrupting model convergence during training. This paper addresses these challenges by proposing a novel base-and-auxiliary detector architecture. We introduce a modality interaction module to adaptively weigh modalities based on their quality and handle imbalanced samples effectively. Additionally, we leverage modality pseudo-degradation to simulate real-world imbalances in training data. The base detector, trained on high-quality pairs, provides a consistency constraint for the auxiliary detector, which receives degraded samples. This framework enhances model robustness, ensuring reliable performance even under severe modality degradation. Experimental results demonstrate the effectiveness of our method in handling extreme modality imbalances~(decreasing the Missing Rate by 55%) and improving performance across various baseline detectors.

Related papers

• RA-SOD: Reliability-Aware RGB-T Salient Object Detection under Modality Degradation [23.43264234749439]
  We propose RA-SOD, a reliability-aware RGB-T salient object detection framework.<n> RA-SOD integrates modality reliability into feature learning and cross-modal fusion.<n>Experiments show RA-SOD achieves state-of-the-art performance and exhibits strong robustness under severe modality degradation.
  arXiv  Detail & Related papers  (2026-09-11T09:22:20Z)
• Active Adversarial Perturbation-driven Associative Memory Retrieval for RGB-Event Visual Object Tracking [58.338848435042536]
  RGB-Event tracking improves localization by fusing RGB appearance textures and dense temporal motion cues from event sensors.<n>Real-world scenes suffer diverse structured signal degradations that hinder traditional multi-modal fusion.<n>We present a hierarchical perturbation and retrieval framework tailored for RGB-Event tracking with robustness against partial target missing and modal degradation.
  arXiv  Detail & Related papers  (2026-06-24T23:34:06Z)
• ALCL: An Adaptive Log-Correntropy Loss for Robust Learning under Non-Gaussian Noise [0.4078247440919472]
  We propose an Adaptive Log-Correntropy Loss (ALCL) that adaptively learns its robustness geometry during optimization.<n>ALCL consistently outperforms MSE and optimally generalized correntropy losses in both reconstruction fidelity and downstream classification accuracy.<n>These results demonstrate that adaptive robustness through joint learning of loss parameters provides a computationally efficient alternative to static correntropy-based losses for deep learning in non-Gaussian environments.
  arXiv  Detail & Related papers  (2026-06-14T22:49:57Z)
• Rewarding the Scientific Process: Process-Level Reward Modeling for Agentic Data Analysis [68.28714988482703]
  Process Reward Models (PRMs) have achieved remarkable success in augmenting the reasoning capabilities of Large Language Models (LLMs)<n>We present a study revealing that general-domain PRMs struggle to supervise data analysis agents.<n>We introduce DataPRM, a novel environment-aware generative process reward model.
  arXiv  Detail & Related papers  (2026-04-27T09:00:30Z)
• Boosting Robust AIGI Detection with LoRA-based Pairwise Training [55.076681464804636]
  Current AIGI detectors perform admirably on clean datasets, but their detection performance decreases when deployed "in the wild"<n>We propose a novel Lo-based Pairwise Training (RA) strategy designed specifically to achieve robust detection for AIGI under severe distortions.
  arXiv  Detail & Related papers  (2026-04-14T05:35:32Z)
• Implicit Neural Representation-Based Continuous Single Image Super Resolution: An Empirical Study [50.15623093332659]
  Implicit neural representation (INR) has become the standard approach for arbitrary-scale image super-resolution (ASSR)<n>We compare existing techniques across diverse settings and present aggregated performance results on multiple image quality metrics.<n>We examine a new loss function that penalizes intensity variations while preserving edges, textures, and finer details during training.
  arXiv  Detail & Related papers  (2026-01-25T07:09:20Z)
• Noisy Analysis of Quantum SMOTE on Condition Monitoring and Fault Classification in Industrial and Energy Systems [0.5505634045241289]
  Imbalanced machine learning models are a fundamental issue in industrial condition monitoring and fault classification pipelines.<n>This work presents a detailed benchmarking and investigation of classical classifiers under class imbalance mitigation.<n>The results show that QSMOTE consistently corrects distributional skew and significantly enhances the performance of non-linear classifiers.
  arXiv  Detail & Related papers  (2026-01-16T16:44:38Z)
• Modality-Decoupled RGB-Thermal Object Detector via Query Fusion [15.717929078660227]
  We propose a Modality-Decoupled RGB-T detection framework with Query Fusion (MDQF) to balance modality complementation and separation.<n>Our approach delivers superior performance to existing RGB-T detectors and achieves better modality independence.
  arXiv  Detail & Related papers  (2026-01-13T11:32:29Z)
• Application Research of a Deep Learning Model Integrating CycleGAN and YOLO in PCB Infrared Defect Detection [7.407155043542133]
  This paper proposes a cross-modal data augmentation framework integrating CycleGAN and YOLOv8.<n>We leverage CycleGAN to perform unpaired image-to-image translation, mapping abundant visible-light PCB images into the infrared domain.<n>We construct a heterogeneous training strategy that fuses generated pseudo-IR data with limited real IR samples to train a lightweight YOLOv8 detector.
  arXiv  Detail & Related papers  (2026-01-01T07:01:47Z)
• Physics-Informed Multimodal Bearing Fault Classification under Variable Operating Conditions using Transfer Learning [0.46085106405479537]
  This study proposes a physics-informed multimodal convolutional neural network (CNN) with a late fusion architecture.<n>The model incorporates a novel physics-informed loss function that penalizes physically implausible predictions.<n>Experiments on the Paderborn University dataset demonstrate that the proposed physics-informed approach consistently outperforms a non-physics-informed baseline.
  arXiv  Detail & Related papers  (2025-08-11T01:32:09Z)
• Learning from Heterogeneity: Generalizing Dynamic Facial Expression Recognition via Distributionally Robust Optimization [23.328511708942045]
  Heterogeneity-aware Distributional Framework (HDF) designed to enhance time-frequency modeling and mitigate imbalance caused by hard samples.<n>Time-Frequency Distributional Attention Module (DAM) captures both temporal consistency and frequency robustness.<n> adaptive optimization module Distribution-aware Scaling Module (DSM) introduced to dynamically balance classification and contrastive losses.
  arXiv  Detail & Related papers  (2025-07-21T16:21:47Z)
• RoHOI: Robustness Benchmark for Human-Object Interaction Detection [84.78366452133514]
  Human-Object Interaction (HOI) detection is crucial for robot-human assistance, enabling context-aware support.<n>We introduce the first benchmark for HOI detection, evaluating model resilience under diverse challenges.<n>Our benchmark, RoHOI, includes 20 corruption types based on the HICO-DET and V-COCO datasets and a new robustness-focused metric.
  arXiv  Detail & Related papers  (2025-07-12T01:58:04Z)
• DINAMO: Dynamic and INterpretable Anomaly MOnitoring for Large-Scale Particle Physics Experiments [0.0]
  We present novel, interpretable, robust, and scalable DQM algorithms designed to automate anomaly detection.<n>Our approach constructs evolving histogram templates with built-in uncertainties, featuring a statistical variant.<n>Experiments on synthetic datasets demonstrate the high accuracy, adaptability, and interpretability of these methods.
  arXiv  Detail & Related papers  (2025-01-31T15:51:41Z)
• Diffusion Prior Interpolation for Flexibility Real-World Face Super-Resolution [48.34173818491552]
  Diffusion Prior Interpolation (DPI) can balance consistency and diversity and can be seamlessly integrated into pre-trained models.<n>In extensive experiments conducted on synthetic and real datasets, DPI demonstrates superiority over SOTA FSR methods.
  arXiv  Detail & Related papers  (2024-12-21T09:28:44Z)
• Energy Score-based Pseudo-Label Filtering and Adaptive Loss for Imbalanced Semi-supervised SAR target recognition [1.2035771704626825]
  Existing semi-supervised SAR ATR algorithms show low recognition accuracy in the case of class imbalance. This work offers a non-balanced semi-supervised SAR target recognition approach using dynamic energy scores and adaptive loss.
  arXiv  Detail & Related papers  (2024-11-06T14:45:16Z)
• Automatically Adaptive Conformal Risk Control [49.95190019041905]
  We propose a methodology for achieving approximate conditional control of statistical risks by adapting to the difficulty of test samples.<n>Our framework goes beyond traditional conditional risk control based on user-provided conditioning events to the algorithmic, data-driven determination of appropriate function classes for conditioning.
  arXiv  Detail & Related papers  (2024-06-25T08:29:32Z)
• Feature Corrective Transfer Learning: End-to-End Solutions to Object Detection in Non-Ideal Visual Conditions [11.90136900277127]
  "Feature Corrective Transfer Learning" is a novel approach to facilitate the end-to-end detection of objects in challenging scenarios. Non-ideal images are processed by comparing their feature maps against those from the initial ideal RGB model. This approach refines the model's ability to perform object detection across varying conditions through direct feature map correction.
  arXiv  Detail & Related papers  (2024-04-17T09:58:53Z)
• Tensor Factorization for Leveraging Cross-Modal Knowledge in Data-Constrained Infrared Object Detection [22.60228799622782]
  Key bottleneck in object detection in IR images is lack of sufficient labeled training data. We seek to leverage cues from the RGB modality to scale object detectors to the IR modality, while preserving model performance in the RGB modality. We first pretrain these factor matrices on the RGB modality, for which plenty of training data are assumed to exist and then augment only a few trainable parameters for training on the IR modality to avoid over-fitting.
  arXiv  Detail & Related papers  (2023-09-28T16:55:52Z)
• Single-Trajectory Distributionally Robust Reinforcement Learning [21.955807398493334]
  We propose Distributionally Robust RL (DRRL) to enhance performance across a range of environments. Existing DRRL algorithms are either model-based or fail to learn from a single sample trajectory. We design a first fully model-free DRRL algorithm, called distributionally robust Q-learning with single trajectory (DRQ)
  arXiv  Detail & Related papers  (2023-01-27T14:08:09Z)
• Improve Noise Tolerance of Robust Loss via Noise-Awareness [60.34670515595074]
  We propose a meta-learning method which is capable of adaptively learning a hyper parameter prediction function, called Noise-Aware-Robust-Loss-Adjuster (NARL-Adjuster for brevity) Four SOTA robust loss functions are attempted to be integrated with our algorithm, and comprehensive experiments substantiate the general availability and effectiveness of the proposed method in both its noise tolerance and performance.
  arXiv  Detail & Related papers  (2023-01-18T04:54:58Z)
• Model-based Deep Learning Receiver Design for Rate-Splitting Multiple Access [65.21117658030235]
  This work proposes a novel design for a practical RSMA receiver based on model-based deep learning (MBDL) methods. The MBDL receiver is evaluated in terms of uncoded Symbol Error Rate (SER), throughput performance through Link-Level Simulations (LLS) and average training overhead. Results reveal that the MBDL outperforms by a significant margin the SIC receiver with imperfect CSIR.
  arXiv  Detail & Related papers  (2022-05-02T12:23:55Z)
• Self-Damaging Contrastive Learning [92.34124578823977]
  Unlabeled data in reality is commonly imbalanced and shows a long-tail distribution. This paper proposes a principled framework called Self-Damaging Contrastive Learning to automatically balance the representation learning without knowing the classes. Our experiments show that SDCLR significantly improves not only overall accuracies but also balancedness.
  arXiv  Detail & Related papers  (2021-06-06T00:04:49Z)
• Learning perturbation sets for robust machine learning [97.6757418136662]
  We use a conditional generator that defines the perturbation set over a constrained region of the latent space. We measure the quality of our learned perturbation sets both quantitatively and qualitatively. We leverage our learned perturbation sets to train models which are empirically and certifiably robust to adversarial image corruptions and adversarial lighting variations.
  arXiv  Detail & Related papers  (2020-07-16T16:39:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.