ReSPEC: A Framework for Online Multispectral Sensor Reconfiguration in Dynamic Environments

• URL: http://arxiv.org/abs/2602.10547v1
• Date: Wed, 11 Feb 2026 05:39:24 GMT
• Title: ReSPEC: A Framework for Online Multispectral Sensor Reconfiguration in Dynamic Environments
• Abstract summary: We introduce a framework that unifies sensing, learning, and actuation into a closed reconfiguration loop.<n>We implement and evaluate this framework on a mobile rover, showing that adaptive control reduces load by 29.3% with only a 5.3% accuracy drop compared to a baseline.
• Score: 6.718801307135799
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-sensor fusion is central to robust robotic perception, yet most existing systems operate under static sensor configurations, collecting all modalities at fixed rates and fidelity regardless of their situational utility. This rigidity wastes bandwidth, computation, and energy, and prevents systems from prioritizing sensors under challenging conditions such as poor lighting or occlusion. Recent advances in reinforcement learning (RL) and modality-aware fusion suggest the potential for adaptive perception, but prior efforts have largely focused on re-weighting features at inference time, ignoring the physical cost of sensor data collection. We introduce a framework that unifies sensing, learning, and actuation into a closed reconfiguration loop. A task-specific detection backbone extracts multispectral features (e.g. RGB, IR, mmWave, depth) and produces quantitative contribution scores for each modality. These scores are passed to an RL agent, which dynamically adjusts sensor configurations, including sampling frequency, resolution, sensing range, and etc., in real time. Less informative sensors are down-sampled or deactivated, while critical sensors are sampled at higher fidelity as environmental conditions evolve. We implement and evaluate this framework on a mobile rover, showing that adaptive control reduces GPU load by 29.3\% with only a 5.3\% accuracy drop compared to a heuristic baseline. These results highlight the potential of resource-aware adaptive sensing for embedded robotic platforms.

Related papers

• Robust Slip Detection and Material Classification via Spatiotemporal Transformers on a Uniformly-Illuminated Visuo-Tactile Sensor [52.33204893988409]
  We propose a visuo-tactile sensor featuring customized uniform illumination, alongside a unified perception framework.<n>The proposed hardware-software framework provides high-fidelity feedback and a powerful multi-modal perception baseline for complex robotic manipulation.
  arXiv  Detail & Related papers  (2026-08-25T07:25:27Z)
• Sen-Cap: Sensor-Flexible and Noise-Resilient Human Motion Capture via LiDAR-Camera Integration [50.18488308004802]
  Sen-Cap is a 3D motion capture framework that integrates multi-modal data from LiDAR and camera.<n>Sen-Cap achieves state-of-the-art performance on major metrics on Human-M3 and FreeMotion.<n>This combination of flexibility and robustness opens new opportunities for motion capture in real-world scenarios.
  arXiv  Detail & Related papers  (2026-08-03T14:16:18Z)
• RAWild: Sensor-Agnostic RAW Object Detection via Physics-Guided Curve and Grid Modeling [69.96407948879015]
  textbfRAWild is a physics-guided global-local tone mapping framework for sensor-agnostic RAW object detection.<n>We construct a physics-based RAW simulation pipeline that synthesizes realistic sensor outputs spanning diverse spectral sensitivities, illuminants, and sensor non-idealities.
  arXiv  Detail & Related papers  (2026-05-07T09:51:47Z)
• Information Density as a Quantitative Measure for AI-enabled Virtual Sensing: Feasibility and Limits [3.2957112871590777]
  This paper introduces Information Density as a quantitative metric to support sensor deployment and enable AI-driven virtual sensing.<n>We propose a framework that leverages spatial, temporal and inter-modal correlations among sensor signals to perform sensing tasks even in the absence of physical sensors.
  arXiv  Detail & Related papers  (2026-05-05T09:03:37Z)
• UniForce: A Unified Latent Force Model for Robot Manipulation with Diverse Tactile Sensors [51.88112610411651]
  We propose a novel unified tactile representation learning framework that learns a shared latent force space across diverse tactile sensors.<n>UniForce reduces cross-sensor domain shift by jointly modeling inverse dynamics (image-to-force) and forward dynamics (force-to-image)<n>To avoid reliance on expensive external force/torque (F/T) sensors, we exploit static equilibrium and collect force-paired data via direct sensor--object--sensor interactions.
  arXiv  Detail & Related papers  (2026-02-01T11:03:01Z)
• Occluded nuScenes: A Multi-Sensor Dataset for Evaluating Perception Robustness in Automated Driving [4.60199848319746]
  Occluded nuScenes dataset is a novel extension of the widely used nuScenes benchmark.<n>This resource supports consistent, reproducible evaluation of perception models under partial sensor failures and environmental interference.<n>We aim to advance research on robust sensor fusion, resilience analysis, and safety-critical perception in automated driving.
  arXiv  Detail & Related papers  (2025-10-21T12:02:26Z)
• Progressive Inertial Poser: Progressive Real-Time Kinematic Chain Estimation for 3D Full-Body Pose from Three IMU Sensors [25.67875816218477]
  Full-body pose estimation from sparse tracking signals is not limited by environmental conditions or recording range.<n>Previous works either face the challenge of wearing additional sensors on the pelvis and lower-body or rely on external visual sensors to obtain global positions of key joints.<n>To improve the practicality of the technology for virtual reality applications, we estimate full-body poses using only inertial data obtained from three Inertial Measurement Unit (IMU) sensors worn on the head and wrists.
  arXiv  Detail & Related papers  (2025-05-08T15:28:09Z)
• Optimizing Sensor Redundancy in Sequential Decision-Making Problems [6.801212913428521]
  Reinforcement Learning (RL) policies are designed to predict actions based on current observations to maximize cumulative future rewards.<n>In real-world applications, sensors are essential for measuring the current state and providing the observations on which RL policies rely to make decisions.<n>This paper explores the optimization of backup sensor configurations to maximize expected returns while keeping costs below a specified threshold.
  arXiv  Detail & Related papers  (2024-12-10T17:20:44Z)
• CAFuser: Condition-Aware Multimodal Fusion for Robust Semantic Perception of Driving Scenes [56.52618054240197]
  We propose a novel, condition-aware multimodal fusion approach for robust semantic perception of driving scenes.<n>Our method, CAFuser, uses an RGB camera input to classify environmental conditions and generate a Condition Token.<n>Our model significantly improves robustness and accuracy, especially in adverse-condition scenarios.
  arXiv  Detail & Related papers  (2024-10-14T17:56:20Z)
• DiffusionPoser: Real-time Human Motion Reconstruction From Arbitrary Sparse Sensors Using Autoregressive Diffusion [10.439802168557513]
  Motion capture from a limited number of body-worn sensors has important applications in health, human performance, and entertainment. Recent work has focused on accurately reconstructing whole-body motion from a specific sensor configuration using six IMUs. We propose a single diffusion model, DiffusionPoser, which reconstructs human motion in real-time from an arbitrary combination of sensors.
  arXiv  Detail & Related papers  (2023-08-31T12:36:50Z)
• Joint Sensing, Communication, and AI: A Trifecta for Resilient THz User Experiences [118.91584633024907]
  A novel joint sensing, communication, and artificial intelligence (AI) framework is proposed so as to optimize extended reality (XR) experiences over terahertz (THz) wireless systems.
  arXiv  Detail & Related papers  (2023-04-29T00:39:50Z)
• Real-time detection of uncalibrated sensors using Neural Networks [62.997667081978825]
  An online machine-learning based uncalibration detector for temperature, humidity and pressure sensors was developed. The solution integrates an Artificial Neural Network as main component which learns from the behavior of the sensors under calibrated conditions. The obtained results show that the proposed solution is able to detect uncalibrations for deviation values of 0.25 degrees, 1% RH and 1.5 Pa, respectively.
  arXiv  Detail & Related papers  (2021-02-02T15:44:39Z)
• Learning Camera Miscalibration Detection [83.38916296044394]
  This paper focuses on a data-driven approach to learn the detection of miscalibration in vision sensors, specifically RGB cameras. Our contributions include a proposed miscalibration metric for RGB cameras and a novel semi-synthetic dataset generation pipeline based on this metric. By training a deep convolutional neural network, we demonstrate the effectiveness of our pipeline to identify whether a recalibration of the camera's intrinsic parameters is required or not.
  arXiv  Detail & Related papers  (2020-05-24T10:32:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.