A Confidence-Aware Multimodal Fusion Framework for Industrial Human-Robot Collaboration

• URL: http://arxiv.org/abs/2609.10339v1
• Date: Wed, 09 Sep 2026 15:36:22 GMT
• Title: A Confidence-Aware Multimodal Fusion Framework for Industrial Human-Robot Collaboration
• Abstract summary: This framework fuses four heterogeneous modalities including object 6D pose, gaze, skeletal motion and IMU-based hand motion.<n>It embeds a confidence-trend-driven dynamic fusion mechanism into BiLSTM to adaptively balance bidirectional temporal features.<n>In practical assembly tasks, the framework enables proactive and stable human-robot cooperation with strong environmental adaptability.
• Score: 4.927894263207565
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A confidence-aware multimodal fusion framework (CAMF) is proposed to realize reliable human intention prediction for industrial human-robot collaboration. This framework fuses four heterogeneous modalities including object 6D pose, gaze, skeletal motion and IMU-based hand motion. It embeds a confidence-trend-driven dynamic fusion mechanism into BiLSTM to adaptively balance bidirectional temporal features according to real-time modality reliability. A confidence-guided balanced learning strategy combined with a confidence freezing mechanism is further adopted to adjust network gradients dynamically, suppress noise from low-quality modalities and mitigate cross-modal learning bias. A physical platform based on the UR3 collaborative robot is built for experimental validation. Comparative results show that the proposed method reaches an intention recognition accuracy of 91.86% and outperforms existing multimodal fusion approaches in overall performance and stability. It also maintains satisfactory accuracy under low light and partial occlusion interference. In practical assembly tasks, the framework enables proactive and stable human-robot cooperation with strong environmental adaptability.

Related papers

• Adaptive Confidence-weighted Expansion for Trustworthy Multi-Omics Multimodal Fusion [0.8749675983608171]
  We introduce Adaptive Confidence-weighted Expansion (ACE) to enhance the trustworthiness of multimodal fusion models.<n>ACE significantly outperforms existing state-of-the-art algorithms in both classification performance and confidence calibration.
  arXiv  Detail & Related papers  (2026-07-22T21:42:16Z)
• Distributed Motion Planning with Safety Guarantees for Self-Reconfiguring Robotic Boats [76.98547580312116]
  This paper proposes a hybrid framework that combines distributed Model Predictive Control (MPC) with Control Barrier Functions (CBFs)<n>To ensure safety in real time, distributed CBF-based filters are applied to enforce inter-agent avoidance.
  arXiv  Detail & Related papers  (2026-07-22T16:37:05Z)
• Consistency-Driven Dual LSTM Models for Kinematic Control of a Wearable Soft Robotic Arm [33.333272339511446]
  We introduce a consistency-driven dual LSTM framework for accurately learning both the forward and inverse kinematics of a pneumatically actuated soft robotic arm integrated into a wearable device.<n>This approach effectively captures the nonlinear and hysteretic behaviors of soft pneumatic actuators while addressing the one-to-many mapping challenge between actuation inputs and end-effector positions.
  arXiv  Detail & Related papers  (2026-03-18T12:51:12Z)
• Stage-Adaptive Reliability Modeling for Continuous Valence-Arousal Estimation [11.761508894410182]
  We propose a Stage-Adaptive reliability modeling framework that explicitly estimates and calibrates modality-wise confidence during multimodal integration.<n>SAGE introduces a reliability-aware fusion mechanism that dynamically rebalances audio and visual representations according to their stage-dependent informativeness.<n>Experiments on the Aff-Wild2 benchmark demonstrate that SAGE consistently improves concordance correlation coefficient scores compared with existing multimodal fusion approaches.
  arXiv  Detail & Related papers  (2026-03-12T02:45:41Z)
• BHaRNet: Reliability-Aware Body-Hand Modality Expertized Networks for Fine-grained Skeleton Action Recognition [12.973657570368317]
  This work presents a probabilistic dual-stream framework that unifies reliability modeling and multi-modal integration, generalizing expertized learning under uncertainty across both intra-skeleton and cross-modal domains.<n>The framework comprises three key components: (1) a calibration-free preprocessing pipeline that removes canonical-space transformations and learns directly from native coordinates; (2) a probabilistic Noisy-OR fusion that stabilizes reliability-aware dual-stream learning without requiring explicit confidence supervision; and (3) an intra- to cross-modal ensemble that couples four skeleton modalities (Joint, Bone, Joint Motion, and Bone Motion) to RGB representations, bridging structural and visual motion
  arXiv  Detail & Related papers  (2026-01-01T15:13:11Z)
• Empower Low-Altitude Economy: A Reliability-Aware Dynamic Weighting Allocation for Multi-modal UAV Beam Prediction [57.04985443535312]
  Low-altitude economy (LAE) is rapidly expanding driven by urban air mobility, logistics drones, and aerial sensing.<n>Current research is shifting from single-signal to multi-modal collaborative approaches.<n>We propose a reliability-aware dynamic weighting scheme applied to a semantic-aware multi-modal beam prediction framework, named SaM2B.
  arXiv  Detail & Related papers  (2025-12-30T16:24:34Z)
• Drift No More? Context Equilibria in Multi-Turn LLM Interactions [58.69551510148673]
  contexts drift is the gradual divergence of a model's outputs from goal-consistent behavior across turns.<n>Unlike single-turn errors, drift unfolds temporally and is poorly captured by static evaluation metrics.<n>We show that multi-turn drift can be understood as a controllable equilibrium phenomenon rather than as inevitable decay.
  arXiv  Detail & Related papers  (2025-10-09T04:48:49Z)
• Feed Two Birds with One Scone: Exploiting Function-Space Regularization for Both OOD Robustness and ID Fine-Tuning Performance [72.57668440744301]
  We propose a novel regularization that constrains the distance of fine-tuning and pre-trained model in the function space with simulated OOD samples.<n>Our approach could consistently improve both downstream task ID fine-tuning performance and OOD robustness across a variety of CLIP backbones.
  arXiv  Detail & Related papers  (2025-08-31T12:14:34Z)
• Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics [50.191655141020505]
  This work advances model-based reinforcement learning by addressing the challenges of long-horizon prediction, error accumulation, and sim-to-real transfer.<n>By providing a scalable and robust framework, the introduced methods pave the way for adaptive and efficient robotic systems in real-world applications.
  arXiv  Detail & Related papers  (2025-01-17T10:39:09Z)
• Learning Multimodal Confidence for Intention Recognition in Human-Robot Interaction [6.658036685588584]
  We propose a novel learning-based multimodal fusion framework Batch Multimodal Confidence Learning for Opinion Pool (BMCLOP) Our approach combines Bayesian multimodal fusion method and batch confidence learning algorithm to improve accuracy, uncertainty reduction and success rate. Our desired assistive scenarios consider three modalities gestures, speech and gaze, all of which produce categorical distributions over all the finite intentions.
  arXiv  Detail & Related papers  (2024-05-23T02:43:31Z)
• Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions [91.63716984911278]
  We introduce a novel Mixture of Normal-Inverse Gamma distributions (MoNIG) algorithm, which efficiently estimates uncertainty in principle for adaptive integration of different modalities and produces a trustworthy regression result. Experimental results on both synthetic and different real-world data demonstrate the effectiveness and trustworthiness of our method on various multimodal regression tasks.
  arXiv  Detail & Related papers  (2021-11-11T14:28:12Z)
• Efficient Empowerment Estimation for Unsupervised Stabilization [75.32013242448151]
  empowerment principle enables unsupervised stabilization of dynamical systems at upright positions. We propose an alternative solution based on a trainable representation of a dynamical system as a Gaussian channel. We show that our method has a lower sample complexity, is more stable in training, possesses the essential properties of the empowerment function, and allows estimation of empowerment from images.
  arXiv  Detail & Related papers  (2020-07-14T21:10:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.