Asynchronous Multimodal Diffusion Policy Composition via Latency-Aware Guidance Fusion

• URL: http://arxiv.org/abs/2607.17257v1
• Date: Sun, 19 Jul 2026 13:48:52 GMT
• Title: Asynchronous Multimodal Diffusion Policy Composition via Latency-Aware Guidance Fusion
• Abstract summary: We propose LAG-Fusion, a latency-aware guidance fusion framework for asynchronous multimodal diffusion policy composition.<n>We instantiate LAG-Fusion in contact-rich manipulation by composing a low-frequency vision policy with a high-frequency force policy.
• Score: 52.50952605013304
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Diffusion policies have shown strong potential for robotic imitation learning, and recent extensions incorporate additional modalities to improve manipulation performance. However, these modalities often differ not only in information content but also in sensing rates and inference latencies. Existing multimodal diffusion policies typically rely on synchronous fusion or manually designed multi-frequency architectures, which either slow down high-frequency feedback or limit extensibility to new modality combinations. We propose LAG-Fusion, a latency-aware guidance fusion framework for asynchronous multimodal diffusion policy composition. LAG-Fusion allows modality-specific policies to operate at their native inference rates and contribute denoising guidance whenever available. To make asynchronous composition consistent, we derive a reference-frame rebasing rule for diffusion variables under relative action representations, enabling delayed guidance to be aligned before fusion. We instantiate LAG-Fusion in contact-rich manipulation by composing a low-frequency vision policy with a high-frequency force policy. Experiments under heterogeneous modality latencies show that LAG-Fusion improves policy responsiveness and task performance over synchronous fusion and specially designed force-aware baselines.

Related papers

• A Hierarchical Approach to Imitation Learning for Manipulation Tasks Requiring Time Varying Forces [14.372073538832803]
  We introduce the Diffusion Policy Augmented by Fast Trajectory Generation (DPA-FTG)<n>Compared to recent visual-tactile approaches that focus on positional correction, DPA-FTG decouples low-frequency planning from high-frequency force regulation.<n>We validate our approach on a bimanual battery disassembly task involving the separation of a compliant sheet.
  arXiv  Detail & Related papers  (2026-08-04T04:21:00Z)
• Boosting Multimodal Federated Learning via Chained Modality Optimization [69.53075328069504]
  Multimodal Federated Learning (MMFL) enables privacy-preserving collaborative learning across decentralized clients with heterogeneous data and modality availability.<n>To address this, we propose FedMChain, a balanced MMFL framework that structures federated multimodal training as a chain of modality-wise phases.<n>This phase-wise design gives each modality a dedicated local optimization window on multimodal clients to mitigate modality competition, and further promotes cross-modal complementarity via an error-compensated regularizer.
  arXiv  Detail & Related papers  (2026-06-01T08:07:09Z)
• From Noise to Control: Parameterized Diffusion Policies [54.941343992110426]
  We propose a framework for learning diffusion policies conditioned on continuous parameters embedded in a learned behavior manifold.<n>Our approach enables smooth distances between known strategies and efficient adaptation to novel constraints without updating policy weights.
  arXiv  Detail & Related papers  (2026-05-29T20:21:50Z)
• From Flow to One Step: Real-Time Multi-Modal Trajectory Policies via Implicit Maximum Likelihood Estimation-based Distribution Distillation [18.70033095161235]
  We propose a framework that distills a Conditional Flow Matching expert into a fast single-step student via Implicit Likelihood Estimation (IMLE)<n>A bi-directional Chamfer distance provides a set-level objective that promotes both mode coverage and fidelity.<n>A unified perception encoder further integrates multi-view RGB, depth, point clouds, and proprioception into a geometry-aware representation.
  arXiv  Detail & Related papers  (2026-03-10T09:30:05Z)
• SG-RIFE: Semantic-Guided Real-Time Intermediate Flow Estimation with Diffusion-Competitive Perceptual Quality [0.0]
  Real-time Video Frame Interpolation (VFI) has long been dominated by flow-based methods like RIFE.<n>Recent diffusion-based approaches achieve state-of-the-art perceptual quality but suffer from prohibitive latency, rendering them impractical for real-time applications.<n>We propose Semantic-Guided RIFE (SG-RIFE), which augments a pre-trained RIFE backbone with semantic priors from a frozen DINOv3 Vision Transformer.
  arXiv  Detail & Related papers  (2025-12-20T06:50:55Z)
• ImplicitRDP: An End-to-End Visual-Force Diffusion Policy with Structural Slow-Fast Learning [52.86018040861575]
  We propose a unified end-to-end visual-force diffusion policy that integrates visual planning and reactive force control within a single network.<n>We introduce Structural Slow-Fast Learning, a mechanism utilizing causal attention to simultaneously process asynchronous visual and force tokens.<n>Experiments on contact-rich tasks demonstrate that ImplicitRDP significantly outperforms both vision-only and hierarchical baselines.
  arXiv  Detail & Related papers  (2025-12-11T18:59:46Z)
• UniDiff: A Unified Diffusion Framework for Multimodal Time Series Forecasting [90.47915032778366]
  We propose UniDiff, a unified diffusion framework for multimodal time series forecasting.<n>At its core lies a unified and parallel fusion module, where a single cross-attention mechanism integrates structural information from timestamps and semantic context from texts.<n>Experiments on real-world benchmark datasets across eight domains demonstrate that the proposed UniDiff model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-12-08T05:36:14Z)
• Towards Unified Semantic and Controllable Image Fusion: A Diffusion Transformer Approach [99.80480649258557]
  DiTFuse is an instruction-driven framework that performs semantics-aware fusion within a single model.<n>Experiments on public IVIF, MFF, and MEF benchmarks confirm superior quantitative and qualitative performance, sharper textures, and better semantic retention.
  arXiv  Detail & Related papers  (2025-12-08T05:04:54Z)
• IRDFusion: Iterative Relation-Map Difference guided Feature Fusion for Multispectral Object Detection [23.256601188227865]
  We propose an innovative feature fusion framework based on cross-modal feature contrastive and screening strategy.<n>The proposed method adaptively enhances salient structures by fusing object-aware complementary cross-modal features.<n>IRDFusion consistently outperforms existing methods across diverse challenging scenarios.
  arXiv  Detail & Related papers  (2025-09-11T01:22:35Z)
• FindRec: Stein-Guided Entropic Flow for Multi-Modal Sequential Recommendation [57.577843653775]
  We propose textbfFindRec (textbfFlexible unified textbfinformation textbfdisentanglement for multi-modal sequential textbfRecommendation)<n>A Stein kernel-based Integrated Information Coordination Module (IICM) theoretically guarantees distribution consistency between multimodal features and ID streams.<n>A cross-modal expert routing mechanism that adaptively filters and combines multimodal features based on their contextual relevance.
  arXiv  Detail & Related papers  (2025-07-07T04:09:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.