$x^2$-Fusion: Cross-Modality and Cross-Dimension Flow Estimation in Event Edge Space

• URL: http://arxiv.org/abs/2603.16671v1
• Date: Tue, 17 Mar 2026 15:36:57 GMT
• Title: $x^2$-Fusion: Cross-Modality and Cross-Dimension Flow Estimation in Event Edge Space
• Abstract summary: Estimating dense 2D optical flow and 3D scene flow is essential for dynamic scene understanding.<n>Recent work combines images, LiDAR, and event data to jointly predict 2D and 3D motion, yet most approaches operate in separate heterogeneous feature spaces.<n>We introduce $x2$-Fusion, which reframes multimodal fusion as representation unification.
• Score: 22.76310468233763
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Estimating dense 2D optical flow and 3D scene flow is essential for dynamic scene understanding. Recent work combines images, LiDAR, and event data to jointly predict 2D and 3D motion, yet most approaches operate in separate heterogeneous feature spaces. Without a shared latent space that all modalities can align to, these systems rely on multiple modality-specific blocks, leaving cross-sensor mismatches unresolved and making fusion unnecessarily complex.Event cameras naturally provide a spatiotemporal edge signal, which we can treat as an intrinsic edge field to anchor a unified latent representation, termed the Event Edge Space. Building on this idea, we introduce $x^2$-Fusion, which reframes multimodal fusion as representation unification: event-derived spatiotemporal edges define an edge-centric homogeneous space, and image and LiDAR features are explicitly aligned in this shared representation.Within this space, we perform reliability-aware adaptive fusion to estimate modality reliability and emphasize stable cues under degradation. We further employ cross-dimension contrast learning to tightly couple 2D optical flow with 3D scene flow. Extensive experiments on both synthetic and real benchmarks show that $x^2$-Fusion achieves state-of-the-art accuracy under standard conditions and delivers substantial improvements in challenging scenarios.

Related papers

• Weaving Light and Time: Unified Harmonic-Geometric Representation Learning for Dense RGB-Event Parsing [55.13802890769086]
  We present Evita, the first unified backbone specifically engineered for dedicated dense RGB-Event parsing.<n>To achieve profound modal synergy, Evita embeds a suite of intrinsic co-learning modules directly into every encoder layer.<n>Evita establishes new state-of-the-art metrics while delivering a superior accuracy-latency trade-off for real-time multimodal perception.
  arXiv  Detail & Related papers  (2026-07-10T06:52:09Z)
• Bidirectional Cross-Modal Prompting for Event-Frame Asymmetric Stereo [54.58078274676216]
  Event cameras offer an alternative visual representation with higher dynamic range free of such limitations.<n>In this paper, we introduce Bi-CMPStereo, a novel bidirectional cross-modal prompting framework.<n>Our approach learns finely aligned stereo representations within a target canonical space and integrates complementary representations by projecting each modality into both event and frame domains.
  arXiv  Detail & Related papers  (2026-04-16T17:59:58Z)
• FUSER: Feed-Forward MUltiview 3D Registration Transformer and SE(3)$^N$ Diffusion Refinement [39.19949818461193]
  F is the first feed-forward multiview registration transformer that processes all scans in a unified, compact latent space.<n>F predicts global poses without any pairwise estimation.<n>Experiments on 3DMatch, ScanNet and ArkitScenes demonstrate that our approach achieves the superior registration accuracy and outstanding computational efficiency.
  arXiv  Detail & Related papers  (2025-12-10T07:11:22Z)
• OpenInsGaussian: Open-vocabulary Instance Gaussian Segmentation with Context-aware Cross-view Fusion [89.98812408058336]
  We introduce textbfOpenInsGaussian, an textbfOpen-vocabulary textbfInstance textbfGaussian segmentation framework with Context-aware Cross-view Fusion.<n>OpenInsGaussian achieves state-of-the-art results in open-vocabulary 3D Gaussian segmentation, outperforming existing baselines by a large margin.
  arXiv  Detail & Related papers  (2025-10-21T03:24:12Z)
• Cross-Modal Geometric Hierarchy Fusion: An Implicit-Submap Driven Framework for Resilient 3D Place Recognition [9.411542547451193]
  We propose a novel framework that redefines 3D place recognition through density-agnostic geometric reasoning.<n>Specifically, we introduce an implicit 3D representation based on elastic points, which is immune to the interference of original scene point cloud density.<n>With the aid of these two types of information, we obtain descriptors that fuse geometric information from both bird's-eye view and 3D segment perspectives.
  arXiv  Detail & Related papers  (2025-06-17T07:04:07Z)
• FUSE: Label-Free Image-Event Joint Monocular Depth Estimation via Frequency-Decoupled Alignment and Degradation-Robust Fusion [92.4205087439928]
  Image-event joint depth estimation methods leverage complementary modalities for robust perception, yet face challenges in generalizability.<n>We propose the Self-supervised Transfer (PST) and the FrequencyDe-coupled Fusion module (FreDF)<n>PST establishes cross-modal knowledge transfer through latent space alignment with image foundation models, effectively mitigating data scarcity.<n>FreDF explicitly decouples high-frequency edge features from low-frequency structural components, resolving modality-specific frequency mismatches.<n>This combined approach enables FUSE to construct a universal image-event that only requires lightweight decoder adaptation for target datasets.
  arXiv  Detail & Related papers  (2025-03-25T15:04:53Z)
• A Unified Diffusion Framework for Scene-aware Human Motion Estimation from Sparse Signals [47.72778485845676]
  We introduce a new framework to combine rich contextual information provided by scenes to benefit full-body motion tracking from sparse observations. $textS2$Fusion first extracts the spatial-temporal relations residing in the sparse signals via a periodic autoencoder. $textS2$Fusion utilizes conditional diffusion to fuse scene geometry and sparse tracking signals to generate full-body scene-aware motions.
  arXiv  Detail & Related papers  (2024-04-07T09:15:45Z)
• RPEFlow: Multimodal Fusion of RGB-PointCloud-Event for Joint Optical Flow and Scene Flow Estimation [43.358140897849616]
  In this paper, we incorporate RGB images, Point clouds and Events for joint optical flow and scene flow estimation with our proposed multi-stage multimodal fusion model, RPEFlow. Experiments on both synthetic and real datasets show that our model outperforms the existing state-of-the-art by a wide margin.
  arXiv  Detail & Related papers  (2023-09-26T17:23:55Z)
• On Robust Cross-View Consistency in Self-Supervised Monocular Depth Estimation [56.97699793236174]
  We study two kinds of robust cross-view consistency in this paper. We exploit the temporal coherence in both depth feature space and 3D voxel space for self-supervised monocular depth estimation. Experimental results on several outdoor benchmarks show that our method outperforms current state-of-the-art techniques.
  arXiv  Detail & Related papers  (2022-09-19T03:46:13Z)
• Similarity-Aware Fusion Network for 3D Semantic Segmentation [87.51314162700315]
  We propose a similarity-aware fusion network (SAFNet) to adaptively fuse 2D images and 3D point clouds for 3D semantic segmentation. We employ a late fusion strategy where we first learn the geometric and contextual similarities between the input and back-projected (from 2D pixels) point clouds. We show that SAFNet significantly outperforms existing state-of-the-art fusion-based approaches across various data integrity.
  arXiv  Detail & Related papers  (2021-07-04T09:28:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.