Cross-Modal Hierarchical Fusion for from Multi-Sensor Ground Observation

• URL: http://arxiv.org/abs/2606.30647v1
• Date: Sun, 31 May 2026 14:17:43 GMT
• Title: Cross-Modal Hierarchical Fusion for from Multi-Sensor Ground Observation
• Abstract summary: AtmoFuseNet is a framework that fuses multi-view sky camera imagery with millimeter-wave cloud radar and ceilometer observations to produce 4D estimates of cloud state and wind.<n>On collocated observations from a semi-arid site, AtmoFuseNet reaches 0.026 g m-3 liquid water content MAE and 1.18 ms-1 wind speed MAE.
• Score: 2.538209532048867
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dense volumetric reconstruction of cloud microphysical fields from sparse ground-based instruments remains an open problem, largely because the available measurements are heterogeneous in both modality and spatial coverage. We present AtmoFuseNet, a framework that fuses multi-view sky camera imagery with millimeter-wave cloud radar and ceilometer observations to produce 4D (three spatial dimensions plus time) estimates of cloud state and wind. The method operates in three stages: a cross-modal hierarchical aggregation module that combines image feature pyramids with instrument-derived vertical profiles through layer-wise cross-attention; a conditional variational refinement module that maps the resulting volume to physically consistent microphysical fields under differentiable radar and image forward models; and a correlation-based motion estimator that recovers per-voxel 3D wind vectors from consecutive volumetric reconstructions. On collocated observations from a semi-arid site, AtmoFuseNet reaches 0.026 g m^-3 liquid water content MAE and 1.18 m s^-1 wind speed MAE, improving over existing retrieval baselines. Ablation experiments isolate the contribution of each module.

Related papers

• OmniLiDAR: A Unified Diffusion Framework for Multi-Domain 3D LiDAR Generation [100.36917520403227]
  We present a unified text-conditioned diffusion framework that generates LiDAR scans in a shared range-image representation across eight representative domains.<n>In the absence of a public consolidated benchmark, we construct an 8-domain dataset by combining real-world scans with physically based weather simulation and systematic beam reduction.
  arXiv  Detail & Related papers  (2026-05-13T17:42:20Z)
• RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow [51.43025173196566]
  Monocular scene flow estimation aims to recover dense 3D motion from image sequences.<n>We propose RAFT-MSF++, a self-supervised multi-frame framework that fuses temporal features to jointly estimate depth and scene flow.<n>Experiments show that RAFT-MSF++ achieves 24.14% SF-all on the KITTI Scene Flow benchmark, with a 30.99% improvement over the baseline and better robustness in occluded regions.
  arXiv  Detail & Related papers  (2026-04-21T11:32:49Z)
• ZeD-MAP: Bundle Adjustment Guided Zero-Shot Depth Maps for Real-Time Aerial Imaging [1.9247157750972368]
  ZeD-MAP is a cluster-level framework that converts a test-time diffusion depth model into a metrically consistent, SLAM-like mapping pipeline.<n>We show that our method achieves sub-meter accuracy, with approximately 0.87 m error in the horizontal (XY) plane and 0.12 m in the vertical (Z) direction.<n>Results are subject to minor noise from manual point-cloud annotation.
  arXiv  Detail & Related papers  (2026-04-06T13:21:17Z)
• Learning Pore-scale Multiphase Flow from 4D Velocimetry [21.658780881627536]
  Multiphase flow in porous media underpins subsurface energy and environmental technologies.<n>We introduce a framework that infers multiphase pore-scale flow directly from time-resolved micro-velocimetry measurements.
  arXiv  Detail & Related papers  (2026-03-12T23:28:20Z)
• SDCM: Simulated Densifying and Compensatory Modeling Fusion for Radar-Vision 3-D Object Detection in Internet of Vehicles [3.0945968470179754]
  A framework named SDCM is proposed, which contains Simulated Densifying and Compensatory Modeling Fusion for radar-vision 3-D object detection in IoV.<n>Experiment results on the VoD, TJ4DRadSet and Astyx HiRes 2019 dataset show that SDCM achieves best performance with lower parameter quantity and faster inference speed.
  arXiv  Detail & Related papers  (2026-01-29T12:52:16Z)
• Seabed-to-Sky Mapping of Maritime Environments with a Dual Orthogonal SONAR and LiDAR Sensor Suite [0.3694429692322631]
  Critical maritime infrastructure increasingly demands situational awareness both above and below the surface.<n>We present a unified, independent mapping system that fuses LiDAR-IMU with a dual, orthogonally mounted Forward Looking Sonars.<n>On the acoustic side, we extend wide-aperture fusion to handle arbitrary inter-sonar translations and extract a leading edge from each FLS to form line-scans.<n>We validate the system on real-world data from Belvederekanalen (Copenhagen) demonstrating real-time operation with approx. 2.65 Hz Hz updates and approx. 2.85 Hz odometry.
  arXiv  Detail & Related papers  (2025-12-04T22:53:32Z)
• PFDepth: Heterogeneous Pinhole-Fisheye Joint Depth Estimation via Distortion-aware Gaussian-Splatted Volumetric Fusion [61.6340987158734]
  We present the first pinhole-fisheye framework for heterogeneous multi-view depth estimation, PFDepth.<n> PFDepth employs a unified architecture capable of processing arbitrary combinations of pinhole and fisheye cameras with varied intrinsics and extrinsics.<n>We show that PFDepth sets a state-of-the-art performance on KITTI-360 and RealHet datasets over current mainstream depth networks.
  arXiv  Detail & Related papers  (2025-09-30T09:38:59Z)
• SGMAGNet: A Baseline Model for 3D Cloud Phase Structure Reconstruction on a New Passive Active Satellite Benchmark [17.3424418972935]
  We present a benchmark dataset for transforming satellite observations into detailed 3D cloud phase structures.<n>We adopt SGMAGNet as the main model and compare it with several baseline architectures.<n>The results demonstrate that SGMAGNet achieves superior performance in cloud phase reconstruction.
  arXiv  Detail & Related papers  (2025-09-19T07:29:23Z)
• VFMM3D: Releasing the Potential of Image by Vision Foundation Model for Monocular 3D Object Detection [80.62052650370416]
  monocular 3D object detection holds significant importance across various applications, including autonomous driving and robotics. In this paper, we present VFMM3D, an innovative framework that leverages the capabilities of Vision Foundation Models (VFMs) to accurately transform single-view images into LiDAR point cloud representations.
  arXiv  Detail & Related papers  (2024-04-15T03:12:12Z)
• M3DeTR: Multi-representation, Multi-scale, Mutual-relation 3D Object Detection with Transformers [78.48081972698888]
  We present M3DeTR, which combines different point cloud representations with different feature scales based on multi-scale feature pyramids. M3DeTR is the first approach that unifies multiple point cloud representations, feature scales, as well as models mutual relationships between point clouds simultaneously using transformers.
  arXiv  Detail & Related papers  (2021-04-24T06:48:23Z)
• Volumetric Propagation Network: Stereo-LiDAR Fusion for Long-Range Depth Estimation [81.08111209632501]
  We propose a geometry-aware stereo-LiDAR fusion network for long-range depth estimation. We exploit sparse and accurate point clouds as a cue for guiding correspondences of stereo images in a unified 3D volume space. Our network achieves state-of-the-art performance on the KITTI and the Virtual- KITTI datasets.
  arXiv  Detail & Related papers  (2021-03-24T03:24:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.