ESMStereo: Enhanced ShuffleMixer Disparity Upsampling for Real-Time and Accurate Stereo Matching

• URL: http://arxiv.org/abs/2506.21091v1
• Date: Thu, 26 Jun 2025 08:34:51 GMT
• Title: ESMStereo: Enhanced ShuffleMixer Disparity Upsampling for Real-Time and Accurate Stereo Matching
• Abstract summary: We propose the Enhanced Shuffle Mixer (ESM) to mitigate information loss associated with small-scale cost volumes.<n>ESM restores critical details by integrating primary features into the disparity upsampling unit.<n> compact version of ESMStereo achieves an inference speed of 116 FPS on high-end GPUs and 91 FPS on the AGX Orin.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Stereo matching has become an increasingly important component of modern autonomous systems. Developing deep learning-based stereo matching models that deliver high accuracy while operating in real-time continues to be a major challenge in computer vision. In the domain of cost-volume-based stereo matching, accurate disparity estimation depends heavily on large-scale cost volumes. However, such large volumes store substantial redundant information and also require computationally intensive aggregation units for processing and regression, making real-time performance unattainable. Conversely, small-scale cost volumes followed by lightweight aggregation units provide a promising route for real-time performance, but lack sufficient information to ensure highly accurate disparity estimation. To address this challenge, we propose the Enhanced Shuffle Mixer (ESM) to mitigate information loss associated with small-scale cost volumes. ESM restores critical details by integrating primary features into the disparity upsampling unit. It quickly extracts features from the initial disparity estimation and fuses them with image features. These features are mixed by shuffling and layer splitting then refined through a compact feature-guided hourglass network to recover more detailed scene geometry. The ESM focuses on local contextual connectivity with a large receptive field and low computational cost, leading to the reconstruction of a highly accurate disparity map at real-time. The compact version of ESMStereo achieves an inference speed of 116 FPS on high-end GPUs and 91 FPS on the AGX Orin.

Related papers

• LiteMatch: Lightweight Zero-Shot Stereo Matching via Cost Volume Stabilization [21.209111801863546]
  We present textitLiteMatch, a lightweight stereo matching framework that achieves strong zero-shot generalization without expensive 3D convolutions.<n>LiteMatch employs two complementary encoders: a Cross-View Correspondence (CVCE) to capture global cross-view interactions, and a High-Frequency Driving (HFE) that enhances fine structural details via FFT-based frequency cues.<n>With a flexible design ranging from 3.36M to 9.58M parameters, LiteMatch achieves exceptional zero-shot generalization, delivering competitive EPE and D1 performance across Scene Flow, KITTI, Middlebury, ETH3D
  arXiv  Detail & Related papers  (2026-06-30T13:20:08Z)
• Timestep Rescheduling in Diffusion Inversion [79.30105376216544]
  We show how the selection of the diffusion timestep in the noise scheduler influences inversion fidelity.<n>We propose a simple yet effective nonuniform timestep scheduler that integrates a global rescaling with a local dynamic programming based rescheduling.<n>Our method serves as an off-the-shelf enhancement for existing inversion techniques and requires no extra parameters or computational overhead.
  arXiv  Detail & Related papers  (2026-06-13T16:32:01Z)
• MAFNet:Multi-frequency Adaptive Fusion Network for Real-time Stereo Matching [2.0202997482095593]
  We propose a Multi-frequency Adaptive Fusion Network (MAFNet) to produce high-quality disparity maps using only efficient 2D convolutions.<n>We show that the proposed MAFNet significantly outperforms existing real-time methods on public datasets such as Scene Flow and KITTI 2015.
  arXiv  Detail & Related papers  (2025-12-04T01:08:38Z)
• PPMStereo: Pick-and-Play Memory Construction for Consistent Dynamic Stereo Matching [51.98089287914147]
  textbfPick-and-textbflay textbfMemory (PM) construction module for dynamic bfStereo matching, dubbed as bftextPPMStereo.<n>Inspired by the two-stage decision-making process in humans, we propose a textbfPick-and-textbflay textbfMemory (PM) construction module for dynamic bfStereo matching, dubbed as bftextPPMStereo.
  arXiv  Detail & Related papers  (2025-10-23T03:52:39Z)
• Any Image Restoration via Efficient Spatial-Frequency Degradation Adaptation [158.37640586809187]
  Restoring any degraded image efficiently via just one model has become increasingly significant.<n>Our approach, termed AnyIR, takes a unified path that leverages inherent similarity across various degradations.<n>To fuse the degradation awareness and the contextualized attention, a spatial-frequency parallel fusion strategy is proposed.
  arXiv  Detail & Related papers  (2025-04-19T09:54:46Z)
• Temporal Feature Matters: A Framework for Diffusion Model Quantization [105.3033493564844]
  Diffusion models rely on the time-step for the multi-round denoising.<n>We introduce a novel quantization framework that includes three strategies.<n>This framework preserves most of the temporal information and ensures high-quality end-to-end generation.
  arXiv  Detail & Related papers  (2024-07-28T17:46:15Z)
• TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models [52.454274602380124]
  Diffusion models heavily depend on the time-step $t$ to achieve satisfactory multi-round denoising. We propose a Temporal Feature Maintenance Quantization (TFMQ) framework building upon a Temporal Information Block. Powered by the pioneering block design, we devise temporal information aware reconstruction (TIAR) and finite set calibration (FSC) to align the full-precision temporal features.
  arXiv  Detail & Related papers  (2023-11-27T12:59:52Z)
• ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution [76.7408734079706]
  Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation. We propose ESSAformer, an ESSA attention-embedded Transformer network for single-HSI-SR with an iterative refining structure.
  arXiv  Detail & Related papers  (2023-07-26T07:45:14Z)
• ACVNet: Attention Concatenation Volume for Accurate and Efficient Stereo Matching [7.39503547452922]
  We present a novel cost volume construction method which generates attention weights from correlation clues to suppress redundant information. To generate reliable attention weights, we propose multi-level adaptive patch matching to improve the distinctiveness of the matching cost. The proposed cost volume is named attention concatenation volume (ACV) which can be seamlessly embedded into most stereo matching networks.
  arXiv  Detail & Related papers  (2022-03-04T06:28:58Z)
• Coarse-to-Fine Embedded PatchMatch and Multi-Scale Dynamic Aggregation for Reference-based Super-Resolution [48.093500219958834]
  We propose an Accelerated Multi-Scale Aggregation network (AMSA) for Reference-based Super-Resolution. The proposed AMSA achieves superior performance over state-of-the-art approaches on both quantitative and qualitative evaluations.
  arXiv  Detail & Related papers  (2022-01-12T08:40:23Z)
• Multi-View Stereo Network with attention thin volume [0.0]
  We propose an efficient multi-view stereo (MVS) network for infering depth value from multiple RGB images. We introduce the self-attention mechanism to fully aggregate the dominant information from input images. We also introduce the group-wise correlation to feature aggregation, which greatly reduces the memory and calculation burden.
  arXiv  Detail & Related papers  (2021-10-16T11:51:23Z)
• Correlate-and-Excite: Real-Time Stereo Matching via Guided Cost Volume Excitation [65.83008812026635]
  We construct Guided Cost volume Excitation (GCE) and show that simple channel excitation of cost volume guided by image can improve performance considerably. We present an end-to-end network that we call Correlate-and-Excite (CoEx)
  arXiv  Detail & Related papers  (2021-08-12T14:32:26Z)
• A Decomposition Model for Stereo Matching [43.04003114948216]
  We present a decomposition model for stereo matching to solve the problem of excessive growth in computational cost as the resolution increases. Our model only runs dense matching at a very low resolution and uses sparse matching at different higher resolutions to recover the disparity of lost details scale-by-scale.
  arXiv  Detail & Related papers  (2021-04-15T15:16:23Z)
• CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching [27.313740022587442]
  We propose CFNet, a Cascade and Fused cost volume based network to improve the robustness of the stereo matching network. We employ a variance-based uncertainty estimation to adaptively adjust the next stage disparity search space. Our proposed method achieves the state-of-the-art overall performance and obtains the 1st place on the stereo task of Robust Vision Challenge 2020.
  arXiv  Detail & Related papers  (2021-04-09T11:38:59Z)
• SMD-Nets: Stereo Mixture Density Networks [68.56947049719936]
  We propose Stereo Mixture Density Networks (SMD-Nets), a simple yet effective learning framework compatible with a wide class of 2D and 3D architectures. Specifically, we exploit bimodal mixture densities as output representation and show that this allows for sharp and precise disparity estimates near discontinuities. We carry out comprehensive experiments on a new high-resolution and highly realistic synthetic stereo dataset, consisting of stereo pairs at 8Mpx resolution, as well as on real-world stereo datasets.
  arXiv  Detail & Related papers  (2021-04-08T16:15:46Z)
• Learning Inverse Depth Regression for Multi-View Stereo with Correlation Cost Volume [32.41293572426403]
  Deep learning has shown to be effective for depth inference in multi-view stereo (MVS) However, the scalability and accuracy still remain an open problem in this domain. Inspired by the group-wise correlation in stereo matching, we propose an average group-wise correlation similarity measure to construct a lightweight cost volume.
  arXiv  Detail & Related papers  (2019-12-26T01:40:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.