Lotus-2: Advancing Geometric Dense Prediction with Powerful Image Generative Model

• URL: http://arxiv.org/abs/2512.01030v2
• Date: Thu, 04 Dec 2025 20:59:28 GMT
• Title: Lotus-2: Advancing Geometric Dense Prediction with Powerful Image Generative Model
• Authors: Jing He, Haodong Li, Mingzhi Sheng, Ying-Cong Chen,
• Abstract summary: We propose a two-stage deterministic framework for stable, accurate and fine-grained geometric dense prediction.<n>Specifically, in the first stage, the core predictor employs a single-step deterministic formulation with a clean-data objective.<n>In the second stage, the detail sharpener performs a constrained multi-step rectified-flow refinement within the manifold defined by the core predictor.
• Score: 32.831576387973875
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recovering pixel-wise geometric properties from a single image is fundamentally ill-posed due to appearance ambiguity and non-injective mappings between 2D observations and 3D structures. While discriminative regression models achieve strong performance through large-scale supervision, their success is bounded by the scale, quality and diversity of available data and limited physical reasoning. Recent diffusion models exhibit powerful world priors that encode geometry and semantics learned from massive image-text data, yet directly reusing their stochastic generative formulation is suboptimal for deterministic geometric inference: the former is optimized for diverse and high-fidelity image generation, whereas the latter requires stable and accurate predictions. In this work, we propose Lotus-2, a two-stage deterministic framework for stable, accurate and fine-grained geometric dense prediction, aiming to provide an optimal adaption protocol to fully exploit the pre-trained generative priors. Specifically, in the first stage, the core predictor employs a single-step deterministic formulation with a clean-data objective and a lightweight local continuity module (LCM) to generate globally coherent structures without grid artifacts. In the second stage, the detail sharpener performs a constrained multi-step rectified-flow refinement within the manifold defined by the core predictor, enhancing fine-grained geometry through noise-free deterministic flow matching. Using only 59K training samples, less than 1% of existing large-scale datasets, Lotus-2 establishes new state-of-the-art results in monocular depth estimation and highly competitive surface normal prediction. These results demonstrate that diffusion models can serve as deterministic world priors, enabling high-quality geometric reasoning beyond traditional discriminative and generative paradigms.

Related papers

• VFMF: World Modeling by Forecasting Vision Foundation Model Features [67.09340259579761]
  We introduce a generative forecaster that performs autoregressive flow matching in vision foundation models feature space.<n>We show that this latent information more effectively than previously used PCA-based alternatives, both for forecasting and other applications.<n>With matched architecture and compute, our method produces sharper and more accurate predictions than regression across all modalities.
  arXiv  Detail & Related papers  (2025-12-12T02:10:05Z)
• Generative Human Geometry Distribution [49.58025398670139]
  We build upon Geometry distributions, a recently proposed representation that can model a single human geometry with high fidelity.<n>We propose a new geometry distribution model by two key techniques: encoding distributions as 2D feature maps rather than network parameters, and using SMPL models as the domain instead of Gaussian.<n> Experimental results demonstrate that our method outperforms existing state-of-the-art methods, achieving a 57% improvement in geometry quality.
  arXiv  Detail & Related papers  (2025-03-03T11:55:19Z)
• Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithms [31.42317398879432]
  Current inference approaches mainly fall into two categories: exact simulation and approximate methods such as $tau$-leaping.<n>In this work, we advance the latter category by tailoring the first extension of high-order numerical inference schemes to discrete diffusion models.<n>We rigorously analyze the proposed schemes and establish the second-order accuracy of the $theta$-trapezoidal method in KL divergence.
  arXiv  Detail & Related papers  (2025-02-01T00:25:21Z)
• GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image [94.56927147492738]
  We introduce GeoWizard, a new generative foundation model designed for estimating geometric attributes from single images. We show that leveraging diffusion priors can markedly improve generalization, detail preservation, and efficiency in resource usage. We propose a simple yet effective strategy to segregate the complex data distribution of various scenes into distinct sub-distributions.
  arXiv  Detail & Related papers  (2024-03-18T17:50:41Z)
• Latent Semantic Consensus For Deterministic Geometric Model Fitting [109.44565542031384]
  We propose an effective method called Latent Semantic Consensus (LSC) LSC formulates the model fitting problem into two latent semantic spaces based on data points and model hypotheses. LSC is able to provide consistent and reliable solutions within only a few milliseconds for general multi-structural model fitting.
  arXiv  Detail & Related papers  (2024-03-11T05:35:38Z)
• Exploiting Diffusion Prior for Generalizable Dense Prediction [85.4563592053464]
  Recent advanced Text-to-Image (T2I) diffusion models are sometimes too imaginative for existing off-the-shelf dense predictors to estimate. We introduce DMP, a pipeline utilizing pre-trained T2I models as a prior for dense prediction tasks. Despite limited-domain training data, the approach yields faithful estimations for arbitrary images, surpassing existing state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2023-11-30T18:59:44Z)
• RGM: A Robust Generalizable Matching Model [49.60975442871967]
  We propose a deep model for sparse and dense matching, termed RGM (Robust Generalist Matching) To narrow the gap between synthetic training samples and real-world scenarios, we build a new, large-scale dataset with sparse correspondence ground truth. We are able to mix up various dense and sparse matching datasets, significantly improving the training diversity.
  arXiv  Detail & Related papers  (2023-10-18T07:30:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.