MHE-Former: Multi-Hypothesis Transformers via Entropy Maximization for 3D Mesh Recovery

• URL: http://arxiv.org/abs/2609.10743v1
• Date: Wed, 09 Sep 2026 18:39:03 GMT
• Title: MHE-Former: Multi-Hypothesis Transformers via Entropy Maximization for 3D Mesh Recovery
• Abstract summary: We introduce an exploration--exploitation paradigm for ambiguous mesh recovery with multi-hypothesis learning and selection.<n>During exploitation, we propose Hypothesis Selection, a context-aware process for multiple predictions.<n>Our framework achieves state-of-the-art performance in accuracy and diversity across multiple datasets.
• Score: 80.22216005340283
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Monocular 3D hand and body mesh recovery often suffers from severe occlusion and ambiguity. Traditional deterministic methods typically regress a single optimal solution, leading to overconfident predictions. In this paper, we introduce an exploration--exploitation paradigm for ambiguous mesh recovery with multi-hypothesis learning and selection. Specifically, during exploration, based on our probabilistic formulation and entropy maximization, we propose a novel multi-hypothesis method referred to as MHE-Former. It is a Transformer-based multi-hypothesis framework, ensuring high training efficiency and label friendliness while generating plausible and diverse hypotheses. During exploitation, we propose Hypothesis Selection, a context-aware process for multiple predictions. Especially leveraging VLM's powerful visual understanding and reasoning capabilities, it allows users to choose the most plausible and desired estimate with additional evidence and natural language intent. Extensive experiments demonstrate that our framework achieves state-of-the-art performance in accuracy and diversity across multiple datasets. The user preference study further shows the practicality of our hypothesis selection process.

Related papers

• DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning [5.070905256010651]
  We present DLWM, a multimodal reasoning framework that combines latent-space reasoning with reinforcement learning.<n>First, we construct a set of diverse latent world hypotheses in continuous latent space, each capturing a different plausible interpretation of the visual input.<n>Second, we formulate the latent reasoning process as a resource-constrained sequential decision problem.
  arXiv  Detail & Related papers  (2026-06-13T07:12:46Z)
• Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Models [53.15468578562038]
  We propose an Uncertainty-aware Exploratory Direct Preference Optimization (UE-DPO) method for Multimodal Large Language Models (MLLMs)<n>We first quantify the uncertainty from the model's failure to ground token predictions in the given image.<n>We then encourage more learning pressure on visually deficient tokens in preferred samples, and alleviate the over-penalization of beneficial knowledge in dispreferred samples.
  arXiv  Detail & Related papers  (2026-05-06T13:08:12Z)
• Preferential Normalizing Flows [9.073645394501082]
  Eliciting a high-dimensional probability distribution from an expert via noisy judgments is notoriously challenging, yet useful for many applications. We introduce a method for eliciting the expert's belief density as a normalizing flow based solely on preferential questions. We show that the belief density can be inferred as the function-space maximum a posteriori estimate.
  arXiv  Detail & Related papers  (2024-10-11T10:53:38Z)
• Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling [53.7117640028211]
  We present a geometry-constrained probabilistic modeling treatment to resolve the identified issues. We incorporate a suite of critical geometric properties to impose proper constraints on the layout of constructed embedding space. A spectral graph-theoretic method is devised to estimate the number of potential novel classes.
  arXiv  Detail & Related papers  (2024-03-02T00:56:05Z)
• Prototype-based Aleatoric Uncertainty Quantification for Cross-modal Retrieval [139.21955930418815]
  Cross-modal Retrieval methods build similarity relations between vision and language modalities by jointly learning a common representation space. However, the predictions are often unreliable due to the Aleatoric uncertainty, which is induced by low-quality data, e.g., corrupt images, fast-paced videos, and non-detailed texts. We propose a novel Prototype-based Aleatoric Uncertainty Quantification (PAU) framework to provide trustworthy predictions by quantifying the uncertainty arisen from the inherent data ambiguity.
  arXiv  Detail & Related papers  (2023-09-29T09:41:19Z)
• Bayesian Hierarchical Models for Counterfactual Estimation [12.159830463756341]
  We propose a probabilistic paradigm to estimate a diverse set of counterfactuals. We treat the perturbations as random variables endowed with prior distribution functions. A gradient based sampler with superior convergence characteristics efficiently computes the posterior samples.
  arXiv  Detail & Related papers  (2023-01-21T00:21:11Z)
• Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions [91.63716984911278]
  We introduce a novel Mixture of Normal-Inverse Gamma distributions (MoNIG) algorithm, which efficiently estimates uncertainty in principle for adaptive integration of different modalities and produces a trustworthy regression result. Experimental results on both synthetic and different real-world data demonstrate the effectiveness and trustworthiness of our method on various multimodal regression tasks.
  arXiv  Detail & Related papers  (2021-11-11T14:28:12Z)
• Probabilistic Modeling of Semantic Ambiguity for Scene Graph Generation [38.30703975408238]
  We argue that visual relationships are often semantically ambiguous. The ambiguity naturally leads to the issue of emphimplicit multi-label, motivating the need for diverse predictions. In this work, we propose a novel plug-and-play Probabilistic Uncertainty Modeling (PUM) module.
  arXiv  Detail & Related papers  (2021-03-09T07:36:09Z)
• Ambiguity in Sequential Data: Predicting Uncertain Futures with Recurrent Models [110.82452096672182]
  We propose an extension of the Multiple Hypothesis Prediction (MHP) model to handle ambiguous predictions with sequential data. We also introduce a novel metric for ambiguous problems, which is better suited to account for uncertainties.
  arXiv  Detail & Related papers  (2020-03-10T09:15:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.