DB-3DME: From Dataset to Benchmark for Human-aligned Automatic 3D Mesh Evaluation

• URL: http://arxiv.org/abs/2606.10142v1
• Date: Mon, 08 Jun 2026 20:17:49 GMT
• Title: DB-3DME: From Dataset to Benchmark for Human-aligned Automatic 3D Mesh Evaluation
• Abstract summary: We introduce DB-3DME, the dataset and benchmark for 3D mesh evaluation.<n>DB-3DME contains 2,619 synthetic 3D meshes paired with human ratings on Geometry and Prompt Adherence.<n>We fine-tune an open-weight VLM, Qwen-2.5-VL-7B, for 3D mesh evaluation by adapting the visual encoder while freezing the language model.
• Score: 16.60840648468348
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in 3D generation have led to substantial improvements in realism, controllability, and efficiency, yet the evaluation of 3D assets remains underexplored. Existing evaluation paradigms, including human evaluation, learned metrics, and vision-language models (VLMs) as judges, suffer from limitations in cost, scalability, resolution handling, or task-specific alignment. In this work, we focus on 3D mesh evaluation and introduce DB-3DME, the Dataset and Benchmark for 3D Mesh Evaluation. DB-3DME contains 2,619 synthetic 3D meshes paired with human ratings on Geometry and Prompt Adherence. Using this dataset, we systematically benchmark state-of-the-art VLMs and identify visual encoding of 3D representations as a key factor for human-aligned evaluation performance. Motivated by this finding, we fine-tune an open-weight VLM, Qwen-2.5-VL-7B, for 3D mesh evaluation by adapting the visual encoder while freezing the language model. The fine-tuned model substantially outperforms existing pre-trained VLMs across multiple evaluation dimensions, establishing a new benchmark for automatic 3D mesh evaluation. We publicly release the benchmark dataset on GitHub and Hugging Face to facilitate future research.

Related papers

• 3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code [25.904997126319547]
  3DCodeBench is a systematic benchmark for evaluating vision-language model (VLM) agents for procedural 3D generation in 3D modeling software.<n>We build 3DCodeArena, a ranking platform based on pairwise human preferences over generated 3D outputs.
  arXiv  Detail & Related papers  (2026-05-31T06:59:49Z)
• Hi3DEval: Advancing 3D Generation Evaluation with Hierarchical Validity [78.7107376451476]
  Hi3DEval is a hierarchical evaluation framework tailored for 3D generative content.<n>We extend texture evaluation beyond aesthetic appearance by explicitly assessing material realism.<n>We propose a 3D-aware automated scoring system based on hybrid 3D representations.
  arXiv  Detail & Related papers  (2025-08-07T17:50:13Z)
• 3D-MOOD: Lifting 2D to 3D for Monocular Open-Set Object Detection [62.57179069154312]
  We introduce the first end-to-end 3D Monocular Open-set Object Detector (3D-MOOD)<n>We lift the open-set 2D detection into 3D space through our designed 3D bounding box head.<n>We condition the object queries with geometry prior and overcome the generalization for 3D estimation across diverse scenes.
  arXiv  Detail & Related papers  (2025-07-31T13:56:41Z)
• E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Models [78.1674905950243]
  We present the first comprehensive benchmark for 3D geometric foundation models (GFMs)<n>GFMs directly predict dense 3D representations in a single feed-forward pass, eliminating the need for slow or unavailable precomputed camera parameters.<n>We evaluate 16 state-of-the-art GFMs, revealing their strengths and limitations across tasks and domains.<n>All code, evaluation scripts, and processed data will be publicly released to accelerate research in 3D spatial intelligence.
  arXiv  Detail & Related papers  (2025-06-02T17:53:09Z)
• Eval3D: Interpretable and Fine-grained Evaluation for 3D Generation [134.53804996949287]
  We introduce Eval3D, a fine-grained, interpretable evaluation tool that can faithfully evaluate the quality of generated 3D assets.<n>Our key observation is that many desired properties of 3D generation, such as semantic and geometric consistency, can be effectively captured.<n>Compared to prior work, Eval3D provides pixel-wise measurement, enables accurate 3D spatial feedback, and aligns more closely with human judgments.
  arXiv  Detail & Related papers  (2025-04-25T17:22:05Z)
• 3DGen-Bench: Comprehensive Benchmark Suite for 3D Generative Models [94.48803082248872]
  3D generation is experiencing rapid advancements, while the development of 3D evaluation has not kept pace.<n>We develop 3DGen-Arena, an integrated platform to gather human preferences from both public users and expert annotators.<n>Using this dataset, we further train a CLIP-based scoring model, 3DGen-Score, and a MLLM-based automatic evaluator, 3DGen-Eval.
  arXiv  Detail & Related papers  (2025-03-27T17:53:00Z)
• From 2D to 3D: Re-thinking Benchmarking of Monocular Depth Prediction [80.67873933010783]
  We argue that MDP is currently witnessing benchmark over-fitting and relying on metrics that are only partially helpful to gauge the usefulness of the predictions for 3D applications. This limits the design and development of novel methods that are truly aware of - and improving towards estimating - the 3D structure of the scene rather than optimizing 2D-based distances. We propose a set of metrics well suited to evaluate the 3D geometry of MDP approaches and a novel indoor benchmark, RIO-D3D, crucial for the proposed evaluation methodology.
  arXiv  Detail & Related papers  (2022-03-15T17:50:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.