ELSA3D: Elastic Semantic Anchoring for Unified 3D Understanding and Generation

• URL: http://arxiv.org/abs/2607.06565v1
• Date: Tue, 07 Jul 2026 17:59:50 GMT
• Title: ELSA3D: Elastic Semantic Anchoring for Unified 3D Understanding and Generation
• Abstract summary: Unified 3D foundation models aspire to generate 3D assets and reason about them in language within a single backbone.<n>We introduce ELSA3D, a unified 3D model that addresses this with elastic semantic anchoring, structuring language and geometric reasoning jointly along matched abstraction scales.<n>A lightweight per-block router makes both computation and reasoning elastic, choosing which text tokens instantiate anchors at which geometric scale so that cross-modal capacity concentrates where alignment is most needed.
• Score: 6.660269100200303
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Unified 3D foundation models aspire to generate 3D assets and reason about them in language within a single backbone, but their text-3D interaction remains largely implicit. Existing methods concatenate text and 3D tokens into a flat sequence and rely on self-attention, collapsing coarse structural cues and fine geometric details into one undifferentiated representation. We introduce ELSA3D, a unified 3D model that addresses this with elastic semantic anchoring, structuring language and geometric reasoning jointly along matched abstraction scales. ELSA3D represents geometry with a scale-aware octree tokenizer and introduces Anchor Tokens, sparse cross-modal units that select semantic cues, route them to the most relevant 3D scale, retrieve scale-specific geometric evidence, and write the fused signal back into the unified representation, keeping interaction sparse yet precise. A lightweight per-block router makes both computation and reasoning elastic, choosing which text tokens instantiate anchors at which geometric scale so that cross-modal capacity concentrates where alignment is most needed. ELSA3D achieves state-of-the-art performance across image-to-3D generation, text-to-3D generation, and 3D captioning, outperforming the strongest unified baseline while roughly halving FLOPs and inference latency relative to the non-elastic version of the same model.

Related papers

• P3D-Bench: Benchmarking MLLMs for Parametric 3D Generation and Structural Reasoning [37.03209423305997]
  We introduce P3D-Bench, a benchmark for parametric 3D generation.<n>Under a unified protocol, P3D-Bench covers three task families (Text-to-3D, Image-to-3D and Assembly-3D)<n>We evaluate frontier MLLMs and text-only LLMs on 400 text cases, 400 image cases and 203 annotated assemblies.
  arXiv  Detail & Related papers  (2026-06-09T17:36:34Z)
• Omni123: Exploring 3D Native Foundation Models with Limited 3D Data by Unifying Text to 2D and 3D Generation [15.607138815246387]
  We present Omni123, a 3D-native foundation model that unifies text-to-2D and text-to-3D generation within a single autoregressive framework.<n> Experiments show that Omni123 significantly improves text-guided 3D generation and editing, demonstrating a scalable path toward multimodal 3D world models.
  arXiv  Detail & Related papers  (2026-04-02T17:29:38Z)
• LoST: Level of Semantics Tokenization for 3D Shapes [50.847769883816085]
  State-of-the-art methods rely on geometric level-of-detail (LoD) hierarchies, originally designed for rendering and compression.<n>We propose Level-of-Semantics Tokenization (LoST), which orders salience, such that early prefixes decode complete, plausible shapes.<n>LoST achieves SOTA reconstruction, surpassing previous LoD-based 3D shape tokenizers by large margins on both geometric and semantic reconstruction metrics.
  arXiv  Detail & Related papers  (2026-03-18T17:56:06Z)
• IGGT: Instance-Grounded Geometry Transformer for Semantic 3D Reconstruction [82.53307702809606]
  Humans naturally perceive the geometric structure and semantic content of a 3D world as intertwined dimensions.<n>We propose InstanceGrounded Geometry Transformer (IGGT) to unify the knowledge for both spatial reconstruction and instance-level contextual understanding.
  arXiv  Detail & Related papers  (2025-10-26T14:57:44Z)
• Seeing 3D Through 2D Lenses: 3D Few-Shot Class-Incremental Learning via Cross-Modal Geometric Rectification [59.17489431187807]
  We propose a framework that enhances 3D geometric fidelity by leveraging CLIP's hierarchical spatial semantics.<n>Our method significantly improves 3D few-shot class-incremental learning, achieving superior geometric coherence and robustness to texture bias.
  arXiv  Detail & Related papers  (2025-09-18T13:45:08Z)
• Point Linguist Model: Segment Any Object via Bridged Large 3D-Language Model [51.02616473941499]
  3D object segmentation with Large Language Models (LLMs) has become a prevailing paradigm due to its broad semantics, task flexibility, and strong generalization.<n>However, this paradigm is hindered by representation misalignment: LLMs process high-level semantic tokens, whereas 3D point clouds convey only dense geometric structures.<n>We present the Point Linguist Model (PLM), a general framework that bridges the representation gap between LLMs and dense 3D point clouds.
  arXiv  Detail & Related papers  (2025-09-09T15:01:28Z)
• Cues3D: Unleashing the Power of Sole NeRF for Consistent and Unique Instances in Open-Vocabulary 3D Panoptic Segmentation [48.231573110948]
  Open-vocabulary 3D panoptic segmentation has recently emerged as a significant trend.<n>We present Cues3D, a compact approach that relies solely on Neural Radiance Field (NeRF) instead of pre-associations.<n>Our experiments are conducted on ScanNet v2, ScanNet200, ScanNet++, and Replica datasets for 3D instance, panoptic, and semantic segmentation tasks.
  arXiv  Detail & Related papers  (2025-05-01T08:12:03Z)
• SCA3D: Enhancing Cross-modal 3D Retrieval via 3D Shape and Caption Paired Data Augmentation [21.070154402838906]
  Cross-modal 3D retrieval aims to achieve mutual matching between text descriptions and 3D shapes.<n>The scarcity and expensiveness of 3D data constrain the performance of existing cross-modal 3D retrieval methods.<n>We introduce SCA3D, a novel 3D shape and caption online data augmentation method for cross-modal 3D retrieval.
  arXiv  Detail & Related papers  (2025-02-26T13:36:40Z)
• Shape from Semantics: 3D Shape Generation from Multi-View Semantics [30.969299308083723]
  Existing 3D reconstruction methods utilize guidances such as 2D images, 3D point clouds, shape contours and single semantics to recover the 3D surface.<n>We propose a novel 3D modeling task called Shape from Semantics'', which aims to create 3D models whose geometry and appearance are consistent with the given text semantics when viewed from different views.
  arXiv  Detail & Related papers  (2025-02-01T07:51:59Z)
• A3D: Does Diffusion Dream about 3D Alignment? [73.97853402817405]
  We tackle the problem of text-driven 3D generation from a geometry alignment perspective.<n>Given a set of text prompts, we aim to generate a collection of objects with semantically corresponding parts aligned across them.<n>We propose to embed these objects into a common latent space and optimize the continuous transitions between these objects.
  arXiv  Detail & Related papers  (2024-06-21T09:49:34Z)
• DIRECT-3D: Learning Direct Text-to-3D Generation on Massive Noisy 3D Data [50.164670363633704]
  We present DIRECT-3D, a diffusion-based 3D generative model for creating high-quality 3D assets from text prompts. Our model is directly trained on extensive noisy and unaligned in-the-wild' 3D assets. We achieve state-of-the-art performance in both single-class generation and text-to-3D generation.
  arXiv  Detail & Related papers  (2024-06-06T17:58:15Z)
• ID-to-3D: Expressive ID-guided 3D Heads via Score Distillation Sampling [96.87575334960258]
  ID-to-3D is a method to generate identity- and text-guided 3D human heads with disentangled expressions. Results achieve an unprecedented level of identity-consistent and high-quality texture and geometry generation.
  arXiv  Detail & Related papers  (2024-05-26T13:36:45Z)
• Point2Seq: Detecting 3D Objects as Sequences [58.63662049729309]
  We present a simple and effective framework, named Point2Seq, for 3D object detection from point clouds. We view each 3D object as a sequence of words and reformulate the 3D object detection task as decoding words from 3D scenes in an auto-regressive manner.
  arXiv  Detail & Related papers  (2022-03-25T00:20:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.