Multi-View Hierarchical Graph Neural Network for Sketch-Based 3D Shape Retrieval

• URL: http://arxiv.org/abs/2604.18019v1
• Date: Mon, 20 Apr 2026 09:46:00 GMT
• Title: Multi-View Hierarchical Graph Neural Network for Sketch-Based 3D Shape Retrieval
• Abstract summary: Sketch-based 3D shape retrieval aims to retrieve 3D shapes consistent with the category of the input hand-drawn sketch.<n>We propose Multi-View Hierarchical Graph Neural Network (MV-HGNN), a novel framework for SBSR.<n>Under both category-level and zero-shot settings, MV-HGNN outperforms state-of-the-art methods.
• Score: 8.680040031590362
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sketch-based 3D shape retrieval (SBSR) aims to retrieve 3D shapes that are consistent with the category of the input hand-drawn sketch. The core challenge of this task lies in two aspects: existing methods typically employ simplified aggregation strategies for independently encoded 3D multi-view features, which ignore the geometric relationships between views and multi-level details, resulting in weak 3D representation. Simultaneously, traditional SBSR methods are constrained by visible category limitations, leading to poor performance in zero-shot scenarios. To address these challenges, we propose Multi-View Hierarchical Graph Neural Network (MV-HGNN), a novel framework for SBSR. Specifically, we construct a view-level graph and capture adjacent geometric dependencies and cross-view message passing via local graph convolution and global attention. A view selector is further introduced to perform hierarchical graph coarsening, enabling a progressively larger receptive field for graph convolution and mitigating the interference of redundant views, which leads to more discriminate discriminative hierarchical 3D representation. To enable category agnostic alignment and mitigate overfitting to seen classes, we leverage CLIP text embeddings as semantic prototypes and project both sketch and 3D features into a shared semantic space. We use a two-stage training strategy for category-level retrieval and a one-stage strategy for zero-shot retrieval under the same model architecture. Under both category-level and zero-shot settings, extensive experiments on two public benchmarks demonstrate that MV-HGNN outperforms state-of-the-art methods.

Related papers

• Hierarchical and Holistic Open-Vocabulary Functional 3D Scene Graphs for Indoor Spaces [116.57196064763924]
  We extend the benchmark coverage by introducing dense tabletop objects and explicit multi-level functional relationships.<n>This work introduces critical challenges involving small-scale, dense, and similar instances.<n>We propose an open-vocabulary pipeline based on 2D visual grounding and 3D graph optimization.
  arXiv  Detail & Related papers  (2026-05-15T09:14:50Z)
• Proxy3D: Efficient 3D Representations for Vision-Language Models via Semantic Clustering and Alignment [50.02189698630855]
  We propose a Proxy3D method with compact yet comprehensive 3D proxy representations for the vision modality.<n>Our method achieves competitive or state-of-the-art performance in 3D visual question answering, visual grounding and general spatial intelligence benchmarks.
  arXiv  Detail & Related papers  (2026-05-08T17:50:47Z)
• 3D-LENS: A 3D Lifting-based Elevated Novel-view Synthesis method for Single-View Aerial-Ground Re-Identification [1.621606615628714]
  Aerial-Ground Re-Identification (AG-ReID) is constrained by the viewpoint-domain gap.<n>We are first to address this challenge by formalizing the Single-View AG-ReID setting.<n>We propose 3D Lifting-based Elevated Novel-view Synthesis (3D-LENS)
  arXiv  Detail & Related papers  (2026-04-29T10:38:48Z)
• Cross-Attentive Multiview Fusion of Vision-Language Embeddings [28.984855441235776]
  We introduce a novel multiview transformer architecture that cross-attends across vision-language descriptors from multiple viewpoints.<n>We leverage multiview consistency as a self-supervision signal for this fusion.<n>Our Cross-Attentive Multiview Fusion consistently outperforms naive averaging or single-view descriptor selection.
  arXiv  Detail & Related papers  (2026-04-14T10:25:32Z)
• Pre-training Graph Neural Networks on 2D and 3D Molecular Structures by using Multi-View Conditional Information Bottleneck [8.42839603549236]
  We propose a Multi-View Conditional Information Bottleneck framework for pre-training graph neural networks on 2D and 3D molecular structures.<n>Our idea is to discover the shared information while minimizing irrelevant features from each view under the MVCIB principle.<n>To enhance semantic and structural consistency across views, we utilize key substructures, e.g., functional groups and ego-networks, as anchors between the two views.
  arXiv  Detail & Related papers  (2025-11-23T11:18:35Z)
• SAI3D: Segment Any Instance in 3D Scenes [68.57002591841034]
  We introduce SAI3D, a novel zero-shot 3D instance segmentation approach. Our method partitions a 3D scene into geometric primitives, which are then progressively merged into 3D instance segmentations. Empirical evaluations on ScanNet, Matterport3D and the more challenging ScanNet++ datasets demonstrate the superiority of our approach.
  arXiv  Detail & Related papers  (2023-12-17T09:05:47Z)
• Beyond First Impressions: Integrating Joint Multi-modal Cues for Comprehensive 3D Representation [72.94143731623117]
  Existing methods simply align 3D representations with single-view 2D images and coarse-grained parent category text. Insufficient synergy neglects the idea that a robust 3D representation should align with the joint vision-language space. We propose a multi-view joint modality modeling approach, termed JM3D, to obtain a unified representation for point cloud, text, and image.
  arXiv  Detail & Related papers  (2023-08-06T01:11:40Z)
• Single-view 3D Mesh Reconstruction for Seen and Unseen Categories [69.29406107513621]
  Single-view 3D Mesh Reconstruction is a fundamental computer vision task that aims at recovering 3D shapes from single-view RGB images. This paper tackles Single-view 3D Mesh Reconstruction, to study the model generalization on unseen categories. We propose an end-to-end two-stage network, GenMesh, to break the category boundaries in reconstruction.
  arXiv  Detail & Related papers  (2022-08-04T14:13:35Z)
• Graph-Based 3D Multi-Person Pose Estimation Using Multi-View Images [79.70127290464514]
  We decompose the task into two stages, i.e. person localization and pose estimation. And we propose three task-specific graph neural networks for effective message passing. Our approach achieves state-of-the-art performance on CMU Panoptic and Shelf datasets.
  arXiv  Detail & Related papers  (2021-09-13T11:44:07Z)
• Fine-Grained 3D Shape Classification with Hierarchical Part-View Attentions [70.0171362989609]
  We propose a novel fine-grained 3D shape classification method named FG3D-Net to capture the fine-grained local details of 3D shapes from multiple rendered views. Our results under the fine-grained 3D shape dataset show that our method outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-05-26T06:53:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.