IOSVLM: A 3D Vision-Language Model for Unified Dental Diagnosis from Intraoral Scans

• URL: http://arxiv.org/abs/2603.16781v1
• Date: Tue, 17 Mar 2026 16:57:02 GMT
• Title: IOSVLM: A 3D Vision-Language Model for Unified Dental Diagnosis from Intraoral Scans
• Abstract summary: 3D intraoral scans (IOS) are increasingly adopted in routine dentistry due to abundant geometric evidence.<n>We present IOSVLM, an end-to-end 3D VLM that represents scans as point clouds.
• Score: 29.9935636058314
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D intraoral scans (IOS) are increasingly adopted in routine dentistry due to abundant geometric evidence, and unified multi-disease diagnosis is desirable for clinical documentation and communication. While recent works introduce dental vision-language models (VLMs) to enable unified diagnosis and report generation on 2D images or multi-view images rendered from IOS, they do not fully leverage native 3D geometry. Such work is necessary and also challenging, due to: (i) heterogeneous scan forms and the complex IOS topology, (ii) multi-disease co-occurrence with class imbalance and fine-grained morphological ambiguity, (iii) limited paired 3D IOS-text data. Thus, we present IOSVLM, an end-to-end 3D VLM that represents scans as point clouds and follows a 3D encoder-projector-LLM design for unified diagnosis and generative visual question-answering (VQA), together with IOSVQA, a large-scale multi-source IOS diagnosis VQA dataset comprising 19,002 cases and 249,055 VQA pairs over 23 oral diseases and heterogeneous scan types. To address the distribution gap between color-free IOS data and color-dependent 3D pre-training, we propose a geometry-to-chromatic proxy that stabilizes fine-grained geometric perception and cross-modal alignment. A two-stage curriculum training strategy further enhances robustness. IOSVLM consistently outperforms strong baselines, achieving gains of at least +9.58% macro accuracy and +1.46% macro F1, indicating the effectiveness of direct 3D geometry modeling for IOS-based diagnosis.

Related papers

• Mr3D-VL: A generalist vision language foundation model for Multiparametric 3D Magnetic Resonance Imaging [3.8658612593287525]
  Mr3D-VL is a visual-language foundation model for multi-parametric 3D MRI.<n>It employs an unsupervised pre-trained shared 3D encoder and 4D rotational positional embedding for dual modality-spatial integration.
  arXiv  Detail & Related papers  (2026-08-13T01:12:34Z)
• Adapting 2D Multi-Modal Large Language Model for 3D CT Image Analysis [52.32262701215441]
  multimodal large language models (MLLMs) have exhibited robust perceptual capacity, strong cross-modal alignment, and promising generalizability.<n>Due to the scarcity of 3D medical images, existing 3D medical MLLMs suffer from insufficiently pretrained vision encoder and inability to extract customized image features.<n>We propose a Text-Guided Hierarchical MoE framework, which can distinguish tasks under the guidance of the text prompt.
  arXiv  Detail & Related papers  (2026-04-11T14:36:05Z)
• MedPruner: Training-Free Hierarchical Token Pruning for Efficient 3D Medical Image Understanding in Vision-Language Models [59.180043227905294]
  We propose MedPruner, a training-free and model-agnostic hierarchical token pruning framework for efficient 3D medical image understanding.<n>We show that MedPruner enables models such as MedGemma to maintain or even exceed their original performance while retaining fewer than 5% of visual tokens.
  arXiv  Detail & Related papers  (2026-03-12T07:37:00Z)
• Multimodal Visual Surrogate Compression for Alzheimer's Disease Classification [69.87877580725768]
  Multimodal Visual Surrogate Compression (MVSC) learns to compress and adapt large 3D sMRI volumes into compact 2D features.<n>MVSC has two key components: a Volume Context that captures global cross-slice context under textual guidance, and an Adaptive Slice Fusion module that aggregates slice-level information in a text-enhanced, patch-wise manner.
  arXiv  Detail & Related papers  (2026-01-29T13:05:46Z)
• 3D-RAD: A Comprehensive 3D Radiology Med-VQA Dataset with Multi-Temporal Analysis and Diverse Diagnostic Tasks [22.761853189757772]
  Medical Visual Question Answering (Med-VQA) holds significant potential for clinical decision support.<n>This paper presents 3D-RAD, a large-scale dataset designed to advance 3D Med-VQA using radiology CT scans.
  arXiv  Detail & Related papers  (2025-06-11T09:55:42Z)
• Med3DVLM: An Efficient Vision-Language Model for 3D Medical Image Analysis [6.464464511743737]
  Vision-language models (VLMs) have shown promise in 2D medical image analysis, but extending them to 3D remains challenging.<n>We present Med3DVLM, a 3D VLM designed to address these challenges through three key innovations.<n>We evaluate our model on the M3D dataset, which includes radiology reports and VQA data for 120,084 3D medical images.
  arXiv  Detail & Related papers  (2025-03-25T20:09:30Z)
• E3D-GPT: Enhanced 3D Visual Foundation for Medical Vision-Language Model [23.56751925900571]
  The development of 3D medical vision-language models holds significant potential for disease diagnosis and patient treatment. We utilize self-supervised learning to construct a 3D visual foundation model for extracting 3D visual features. We apply 3D spatial convolutions to aggregate and project high-level image features, reducing computational complexity. Our model demonstrates superior performance compared to existing methods in report generation, visual question answering, and disease diagnosis.
  arXiv  Detail & Related papers  (2024-10-18T06:31:40Z)
• M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Models [49.5030774873328]
  Previous research has primarily focused on 2D medical images, leaving 3D images under-explored, despite their richer spatial information. We present a large-scale 3D multi-modal medical dataset, M3D-Data, comprising 120K image-text pairs and 662K instruction-response pairs. We also introduce a new 3D multi-modal medical benchmark, M3D-Bench, which facilitates automatic evaluation across eight tasks.
  arXiv  Detail & Related papers  (2024-03-31T06:55:12Z)
• Spatiotemporal Modeling Encounters 3D Medical Image Analysis: Slice-Shift UNet with Multi-View Fusion [0.0]
  We propose a new 2D-based model dubbed Slice SHift UNet which encodes three-dimensional features at 2D CNN's complexity. More precisely multi-view features are collaboratively learned by performing 2D convolutions along the three planes of a volume. The effectiveness of our approach is validated in Multi-Modality Abdominal Multi-Organ axis (AMOS) and Multi-Atlas Labeling Beyond the Cranial Vault (BTCV) datasets.
  arXiv  Detail & Related papers  (2023-07-24T14:53:23Z)
• Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation [133.02190910009384]
  We propose a two-stream graph convolutional network (i.e., TSGCN) to handle inter-view confusion between different raw attributes. Our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.
  arXiv  Detail & Related papers  (2022-04-19T10:41:09Z)
• Automated Model Design and Benchmarking of 3D Deep Learning Models for COVID-19 Detection with Chest CT Scans [72.04652116817238]
  We propose a differentiable neural architecture search (DNAS) framework to automatically search for the 3D DL models for 3D chest CT scans classification. We also exploit the Class Activation Mapping (CAM) technique on our models to provide the interpretability of the results.
  arXiv  Detail & Related papers  (2021-01-14T03:45:01Z)
• TSGCNet: Discriminative Geometric Feature Learning with Two-Stream GraphConvolutional Network for 3D Dental Model Segmentation [141.2690520327948]
  We propose a two-stream graph convolutional network (TSGCNet) to learn multi-view information from different geometric attributes. We evaluate our proposed TSGCNet on a real-patient dataset of dental models acquired by 3D intraoral scanners.
  arXiv  Detail & Related papers  (2020-12-26T08:02:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.