TeethDreamer: 3D Teeth Reconstruction from Five Intra-oral Photographs

• URL: http://arxiv.org/abs/2407.11419v1
• Date: Tue, 16 Jul 2024 06:24:32 GMT
• Title: TeethDreamer: 3D Teeth Reconstruction from Five Intra-oral Photographs
• Authors: Chenfan Xu, Zhentao Liu, Yuan Liu, Yulong Dou, Jiamin Wu, Jiepeng Wang, Minjiao Wang, Dinggang Shen, Zhiming Cui,
• Abstract summary: We propose a 3D teeth reconstruction framework, named TeethDreamer, to restore the shape and position of the upper and lower teeth. Given five intra-oral photographs, our approach first leverages a large diffusion model's prior knowledge to generate novel multi-view images. To ensure the 3D consistency across generated views, we integrate a 3D-aware feature attention mechanism in the reverse diffusion process.
• Score: 45.0864129371874
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Orthodontic treatment usually requires regular face-to-face examinations to monitor dental conditions of the patients. When in-person diagnosis is not feasible, an alternative is to utilize five intra-oral photographs for remote dental monitoring. However, it lacks of 3D information, and how to reconstruct 3D dental models from such sparse view photographs is a challenging problem. In this study, we propose a 3D teeth reconstruction framework, named TeethDreamer, aiming to restore the shape and position of the upper and lower teeth. Given five intra-oral photographs, our approach first leverages a large diffusion model's prior knowledge to generate novel multi-view images with known poses to address sparse inputs and then reconstructs high-quality 3D teeth models by neural surface reconstruction. To ensure the 3D consistency across generated views, we integrate a 3D-aware feature attention mechanism in the reverse diffusion process. Moreover, a geometry-aware normal loss is incorporated into the teeth reconstruction process to enhance geometry accuracy. Extensive experiments demonstrate the superiority of our method over current state-of-the-arts, giving the potential to monitor orthodontic treatment remotely. Our code is available at https://github.com/ShanghaiTech-IMPACT/TeethDreamer

Related papers

• 3D Structure-guided Network for Tooth Alignment in 2D Photograph [47.51314162367702]
  A 2D photograph depicting aligned teeth prior to orthodontic treatment is crucial for effective dentist-patient communication. We propose a 3D structure-guided tooth alignment network that takes 2D photographs as input and aligns the teeth within the 2D image space. We evaluate our network on various facial photographs, demonstrating its exceptional performance and strong applicability within the orthodontic industry.
  arXiv  Detail & Related papers  (2023-10-17T09:44:30Z)
• Oral-3Dv2: 3D Oral Reconstruction from Panoramic X-Ray Imaging with Implicit Neural Representation [3.8215162658168524]
  Oral-3Dv2 is a non-adversarial-learning-based model in 3D radiology reconstruction from a single panoramic X-ray image. Our model learns to represent the 3D oral structure in an implicit way by mapping 2D coordinates into density values of voxels in the 3D space. To the best of our knowledge, this is the first work of a non-adversarial-learning-based model in 3D radiology reconstruction from a single panoramic X-ray image.
  arXiv  Detail & Related papers  (2023-03-21T18:17:27Z)
• OrthoGAN:High-Precision Image Generation for Teeth Orthodontic Visualization [17.647161676763478]
  We build an efficient system for simulating virtual teeth alignment effects in a frontal facial image. We design a multi-modal encoder-decoder based generative model to synthesize identity-preserving frontal facial images with aligned teeth.
  arXiv  Detail & Related papers  (2022-12-29T03:12:47Z)
• ToothInpaintor: Tooth Inpainting from Partial 3D Dental Model and 2D Panoramic Image [35.72913439096702]
  In orthodontic treatment, a full tooth model consisting of both the crown and root is indispensable. In this paper, we propose a neural network, called ToothInpaintor, that takes as input a partial 3D dental model and a 2D panoramic image. We successfully project an input to the learned latent space via neural optimization to obtain the full tooth model conditioned on the input.
  arXiv  Detail & Related papers  (2022-11-25T18:15:22Z)
• An Implicit Parametric Morphable Dental Model [79.29420177904022]
  We present the first parametric 3D morphable dental model for both teeth and gum. It is based on a component-wise representation for each tooth and the gum, together with a learnable latent code for each of such components. Our reconstruction quality is on par with the most advanced global implicit representations while enabling novel applications.
  arXiv  Detail & Related papers  (2022-11-21T12:23:54Z)
• 3D Reconstruction of Curvilinear Structures with Stereo Matching DeepConvolutional Neural Networks [52.710012864395246]
  We propose a fully automated pipeline for both detection and matching of curvilinear structures in stereo pairs. We mainly focus on 3D reconstruction of dislocations from stereo pairs of TEM images.
  arXiv  Detail & Related papers  (2021-10-14T23:05:47Z)
• Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction [76.1612334630256]
  We harness the power of Generative Adversarial Networks (GANs) and Deep Convolutional Neural Networks (DCNNs) to reconstruct the facial texture and shape from single images. We demonstrate excellent results in photorealistic and identity preserving 3D face reconstructions and achieve for the first time, facial texture reconstruction with high-frequency details.
  arXiv  Detail & Related papers  (2021-05-16T16:35:44Z)
• OSTeC: One-Shot Texture Completion [86.23018402732748]
  We propose an unsupervised approach for one-shot 3D facial texture completion. The proposed approach rotates an input image in 3D and fill-in the unseen regions by reconstructing the rotated image in a 2D face generator. We frontalize the target image by projecting the completed texture into the generator.
  arXiv  Detail & Related papers  (2020-12-30T23:53:26Z)
• Oral-3D: Reconstructing the 3D Bone Structure of Oral Cavity from 2D Panoramic X-ray [17.34835093235681]
  We propose a framework, named Oral-3D, to reconstruct the 3D oral cavity from a single PX image and prior information of the dental arch. We show that Oral-3D can efficiently and effectively reconstruct the 3D oral structure and show critical information in clinical applications.
  arXiv  Detail & Related papers  (2020-03-18T18:02:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.