A Generative Approach for Image Registration of Visible-Thermal (VT) Cancer Faces

• URL: http://arxiv.org/abs/2308.12271v1
• Date: Wed, 23 Aug 2023 17:39:58 GMT
• Title: A Generative Approach for Image Registration of Visible-Thermal (VT) Cancer Faces
• Authors: Catherine Ordun, Alexandra Cha, Edward Raff, Sanjay Purushotham, Karen Kwok, Mason Rule, James Gulley
• Abstract summary: We modernize the classic computer vision task of image registration by applying and modifying a generative alignment algorithm. We demonstrate that the quality of thermal images produced in the generative AI downstream task of Visible-to-Thermal (V2T) image translation significantly improves up to 52.5%.
• Score: 77.77475333490744
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Since thermal imagery offers a unique modality to investigate pain, the U.S. National Institutes of Health (NIH) has collected a large and diverse set of cancer patient facial thermograms for AI-based pain research. However, differing angles from camera capture between thermal and visible sensors has led to misalignment between Visible-Thermal (VT) images. We modernize the classic computer vision task of image registration by applying and modifying a generative alignment algorithm to register VT cancer faces, without the need for a reference or alignment parameters. By registering VT faces, we demonstrate that the quality of thermal images produced in the generative AI downstream task of Visible-to-Thermal (V2T) image translation significantly improves up to 52.5\%, than without registration. Images in this paper have been approved by the NIH NCI for public dissemination.

Related papers

• Deep learning for automated detection of breast cancer in deep ultraviolet fluorescence images with diffusion probabilistic model [6.658963545934998]
  diffusion probabilistic model (DPM) has shown potential to generate high-quality images. In this paper, we apply DPM to augment the deep ultraviolet fluorescence (DUV) image dataset with an aim to improve breast cancer classification.
  arXiv  Detail & Related papers  (2024-07-01T05:00:26Z)
• Vista-Morph: Unsupervised Image Registration of Visible-Thermal Facial Pairs [36.33347149799959]
  We introduce our approach for Visible-Thermal (VT) image registration called Vista Morph. By learning the affine matrix through a Vision Transformer (ViT)-based Spatial Transformer Network (STN) and Generative Adversarial Networks (GAN), Vista Morph successfully aligns facial and non-facial VT images. We conduct a downstream generative AI task to show that registering training data with Vista Morph improves subject identity of generated thermal faces when performing V2T image translation.
  arXiv  Detail & Related papers  (2023-06-10T18:42:36Z)
• Generation of Artificial CT Images using Patch-based Conditional Generative Adversarial Networks [0.0]
  We present an image generation approach that uses generative adversarial networks with a conditional discriminator. We validate the feasibility of GAN-enhanced medical image generation on whole heart computed tomography (CT) images.
  arXiv  Detail & Related papers  (2022-05-19T20:29:25Z)
• Harmonizing Pathological and Normal Pixels for Pseudo-healthy Synthesis [68.5287824124996]
  We present a new type of discriminator, the segmentor, to accurately locate the lesions and improve the visual quality of pseudo-healthy images. We apply the generated images into medical image enhancement and utilize the enhanced results to cope with the low contrast problem. Comprehensive experiments on the T2 modality of BraTS demonstrate that the proposed method substantially outperforms the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-03-29T08:41:17Z)
• Texture Characterization of Histopathologic Images Using Ecological Diversity Measures and Discrete Wavelet Transform [82.53597363161228]
  This paper proposes a method for characterizing texture across histopathologic images with a considerable success rate. It is possible to quantify the intrinsic properties of such images with promising accuracy on two HI datasets.
  arXiv  Detail & Related papers  (2022-02-27T02:19:09Z)
• A Deep Discontinuity-Preserving Image Registration Network [73.03885837923599]
  Most deep learning-based registration methods assume that the desired deformation fields are globally smooth and continuous. We propose a weakly-supervised Deep Discontinuity-preserving Image Registration network (DDIR) to obtain better registration performance and realistic deformation fields. We demonstrate that our method achieves significant improvements in registration accuracy and predicts more realistic deformations, in registration experiments on cardiac magnetic resonance (MR) images.
  arXiv  Detail & Related papers  (2021-07-09T13:35:59Z)
• Generating Thermal Human Faces for Physiological Assessment Using Thermal Sensor Auxiliary Labels [21.920079976038163]
  Thermal images reveal medically important physiological information about human stress, signs of inflammation, and emotional mood that cannot be seen on visible images. We introduce favtGAN, a VT GAN which uses the pix2pix image translation model with an auxiliary sensor label prediction network for generating thermal faces from visible images. Experiments on these combined datasets show that favtGAN demonstrates an increase in SSIM and PSNR scores of generated thermal faces, compared to training on a single face dataset alone.
  arXiv  Detail & Related papers  (2021-06-15T12:32:52Z)
• Simultaneous Face Hallucination and Translation for Thermal to Visible Face Verification using Axial-GAN [74.22129648654783]
  We introduce the task of thermal-to-visible face verification from low-resolution thermal images. We propose Axial-Generative Adversarial Network (Axial-GAN) to synthesize high-resolution visible images for matching.
  arXiv  Detail & Related papers  (2021-04-13T22:34:28Z)
• Prediction of low-keV monochromatic images from polyenergetic CT scans for improved automatic detection of pulmonary embolism [21.47219330040151]
  We are training convolutional neural networks that can emulate the generation of monoE images from conventional single energy CT acquisitions. We expand on these methods through the use of a multi-task optimization approach, under which the networks achieve improved classification as well as generation results.
  arXiv  Detail & Related papers  (2021-02-02T11:42:31Z)
• A Large-Scale, Time-Synchronized Visible and Thermal Face Dataset [62.193924313292875]
  We present the DEVCOM Army Research Laboratory Visible-Thermal Face dataset (ARL-VTF) With over 500,000 images from 395 subjects, the ARL-VTF dataset represents to the best of our knowledge, the largest collection of paired visible and thermal face images to date. This paper presents benchmark results and analysis on thermal face landmark detection and thermal-to-visible face verification by evaluating state-of-the-art models on the ARL-VTF dataset.
  arXiv  Detail & Related papers  (2021-01-07T17:17:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.