X-RAFT: Cross-Modal Non-Rigid Registration of Blue and White Light Neurosurgical Hyperspectral Images

• URL: http://arxiv.org/abs/2507.07747v1
• Date: Thu, 10 Jul 2025 13:25:29 GMT
• Title: X-RAFT: Cross-Modal Non-Rigid Registration of Blue and White Light Neurosurgical Hyperspectral Images
• Authors: Charlie Budd, Silvère Ségaud, Matthew Elliot, Graeme Stasiuk, Yijing Xie, Jonathan Shapey, Tom Vercauteren,
• Abstract summary: Integration of hyperspectral imaging into fluorescence-guided neurosurgery has the potential to improve surgical decision making.<n>We propose using distinct image encoders for each modality pair, and fine-tune these in a self-supervised manner using flow-cycle-consistency on our neurosurgical hyperspectral data.
• Score: 2.4461177743588265
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Integration of hyperspectral imaging into fluorescence-guided neurosurgery has the potential to improve surgical decision making by providing quantitative fluorescence measurements in real-time. Quantitative fluorescence requires paired spectral data in fluorescence (blue light) and reflectance (white light) mode. Blue and white image acquisition needs to be performed sequentially in a potentially dynamic surgical environment. A key component to the fluorescence quantification process is therefore the ability to find dense cross-modal image correspondences between two hyperspectral images taken under these drastically different lighting conditions. We address this challenge with the introduction of X-RAFT, a Recurrent All-Pairs Field Transforms (RAFT) optical flow model modified for cross-modal inputs. We propose using distinct image encoders for each modality pair, and fine-tune these in a self-supervised manner using flow-cycle-consistency on our neurosurgical hyperspectral data. We show an error reduction of 36.6% across our evaluation metrics when comparing to a naive baseline and 27.83% reduction compared to an existing cross-modal optical flow method (CrossRAFT). Our code and models will be made publicly available after the review process.

Related papers

• Quantum-inspired super-resolution of fluorescent point-like sources [0.0]
  We report the experimental super-resolution of pairs of point-like fluorescent sources using a modified image inversion interferometer microscope.<n>The technique is inspired by recent developments in the application of quantum parameter estimation theory to semiclassical imaging problems.
  arXiv  Detail & Related papers  (2024-12-22T02:58:40Z)
• Deep intra-operative illumination calibration of hyperspectral cameras [73.08443963791343]
  Hyperspectral imaging (HSI) is emerging as a promising novel imaging modality with various potential surgical applications. We show that dynamically changing lighting conditions in the operating room dramatically affect the performance of HSI applications. We propose a novel learning-based approach to automatically recalibrating hyperspectral images during surgery.
  arXiv  Detail & Related papers  (2024-09-11T08:30:03Z)
• Step-Calibrated Diffusion for Biomedical Optical Image Restoration [47.191704042917394]
  Restorative Step-Calibrated Diffusion (RSCD) is an unpaired diffusion-based image restoration method.<n>RSCD outperforms other widely used unpaired image restoration methods on both image quality and perceptual evaluation.<n>RSCD improves performance on downstream clinical imaging tasks, including automated brain tumor diagnosis and deep tissue imaging.
  arXiv  Detail & Related papers  (2024-03-20T15:38:53Z)
• Spatial super-resolution in nanosensing with blinking emitters [79.16635054977068]
  We propose a method of spatial resolution enhancement in metrology (thermometry, magnetometry, pH estimation, and similar methods) with blinking fluorescent nanosensors.<n>We believe that blinking fluorescent sensing agents being complemented with the developed image analysis technique could be utilized routinely in the life science sector.
  arXiv  Detail & Related papers  (2024-02-27T10:38:05Z)
• A Spatiotemporal Illumination Model for 3D Image Fusion in Optical Coherence Tomography [2.8532140618225337]
  Optical-scanning standard tomography ( OCT) is a non-invasive, micrometer-scale imaging modality in ophthalmology. We present a novel parametrization that exploits continuity in sequentially-scanned volume data. In 68 volumes from eyes with pathology showed reduction of illumination artifacts in 88% of the data, and only 6% moderate illumination artifacts.
  arXiv  Detail & Related papers  (2024-02-19T13:08:31Z)
• Fluctuation-based deconvolution in fluorescence microscopy using plug-and-play denoisers [2.236663830879273]
  spatial resolution of images of living samples obtained by fluorescence microscopes is physically limited due to the diffraction of visible light. Several deconvolution and super-resolution techniques have been proposed to overcome this limitation.
  arXiv  Detail & Related papers  (2023-03-20T15:43:52Z)
• OADAT: Experimental and Synthetic Clinical Optoacoustic Data for Standardized Image Processing [62.993663757843464]
  Optoacoustic (OA) imaging is based on excitation of biological tissues with nanosecond-duration laser pulses followed by detection of ultrasound waves generated via light-absorption-mediated thermoelastic expansion. OA imaging features a powerful combination between rich optical contrast and high resolution in deep tissues. No standardized datasets generated with different types of experimental set-up and associated processing methods are available to facilitate advances in broader applications of OA in clinical settings.
  arXiv  Detail & Related papers  (2022-06-17T08:11:26Z)
• Dense Optical Flow from Event Cameras [55.79329250951028]
  We propose to incorporate feature correlation and sequential processing into dense optical flow estimation from event cameras. Our proposed approach computes dense optical flow and reduces the end-point error by 23% on MVSEC.
  arXiv  Detail & Related papers  (2021-08-24T07:39:08Z)
• Quantum Optical Coherence Tomography using two photon joint spectrum detection (JS-Q-OCT) [0.0]
  Quantum Optical Coherence Tomography (Q- OCT) is a high-resolution 3D imaging technique based on white-light interferometry. Here, we present a theoretical analysis of a novel approach that is free of image-scrambling artefacts and slow acquisition times. We show that all the information about the internal structures of the object is encoded in the joint spectrum and can be easily retrieved through Fourier transformation.
  arXiv  Detail & Related papers  (2020-05-27T04:02:10Z)
• Hyperspectral-Multispectral Image Fusion with Weighted LASSO [68.04032419397677]
  We propose an approach for fusing hyperspectral and multispectral images to provide high-quality hyperspectral output. We demonstrate that the proposed sparse fusion and reconstruction provides quantitatively superior results when compared to existing methods on publicly available images.
  arXiv  Detail & Related papers  (2020-03-15T23:07:56Z)
• Multifold Acceleration of Diffusion MRI via Slice-Interleaved Diffusion Encoding (SIDE) [50.65891535040752]
  We propose a diffusion encoding scheme, called Slice-Interleaved Diffusion. SIDE, that interleaves each diffusion-weighted (DW) image volume with slices encoded with different diffusion gradients. We also present a method based on deep learning for effective reconstruction of DW images from the highly slice-undersampled data.
  arXiv  Detail & Related papers  (2020-02-25T14:48:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.