3-D Image-to-Image Fusion in Lightsheet Microscopy by Two-Step Adversarial Network: Contribution to the FuseMyCells Challenge

• URL: http://arxiv.org/abs/2503.16075v1
• Date: Thu, 20 Mar 2025 12:12:01 GMT
• Title: 3-D Image-to-Image Fusion in Lightsheet Microscopy by Two-Step Adversarial Network: Contribution to the FuseMyCells Challenge
• Authors: Marek Wodzinski, Henning Müller,
• Abstract summary: Multiview lightsheet microscopy improves 3-D resolution by combining multiple views but simultaneously increasing the complexity and the photon budget.<n>We propose a contribution to the FuseMyCells challenge based on a two-step procedure.<n> Experimental results demonstrate the effectiveness of our approach.
• Score: 2.1640167354628046
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Lightsheet microscopy is a powerful 3-D imaging technique that addresses limitations of traditional optical and confocal microscopy but suffers from a low penetration depth and reduced image quality at greater depths. Multiview lightsheet microscopy improves 3-D resolution by combining multiple views but simultaneously increasing the complexity and the photon budget, leading to potential photobleaching and phototoxicity. The FuseMyCells challenge, organized in conjunction with the IEEE ISBI 2025 conference, aims to benchmark deep learning-based solutions for fusing high-quality 3-D volumes from single 3-D views, potentially simplifying procedures and conserving the photon budget. In this work, we propose a contribution to the FuseMyCells challenge based on a two-step procedure. The first step processes a downsampled version of the image to capture the entire region of interest, while the second step uses a patch-based approach for high-resolution inference, incorporating adversarial loss to enhance visual outcomes. This method addresses challenges related to high data resolution, the necessity of global context, and the preservation of high-frequency details. Experimental results demonstrate the effectiveness of our approach, highlighting its potential to improve 3-D image fusion quality and extend the capabilities of lightsheet microscopy. The average SSIM for the nucleus and membranes is greater than 0.85 and 0.91, respectively.

Related papers

• The Quest for Early Detection of Retinal Disease: 3D CycleGAN-based Translation of Optical Coherence Tomography into Confocal Microscopy [11.321411104729002]
  We propose a novel framework based on unsupervised 3D CycleGAN for translating unpaired in vivo OCT to ex vivo confocal microscopy images. This marks the first attempt to exploit the inherent 3D information of OCT and translate it into the rich, detailed color domain of confocal microscopy.
  arXiv  Detail & Related papers  (2024-08-07T21:13:49Z)
• SR-CACO-2: A Dataset for Confocal Fluorescence Microscopy Image Super-Resolution [7.770202118479678]
  Machine/deep learning methods for single-image super-resolution (SISR) can be applied to restore image quality by upscaling lower-resolution images to yield high-resolution images (HR) SISR methods have been successfully applied to photo-realistic images due partly to the abundance of publicly available data. We introduce a large scanning confocal microscopy dataset named SR-CACO-2 that is comprised of low- and high-resolution image pairs marked for three different fluorescent markers.
  arXiv  Detail & Related papers  (2024-06-13T14:30:35Z)
• Hybrid-Supervised Dual-Search: Leveraging Automatic Learning for Loss-free Multi-Exposure Image Fusion [60.221404321514086]
  Multi-exposure image fusion (MEF) has emerged as a prominent solution to address the limitations of digital imaging in representing varied exposure levels. This paper presents a Hybrid-Supervised Dual-Search approach for MEF, dubbed HSDS-MEF, which introduces a bi-level optimization search scheme for automatic design of both network structures and loss functions.
  arXiv  Detail & Related papers  (2023-09-03T08:07:26Z)
• MLF-DET: Multi-Level Fusion for Cross-Modal 3D Object Detection [54.52102265418295]
  We propose a novel and effective Multi-Level Fusion network, named as MLF-DET, for high-performance cross-modal 3D object DETection. For the feature-level fusion, we present the Multi-scale Voxel Image fusion (MVI) module, which densely aligns multi-scale voxel features with image features. For the decision-level fusion, we propose the lightweight Feature-cued Confidence Rectification (FCR) module, which exploits image semantics to rectify the confidence of detection candidates.
  arXiv  Detail & Related papers  (2023-07-18T11:26:02Z)
• Searching a Compact Architecture for Robust Multi-Exposure Image Fusion [55.37210629454589]
  Two major stumbling blocks hinder the development, including pixel misalignment and inefficient inference. This study introduces an architecture search-based paradigm incorporating self-alignment and detail repletion modules for robust multi-exposure image fusion. The proposed method outperforms various competitive schemes, achieving a noteworthy 3.19% improvement in PSNR for general scenarios and an impressive 23.5% enhancement in misaligned scenarios.
  arXiv  Detail & Related papers  (2023-05-20T17:01:52Z)
• Semi- and Self-Supervised Multi-View Fusion of 3D Microscopy Images using Generative Adversarial Networks [0.11719282046304678]
  Recent developments in fluorescence microscopy allow capturing high-resolution 3D images over time for living model organisms. To be able to image even large specimens, techniques like multi-view light-sheet imaging record different orientations at each time point. CNN-based multi-view deconvolution and fusion with two synthetic data sets mimic developing embryos and involve either two or four complementary 3D views.
  arXiv  Detail & Related papers  (2021-08-05T17:21:01Z)
• Axial-to-lateral super-resolution for 3D fluorescence microscopy using unsupervised deep learning [19.515134844947717]
  We present a deep-learning-enabled unsupervised super-resolution technique that enhances anisotropic images in fluorescence microscopy. Our method greatly reduces the effort to put into practice as the training of a network requires as little as a single 3D image stack. We demonstrate that the trained network not only enhances axial resolution beyond the diffraction limit, but also enhances suppressed visual details between the imaging planes and removes imaging artifacts.
  arXiv  Detail & Related papers  (2021-04-19T16:31:12Z)
• Light Field Reconstruction via Deep Adaptive Fusion of Hybrid Lenses [67.01164492518481]
  This paper explores the problem of reconstructing high-resolution light field (LF) images from hybrid lenses. We propose a novel end-to-end learning-based approach, which can comprehensively utilize the specific characteristics of the input. Our framework could potentially decrease the cost of high-resolution LF data acquisition and benefit LF data storage and transmission.
  arXiv  Detail & Related papers  (2021-02-14T06:44:47Z)
• Correlation Plenoptic Imaging between Arbitrary Planes [52.77024349608834]
  We show that the protocol enables to change the focused planes, in post-processing, and to achieve an unprecedented combination of image resolution and depth of field. Results lead the way towards the development of compact designs for correlation plenoptic imaging devices based on chaotic light, as well as high-SNR plenoptic imaging devices based on entangled photon illumination.
  arXiv  Detail & Related papers  (2020-07-23T14:26:14Z)
• Gated Fusion Network for Degraded Image Super Resolution [78.67168802945069]
  We propose a dual-branch convolutional neural network to extract base features and recovered features separately. By decomposing the feature extraction step into two task-independent streams, the dual-branch model can facilitate the training process.
  arXiv  Detail & Related papers  (2020-03-02T13:28:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.