Distilling Knowledge for Designing Computational Imaging Systems

• URL: http://arxiv.org/abs/2501.17898v1
• Date: Wed, 29 Jan 2025 03:49:21 GMT
• Title: Distilling Knowledge for Designing Computational Imaging Systems
• Authors: Leon Suarez-Rodriguez, Roman Jacome, Henry Arguello,
• Abstract summary: The performance of E2E optimization is significantly reduced by the physical constraints imposed on the encoder. We reinterpret the concept of knowledge distillation for designing a physically constrained CI system by transferring the knowledge of a pretrained, less-constrained CI system. Our approach achieves significantly improved reconstruction performance and encoder design, outperforming both E2E optimization and traditional non-data-driven encoder designs.
• Score: 15.662108754691864
• License:
• Abstract: Designing the physical encoder is crucial for accurate image reconstruction in computational imaging (CI) systems. Currently, these systems are designed via end-to-end (E2E) optimization, where the encoder is modeled as a neural network layer and is jointly optimized with the decoder. However, the performance of E2E optimization is significantly reduced by the physical constraints imposed on the encoder. Also, since the E2E learns the parameters of the encoder by backpropagating the reconstruction error, it does not promote optimal intermediate outputs and suffers from gradient vanishing. To address these limitations, we reinterpret the concept of knowledge distillation (KD) for designing a physically constrained CI system by transferring the knowledge of a pretrained, less-constrained CI system. Our approach involves three steps: (1) Given the original CI system (student), a teacher system is created by relaxing the constraints on the student's encoder. (2) The teacher is optimized to solve a less-constrained version of the student's problem. (3) The teacher guides the training of the student through two proposed knowledge transfer functions, targeting both the encoder and the decoder feature space. The proposed method can be employed to any imaging modality since the relaxation scheme and the loss functions can be adapted according to the physical acquisition and the employed decoder. This approach was validated on three representative CI modalities: magnetic resonance, single-pixel, and compressive spectral imaging. Simulations show that a teacher system with an encoder that has a structure similar to that of the student encoder provides effective guidance. Our approach achieves significantly improved reconstruction performance and encoder design, outperforming both E2E optimization and traditional non-data-driven encoder designs.

Related papers

• END$^2$: Robust Dual-Decoder Watermarking Framework Against Non-Differentiable Distortions [15.774214187916423]
  Real-world distortions are often non-differentiable, leading to challenges in end-to-end training. We propose a novel dual-decoder architecture (END$2$) to better incorporate non-differentiable distortions into training. Our scheme outperforms state-of-the-art algorithms under various non-differentiable distortions.
  arXiv  Detail & Related papers  (2024-12-13T08:37:30Z)
• Highly Constrained Coded Aperture Imaging Systems Design Via a Knowledge Distillation Approach [15.662108754691864]
  This paper proposes a knowledge distillation (KD) framework for the design of highly physically constrained COI systems. We validate the proposed approach, using a binary coded apertures single pixel camera for monochromatic and multispectral image reconstruction.
  arXiv  Detail & Related papers  (2024-06-25T23:03:48Z)
• Joint Beam Search Integrating CTC, Attention, and Transducer Decoders [53.297697898510194]
  We propose a joint modeling scheme where four decoders share the same encoder. The 4D model is trained jointly, which will bring model regularization and maximize the model robustness. In addition, we propose three novel joint beam search algorithms by combining three decoders.
  arXiv  Detail & Related papers  (2024-06-05T05:18:20Z)
• Learning Linear Block Error Correction Codes [62.25533750469467]
  We propose for the first time a unified encoder-decoder training of binary linear block codes. We also propose a novel Transformer model in which the self-attention masking is performed in a differentiable fashion for the efficient backpropagation of the code gradient.
  arXiv  Detail & Related papers  (2024-05-07T06:47:12Z)
• Efficient Transformer Encoders for Mask2Former-style models [57.54752243522298]
  ECO-M2F is a strategy to self-select the number of hidden layers in the encoder conditioned on the input image. The proposed approach reduces expected encoder computational cost while maintaining performance. It is flexible in architecture configurations, and can be extended beyond the segmentation task to object detection.
  arXiv  Detail & Related papers  (2024-04-23T17:26:34Z)
• A Principled Hierarchical Deep Learning Approach to Joint Image Compression and Classification [27.934109301041595]
  This work proposes a three-step joint learning strategy to guide encoders to extract features that are compact, discriminative, and amenable to common augmentations/transformations. Tests show that our proposed method achieves accuracy improvement of up to 1.5% on CIFAR-10 and 3% on CIFAR-100 over conventional E2E cross-entropy training.
  arXiv  Detail & Related papers  (2023-10-30T15:52:18Z)
• Triple-View Knowledge Distillation for Semi-Supervised Semantic Segmentation [54.23510028456082]
  We propose a Triple-view Knowledge Distillation framework, termed TriKD, for semi-supervised semantic segmentation. The framework includes the triple-view encoder and the dual-frequency decoder.
  arXiv  Detail & Related papers  (2023-09-22T01:02:21Z)
• Deep Optical Coding Design in Computational Imaging [16.615106763985942]
  Computational optical imaging (COI) systems leverage optical coding elements (CE) in their setups to encode a high-dimensional scene in a single or multiple snapshots and decode it by using computational algorithms. The performance of COI systems highly depends on the design of its main components: the CE pattern and the computational method used to perform a given task. Deep neural networks (DNNs) have opened a new horizon in CE data-driven designs that jointly consider the optical encoder and computational decoder.
  arXiv  Detail & Related papers  (2022-06-27T04:41:48Z)
• Reducing Redundancy in the Bottleneck Representation of the Autoencoders [98.78384185493624]
  Autoencoders are a type of unsupervised neural networks, which can be used to solve various tasks. We propose a scheme to explicitly penalize feature redundancies in the bottleneck representation. We tested our approach across different tasks: dimensionality reduction using three different dataset, image compression using the MNIST dataset, and image denoising using fashion MNIST.
  arXiv  Detail & Related papers  (2022-02-09T18:48:02Z)
• Machine Learning Framework for Quantum Sampling of Highly-Constrained, Continuous Optimization Problems [101.18253437732933]
  We develop a generic, machine learning-based framework for mapping continuous-space inverse design problems into surrogate unconstrained binary optimization problems. We showcase the framework's performance on two inverse design problems by optimizing thermal emitter topologies for thermophotovoltaic applications and (ii) diffractive meta-gratings for highly efficient beam steering.
  arXiv  Detail & Related papers  (2021-05-06T02:22:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.