InDeed: Interpretable image deep decomposition with guaranteed generalizability

• URL: http://arxiv.org/abs/2501.01127v1
• Date: Thu, 02 Jan 2025 07:58:26 GMT
• Title: InDeed: Interpretable image deep decomposition with guaranteed generalizability
• Authors: Sihan Wang, Shangqi Gao, Fuping Wu, Xiahai Zhuang,
• Abstract summary: Image decomposition aims to analyze an image into elementary components. Deep learning can be powerful for such tasks, but its combination with a focus on interpretability and generalizability is rarely explored. We introduce a novel framework for interpretable deep image decomposition, combining hierarchical Bayesian modeling and deep learning.
• Score: 28.595151003310452
• License:
• Abstract: Image decomposition aims to analyze an image into elementary components, which is essential for numerous downstream tasks and also by nature provides certain interpretability to the analysis. Deep learning can be powerful for such tasks, but surprisingly their combination with a focus on interpretability and generalizability is rarely explored. In this work, we introduce a novel framework for interpretable deep image decomposition, combining hierarchical Bayesian modeling and deep learning to create an architecture-modularized and model-generalizable deep neural network (DNN). The proposed framework includes three steps: (1) hierarchical Bayesian modeling of image decomposition, (2) transforming the inference problem into optimization tasks, and (3) deep inference via a modularized Bayesian DNN. We further establish a theoretical connection between the loss function and the generalization error bound, which inspires a new test-time adaptation approach for out-of-distribution scenarios. We instantiated the application using two downstream tasks, \textit{i.e.}, image denoising, and unsupervised anomaly detection, and the results demonstrated improved generalizability as well as interpretability of our methods. The source code will be released upon the acceptance of this paper.

Related papers

• Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditions [30.148969711689773]
  We present a novel approach designed to address the complexities posed by challenging, out-of-distribution data in the single-image depth estimation task. We systematically generate new, user-defined scenes with a comprehensive set of challenges and associated depth information. This is achieved by leveraging cutting-edge text-to-image diffusion models with depth-aware control.
  arXiv  Detail & Related papers  (2024-07-23T17:59:59Z)
• Rotation Equivariant Proximal Operator for Deep Unfolding Methods in Image Restoration [62.41329042683779]
  We propose a high-accuracy rotation equivariant proximal network that embeds rotation symmetry priors into the deep unfolding framework. This study makes efforts to suggest a high-accuracy rotation equivariant proximal network that effectively embeds rotation symmetry priors into the deep unfolding framework.
  arXiv  Detail & Related papers  (2023-12-25T11:53:06Z)
• Intrinsic Image Decomposition via Ordinal Shading [0.0]
  Intrinsic decomposition is a fundamental mid-level vision problem that plays a crucial role in inverse rendering and computational photography pipelines. We present a dense ordinal shading formulation using a shift- and scale-invariant loss to estimate ordinal shading cues. We then combine low- and high-resolution ordinal estimations using a second network to generate a shading estimate with both global coherency and local details.
  arXiv  Detail & Related papers  (2023-11-21T18:58:01Z)
• Understanding and Constructing Latent Modality Structures in Multi-modal Representation Learning [53.68371566336254]
  We argue that the key to better performance lies in meaningful latent modality structures instead of perfect modality alignment. Specifically, we design 1) a deep feature separation loss for intra-modality regularization; 2) a Brownian-bridge loss for inter-modality regularization; and 3) a geometric consistency loss for both intra- and inter-modality regularization.
  arXiv  Detail & Related papers  (2023-03-10T14:38:49Z)
• In-N-Out: Faithful 3D GAN Inversion with Volumetric Decomposition for Face Editing [28.790900756506833]
  3D-aware GANs offer new capabilities for view synthesis while preserving the editing functionalities of their 2D counterparts. GAN inversion is a crucial step that seeks the latent code to reconstruct input images or videos, subsequently enabling diverse editing tasks through manipulation of this latent code. We address this issue by explicitly modeling OOD objects from the input in 3D-aware GANs.
  arXiv  Detail & Related papers  (2023-02-09T18:59:56Z)
• DeepMLE: A Robust Deep Maximum Likelihood Estimator for Two-view Structure from Motion [9.294501649791016]
  Two-view structure from motion (SfM) is the cornerstone of 3D reconstruction and visual SLAM (vSLAM) We formulate the two-view SfM problem as a maximum likelihood estimation (MLE) and solve it with the proposed framework, denoted as DeepMLE. Our method significantly outperforms the state-of-the-art end-to-end two-view SfM approaches in accuracy and generalization capability.
  arXiv  Detail & Related papers  (2022-10-11T15:07:25Z)
• HandFlow: Quantifying View-Dependent 3D Ambiguity in Two-Hand Reconstruction with Normalizing Flow [73.7895717883622]
  We explicitly model the distribution of plausible reconstructions in a conditional normalizing flow framework. We show that explicit ambiguity modeling is better-suited for this challenging problem.
  arXiv  Detail & Related papers  (2022-10-04T15:42:22Z)
• Deep Learning for Material Decomposition in Photon-Counting CT [0.5801044612920815]
  We present a novel deep-learning solution for material decomposition in PCCT, based on an unrolled/unfolded iterative network. Our approach outperforms a maximum likelihood estimation, a variational method, as well as a fully-learned network.
  arXiv  Detail & Related papers  (2022-08-05T19:05:16Z)
• Deep Two-View Structure-from-Motion Revisited [83.93809929963969]
  Two-view structure-from-motion (SfM) is the cornerstone of 3D reconstruction and visual SLAM. We propose to revisit the problem of deep two-view SfM by leveraging the well-posedness of the classic pipeline. Our method consists of 1) an optical flow estimation network that predicts dense correspondences between two frames; 2) a normalized pose estimation module that computes relative camera poses from the 2D optical flow correspondences, and 3) a scale-invariant depth estimation network that leverages epipolar geometry to reduce the search space, refine the dense correspondences, and estimate relative depth maps.
  arXiv  Detail & Related papers  (2021-04-01T15:31:20Z)
• Deep Variational Network Toward Blind Image Restoration [60.45350399661175]
  Blind image restoration is a common yet challenging problem in computer vision. We propose a novel blind image restoration method, aiming to integrate both the advantages of them. Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.
  arXiv  Detail & Related papers  (2020-08-25T03:30:53Z)
• A Flexible Framework for Designing Trainable Priors with Adaptive Smoothing and Game Encoding [57.1077544780653]
  We introduce a general framework for designing and training neural network layers whose forward passes can be interpreted as solving non-smooth convex optimization problems. We focus on convex games, solved by local agents represented by the nodes of a graph and interacting through regularization functions. This approach is appealing for solving imaging problems, as it allows the use of classical image priors within deep models that are trainable end to end.
  arXiv  Detail & Related papers  (2020-06-26T08:34:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.