SegDem: Segmentation helps Demosaicing

• URL: http://arxiv.org/abs/2608.07916v1
• Date: Sat, 08 Aug 2026 04:49:20 GMT
• Title: SegDem: Segmentation helps Demosaicing
• Abstract summary: Image demosaicing reconstructs a full-color image from incomplete color measurements produced by a sensor covered with a color filter (CFA)<n>Most existing methods formulate demosaicing as pixel-level reconstruction.<n>SegDem is a cross-task decoder representation transfer framework for demosaicing.
• Score: 52.515779223891705
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Image demosaicing reconstructs a full-color image from incomplete color measurements produced by a sensor covered with a color filter array (CFA). Most existing methods formulate demosaicing as pixel-level reconstruction and mainly rely on local textures, cross-channel correlations, and low-level image statistics. Our core insight is that reconstruction and visual understanding can be viewed as complementary views of shared scene structure: both are grounded in the same underlying physical world, and therefore the structural and physical information inferred from an image should remain consistent across the two tasks. We instantiate this idea with instance segmentation and propose \emph{SegDem}, a cross-task decoder representation transfer framework for demosaicing. SegDem first learns region- and boundary-aware representations through instance-aware structural pretraining and then transfers the decoder to RAW-conditioned reconstruction. Segmentation- and demosaicing-conditioned features are further anchored to a shared frozen DINOv2 representation space to preserve structural organization across tasks. We instantiate SegDem with convolutional, Transformer-based, and state-space backbones for unified Single- and Quad-Bayer demosaicing. Extensive experiments on synthetic, external, and challenging datasets demonstrate consistent improvements across different architectures and CFA layouts.

Related papers

• Bridging Modalities and Tasks: A Unified Hierarchical ViT for SAR-to-Optical Translation and Semantic Segmentation [6.274164868115626]
  Synthetic Aperture Radar (SAR) images have all-weather, day-and-night observation capabilities.<n>Generative models for SAR-to-optical (S2O) conversion can improve visual interpretability, but existing methods often ignore the constraints on semantic structure.<n>We propose a unified collaborative dual-task learning framework, termed BMT (Bridging Modalities and Tasks), that jointly optimize S2O image translation and semantic segmentation.
  arXiv  Detail & Related papers  (2026-09-04T04:50:19Z)
• Structural Guidance for Unified Joint Demosaicing and Denoising [15.554336519344078]
  Joint demosaicing and denoising is a fundamental step in camera image signal processing.<n>Different Bayer-like color filter arrays (CFAs) and sensor noise jointly corrupt both color sampling and image content.<n>We propose a structural-guided unified restoration framework that injects pretrained structural knowledge into CFA-aware image restoration.
  arXiv  Detail & Related papers  (2026-08-07T10:14:20Z)
• StructDiff: A Structure-Preserving and Spatially Controllable Diffusion Model for Single-Image Generation [72.84181869780627]
  StructDiff is a generative framework based on a single-scale diffusion model for single-image generation.<n>It incorporates 3D positional encoding (PE) as a spatial prior, allowing flexible control over positions, scale, and local details of generated objects.<n>It also demonstrates broad applicability across downstream tasks, such as text-guided image generation, image editing, outpainting, and paint-to-image synthesis.
  arXiv  Detail & Related papers  (2026-04-14T10:55:43Z)
• Component-Aware Sketch-to-Image Generation Using Self-Attention Encoding and Coordinate-Preserving Fusion [2.510998372750843]
  Translating freehand sketches into photorealistic images remains a fundamental challenge in image synthesis.<n>Existing approaches, including GAN-based and diffusion-based models, often struggle to reconstruct fine-grained details, maintain spatial alignment, or adapt across different sketch domains.<n>We propose a component-aware, self-refining framework for sketch-to-image generation that addresses these challenges through a novel two-stage architecture.
  arXiv  Detail & Related papers  (2026-03-10T10:39:24Z)
• LAW-Diffusion: Complex Scene Generation by Diffusion with Layouts [107.11267074981905]
  We propose a semantically controllable layout-AWare diffusion model, termed LAW-Diffusion. We show that LAW-Diffusion yields the state-of-the-art generative performance, especially with coherent object relations.
  arXiv  Detail & Related papers  (2023-08-13T08:06:18Z)
• Occlusion-Aware Instance Segmentation via BiLayer Network Architectures [73.45922226843435]
  We propose Bilayer Convolutional Network (BCNet), where the top layer detects occluding objects (occluders) and the bottom layer infers partially occluded instances (occludees) We investigate the efficacy of bilayer structure using two popular convolutional network designs, namely, Fully Convolutional Network (FCN) and Graph Convolutional Network (GCN)
  arXiv  Detail & Related papers  (2022-08-08T21:39:26Z)
• Cross-View Panorama Image Synthesis [68.35351563852335]
  PanoGAN is a novel adversarial feedback GAN framework named. PanoGAN enables high-quality panorama image generation with more convincing details than state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-03-22T15:59:44Z)
• Self-supervised Correlation Mining Network for Person Image Generation [9.505343361614928]
  Person image generation aims to perform non-rigid deformation on source images. We propose a Self-supervised Correlation Mining Network (SCM-Net) to rearrange the source images in the feature space. For improving the fidelity of cross-scale pose transformation, we propose a graph based Body Structure Retaining Loss.
  arXiv  Detail & Related papers  (2021-11-26T03:57:46Z)
• Image Inpainting via Conditional Texture and Structure Dual Generation [26.97159780261334]
  We propose a novel two-stream network for image inpainting, which models the structure-constrained texture synthesis and texture-guided structure reconstruction. To enhance the global consistency, a Bi-directional Gated Feature Fusion (Bi-GFF) module is designed to exchange and combine the structure and texture information. Experiments on the CelebA, Paris StreetView and Places2 datasets demonstrate the superiority of the proposed method.
  arXiv  Detail & Related papers  (2021-08-22T15:44:37Z)
• Retinal Image Segmentation with a Structure-Texture Demixing Network [62.69128827622726]
  The complex structure and texture information are mixed in a retinal image, and distinguishing the information is difficult. Existing methods handle texture and structure jointly, which may lead biased models toward recognizing textures and thus results in inferior segmentation performance. We propose a segmentation strategy that seeks to separate structure and texture components and significantly improve the performance.
  arXiv  Detail & Related papers  (2020-07-15T12:19:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.