Modular Neural Image Signal Processing

• URL: http://arxiv.org/abs/2512.08564v1
• Date: Tue, 09 Dec 2025 13:04:08 GMT
• Title: Modular Neural Image Signal Processing
• Abstract summary: This paper presents a modular neural image signal processing (ISP) framework that processes raw inputs and renders high-quality display-referred images.<n>Our method introduces a high degree of modularity, providing full control over multiple intermediate stages of the rendering process.<n>Our method is a fully learning-based framework with variants of different capacities, all of moderate size.
• Score: 21.19661399053891
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents a modular neural image signal processing (ISP) framework that processes raw inputs and renders high-quality display-referred images. Unlike prior neural ISP designs, our method introduces a high degree of modularity, providing full control over multiple intermediate stages of the rendering process.~This modular design not only achieves high rendering accuracy but also improves scalability, debuggability, generalization to unseen cameras, and flexibility to match different user-preference styles. To demonstrate the advantages of this design, we built a user-interactive photo-editing tool that leverages our neural ISP to support diverse editing operations and picture styles. The tool is carefully engineered to take advantage of the high-quality rendering of our neural ISP and to enable unlimited post-editable re-rendering. Our method is a fully learning-based framework with variants of different capacities, all of moderate size (ranging from ~0.5 M to ~3.9 M parameters for the entire pipeline), and consistently delivers competitive qualitative and quantitative results across multiple test sets. Watch the supplemental video at: https://youtu.be/ByhQjQSjxVM

Related papers

• Self-supervised pretraining for an iterative image size agnostic vision transformer [60.09702846704075]
  Vision Transformers (ViTs) dominate self-supervised learning (SSL)<n>We introduce a novel sequential-to-global SSL framework based on DINO's self-distillation objective.<n>We achieve competitive performance on ImageNet-1K and downstream classification tasks.
  arXiv  Detail & Related papers  (2026-04-22T09:53:28Z)
• Joint Post-Training Quantization of Vision Transformers with Learned Prompt-Guided Data Generation [5.75627633588113]
  We present a framework for joint quantization of Vision Transformers trained on ImageNet.<n>We achieve state-of-the-art W4A4 and W3A3 accuracies on ImageNet.<n>We also introduce a data-free calibration strategy that synthesizes diverse, label-free samples.
  arXiv  Detail & Related papers  (2026-02-21T15:02:21Z)
• MuSASplat: Efficient Sparse-View 3D Gaussian Splats via Lightweight Multi-Scale Adaptation [92.57609195819647]
  MuSASplat is a novel framework that dramatically reduces the computational burden of training pose-free feed-forward 3D Gaussian splats models.<n>Central to our approach is a lightweight Multi-Scale Adapter that enables efficient fine-tuning of ViT-based architectures with only a small fraction of training parameters.
  arXiv  Detail & Related papers  (2025-12-08T04:56:46Z)
• Multi-Head Attention Residual Unfolded Network for Model-Based Pansharpening [2.874893537471256]
  Unfolding fusion methods integrate the powerful representation capabilities of deep learning with the robustness of model-based approaches. In this paper, we propose a model-based deep unfolded method for satellite image fusion. Experimental results on PRISMA, Quickbird, and WorldView2 datasets demonstrate the superior performance of our method.
  arXiv  Detail & Related papers  (2024-09-04T13:05:00Z)
• Generalizable Neural Human Renderer [29.533330467339013]
  Generalizable Human Renderer (GNH) is a novel method for rendering animatable humans from monocular video without any test-time optimization. We demonstrate GNH surpasses current state-of-the-art methods, notably achieving a 31.3% improvement in LPIPS.
  arXiv  Detail & Related papers  (2024-04-22T14:09:53Z)
• Hyper-VolTran: Fast and Generalizable One-Shot Image to 3D Object Structure via HyperNetworks [53.67497327319569]
  We introduce a novel neural rendering technique to solve image-to-3D from a single view. Our approach employs the signed distance function as the surface representation and incorporates generalizable priors through geometry-encoding volumes and HyperNetworks. Our experiments show the advantages of our proposed approach with consistent results and rapid generation.
  arXiv  Detail & Related papers  (2023-12-24T08:42:37Z)
• Multi-scale Transformer Network with Edge-aware Pre-training for Cross-Modality MR Image Synthesis [52.41439725865149]
  Cross-modality magnetic resonance (MR) image synthesis can be used to generate missing modalities from given ones. Existing (supervised learning) methods often require a large number of paired multi-modal data to train an effective synthesis model. We propose a Multi-scale Transformer Network (MT-Net) with edge-aware pre-training for cross-modality MR image synthesis.
  arXiv  Detail & Related papers  (2022-12-02T11:40:40Z)
• Multi-View Photometric Stereo Revisited [100.97116470055273]
  Multi-view photometric stereo (MVPS) is a preferred method for detailed and precise 3D acquisition of an object from images. We present a simple, practical approach to MVPS, which works well for isotropic as well as other object material types such as anisotropic and glossy. The proposed approach shows state-of-the-art results when tested extensively on several benchmark datasets.
  arXiv  Detail & Related papers  (2022-10-14T09:46:15Z)
• Effective Invertible Arbitrary Image Rescaling [77.46732646918936]
  Invertible Neural Networks (INN) are able to increase upscaling accuracy significantly by optimizing the downscaling and upscaling cycle jointly. A simple and effective invertible arbitrary rescaling network (IARN) is proposed to achieve arbitrary image rescaling by training only one model in this work. It is shown to achieve a state-of-the-art (SOTA) performance in bidirectional arbitrary rescaling without compromising perceptual quality in LR outputs.
  arXiv  Detail & Related papers  (2022-09-26T22:22:30Z)
• MAT: Mask-Aware Transformer for Large Hole Image Inpainting [79.67039090195527]
  We present a novel model for large hole inpainting, which unifies the merits of transformers and convolutions. Experiments demonstrate the state-of-the-art performance of the new model on multiple benchmark datasets.
  arXiv  Detail & Related papers  (2022-03-29T06:36:17Z)
• Image-specific Convolutional Kernel Modulation for Single Image Super-resolution [85.09413241502209]
  In this issue, we propose a novel image-specific convolutional modulation kernel (IKM) We exploit the global contextual information of image or feature to generate an attention weight for adaptively modulating the convolutional kernels. Experiments on single image super-resolution show that the proposed methods achieve superior performances over state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-11-16T11:05:10Z)
• Point-Based Neural Rendering with Per-View Optimization [5.306819482496464]
  We introduce a general approach that is with MVS, but allows further optimization of scene properties in the space of input views. A key element of our approach is our new differentiable point-based pipeline. We use these elements together in our neural splatting, that outperforms all previous methods both in quality and speed in almost all scenes we tested.
  arXiv  Detail & Related papers  (2021-09-06T11:19:31Z)
• Conditional Sequential Modulation for Efficient Global Image Retouching [45.99310982782054]
  Photo retouching aims at enhancing the aesthetic visual quality of images that suffer from photographic defects such as over/under exposure, poor contrast, inharmonious saturation. In this paper, we investigate some commonly-used retouching operations and mathematically find that these pixel-independent operations can be approximated or formulated by multi-layer perceptrons (MLPs) We propose an extremely light-weight framework - Sequential Retouching Network (CSRNet) - for efficient global image retouching.
  arXiv  Detail & Related papers  (2020-09-22T08:32:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.