HyPER-GAN: Hybrid Patch-Based Image-to-Image Translation for Real-Time Photorealism Enhancement

• URL: http://arxiv.org/abs/2603.10604v2
• Date: Tue, 17 Mar 2026 21:17:37 GMT
• Title: HyPER-GAN: Hybrid Patch-Based Image-to-Image Translation for Real-Time Photorealism Enhancement
• Abstract summary: We propose a lightweight image-to-image translation method based on a U-Net-style generator designed for real-time inference.<n>HyPER-GAN outperforms state-of-the-art lightweight paired image-to-image translation methods in terms of inference latency, visual realism, and semantic robustness.
• Score: 2.478819644330144
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative models are widely employed to enhance the photorealism of visual synthetic data for training computer vision algorithms. However, they often introduce visual artifacts that degrade the accuracy of these algorithms and require high computational resources, limiting their applicability in real-time training or evaluation scenarios. In this paper, we propose Hybrid Patch Enhanced Realism Generative Adversarial Network (HyPER-GAN), a lightweight image-to-image translation method based on a U-Net-style generator designed for real-time inference. The model is trained using paired synthetic and photorealism-enhanced images, complemented by a hybrid training strategy that incorporates matched patches from real-world images to improve visual realism and semantic consistency. Experimental results demonstrate that HyPER-GAN outperforms state-of-the-art lightweight paired image-to-image translation methods in terms of inference latency, visual realism, and semantic robustness. Moreover, it is illustrated that the proposed hybrid training strategy indeed improves visual quality and semantic consistency compared to training the model solely with paired synthetic and photorealism-enhanced images. Code and pretrained models are publicly available for download at: https://github.com/stefanos50/HyPER-GAN

Related papers

• A Hybrid Approach for Closing the Sim2real Appearance Gap in Game Engine Synthetic Datasets [0.0]
  Video game engines have been an important source for generating large volumes of visual synthetic datasets.<n>A notable sim2real appearance gap between the synthetic and the real-world images still remains.<n>We investigate the ability of a state-of-the-art image generation and editing diffusion model to enhance the photorealism of synthetic datasets.
  arXiv  Detail & Related papers  (2026-05-04T07:28:01Z)
• Computer vision training dataset generation for robotic environments using Gaussian splatting [0.0]
  This paper introduces a novel pipeline for generating large-scale, highly realistic, and automatically labeled datasets for computer vision tasks in robotic environments.<n>We leverage 3D Gaussian Splatting (3DGS) to create photorealistic representations of the operational environment and objects.<n>A novel, two-pass rendering technique combines the realism of splats with a shadow map generated from proxy meshes.<n> Pixel-perfect segmentation masks are generated automatically and formatted for direct use with object detection models like YOLO.
  arXiv  Detail & Related papers  (2025-12-15T15:00:17Z)
• RealGen: Photorealistic Text-to-Image Generation via Detector-Guided Rewards [53.25632969696776]
  We propose RealGen, a text-to-image framework for photorealistic image generation.<n>Inspired by adversarial generation, RealGen introduces a "Detector Reward" mechanism, which quantifies artifacts and assesses realism.<n>Experiments demonstrate that RealGen significantly outperforms general models like GPT-Image-1 and Qwen-Image, as well as specialized photorealistic models like FLUX-Krea.
  arXiv  Detail & Related papers  (2025-11-29T12:52:26Z)
• HazeCLIP: Towards Language Guided Real-World Image Dehazing [62.4454483961341]
  Existing methods have achieved remarkable performance in image dehazing, particularly on synthetic datasets.<n>This paper introduces HazeCLIP, a language-guided adaptation framework designed to enhance the real-world performance of pre-trained dehazing networks.
  arXiv  Detail & Related papers  (2024-07-18T17:18:25Z)
• Is Synthetic Image Useful for Transfer Learning? An Investigation into Data Generation, Volume, and Utilization [62.157627519792946]
  We introduce a novel framework called bridged transfer, which initially employs synthetic images for fine-tuning a pre-trained model to improve its transferability. We propose dataset style inversion strategy to improve the stylistic alignment between synthetic and real images. Our proposed methods are evaluated across 10 different datasets and 5 distinct models, demonstrating consistent improvements.
  arXiv  Detail & Related papers  (2024-03-28T22:25:05Z)
• Generalizable Synthetic Image Detection via Language-guided Contrastive Learning [22.533225521726116]
  malevolent use of synthetic images, such as the dissemination of fake news or the creation of fake profiles, raises significant concerns regarding the authenticity of images.<n>We propose a simple yet very effective synthetic image detection method via a language-guided contrastive learning.<n>It is shown that our proposed LanguAge-guided SynThEsis Detection (LASTED) model achieves much improved generalizability to unseen image generation models.
  arXiv  Detail & Related papers  (2023-05-23T08:13:27Z)
• Automated Synthetic-to-Real Generalization [142.41531132965585]
  We propose a textitlearning-to-optimize (L2O) strategy to automate the selection of layer-wise learning rates. We demonstrate that the proposed framework can significantly improve the synthetic-to-real generalization performance without seeing and training on real data.
  arXiv  Detail & Related papers  (2020-07-14T10:57:34Z)
• Intrinsic Autoencoders for Joint Neural Rendering and Intrinsic Image Decomposition [67.9464567157846]
  We propose an autoencoder for joint generation of realistic images from synthetic 3D models while simultaneously decomposing real images into their intrinsic shape and appearance properties. Our experiments confirm that a joint treatment of rendering and decomposition is indeed beneficial and that our approach outperforms state-of-the-art image-to-image translation baselines both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2020-06-29T12:53:58Z)
• Deep CG2Real: Synthetic-to-Real Translation via Image Disentanglement [78.58603635621591]
  Training an unpaired synthetic-to-real translation network in image space is severely under-constrained. We propose a semi-supervised approach that operates on the disentangled shading and albedo layers of the image. Our two-stage pipeline first learns to predict accurate shading in a supervised fashion using physically-based renderings as targets.
  arXiv  Detail & Related papers  (2020-03-27T21:45:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.