AlphaVAE: Unified End-to-End RGBA Image Reconstruction and Generation with Alpha-Aware Representation Learning

• URL: http://arxiv.org/abs/2507.09308v1
• Date: Sat, 12 Jul 2025 14:53:42 GMT
• Title: AlphaVAE: Unified End-to-End RGBA Image Reconstruction and Generation with Alpha-Aware Representation Learning
• Abstract summary: We propose ALPHA, the first comprehensive RGBA benchmark that adapts standard RGB metrics to four-channel images via alpha blending over canonical backgrounds.<n>We further introduce ALPHAVAE, a unified end-to-end RGBA VAE that extends a pretrained RGB VAE by incorporating a dedicated alpha channel.<n>Our RGBA VAE, trained on only 8K images in contrast to 1M used by prior methods, achieves a +4.9 dB improvement in PSNR and a +3.2% increase in SSIM over LayerDiffuse in reconstruction.
• Score: 32.798523698352916
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in latent diffusion models have achieved remarkable results in high-fidelity RGB image synthesis by leveraging pretrained VAEs to compress and reconstruct pixel data at low computational cost. However, the generation of transparent or layered content (RGBA image) remains largely unexplored, due to the lack of large-scale benchmarks. In this work, we propose ALPHA, the first comprehensive RGBA benchmark that adapts standard RGB metrics to four-channel images via alpha blending over canonical backgrounds. We further introduce ALPHAVAE, a unified end-to-end RGBA VAE that extends a pretrained RGB VAE by incorporating a dedicated alpha channel. The model is trained with a composite objective that combines alpha-blended pixel reconstruction, patch-level fidelity, perceptual consistency, and dual KL divergence constraints to ensure latent fidelity across both RGB and alpha representations. Our RGBA VAE, trained on only 8K images in contrast to 1M used by prior methods, achieves a +4.9 dB improvement in PSNR and a +3.2% increase in SSIM over LayerDiffuse in reconstruction. It also enables superior transparent image generation when fine-tuned within a latent diffusion framework. Our code, data, and models are released on https://github.com/o0o0o00o0/AlphaVAE for reproducibility.

Related papers

• Alpha as an Efficiency Signal: Visibility-Routed RGBA Image-to-Video Generation [10.796062926739511]
  RGBA videos combine RGB appearance with an alpha channel, enabling animated assets to be applied across arbitrary backgrounds.<n>Most existing RGBA video datasets are dominated by photorealistic content.<n>Traditional generate-then-matte pipelines estimate alpha only after RGB synthesis.
  arXiv  Detail & Related papers  (2026-08-10T09:35:36Z)
• TransText: Alpha-as-RGB Representation for Transparent Text Animation [35.1650602838868]
  We introduce the first method, for adapting image-to-video models to layer-aware text (glyph) animation.<n>We propose TransText, a framework based on a novel Alpha-as-RGB paradigm to jointly model appearance and transparency.<n>Our experiments demonstrate that TransText significantly outperforms baselines, generating coherent, high-fidelity transparent animations.
  arXiv  Detail & Related papers  (2026-03-18T17:16:40Z)
• RAW-Flow: Advancing RGB-to-RAW Image Reconstruction with Deterministic Latent Flow Matching [55.03149221192589]
  We introduce a novel framework named RAW-Flow to bridge the gap between RGB and RAW representations.<n>We also introduce a cross-scale context guidance module that injects hierarchical RGB features into the flow estimation process.<n> RAW-Flow outperforms state-of-the-art approaches both quantitatively and visually.
  arXiv  Detail & Related papers  (2026-01-28T08:27:38Z)
• OmniAlpha: A Sequence-to-Sequence Framework for Unified Multi-Task RGBA Generation [43.93970229518124]
  We propose OmniAlpha, the first unified, multi-task generative framework for sequence-to-sequence RGBA image generation and editing.<n>Our work proves that a unified, multi-task model can learn a superior shared representation for RGBA, paving the way for more powerful, layer-aware generative systems.
  arXiv  Detail & Related papers  (2025-11-25T11:34:51Z)
• Wan-Alpha: High-Quality Text-to-Video Generation with Alpha Channel [14.361698701397545]
  Wan-Alpha is a new framework that generates transparent videos by learning both RGB and alpha channels jointly.<n>Compared with state-of-the-art methods, our model demonstrates superior performance in visual quality, motion realism, and transparency rendering.
  arXiv  Detail & Related papers  (2025-09-29T16:08:21Z)
• Vision-Language Alignment from Compressed Image Representations using 2D Gaussian Splatting [4.2390854432099205]
  Modern vision language pipelines are driven by RGB vision encoders trained on massive image text corpora.<n>These pipelines inherit two structural inefficiencies from the pixel domain: (i) transmitting dense RGB images from edge devices to the cloud is energy intensive and costly, and (ii) patch based tokenization explodes sequence length.<n>We explore 2D Gaussian Splatting (2DGS) as an alternative visual substrate for alignment.
  arXiv  Detail & Related papers  (2025-09-26T17:41:57Z)
• TransPixeler: Advancing Text-to-Video Generation with Transparency [43.6546902960154]
  We introduce TransPixeler, a method to extend pretrained video models for RGBA generation while retaining the original RGB capabilities.<n>Our approach effectively generates diverse and consistent RGBA videos, advancing the possibilities for VFX and interactive content creation.
  arXiv  Detail & Related papers  (2025-01-06T13:32:16Z)
• Enhancing RAW-to-sRGB with Decoupled Style Structure in Fourier Domain [27.1716081216131]
  Current methods ignore the difference between cell phone RAW images and DSLR camera RGB images. We present a novel Neural ISP framework, named FourierISP. This approach breaks the image down into style and structure within the frequency domain, allowing for independent optimization.
  arXiv  Detail & Related papers  (2024-01-04T09:18:31Z)
• Ternary-Type Opacity and Hybrid Odometry for RGB NeRF-SLAM [58.736472371951955]
  We introduce a ternary-type opacity (TT) model, which categorizes points on a ray intersecting a surface into three regions: before, on, and behind the surface. This enables a more accurate rendering of depth, subsequently improving the performance of image warping techniques. Our integrated approach of TT and HO achieves state-of-the-art performance on synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2023-12-20T18:03:17Z)
• StereoISP: Rethinking Image Signal Processing for Dual Camera Systems [4.703692756660711]
  StereoISP employs raw measurements from a stereo camera pair to generate a demosaicked, denoised RGB image. Our preliminary results show an improvement in the PSNR of the reconstructed RGB image by at least 2dB on KITTI 2015.
  arXiv  Detail & Related papers  (2022-11-11T18:34:59Z)
• Pyramidal Attention for Saliency Detection [30.554118525502115]
  This paper exploits only RGB images, estimates depth from RGB, and leverages the intermediate depth features. We employ a pyramidal attention structure to extract multi-level convolutional-transformer features to process initial stage representations. We report significantly improved performance against 21 and 40 state-of-the-art SOD methods on eight RGB and RGB-D datasets.
  arXiv  Detail & Related papers  (2022-04-14T06:57:46Z)
• Semantic-embedded Unsupervised Spectral Reconstruction from Single RGB Images in the Wild [48.44194221801609]
  We propose a new lightweight and end-to-end learning-based framework to tackle this challenge. We progressively spread the differences between input RGB images and re-projected RGB images from recovered HS images via effective camera spectral response function estimation. Our method significantly outperforms state-of-the-art unsupervised methods and even exceeds the latest supervised method under some settings.
  arXiv  Detail & Related papers  (2021-08-15T05:19:44Z)
• UltraSR: Spatial Encoding is a Missing Key for Implicit Image Function-based Arbitrary-Scale Super-Resolution [74.82282301089994]
  In this work, we propose UltraSR, a simple yet effective new network design based on implicit image functions. We show that spatial encoding is indeed a missing key towards the next-stage high-accuracy implicit image function. Our UltraSR sets new state-of-the-art performance on the DIV2K benchmark under all super-resolution scales.
  arXiv  Detail & Related papers  (2021-03-23T17:36:42Z)
• Deep Burst Super-Resolution [165.90445859851448]
  We propose a novel architecture for the burst super-resolution task. Our network takes multiple noisy RAW images as input, and generates a denoised, super-resolved RGB image as output. In order to enable training and evaluation on real-world data, we additionally introduce the BurstSR dataset.
  arXiv  Detail & Related papers  (2021-01-26T18:57:21Z)
• Adversarial Generation of Continuous Images [31.92891885615843]
  In this paper, we propose two novel architectural techniques for building INR-based image decoders. We use them to build a state-of-the-art continuous image GAN. Our proposed INR-GAN architecture improves the performance of continuous image generators by several times.
  arXiv  Detail & Related papers  (2020-11-24T11:06:40Z)
• Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation [59.94819184452694]
  Depth information has proven to be a useful cue in the semantic segmentation of RGBD images for providing a geometric counterpart to the RGB representation. Most existing works simply assume that depth measurements are accurate and well-aligned with the RGB pixels and models the problem as a cross-modal feature fusion. In this paper, we propose a unified and efficient Crossmodality Guided to not only effectively recalibrate RGB feature responses, but also to distill accurate depth information via multiple stages and aggregate the two recalibrated representations alternatively.
  arXiv  Detail & Related papers  (2020-07-17T18:35:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.