Accelerating Diffusion Models for Generative AI Applications with Silicon Photonics

• URL: http://arxiv.org/abs/2603.07626v1
• Date: Sun, 08 Mar 2026 13:27:14 GMT
• Title: Accelerating Diffusion Models for Generative AI Applications with Silicon Photonics
• Abstract summary: Diffusion models have revolutionized generative AI, with their inherent capacity to generate highly realistic state-of-the-art synthetic data.<n>These models employ an iterative denoising process over computationally intensive layers such as UNets and attention mechanisms.<n>We present a novel silicon photonics-based accelerator for diffusion models.
• Score: 2.102846336724103
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Diffusion models have revolutionized generative AI, with their inherent capacity to generate highly realistic state-of-the-art synthetic data. However, these models employ an iterative denoising process over computationally intensive layers such as UNets and attention mechanisms. This results in high inference energy on conventional electronic platforms, and thus, there is an emerging need to accelerate these models in a sustainable manner. To address this challenge, we present a novel silicon photonics-based accelerator for diffusion models. Experimental evaluations demonstrate that our photonic accelerator achieves at least 3x better energy efficiency and 5.5x throughput improvement compared to state-of-the-art diffusion model accelerators.

Related papers

• teasr: training-efficient any-step diffusion transformer for real-world image super-resolution [10.733502031936958]
  TEASR is a training-efficient any-step diffusion framework for Real-ISR.<n>Our key idea is to perform self-adversarial distillation within a single diffusion model.<n>We propose a timestep-aware rectification strategy that stabilizes one-step generation across noise levels.
  arXiv  Detail & Related papers  (2026-06-15T04:02:37Z)
• Energy Scaling Laws for Diffusion Models: Quantifying Compute and Carbon Emissions in Image Generation [50.21021246855702]
  We propose an adaptation of Kaplan scaling laws to predict GPU energy consumption for diffusion models based on computational complexity (FLOPs)<n>Our approach decomposes diffusion model inference into text encoding, iterative denoising, and decoding components, with the hypothesis that denoising operations dominate energy consumption due to their repeated execution across multiple inference steps.<n>Our results validate the compute-bound nature of diffusion inference and provide a foundation for sustainable AI deployment planning and carbon footprint estimation.
  arXiv  Detail & Related papers  (2025-11-21T08:12:47Z)
• CHORDS: Diffusion Sampling Accelerator with Multi-core Hierarchical ODE Solvers [72.23291099555459]
  Diffusion-based generative models have become dominant generators of high-fidelity images and videos but remain limited by their computationally expensive inference procedures.<n>This paper explores a general, training-free, and model-agnostic acceleration strategy via multi-core parallelism.<n>ChoRDS significantly accelerates sampling across diverse large-scale image and video diffusion models, yielding up to 2.1x speedup with four cores, improving by 50% over baselines, and 2.9x speedup with eight cores, all without quality degradation.
  arXiv  Detail & Related papers  (2025-07-21T05:48:47Z)
• Iterative Distillation for Reward-Guided Fine-Tuning of Diffusion Models in Biomolecular Design [58.8094854658848]
  We address the problem of fine-tuning diffusion models for reward-guided generation in biomolecular design.<n>We propose an iterative distillation-based fine-tuning framework that enables diffusion models to optimize for arbitrary reward functions.<n>Our off-policy formulation, combined with KL divergence minimization, enhances training stability and sample efficiency compared to existing RL-based methods.
  arXiv  Detail & Related papers  (2025-07-01T05:55:28Z)
• SQ-DM: Accelerating Diffusion Models with Aggressive Quantization and Temporal Sparsity [4.6126713437495495]
  We present a novel diffusion model accelerator featuring a mixed-precision dense-sparse architecture, channel-last address mapping, and a time-step-aware sparsity detector.<n>Our accelerator achieves 6.91x speed-up and 51.5% energy reduction compared to traditional dense accelerators.
  arXiv  Detail & Related papers  (2025-01-26T08:34:26Z)
• Energy-Based Diffusion Language Models for Text Generation [126.23425882687195]
  Energy-based Diffusion Language Model (EDLM) is an energy-based model operating at the full sequence level for each diffusion step.<n>Our framework offers a 1.3$times$ sampling speedup over existing diffusion models.
  arXiv  Detail & Related papers  (2024-10-28T17:25:56Z)
• Taming Diffusion Prior for Image Super-Resolution with Domain Shift SDEs [36.65594293655289]
  DoSSR is a Domain Shift diffusion-based SR model that capitalizes on the generative powers of pretrained diffusion models.<n>At the core of our approach is a domain shift equation that integrates seamlessly with existing diffusion models.<n>Our proposed method achieves state-of-the-art performance on synthetic and real-world datasets, while notably requiring only 5 sampling steps.
  arXiv  Detail & Related papers  (2024-09-26T12:16:11Z)
• Diff-PIC: Revolutionizing Particle-In-Cell Nuclear Fusion Simulation with Diffusion Models [38.46100610494588]
  Nuclear fusion, generally seen as an ultimate solution, has been the focus of intensive research for nearly a century. Recent advancements in Inertial Confinement Fusion have drawn significant attention to fusion research. Laser-Plasma Interaction is critical for ensuring fusion stability and efficiency.
  arXiv  Detail & Related papers  (2024-08-03T19:42:31Z)
• Generative Diffusion Models for Fast Simulations of Particle Collisions at CERN [3.2686289567336235]
  In High Energy Physics simulations play a crucial role in unraveling the complexities of particle collision experiments within CERN's Large Hadron Collider. Recent advancements highlight the efficacy of diffusion models as state-of-the-art generative machine learning methods. We present the first simulation for Zero Degree Calorimeter (ZDC) at the ALICE experiment based on diffusion models, achieving the highest fidelity compared to existing baselines.
  arXiv  Detail & Related papers  (2024-06-05T13:11:53Z)
• Distilling Diffusion Models into Conditional GANs [90.76040478677609]
  We distill a complex multistep diffusion model into a single-step conditional GAN student model. For efficient regression loss, we propose E-LatentLPIPS, a perceptual loss operating directly in diffusion model's latent space. We demonstrate that our one-step generator outperforms cutting-edge one-step diffusion distillation models.
  arXiv  Detail & Related papers  (2024-05-09T17:59:40Z)
• An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization [59.63880337156392]
  Diffusion models have achieved tremendous success in computer vision, audio, reinforcement learning, and computational biology. Despite the significant empirical success, theory of diffusion models is very limited. This paper provides a well-rounded theoretical exposure for stimulating forward-looking theories and methods of diffusion models.
  arXiv  Detail & Related papers  (2024-04-11T14:07:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.