Photonic Fabric Platform for AI Accelerators

• URL: http://arxiv.org/abs/2507.14000v3
• Date: Wed, 23 Jul 2025 15:07:06 GMT
• Title: Photonic Fabric Platform for AI Accelerators
• Authors: Jing Ding, Trung Diep,
• Abstract summary: Photonic Fabric Appliance (PFA) is a photonic-enabled switch and memory subsystem that delivers low latency, high bandwidth, and low per-bit energy.<n>PFA offers up to 32 TB of shared memory alongside 115 Tbps of all-to-all digital switching.<n>Replaces a local stack on an XPU with a chiplet that connects to the Photonic Fabric.
• Score: 0.844067337858849
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: This paper presents the Photonic FabricTM and the Photonic Fabric ApplianceTM (PFA), a photonic-enabled switch and memory subsystem that delivers low latency, high bandwidth, and low per-bit energy. By integrating high-bandwidth HBM3E memory, an on-module photonic switch, and external DDR5 in a 2.5D electro-optical system-in-package, the PFA offers up to 32 TB of shared memory alongside 115 Tbps of all-to-all digital switching. The Photonic FabricTM enables distributed AI training and inference to execute parallelism strategies more efficiently. The Photonic Fabric removes the silicon beachfront constraint that limits the fixed memory-to-compute ratio observed in virtually all current XPU accelerator designs. Replacing a local HBM stack on an XPU with a chiplet that connects to the Photonic Fabric increases its memory capacity and correspondingly its memory bandwidth by offering a flexible path to scaling well beyond the limitations of on-package HBM alone. We introduce CelestiSim, a lightweight analytical simulator validated on NVIDIA H100 and H200 systems. It is used to evaluate the performance of LLM reference and energy savings on PFA, without any significant change to the GPU core design. With the PFA, the simulation results show that up to 3.66x throughput and 1.40x latency improvements in LLM inference at 405B parameters, up to 7.04x throughput and 1.41x latency improvements at 1T parameters, and 60-90% energy savings in data movement for heavy collective operations in all LLM training scenarios. While these results are shown for NVIDIA GPUs, they can be applied similarly to other AI accelerator designs (XPUs) that share the same fundamental limitation of fixed memory to compute.

Related papers

• COMPASS: A Compiler Framework for Resource-Constrained Crossbar-Array Based In-Memory Deep Learning Accelerators [6.172271429579593]
  We propose a compiler framework for resource-constrained crossbar-based processing-in-memory (PIM) deep neural network (DNN) accelerators.<n>We propose an algorithm to determine the optimal partitioning that divides the layers so that each partition can be accelerated on chip.
  arXiv  Detail & Related papers  (2025-01-12T11:31:25Z)
• LiVOS: Light Video Object Segmentation with Gated Linear Matching [116.58237547253935]
  LiVOS is a lightweight memory network that employs linear matching via linear attention. For longer and higher-resolution videos, it matched STM-based methods with 53% less GPU memory and supports 4096p inference on a 32G consumer-grade GPU.
  arXiv  Detail & Related papers  (2024-11-05T05:36:17Z)
• FAMOUS: Flexible Accelerator for the Attention Mechanism of Transformer on UltraScale+ FPGAs [0.0]
  Transformer neural networks (TNNs) are being applied across a widening range of application domains, including natural language processing (NLP), machine translation, and computer vision (CV)<n>This paper proposes textitFAMOUS, a flexible hardware accelerator for dense multi-head attention computation of TNNs on field-programmable gate arrays (FPGAs)<n>It is optimized for high utilization of processing elements and on-chip memories to improve parallelism and reduce latency.
  arXiv  Detail & Related papers  (2024-09-21T05:25:46Z)
• MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Models [58.3342517278868]
  This paper describes the design of Mixed-precision AutoRegressive LINear kernels. It shows that batchsizes up to 16-32 can be supported with close to maximum ($4times$) quantization speedup. MarLIN accomplishes this via a combination of techniques, such as asynchronous memory access, complex task scheduling and pipelining.
  arXiv  Detail & Related papers  (2024-08-21T16:10:41Z)
• Fast Matrix Multiplications for Lookup Table-Quantized LLMs [58.11584672945781]
  FLUTE is a flexible lookup table engine for LUT-quantized LLMs.<n>At batch sizes 32 and quantization group size of 128, the FLUTE kernel can be 2-4x faster than existing GEMM kernels.
  arXiv  Detail & Related papers  (2024-07-15T17:55:42Z)
• Mamba-based Light Field Super-Resolution with Efficient Subspace Scanning [48.99361249764921]
  Transformer-based methods have demonstrated impressive performance in 4D light field (LF) super-resolution. However, their quadratic complexity hinders the efficient processing of high resolution 4D inputs. We propose a Mamba-based Light Field Super-Resolution method, named MLFSR, by designing an efficient subspace scanning strategy.
  arXiv  Detail & Related papers  (2024-06-23T11:28:08Z)
• A Cost-Efficient FPGA Implementation of Tiny Transformer Model using Neural ODE [0.8403582577557918]
  Transformer has been adopted to image recognition tasks and shown to outperform CNNs and RNNs while it suffers from high training cost and computational complexity. We propose a lightweight hybrid model which uses Neural ODE as a backbone instead of ResNet. The proposed model is deployed on a modest-sized FPGA device for edge computing.
  arXiv  Detail & Related papers  (2024-01-05T09:32:39Z)
• ACNPU: A 4.75TOPS/W 1080P@30FPS Super Resolution Accelerator with Decoupled Asymmetric Convolution [0.0502254944841629]
  Deep learning-driven superresolution (SR) outperforms traditional techniques but also faces the challenge of high complexity and memory bandwidth. This paper proposes an energy-efficient SR accelerator, ACNPU, to tackle this challenge. The ACNPU enhances image quality by 0.34dB with a 27-layer model, but needs 36% less complexity than FSRCNN.
  arXiv  Detail & Related papers  (2023-08-30T07:23:32Z)
• RAMP: A Flat Nanosecond Optical Network and MPI Operations for Distributed Deep Learning Systems [68.8204255655161]
  We introduce a near-exascale, full-bisection bandwidth, all-to-all, single-hop, all-optical network architecture with nanosecond reconfiguration called RAMP. RAMP supports large-scale distributed and parallel computing systems (12.8Tbps per node for up to 65,536 nodes.
  arXiv  Detail & Related papers  (2022-11-28T11:24:51Z)
• Interleaving: Modular architectures for fault-tolerant photonic quantum computing [50.591267188664666]
  Photonic fusion-based quantum computing (FBQC) uses low-loss photonic delays. We present a modular architecture for FBQC in which these components are combined to form "interleaving modules" Exploiting the multiplicative power of delays, each module can add thousands of physical qubits to the computational Hilbert space.
  arXiv  Detail & Related papers  (2021-03-15T18:00:06Z)
• Training Large Neural Networks with Constant Memory using a New Execution Algorithm [0.5424799109837065]
  We introduce a new relay-style execution technique called L2L (layer-to-layer) L2L is able to fit models up to 50 Billion parameters on a machine with a single 16GB V100 and 512GB CPU memory.
  arXiv  Detail & Related papers  (2020-02-13T17:29:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.