Event Tensor: A Unified Abstraction for Compiling Dynamic Megakernel

• URL: http://arxiv.org/abs/2604.13327v2
• Date: Tue, 21 Apr 2026 00:31:44 GMT
• Title: Event Tensor: A Unified Abstraction for Compiling Dynamic Megakernel
• Abstract summary: Event Compiler (ETC) applies static and dynamic scheduling to generate high-performance persistent kernels.<n>ETC achieves state-of-the-art serving latency while significantly reducing warmup overhead.
• Score: 17.931727429220874
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Modern GPU workloads, especially large language model (LLM) inference, suffer from kernel launch overheads and coarse synchronization that limit inter-kernel parallelism. Recent megakernel techniques fuse multiple operators into a single persistent kernel to eliminate launch gaps and expose inter-kernel parallelism, but struggle to handle dynamic shapes and data-dependent computation in real workloads. We present Event Tensor, a unified compiler abstraction for dynamic megakernels. Event Tensor encodes dependencies between tiled tasks, and enables first-class support for both shape and data-dependent dynamism. Built atop this abstraction, our Event Tensor Compiler (ETC) applies static and dynamic scheduling transformations to generate high-performance persistent kernels. Evaluations show that ETC achieves state-of-the-art LLM serving latency while significantly reducing system warmup overhead.

Related papers

• Partition-Aware Scheduling for Mobile Heterogeneous Inference Co-Execution [1.3356260369011272]
  We consider the problem of partition-aware DAG scheduling for mobile heterogeneous inference.<n>We propose an online iterative search framework, which decomposes large DAGs into stages, focuses search on critical operators, and uses latency predictors to estimate partitioned execution.
  arXiv  Detail & Related papers  (2026-09-13T00:44:37Z)
• Accelerating Unified Multimodal Models with Core-Expansion Routing and Unified Computation Scheduling [61.28599393707943]
  Unified multimodal models jointly support understanding and generation, but incur substantial redundant computation across tokens, layers, and generation timesteps.<n>We propose a task-shared core scorer and progress-conditioned generation expansions, optimized through generation decomposition and cross-task core alignment.<n> Experiments on two representative UMM architectures demonstrate consistent quality--efficiency improvements across both tasks, retaining 98.03% of dense understanding performance with a 1.93times$ end-to-end inference speedup.
  arXiv  Detail & Related papers  (2026-08-29T14:27:55Z)
• DiscreteRTC: Discrete Diffusion Policies are Natural Asynchronous Executors [57.944744187489185]
  We propose DiscreteRTC, which replaces external corrections with native unmasking.<n>DiscreteRTC is simpler to implement with 0 lines of code for async inpainting, faster at inference with only 0.7x computation compared with generating actions from scratch, and better at execution with 50% higher success rate in real-world dynamic pick task compared with flow-based RTC.
  arXiv  Detail & Related papers  (2026-04-27T23:04:03Z)
• Decoupled DiLoCo for Resilient Distributed Pre-training [19.201912399584813]
  We introduce Decoupled DiLoCo, an evolution of the DiLoCo framework designed to break the lock-step synchronization barrier.<n>We achieve significantly improved training efficiency in failure-prone environments with millions of simulated chips with strictly zero global downtime.
  arXiv  Detail & Related papers  (2026-04-23T08:45:38Z)
• DexWorldModel: Causal Latent World Modeling towards Automated Learning of Embodied Tasks [54.32016216994156]
  We introduce the Causal Latent World Model (CLWM), which employs DINOv3 features as generative targets to disentangle interaction semantics from visual noise.<n>To overcome memory scaling, CLWM features a Dual-State Test-Time Training (TTT) Memory that guarantees a strict $mathcalO(1)$ footprint for long-horizon tasks.<n>We present EmbodiChain, an online framework that establishes the Efficiency Law by injecting an infinite flow of physics-grounded trajectories during training.
  arXiv  Detail & Related papers  (2026-04-13T03:19:36Z)
• AdaFuse: Accelerating Dynamic Adapter Inference via Token-Level Pre-Gating and Fused Kernel Optimization [84.25316984309725]
  Integration of dynamic, sparse structures with parameter-efficient adapters (e.g., LoRA) is a powerful technique for enhancing Large Language Models (LLMs)<n>Despite minimal increases in computational load, the computation latency often skyrockets, leading to decoding speeds slowing by over 2.5 times.<n>AdaFuse is a framework built on a tight co-design between the algorithm and the underlying hardware system to enable efficient dynamic adapter execution.
  arXiv  Detail & Related papers  (2026-03-12T12:46:42Z)
• Analyzing Latency Hiding and Parallelism in an MLIR-based AI Kernel Compiler [0.0]
  AI kernel compilation for edge devices depends on the compiler's ability to exploit parallelism and hide memory latency.<n>This paper reports a benchmark methodology and corresponding results for three compiler-controlled mechanisms in an MLIR-based compilation pipeline.
  arXiv  Detail & Related papers  (2026-02-22T19:14:23Z)
• Mirage Persistent Kernel: A Compiler and Runtime for Mega-Kernelizing Tensor Programs [17.461191811780722]
  Mirage Persistent Kernel (MPK) is the first compiler and runtime system that automatically transforms multi-GPU model inference into a single high-performance megakernel.<n>MPK introduces an SM-level graph representation that captures data dependencies at the granularity of individual streaming multiprocessors.<n>MPK significantly outperforms existing kernel-per-operator serving systems by reducing end-to-end latency by up to 1.7x.
  arXiv  Detail & Related papers  (2025-12-22T14:18:20Z)
• Streaming Tensor Program: A streaming abstraction for dynamic parallelism [3.2194902146668127]
  Streaming Program (STeP) is a new streaming abstraction that enables dynamic tensor workloads to run efficiently on spatial dataflow accelerators.<n> STeP introduces flexible routing operators, an explicit memory hierarchy, and symbolic shape semantics that expose dynamic data rates and tensor dimensions.<n>These capabilities unlock new optimizations-dynamic tiling, dynamic parallelization, and configuration time-multiplexing-that adapt to dynamic behaviors while preserving dataflow efficiency.
  arXiv  Detail & Related papers  (2025-11-11T02:49:10Z)
• Eliminating Multi-GPU Performance Taxes: A Systems Approach to Efficient Distributed LLMs [61.953548065938385]
  We introduce the ''Three Taxes'' (Bulk Synchronous, Inter- Kernel Data Locality, and Kernel Launch Overhead) as an analytical framework.<n>We propose moving beyond the rigid BSP model to address key inefficiencies in distributed GPU execution.<n>We observe a 10-20% speedup in end-to-end latency over BSP-based approaches.
  arXiv  Detail & Related papers  (2025-11-04T01:15:44Z)
• CSGO: Generalized Optimization for Cold Start in Wireless Collaborative Edge LLM Systems [62.24576366776727]
  We propose a latency-aware scheduling framework to minimize total inference latency.<n>We show that the proposed method significantly reduces cold-start latency compared to baseline strategies.
  arXiv  Detail & Related papers  (2025-08-15T07:49:22Z)
• TileLang: A Composable Tiled Programming Model for AI Systems [17.240134151647187]
  We present TileLang, a generalized tiled programming model for more efficient AI programming.<n> TileLang decouples scheduling space (thread binding, layout, tensorize and pipeline) from dataflow, and encapsulated them as a set of customization annotations and primitives.<n>We conduct comprehensive experiments on commonly-used devices, across numerous experiments, our evaluation shows that TileLang can achieve state-of-the-art performance in key kernels.
  arXiv  Detail & Related papers  (2025-04-24T14:08:49Z)
• Latency-aware Unified Dynamic Networks for Efficient Image Recognition [72.8951331472913]
  LAUDNet is a framework to bridge the theoretical and practical efficiency gap in dynamic networks. It integrates three primary dynamic paradigms-spatially adaptive computation, dynamic layer skipping, and dynamic channel skipping. It can notably reduce the latency of models like ResNet by over 50% on platforms such as V100,3090, and TX2 GPUs.
  arXiv  Detail & Related papers  (2023-08-30T10:57:41Z)
• Harnessing Deep Learning and HPC Kernels via High-Level Loop and Tensor Abstractions on CPU Architectures [67.47328776279204]
  This work introduces a framework to develop efficient, portable Deep Learning and High Performance Computing kernels. We decompose the kernel development in two steps: 1) Expressing the computational core using Processing Primitives (TPPs) and 2) Expressing the logical loops around TPPs in a high-level, declarative fashion. We demonstrate the efficacy of our approach using standalone kernels and end-to-end workloads that outperform state-of-the-art implementations on diverse CPU platforms.
  arXiv  Detail & Related papers  (2023-04-25T05:04:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.