PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloud

• URL: http://arxiv.org/abs/2608.03682v2
• Date: Wed, 05 Aug 2026 05:24:32 GMT
• Title: PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloud
• Abstract summary: PhyAI is a Physical AI inference engine with a single runtime.<n>It runs vision-language-action (VLA) models and world-action models (WAMs) on single or multiple GPU.<n>It achieves 1.40x-4.65x speedups over the official implementations of pi0, pi0.5, GR00T N1.7, and MiniCPM-Robot.
• Score: 19.027069678233655
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Physical AI policies require inference throughout their lifecycle, including model evaluation, cloud reinforcement learning rollout, edge GPU serving, and onboard deployment. Although these settings share the same checkpoint and action semantics, they often rely on separate inference programs. To unify them, we build PhyAI, a Physical AI inference engine with a single runtime that keeps architecture-specific conditioning, solver, cache, and output logic in model adapters while sharing graph execution, kernels, memory management, and parallel services. The same codebase runs vision-language-action (VLA) models and world-action models (WAMs) on single or multiple GPUs across onboard, edge, and cloud deployments. We used the adapter interface to add MiniCPM-Robot on the day of its release. PhyAI achieves 1.40x-4.65x speedups over the official implementations of pi0, pi0.5, GR00T N1.7, and MiniCPM-Robot. On Cosmos3-Nano-Policy-DROID it reduces latency from 2.46 to 1.18 s on eight H20 GPUs (CFG=2, TP=4), a 2.08x speedup. Specialized runtimes remain faster in several configurations, so our goal is one runtime with competitive latency rather than the fastest result in every case. Detailed profiles reveal why different models need different execution policies: on a Hopper-series GPU at batch size one, the pi0.5 action expert accounts for 8.8% of FLOPs but 57.2% of latency; at batch size 32 its share drops to 13.5% and throughput reaches about 100 samples/s. Cosmos3 remains generation-dominated and gains only 14.3% throughput as batch size increases from 1 to 16. We further introduce the control-time Roofline, which distinguishes inference-bound from environment-bound control; the measured pi0.5 points on four LIBERO suites are environment-bound while Cosmos3 stays inference-bound. Code and benchmarks: https://github.com/mingti-org/phyai.

Related papers

• Pre-Compiled Pipeline Shards for Distributed LLM Inference on Intel AI PC Fleets [0.0]
  We show that a handful of AIPCs can serve models beyond the capability of any single one.<n>A four-node deployment of Lunar Lake AI PCs on Intel Tiber Cloud serves a single user at interactive speed.
  arXiv  Detail & Related papers  (2026-08-19T17:33:28Z)
• Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control [0.0]
  We formalize the ready-cohort boundary using fixed-share F, exact offline share P*, local upper bound U, and online achieved share A.<n>A separate mechanism study keeps a GPU-computed binary decision on device instead of returning four bytes to the host and redispatching.
  arXiv  Detail & Related papers  (2026-08-12T14:42:15Z)
• Meganeura: Portable GPU Training and Inference through Vulkan and Metal [0.0]
  Meganeura asks whether one compact native compiler can span both phases on consumer GPU.<n>We compare five matched workloads with PyTorch on NVIDIA and AMD discrete GPUs.<n>The results show that general consumer graphics APIs can support a compact shared train-to-deploy stack at useful, sometimes vendor-competitive performance.
  arXiv  Detail & Related papers  (2026-08-03T00:42:55Z)
• MolmoAct2: Action Reasoning Models for Real-world Deployment [67.6315757474802]
  MolmoAct2 is a fully open action reasoning model built for practical deployment.<n>We introduce MolmoER, a VLM backbone specialized for spatial and embodied reasoning.<n>We release three new datasets spanning low-to-medium cost platforms.
  arXiv  Detail & Related papers  (2026-05-04T17:51:21Z)
• RaMP: Runtime-Aware Megakernel Polymorphism for Mixture-of-Experts [0.22625548856057365]
  We present RaMP, a routing-aware dispatch framework.<n>RaMP delivers 1.22x kernel speedup over static dispatch and 1.30x end-to-end speedup in vLLM serving over Triton, 1.41x over DeepGEMM, and 1.13x over FlashInfer CUTLASS.
  arXiv  Detail & Related papers  (2026-04-28T18:20:12Z)
• Scaling DoRA: High-Rank Adaptation via Factored Norms and Fused Kernels [83.99688944263843]
  Weight-De Low-Rank Adaptation (DoRA) extends LoRA by decoupling weight magnitude from direction.<n>At d_in = 8192 and rank r = 384, a single module's norm requires about 512 MB of transient working memory in bf16.<n>A factored norm decomposes the squared norm into base, cross, and Gram terms computable through O(d_out r + r2) intermediates, eliminating the dense product.
  arXiv  Detail & Related papers  (2026-03-23T17:57:24Z)
• Architecture-Aware LLM Inference Optimization on AMD Instinct GPUs: A Comprehensive Benchmark and Deployment Study [0.0]
  Cross-architecture evaluation of production LLM inference on AMD Instinct MI325X GPUs.<n> benchmarking four models spanning 235B to 1 trillion parameters across three architectural families.
  arXiv  Detail & Related papers  (2026-02-27T13:21:48Z)
• Revati: Transparent GPU-Free Time-Warp Emulation for LLM Serving [1.4573878379102423]
  Revati is a time-warp emulator that enables performance modeling by directly executing real serving system code at simulation-like speed.<n>Revati achieves less than 5% prediction error across multiple models and configurations, while running 5-17x faster than real GPU execution.
  arXiv  Detail & Related papers  (2026-01-01T17:19:58Z)
• StrikeWatch: Wrist-worn Gait Recognition with Compact Time-series Models on Low-power FPGAs [10.946464973530214]
  Proper gait patterns can lead to injuries, particularly without expert feedback.<n>Wrist-worn wearables offer a more practical and non-intrusive alternative.<n>This paper introduces StrikeWatch, a compact wrist-worn system that performs entirely on-device, real-time gait recognition.
  arXiv  Detail & Related papers  (2025-10-14T20:28:31Z)
• 70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float [52.079202872069835]
  Large-scale AI models, such as Large Language Models (LLMs) and Diffusion Models (DMs) have grown rapidly in size.<n>We introduce Dynamic-Length Float (DFloat11), a compression framework that reduces LLM and DM size by 30% while preserving outputs that are bit-for-bit identical to the original model.
  arXiv  Detail & Related papers  (2025-04-15T22:38:38Z)
• MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Models [58.3342517278868]
  This paper describes the design of Mixed-precision AutoRegressive LINear kernels. It shows that batchsizes up to 16-32 can be supported with close to maximum ($4times$) quantization speedup. MarLIN accomplishes this via a combination of techniques, such as asynchronous memory access, complex task scheduling and pipelining.
  arXiv  Detail & Related papers  (2024-08-21T16:10:41Z)
• Latency-aware Unified Dynamic Networks for Efficient Image Recognition [72.8951331472913]
  LAUDNet is a framework to bridge the theoretical and practical efficiency gap in dynamic networks. It integrates three primary dynamic paradigms-spatially adaptive computation, dynamic layer skipping, and dynamic channel skipping. It can notably reduce the latency of models like ResNet by over 50% on platforms such as V100,3090, and TX2 GPUs.
  arXiv  Detail & Related papers  (2023-08-30T10:57:41Z)
• Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipelining [58.10436813430554]
  Mini-batch training of graph neural networks (GNNs) requires a lot of computation and data movement. We argue in favor of performing mini-batch training with neighborhood sampling in a distributed multi-GPU environment. We present a sequence of improvements to mitigate these bottlenecks, including a performance-engineered neighborhood sampler. We also conduct an empirical analysis that supports the use of sampling for inference, showing that test accuracies are not materially compromised.
  arXiv  Detail & Related papers  (2021-10-16T02:41:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.