Cloud Is Closer Than It Appears: Revisiting the Tradeoffs of Distributed Real-Time Inference

• URL: http://arxiv.org/abs/2605.00005v1
• Date: Tue, 17 Feb 2026 20:44:13 GMT
• Title: Cloud Is Closer Than It Appears: Revisiting the Tradeoffs of Distributed Real-Time Inference
• Abstract summary: Deep neural networks (DNNs) in cyber-physical systems (CPS) enhance perception fidelity, but impose substantial computational demands on execution platforms.<n>Traditional distributed CPS architectures favor on-device inference to avoid network variability and contention-induced delays on remote platforms.<n>In this work, we revisit the assumption that cloud-based inference is intrinsically unsuitable for latency-sensitive control tasks.
• Score: 3.9986079633630975
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The increasing deployment of deep neural networks (DNNs) in cyber-physical systems (CPS) enhances perception fidelity, but imposes substantial computational demands on execution platforms, posing challenges to real-time control deadlines. Traditional distributed CPS architectures typically favor on-device inference to avoid network variability and contention-induced delays on remote platforms. However, this design choice places significant energy and computational demands on the local hardware. In this work, we revisit the assumption that cloud-based inference is intrinsically unsuitable for latency-sensitive control tasks. We demonstrate that, when provisioned with high-throughput compute resources, cloud platforms can effectively amortize network and queueing delays, enabling them to match or surpass on-device performance for real-time decision-making. Specifically, we develop a formal analytical model that characterizes distributed inference latency as a function of the sensing frequency, platform throughput, network delay, and task-specific safety constraints. We instantiate this model in the context of emergency braking for autonomous driving and validate it through extensive simulations using real-time vehicular dynamics. Our empirical results identify concrete conditions under which cloud-based inference adheres to safety margins more reliably than its on-device counterpart. These findings challenge prevailing design strategies and suggest that the cloud is not merely a feasible option, but often the preferred inference location for distributed CPS architectures. In this light, the cloud is not as distant as traditionally perceived; in fact, it is closer than it appears.

Related papers

• Cloud-Edge Collaborative Large Models for Robust Photovoltaic Power Forecasting [10.936533381499016]
  We propose a condition-adaptive cloud-edge collaborative framework *CAPE* for PV forecasting.<n> *CAPE* consists of three main modules: a site-specific expert model for routine predictions, a lightweight edge-side model for enhanced local inference, and a cloud-based large retrieval model.<n> Experiments on two real-world PV datasets demonstrate that *CAPE* achieves superior performance in terms of forecasting accuracy, robustness, routing quality, and system efficiency.
  arXiv  Detail & Related papers  (2026-03-21T20:09:22Z)
• Speculative Policy Orchestration: A Latency-Resilient Framework for Cloud-Robotic Manipulation [6.709418204725671]
  Cloud robotics enables robots to offload high-dimensional motion planning and reasoning to remote servers.<n>For continuous manipulation tasks requiring high-frequency control, network latency and jitter can severely destabilize the system.<n>We propose Speculative Policy Orchestration (SPO), a latency-resilient cloud-edge framework.
  arXiv  Detail & Related papers  (2026-03-19T19:24:14Z)
• HALO: Semantic-Aware Distributed LLM Inference in Lossy Edge Network [50.33808558714122]
  Large language models' (LLMs) inference at the edge can facilitate prompt service responsiveness while protecting user privacy.<n>We propose HALO, a novel framework that can boost the distributed LLM inference in lossy edge network.<n> Experimental results from a Raspberry Pi cluster demonstrate that HALO achieves a 3.41x end-to-end speedup for LLaMA-series LLMs under unreliable network conditions.
  arXiv  Detail & Related papers  (2026-01-16T07:37:23Z)
• Deep Learning to Identify the Spatio-Temporal Cascading Effects of Train Delays in a High-Density Network [7.850219269698452]
  This paper develops and evaluating a novel XGeoAI framework for live, explainable, multi-step train delay forecasting.<n>The core of this work is a two-stage, autoregressive Graph Attention Network (GAT) model, trained on a real-world dataset covering over 40% of the Dutch railway network.<n>To test its viability for live deployment, the model is rigorously evaluated using a sequential, k-step-ahead forecasting protocol.
  arXiv  Detail & Related papers  (2025-10-10T13:03:00Z)
• CSGO: Generalized Optimization for Cold Start in Wireless Collaborative Edge LLM Systems [62.24576366776727]
  We propose a latency-aware scheduling framework to minimize total inference latency.<n>We show that the proposed method significantly reduces cold-start latency compared to baseline strategies.
  arXiv  Detail & Related papers  (2025-08-15T07:49:22Z)
• The Larger the Merrier? Efficient Large AI Model Inference in Wireless Edge Networks [56.37880529653111]
  The demand for large computation model (LAIM) services is driving a paradigm shift from traditional cloud-based inference to edge-based inference for low-latency, privacy-preserving applications.<n>In this paper, we investigate the LAIM-inference scheme, where a pre-trained LAIM is pruned and partitioned into on-device and on-server sub-models for deployment.
  arXiv  Detail & Related papers  (2025-05-14T08:18:55Z)
• Toward a real-time TCP SYN Flood DDoS mitigation using Adaptive Neuro-Fuzzy classifier and SDN Assistance in Fog Computing [0.31318403497744784]
  We propose mitigation of Fog computing-based SYN Flood DDoS attacks using an Adaptive Neuro-Fuzzy Inference System (ANFIS) and Software Defined Networking (SDN) Assistance (FASA) The simulation results show that FASA system outperforms other algorithms in terms of accuracy, precision, recall, and F1-score.
  arXiv  Detail & Related papers  (2023-11-27T08:54:00Z)
• Leveraging Low-Rank and Sparse Recurrent Connectivity for Robust Closed-Loop Control [63.310780486820796]
  We show how a parameterization of recurrent connectivity influences robustness in closed-loop settings. We find that closed-form continuous-time neural networks (CfCs) with fewer parameters can outperform their full-rank, fully-connected counterparts.
  arXiv  Detail & Related papers  (2023-10-05T21:44:18Z)
• Leveraging the Edge and Cloud for V2X-Based Real-Time Object Detection in Autonomous Driving [0.0]
  Environmental perception is a key element of autonomous driving. In this paper, we investigate the best trade-off between detection quality and latency for real-time perception in autonomous vehicles. We show that models with adequate compression can be run in real-time on the cloud while outperforming local detection performance.
  arXiv  Detail & Related papers  (2023-08-09T21:39:10Z)
• Design and Prototyping Distributed CNN Inference Acceleration in Edge Computing [85.74517957717363]
  HALP accelerates inference by designing a seamless collaboration among edge devices (EDs) in Edge Computing. Experiments show that the distributed inference HALP achieves 1.7x inference acceleration for VGG-16. It is shown that the model selection with distributed inference HALP can significantly improve service reliability.
  arXiv  Detail & Related papers  (2022-11-24T19:48:30Z)
• Fluid Batching: Exit-Aware Preemptive Serving of Early-Exit Neural Networks on Edge NPUs [74.83613252825754]
  "smart ecosystems" are being formed where sensing happens concurrently rather than standalone. This is shifting the on-device inference paradigm towards deploying neural processing units (NPUs) at the edge. We propose a novel early-exit scheduling that allows preemption at run time to account for the dynamicity introduced by the arrival and exiting processes.
  arXiv  Detail & Related papers  (2022-09-27T15:04:01Z)
• SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud [13.315410752311768]
  A popular alternative comprises offloading CNN processing to powerful cloud-based servers. SPINN is a distributed inference system that employs synergistic device-cloud together with a progressive inference method. It provides robust operation under uncertain connectivity conditions and significant energy savings compared to cloud-centric execution.
  arXiv  Detail & Related papers  (2020-08-14T15:00:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.