ExecuTorch -- A Unified PyTorch Solution to Run AI Models On-Device

• URL: http://arxiv.org/abs/2605.08195v1
• Date: Tue, 05 May 2026 18:59:31 GMT
• Title: ExecuTorch -- A Unified PyTorch Solution to Run AI Models On-Device
• Abstract summary: ExecuTorch is a unified PyTorch-native deployment framework for edge AI.<n>It enables seamless deployment of machine learning models across heterogeneous compute environments.
• Score: 9.328077533565924
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Local execution of AI on edge devices is important for low latency and offline operation. However, deploying models on diverse hardware remains fragmented, often requiring model conversion or complete reimplementation outside the PyTorch ecosystem where the model was originally authored. We introduce ExecuTorch, a unified PyTorch-native deployment framework for edge AI. ExecuTorch enables seamless deployment of machine learning models across heterogeneous compute environments. It scales from embedded microcontrollers to complex system-on-chips (SoCs) with dedicated accelerators, powering devices ranging from wearables and smartphones to large compute clusters. ExecuTorch preserves PyTorch semantics while allowing customization, support for optimizations like quantization, and pluggable execution "backends". These features together enable fast experimentation, allowing researchers to validate deployment behavior entirely within PyTorch, bridging the gap between research and production.

Related papers

• Device-Conditioned Neural Architecture Search for Efficient Robotic Manipulation [16.017058636822025]
  We propose a unified framework named textbfDevice-textbfConditioned textbfQuantization-textbfFor-textbfAll (DC-QFA)<n>DC-QFA amortizes deployment effort with the device-conditioned quantization-aware training and hardware-constrained architecture search.
  arXiv  Detail & Related papers  (2026-04-11T11:36:16Z)
• UNIFERENCE: A Discrete Event Simulation Framework for Developing Distributed AI Models [0.42015796373418257]
  UNI is a discrete-event simulation framework for developing, benchmarking, and deploying distributed AI models.<n>It integrates seamlessly with PyTorch Distributed, enabling the same transition from simulation to real deployment.
  arXiv  Detail & Related papers  (2026-03-27T14:35:02Z)
• RAGdb: A Zero-Dependency, Embeddable Architecture for Multimodal Retrieval-Augmented Generation on the Edge [0.0]
  Retrieval-Augmented Generation (RAG) has established itself as the standard paradigm for grounding Large Language Models (LLMs) in domain-specific, up-to-date data.<n>RAGdb consolidates automated multimodal ingestion, ONNX-based extraction, and hybrid vector retrieval into a single, portable container.<n>System reduces disk footprint by approximately 99.5% compared to standard Docker-based RAG stacks.
  arXiv  Detail & Related papers  (2025-12-09T15:12:13Z)
• Elastic ViTs from Pretrained Models without Retraining [74.5386166956142]
  Vision foundation models achieve remarkable performance but are only available in a limited set of pre-determined sizes.<n>We introduce SnapViT: Single-shot network approximation for pruned Vision Transformers.<n>Our approach efficiently combines gradient information with cross-network structure correlations, approximated via an evolutionary algorithm.
  arXiv  Detail & Related papers  (2025-10-20T16:15:03Z)
• PIPer: On-Device Environment Setup via Online Reinforcement Learning [74.52354321028493]
  Automated environment setup methods could assist developers by providing fully configured environments for arbitrary repositories without manual effort.<n>Recent studies reveal that even state-of-the-art Large Language Models (LLMs) achieve limited success in automating this task.<n>We combine supervised fine-tuning for generating correct scripts and Reinforcement Learning with Verifiable Rewards (RLVR) to adapt it to the task of environment setup.<n>On EnvBench-Python, our method enables Qwen3-8B (a model runnable on consumer hardware) to perform on par with larger models-Qwen3-32B and GPT-4
  arXiv  Detail & Related papers  (2025-09-29T20:03:05Z)
• PyLate: Flexible Training and Retrieval for Late Interaction Models [3.737581531719168]
  We introduce PyLate, a library built on top of Sentence Transformers to support multi-vector architectures.<n>By offering multi-vector-specific features such as efficient indexes, PyLate aims to accelerate research and real-world application of late interaction models.<n> PyLate has already enabled the development of state-of-the-art models, including GTE-ModernColBERT and Reason-ModernColBERT.
  arXiv  Detail & Related papers  (2025-08-05T15:23:40Z)
• NNTile: a machine learning framework capable of training extremely large GPT language models on a single node [83.9328245724548]
  NNTile is based on a StarPU library, which implements task-based parallelism and schedules all provided tasks onto all available processing units.<n>It means that a particular operation, necessary to train a large neural network, can be performed on any of the CPU cores or GPU devices.
  arXiv  Detail & Related papers  (2025-04-17T16:22:32Z)
• STAMP: Scalable Task And Model-agnostic Collaborative Perception [24.890993164334766]
  STAMP is a task- and model-agnostic, collaborative perception pipeline for heterogeneous agents.<n>It minimizes computational overhead, enhances scalability, and preserves model security.<n>As a first-of-its-kind framework, STAMP aims to advance research in scalable and secure mobility systems towards Level 5 autonomy.
  arXiv  Detail & Related papers  (2025-01-24T16:27:28Z)
• An Adaptive Device-Edge Co-Inference Framework Based on Soft Actor-Critic [72.35307086274912]
  High-dimension parameter model and large-scale mathematical calculation restrict execution efficiency, especially for Internet of Things (IoT) devices. We propose a new Deep Reinforcement Learning (DRL)-Soft Actor Critic for discrete (SAC-d), which generates the emphexit point, emphexit point, and emphcompressing bits by soft policy iterations. Based on the latency and accuracy aware reward design, such an computation can well adapt to the complex environment like dynamic wireless channel and arbitrary processing, and is capable of supporting the 5G URL
  arXiv  Detail & Related papers  (2022-01-09T09:31:50Z)
• PnP-DETR: Towards Efficient Visual Analysis with Transformers [146.55679348493587]
  Recently, DETR pioneered the solution vision tasks with transformers, it directly translates the image feature map into the object result. Recent transformer-based image recognition model andTT show consistent efficiency gain.
  arXiv  Detail & Related papers  (2021-09-15T01:10:30Z)
• SensiX++: Bringing MLOPs and Multi-tenant Model Serving to Sensory Edge Devices [69.1412199244903]
  We present a multi-tenant runtime for adaptive model execution with integrated MLOps on edge devices, e.g., a camera, a microphone, or IoT sensors. S SensiX++ operates on two fundamental principles - highly modular componentisation to externalise data operations with clear abstractions and document-centric manifestation for system-wide orchestration. We report on the overall throughput and quantified benefits of various automation components of SensiX++ and demonstrate its efficacy to significantly reduce operational complexity and lower the effort to deploy, upgrade, reconfigure and serve embedded models on edge devices.
  arXiv  Detail & Related papers  (2021-09-08T22:06:16Z)
• PyChain: A Fully Parallelized PyTorch Implementation of LF-MMI for End-to-End ASR [65.20342293605472]
  PyChain is an implementation of end-to-end lattice-free maximum mutual information (LF-MMI) training for the so-called emphchain models in the Kaldi automatic speech recognition (ASR) toolkit. Unlike other PyTorch and Kaldi based ASR toolkits, PyChain is designed to be as flexible and light-weight as possible.
  arXiv  Detail & Related papers  (2020-05-20T02:10:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.