MLCommons Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces

• URL: http://arxiv.org/abs/2605.11333v3
• Date: Tue, 19 May 2026 03:40:14 GMT
• Title: MLCommons Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces
• Abstract summary: We present Chakra, an open and portable ecosystem for performance benchmarking and co-design.<n>The core component of Chakra is an open and interoperable graph-based representation of distributed AI/ML workloads, called Chakra execution trace(ET)<n>These ETs represent key operations, such as compute, memory, and communication, data and control dependencies, timing, and resource constraints.
• Score: 12.410424372802689
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The fast pace of artificial intelligence~(AI) innovation demands an agile methodology for observation, reproduction and optimization of distributed machine learning~(ML) workload behavior in production AI systems and enables efficient software-hardware~(SW-HW) co-design for future systems. We present Chakra, an open and portable ecosystem for performance benchmarking and co-design. The core component of Chakra is an open and interoperable graph-based representation of distributed AI/ML workloads, called Chakra execution trace~(ET). These ETs represent key operations, such as compute, memory, and communication, data and control dependencies, timing, and resource constraints. Additionally, Chakra includes a complementary set of tools and capabilities to enable the collection, analysis, generation, and adoption of Chakra ETs by a broad range of simulators, emulators, and replay tools. We present analysis of Chakra ETs collected on production AI clusters and demonstrate value via real-world case studies. Chakra has been adopted by MLCommons and has active contributions and engagement across the industry, including but not limited to NVIDIA, AMD, Meta, Keysight, HPE, and Scala, to name a few.

Related papers

• AstraFlow: Dataflow-Oriented Reinforcement Learning for Agentic LLMs [57.86040075371121]
  Reinforcement learning (RL) is increasingly used to improve the reasoning, coding, and tool-use capabilities of large language models.<n>Agentic RL remains prohibitively expensive.<n>We propose AstraFlow, a dataflow-oriented RL system that replaces conventional trainer-centered control with principled component abstractions.
  arXiv  Detail & Related papers  (2026-05-15T03:13:35Z)
• Quantum Integrated High-Performance Computing: Foundations, Architectural Elements and Future Directions [19.209291403001416]
  We propose a visionary architectural framework that unifies CPUs, GPU, FPGAs, and QPUs as first-class heterogeneous resources.<n>A central aspect of our vision is a strong user requests abstraction that exposes heterogeneous resources through a unified job submission interface.<n>We conclude by highlighting open challenges in building scalable, reliable, and programmable quantum-classical infrastructures.
  arXiv  Detail & Related papers  (2026-04-17T04:29:16Z)
• Energy Efficient Software Hardware CoDesign for Machine Learning: From TinyML to Large Language Models [0.4852108302782534]
  This work reviews energy efficient software hardware codesign methods spanning edge inference and training to datacenter-scale LLM serving.<n>We distill common design levers and trade-offs, and highlight recurring gaps including limited cross-platform generalization, large and costly co-design search spaces, and inconsistent benchmarking across workloads and deployment settings.<n>Finally, we outline a hierarchical decomposition perspective that maps optimization strategies to computational roles and supports incremental adaptation, offering practical guidance for building energy and carbon aware ML systems.
  arXiv  Detail & Related papers  (2026-03-24T19:10:42Z)
• Towards Agentic Intelligence for Materials Science [73.4576385477731]
  This survey advances a unique pipeline-centric view that spans from corpus curation and pretraining to goal-conditioned agents interfacing with simulation and experimental platforms.<n>To bridge communities and establish a shared frame of reference, we first present an integrated lens that aligns terminology, evaluation, and workflow stages across AI and materials science.
  arXiv  Detail & Related papers  (2026-01-29T23:48:43Z)
• The AI_INFN Platform: Artificial Intelligence Development in the Cloud [0.0]
  The INFN initiative AI_INFN (Artificial Intelligence at INFN) seeks to promote the use of ML methods across various INFN research scenarios.<n>We will present preliminary benchmarks, functional tests, and case studies, demonstrating both performance and integration outcomes.
  arXiv  Detail & Related papers  (2025-09-26T09:40:51Z)
• VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use [78.29315418819074]
  We introduce VerlTool, a unified and modular framework that addresses limitations through systematic design principles.<n>Our framework formalizes ARLT as multi-turn trajectories with multi-modal observation tokens (text/image/video), extending beyond single-turn RLVR paradigms.<n>The modular plugin architecture enables rapid tool integration requiring only lightweight Python definitions.
  arXiv  Detail & Related papers  (2025-09-01T01:45:18Z)
• A Survey on Cloud-Edge-Terminal Collaborative Intelligence in AIoT Networks [49.90474228895655]
  Cloud-edge-terminal collaborative intelligence (CETCI) is a fundamental paradigm within the artificial intelligence of things (AIoT) community.<n>CETCI has made significant progress with emerging AIoT applications, moving beyond isolated layer optimization to deployable collaborative intelligence systems.<n>This survey describes foundational architectures, enabling technologies, and scenarios of CETCI paradigms, offering a tutorial-style review for CISAIOT beginners.
  arXiv  Detail & Related papers  (2025-08-26T08:38:01Z)
• Edge-Cloud Collaborative Computing on Distributed Intelligence and Model Optimization: A Survey [58.50944604905037]
  Edge-cloud collaborative computing (ECCC) has emerged as a pivotal paradigm for addressing the computational demands of modern intelligent applications.<n>Recent advancements in AI, particularly deep learning and large language models (LLMs), have dramatically enhanced the capabilities of these distributed systems.<n>This survey provides a structured tutorial on fundamental architectures, enabling technologies, and emerging applications.
  arXiv  Detail & Related papers  (2025-05-03T13:55:38Z)
• Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces [5.692357167709513]
  We propose Chakra, an open graph schema for standardizing workload specification capturing key operations and dependencies, also known as Execution Trace (ET) For instance, we use generative AI models to learn latent statistical properties across thousands of Chakra ETs and use these models to synthesize Chakra ETs. Our end-goal is to build a vibrant industry-wide ecosystem of agile benchmarks and tools to drive future AI system co-design.
  arXiv  Detail & Related papers  (2023-05-23T20:45:45Z)
• SOLIS -- The MLOps journey from data acquisition to actionable insights [62.997667081978825]
  In this paper we present a unified deployment pipeline and freedom-to-operate approach that supports all requirements while using basic cross-platform tensor framework and script language engines. This approach however does not supply the needed procedures and pipelines for the actual deployment of machine learning capabilities in real production grade systems.
  arXiv  Detail & Related papers  (2021-12-22T14:45:37Z)
• Enabling Un-/Semi-Supervised Machine Learning for MDSE of the Real-World CPS/IoT Applications [0.5156484100374059]
  We propose a novel approach to support domain-specific Model-Driven Software Engineering (MDSE) for the real-world use-case scenarios of smart Cyber-Physical Systems (CPS) and the Internet of Things (IoT) We argue that the majority of available data in the nature for Artificial Intelligence (AI) are unlabeled. Hence, unsupervised and/or semi-supervised ML approaches are the practical choices. Our proposed approach is fully implemented and integrated with an existing state-of-the-art MDSE tool to serve the CPS/IoT domain.
  arXiv  Detail & Related papers  (2021-07-06T15:51:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.