Towards Decentralized and Sustainable Foundation Model Training with the Edge

• URL: http://arxiv.org/abs/2507.01803v1
• Date: Wed, 02 Jul 2025 15:21:40 GMT
• Title: Towards Decentralized and Sustainable Foundation Model Training with the Edge
• Authors: Leyang Xue, Meghana Madhyastha, Randal Burns, Myungjin Lee, Mahesh K. Marina,
• Abstract summary: Foundation models are at the forefront of AI research, appealing for their ability to learn from vast datasets and cater to diverse tasks.<n>We put forward a vision towards decentralized and sustainable foundation model training that leverages the collective compute of sparingly used connected edge AI devices.
• Score: 2.2815302415385297
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Foundation models are at the forefront of AI research, appealing for their ability to learn from vast datasets and cater to diverse tasks. Yet, their significant computational demands raise issues of environmental impact and the risk of centralized control in their development. We put forward a vision towards decentralized and sustainable foundation model training that leverages the collective compute of sparingly used connected edge AI devices. We present the rationale behind our vision, particularly in support of its sustainability benefit. We further outline a set of challenges that need to be addressed to turn this vision into reality.

Related papers

• Towards Depth Foundation Model: Recent Trends in Vision-Based Depth Estimation [75.30238170051291]
  Depth estimation is a fundamental task in 3D computer vision, crucial for applications such as 3D reconstruction, free-viewpoint rendering, robotics, autonomous driving, and AR/VR technologies.<n>Traditional methods relying on hardware sensors like LiDAR are often limited by high costs, low resolution, and environmental sensitivity, limiting their applicability in real-world scenarios.<n>Recent advances in vision-based methods offer a promising alternative, yet they face challenges in generalization and stability due to either the low-capacity model architectures or the reliance on domain-specific and small-scale datasets.
  arXiv  Detail & Related papers  (2025-07-15T17:59:59Z)
• An Overview of Large Language Models for Statisticians [109.38601458831545]
  Large Language Models (LLMs) have emerged as transformative tools in artificial intelligence (AI)<n>This paper explores potential areas where statisticians can make important contributions to the development of LLMs.<n>We focus on issues such as uncertainty quantification, interpretability, fairness, privacy, watermarking and model adaptation.
  arXiv  Detail & Related papers  (2025-02-25T03:40:36Z)
• Spatio-Temporal Foundation Models: Vision, Challenges, and Opportunities [48.45951497996322]
  Foundation models (STFMs) have revolutionized artificial intelligence, setting new benchmarks in performance and enabling transformative capabilities across a wide range of vision and language tasks.<n>In this paper, we articulate a vision for the future of STFMs, outlining their essential characteristics and generalization capabilities necessary for broad applicability.<n>We explore potential opportunities and directions to advance research towards the aim of effective and broadly applicable STFMs.
  arXiv  Detail & Related papers  (2025-01-15T08:52:28Z)
• Addressing the sustainable AI trilemma: a case study on LLM agents and RAG [7.6212949300713015]
  Large language models (LLMs) have demonstrated significant capabilities, but their widespread deployment and more advanced applications raise critical sustainability challenges.<n>We propose the concept of the Sustainable AI Trilemma, highlighting the tensions between AI capability, digital equity, and environmental sustainability.
  arXiv  Detail & Related papers  (2025-01-14T17:21:16Z)
• Unleashing the Power of Continual Learning on Non-Centralized Devices: A Survey [37.07938402225207]
  Non- Continual Learning (NCCL) has become an emerging paradigm for enabling distributed devices to handle streaming data from a joint non-stationary environment.<n>This survey focuses on the development of the non-centralized continual learning algorithms and the real-world deployment across distributed devices.
  arXiv  Detail & Related papers  (2024-12-18T13:33:28Z)
• Implementation of Big AI Models for Wireless Networks with Collaborative Edge Computing [10.524645516703643]
  Training big AI models poses significant challenges to edge devices. Traditional approaches usually resort to aggregating training data and sending it to a remote cloud for centralized training. We propose collaborative edge training, a novel training mechanism that orchestrates a group of trusted edge devices as a resource pool.
  arXiv  Detail & Related papers  (2024-04-27T03:09:39Z)
• A Survey on Robotics with Foundation Models: toward Embodied AI [30.999414445286757]
  Recent advances in computer vision, natural language processing, and multi-modality learning have shown that the foundation models have superhuman capabilities for specific tasks. This survey aims to provide a comprehensive and up-to-date overview of foundation models in robotics, focusing on autonomous manipulation and encompassing high-level planning and low-level control.
  arXiv  Detail & Related papers  (2024-02-04T07:55:01Z)
• A Survey of Reasoning with Foundation Models [235.7288855108172]
  Reasoning plays a pivotal role in various real-world settings such as negotiation, medical diagnosis, and criminal investigation. We introduce seminal foundation models proposed or adaptable for reasoning. We then delve into the potential future directions behind the emergence of reasoning abilities within foundation models.
  arXiv  Detail & Related papers  (2023-12-17T15:16:13Z)
• Exploring the Robustness of Decentralized Training for Large Language Models [51.41850749014054]
  Decentralized training of large language models has emerged as an effective way to democratize this technology. This paper explores the robustness of decentralized training from three main perspectives.
  arXiv  Detail & Related papers  (2023-12-01T04:04:03Z)
• Causal Reasoning: Charting a Revolutionary Course for Next-Generation AI-Native Wireless Networks [63.246437631458356]
  Next-generation wireless networks (e.g., 6G) will be artificial intelligence (AI)-native. This article introduces a novel framework for building AI-native wireless networks; grounded in the emerging field of causal reasoning. We highlight several wireless networking challenges that can be addressed by causal discovery and representation.
  arXiv  Detail & Related papers  (2023-09-23T00:05:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.