Introduction to Latent Variable Energy-Based Models: A Path Towards Autonomous Machine Intelligence

• URL: http://arxiv.org/abs/2306.02572v1
• Date: Mon, 5 Jun 2023 03:55:26 GMT
• Title: Introduction to Latent Variable Energy-Based Models: A Path Towards Autonomous Machine Intelligence
• Authors: Anna Dawid, Yann LeCun
• Abstract summary: We summarize the main ideas behind the architecture of autonomous intelligence of the future proposed by Yann LeCun. In particular, we introduce energy-based and latent variable models and combine their advantages in the building block of LeCun's proposal.
• Score: 13.27120983899836
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current automated systems have crucial limitations that need to be addressed before artificial intelligence can reach human-like levels and bring new technological revolutions. Among others, our societies still lack Level 5 self-driving cars, domestic robots, and virtual assistants that learn reliable world models, reason, and plan complex action sequences. In these notes, we summarize the main ideas behind the architecture of autonomous intelligence of the future proposed by Yann LeCun. In particular, we introduce energy-based and latent variable models and combine their advantages in the building block of LeCun's proposal, that is, in the hierarchical joint embedding predictive architecture (H-JEPA).

Related papers

• AI Flow: Perspectives, Scenarios, and Approaches [51.38621621775711]
  We introduce AI Flow, a framework that integrates cutting-edge IT and CT advancements.<n>First, device-edge-cloud framework serves as the foundation, which integrates end devices, edge servers, and cloud clusters.<n>Second, we introduce the concept of familial models, which refers to a series of different-sized models with aligned hidden features.<n>Third, connectivity- and interaction-based intelligence emergence is a novel paradigm of AI Flow.
  arXiv  Detail & Related papers  (2025-06-14T12:43:07Z)
• Bayesian Inverse Physics for Neuro-Symbolic Robot Learning [13.63049975599771]
  Real-world robotic applications require adaptive, interpretable, and data-efficient learning paradigms.<n>We introduce a conceptual framework for combining data-driven learning with deliberate, structured reasoning.<n>We argue that such hybrid neuro-symbolic architectures are essential for the next generation of autonomous systems.
  arXiv  Detail & Related papers  (2025-06-10T12:53:31Z)
• Embodied AI with Foundation Models for Mobile Service Robots: A Systematic Review [4.540236408836132]
  We present the first systematic review of the integration of foundation models in mobile service robotics.<n>We explore the role of such models in enabling real-time sensor fusion, language-conditioned control, and adaptive task execution.<n>We also discuss real-world applications in the domestic assistance, healthcare, and service automation sectors.
  arXiv  Detail & Related papers  (2025-05-26T20:08:09Z)
• Generative AI for Autonomous Driving: Frontiers and Opportunities [145.6465312554513]
  This survey delivers a comprehensive synthesis of the emerging role of GenAI across the autonomous driving stack.<n>We begin by distilling the principles and trade-offs of modern generative modeling, encompassing VAEs, GANs, Diffusion Models, and Large Language Models.<n>We categorize practical applications, such as synthetic data generalization, end-to-end driving strategies, high-fidelity digital twin systems, smart transportation networks, and cross-domain transfer to embodied AI.
  arXiv  Detail & Related papers  (2025-05-13T17:59:20Z)
• Model-Based AI planning and Execution Systems for Robotics [8.287206589886878]
  Model-based planning and execution systems offer a principled approach to building flexible autonomous robots.<n>We consider the diverse design choices and issues existing systems attempt to address, the different solutions proposed so far, and suggest avenues for future development.
  arXiv  Detail & Related papers  (2025-05-07T15:17:38Z)
• Personalized Artificial General Intelligence (AGI) via Neuroscience-Inspired Continuous Learning Systems [3.764721243654025]
  Current approaches largely depend on expanding model parameters, which improves task-specific performance but falls short in enabling continuous, adaptable, and generalized learning. This paper reviews the state of continual learning and neuroscience-inspired AI, and proposes a novel architecture for Personalized AGI that integrates brain-like learning mechanisms for edge deployment. Building on these insights, we outline an AI architecture that features complementary fast-and-slow learning modules, synaptic self-optimization, and memory-efficient model updates to support on-device lifelong adaptation.
  arXiv  Detail & Related papers  (2025-04-27T16:10:17Z)
• Redefining Robot Generalization Through Interactive Intelligence [0.0]
  We argue that robot foundation models must evolve to an interactive multi-agent perspective in order to handle the complexities of real-time human-robot co-adaptation. By moving beyond single-agent designs, our position emphasizes how foundation models in robotics can achieve a more robust, personalized, and anticipatory level of performance.
  arXiv  Detail & Related papers  (2025-02-09T17:13:27Z)
• Exploring the Interplay Between Video Generation and World Models in Autonomous Driving: A Survey [61.39993881402787]
  World models and video generation are pivotal technologies in the domain of autonomous driving. This paper investigates the relationship between these two technologies. By analyzing the interplay between video generation and world models, this survey identifies critical challenges and future research directions.
  arXiv  Detail & Related papers  (2024-11-05T08:58:35Z)
• $π_0$: A Vision-Language-Action Flow Model for General Robot Control [77.32743739202543]
  We propose a novel flow matching architecture built on top of a pre-trained vision-language model (VLM) to inherit Internet-scale semantic knowledge. We evaluate our model in terms of its ability to perform tasks in zero shot after pre-training, follow language instructions from people, and its ability to acquire new skills via fine-tuning.
  arXiv  Detail & Related papers  (2024-10-31T17:22:30Z)
• Position Paper: Agent AI Towards a Holistic Intelligence [53.35971598180146]
  We emphasize developing Agent AI -- an embodied system that integrates large foundation models into agent actions. In this paper, we propose a novel large action model to achieve embodied intelligent behavior, the Agent Foundation Model.
  arXiv  Detail & Related papers  (2024-02-28T16:09:56Z)
• A call for embodied AI [1.7544885995294304]
  We propose Embodied AI as the next fundamental step in the pursuit of Artificial General Intelligence. By broadening the scope of Embodied AI, we introduce a theoretical framework based on cognitive architectures. This framework is aligned with Friston's active inference principle, offering a comprehensive approach to EAI development.
  arXiv  Detail & Related papers  (2024-02-06T09:11:20Z)
• A Survey on Robotics with Foundation Models: toward Embodied AI [30.999414445286757]
  Recent advances in computer vision, natural language processing, and multi-modality learning have shown that the foundation models have superhuman capabilities for specific tasks. This survey aims to provide a comprehensive and up-to-date overview of foundation models in robotics, focusing on autonomous manipulation and encompassing high-level planning and low-level control.
  arXiv  Detail & Related papers  (2024-02-04T07:55:01Z)
• Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis [82.59451639072073]
  General-purpose robots operate seamlessly in any environment, with any object, and utilize various skills to complete diverse tasks. As a community, we have been constraining most robotic systems by designing them for specific tasks, training them on specific datasets, and deploying them within specific environments. Motivated by the impressive open-set performance and content generation capabilities of web-scale, large-capacity pre-trained models, we devote this survey to exploring how foundation models can be applied to general-purpose robotics.
  arXiv  Detail & Related papers  (2023-12-14T10:02:55Z)
• RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation [68.70755196744533]
  RoboGen is a generative robotic agent that automatically learns diverse robotic skills at scale via generative simulation. Our work attempts to extract the extensive and versatile knowledge embedded in large-scale models and transfer them to the field of robotics.
  arXiv  Detail & Related papers  (2023-11-02T17:59:21Z)
• Integrating Generative Artificial Intelligence in Intelligent Vehicle Systems [4.724940029079736]
  As the automotive industry progressively integrates AI, generative artificial intelligence technologies hold the potential to revolutionize user interactions. We provide an overview of current applications of generative artificial intelligence in the automotive domain, emphasizing speech, audio, vision, and multimodal interactions. We outline critical future research areas, including domain adaptability, alignment, multimodal integration and others, as well as, address the challenges and risks associated with ethics.
  arXiv  Detail & Related papers  (2023-05-15T09:09:40Z)
• AI Maintenance: A Robustness Perspective [91.28724422822003]
  We introduce highlighted robustness challenges in the AI lifecycle and motivate AI maintenance by making analogies to car maintenance. We propose an AI model inspection framework to detect and mitigate robustness risks. Our proposal for AI maintenance facilitates robustness assessment, status tracking, risk scanning, model hardening, and regulation throughout the AI lifecycle.
  arXiv  Detail & Related papers  (2023-01-08T15:02:38Z)
• Towards open and expandable cognitive AI architectures for large-scale multi-agent human-robot collaborative learning [5.478764356647437]
  A novel cognitive architecture for multi-agent LfD robotic learning is introduced, targeting to enable the reliable deployment of open, scalable and expandable robotic systems. The conceptualization relies on employing multiple AI-empowered cognitive processes that operate at the edge nodes of a network of robotic platforms. The applicability of the proposed framework is explained using an example of a real-world industrial case study.
  arXiv  Detail & Related papers  (2020-12-15T09:49:22Z)
• Cloud2Edge Elastic AI Framework for Prototyping and Deployment of AI Inference Engines in Autonomous Vehicles [1.688204090869186]
  This paper proposes a novel framework for developing AI Inference Engines for autonomous driving applications based on deep learning modules. We introduce a simple yet elegant solution for the AI components development cycle, where prototyping takes place in the cloud according to the Software-in-the-Loop (SiL) paradigm. The effectiveness of the proposed framework is demonstrated using two real-world use-cases of AI inference engines for autonomous vehicles.
  arXiv  Detail & Related papers  (2020-09-23T09:23:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.