Towards Message Brokers for Generative AI: Survey, Challenges, and Opportunities

• URL: http://arxiv.org/abs/2312.14647v2
• Date: Wed, 21 Feb 2024 07:41:10 GMT
• Title: Towards Message Brokers for Generative AI: Survey, Challenges, and Opportunities
• Authors: Alaa Saleh, Roberto Morabito, Sasu Tarkoma, Susanna Pirttikangas and Lauri Lov\'en
• Abstract summary: Generative Artificial Intelligence (GenAI) is becoming increasingly prevalent, extending its reach across diverse applications. This surge in adoption has sparked a significant increase in demand for data-centric GenAI models. Central to this need are message brokers, which serve as essential channels for data transfer within various system components.
• Score: 4.49465498333472
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In today's digital world, Generative Artificial Intelligence (GenAI) such as Large Language Models (LLMs) is becoming increasingly prevalent, extending its reach across diverse applications. This surge in adoption has sparked a significant increase in demand for data-centric GenAI models, highlighting the necessity for robust data communication infrastructures. Central to this need are message brokers, which serve as essential channels for data transfer within various system components. This survey aims to delve into a comprehensive analysis of traditional and modern message brokers, offering a comparative study of prevalent platforms. Our study considers numerous criteria including, but not limited to, open-source availability, integrated monitoring tools, message prioritization mechanisms, capabilities for parallel processing, reliability, distribution and clustering functionalities, authentication processes, data persistence strategies, fault tolerance, and scalability. Furthermore, we explore the intrinsic constraints that the design and operation of each message broker might impose, recognizing that these limitations are crucial in understanding their real-world applicability. Finally, this study examines the enhancement of message broker mechanisms specifically for GenAI contexts, emphasizing the criticality of developing a versatile message broker framework. Such a framework would be poised for quick adaptation, catering to the dynamic and growing demands of GenAI in the foreseeable future. Through this dual-pronged approach, we intend to contribute a foundational compendium that can guide future innovations and infrastructural advancements in the realm of GenAI data communication.

Related papers

• Cutting Through the Confusion and Hype: Understanding the True Potential of Generative AI [0.0]
  This paper explores the nuanced landscape of generative AI (genAI) It focuses on neural network-based models like Large Language Models (LLMs)
  arXiv  Detail & Related papers  (2024-10-22T02:18:44Z)
• Agent-driven Generative Semantic Communication with Cross-Modality and Prediction [57.335922373309074]
  We propose a novel agent-driven generative semantic communication framework based on reinforcement learning. In this work, we develop an agent-assisted semantic encoder with cross-modality capability, which can track the semantic changes, channel condition, to perform adaptive semantic extraction and sampling. The effectiveness of the designed models has been verified using the UA-DETRAC dataset, demonstrating the performance gains of the overall A-GSC framework.
  arXiv  Detail & Related papers  (2024-04-10T13:24:27Z)
• Generative AI for Secure Physical Layer Communications: A Survey [80.0638227807621]
  Generative Artificial Intelligence (GAI) stands at the forefront of AI innovation, demonstrating rapid advancement and unparalleled proficiency in generating diverse content. In this paper, we offer an extensive survey on the various applications of GAI in enhancing security within the physical layer of communication networks. We delve into the roles of GAI in addressing challenges of physical layer security, focusing on communication confidentiality, authentication, availability, resilience, and integrity.
  arXiv  Detail & Related papers  (2024-02-21T06:22:41Z)
• Emergent Explainability: Adding a causal chain to neural network inference [0.0]
  This position paper presents a theoretical framework for enhancing explainable artificial intelligence (xAI) through emergent communication (EmCom) We explore the novel integration of EmCom into AI systems, offering a paradigm shift from conventional associative relationships between inputs and outputs to a more nuanced, causal interpretation. The paper discusses the theoretical underpinnings of this approach, its potential broad applications, and its alignment with the growing need for responsible and transparent AI systems.
  arXiv  Detail & Related papers  (2024-01-29T02:28:39Z)
• Generative AI for Physical Layer Communications: A Survey [76.61956357178295]
  generative artificial intelligence (GAI) has the potential to enhance the efficiency of digital content production. GAI's capability in analyzing complex data distributions offers great potential for wireless communications. This paper presents a comprehensive investigation of GAI's applications for communications at the physical layer, ranging from traditional issues, including signal classification, channel estimation, and equalization, to emerging topics, such as intelligent reflecting surfaces and joint source channel coding.
  arXiv  Detail & Related papers  (2023-12-09T15:20:56Z)
• Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communication [76.04373033082948]
  Large Language Models (LLMs) have recently made significant strides in complex reasoning tasks through the Chain-of-Thought technique. We propose Exchange-of-Thought (EoT), a novel framework that enables cross-model communication during problem-solving.
  arXiv  Detail & Related papers  (2023-12-04T11:53:56Z)
• Towards a Responsible AI Metrics Catalogue: A Collection of Metrics for AI Accountability [28.67753149592534]
  This study bridges the accountability gap by introducing our effort towards a comprehensive metrics catalogue. Our catalogue delineates process metrics that underpin procedural integrity, resource metrics that provide necessary tools and frameworks, and product metrics that reflect the outputs of AI systems.
  arXiv  Detail & Related papers  (2023-11-22T04:43:16Z)
• Data Equity: Foundational Concepts for Generative AI [0.0]
  GenAI promises immense potential to drive digital and social innovation. GenAI has the potential to democratize access and usage of technologies. However, left unchecked, it could deepen inequities.
  arXiv  Detail & Related papers  (2023-10-27T05:19:31Z)
• An In-depth Survey of Large Language Model-based Artificial Intelligence Agents [11.774961923192478]
  We have explored the core differences and characteristics between LLM-based AI agents and traditional AI agents. We conducted an in-depth analysis of the key components of AI agents, including planning, memory, and tool use.
  arXiv  Detail & Related papers  (2023-09-23T11:25:45Z)
• Large Language Models for Information Retrieval: A Survey [58.30439850203101]
  Information retrieval has evolved from term-based methods to its integration with advanced neural models. Recent research has sought to leverage large language models (LLMs) to improve IR systems. We delve into the confluence of LLMs and IR systems, including crucial aspects such as query rewriters, retrievers, rerankers, and readers.
  arXiv  Detail & Related papers  (2023-08-14T12:47:22Z)
• Iterative Zero-Shot LLM Prompting for Knowledge Graph Construction [104.29108668347727]
  This paper proposes an innovative knowledge graph generation approach that leverages the potential of the latest generative large language models. The approach is conveyed in a pipeline that comprises novel iterative zero-shot and external knowledge-agnostic strategies. We claim that our proposal is a suitable solution for scalable and versatile knowledge graph construction and may be applied to different and novel contexts.
  arXiv  Detail & Related papers  (2023-07-03T16:01:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.