Large Language Model Based Multi-Agent System Augmented Complex Event Processing Pipeline for Internet of Multimedia Things

• URL: http://arxiv.org/abs/2501.00906v2
• Date: Fri, 03 Jan 2025 07:47:36 GMT
• Title: Large Language Model Based Multi-Agent System Augmented Complex Event Processing Pipeline for Internet of Multimedia Things
• Authors: Talha Zeeshan, Abhishek Kumar, Susanna Pirttikangas, Sasu Tarkoma,
• Abstract summary: This paper presents the development and evaluation of a Large Language Model (LLM) based system framework for complex event processing (CEP)<n>The primary goal is to create a proof-of-concept that integrates state-of-the-art LLM orchestration frameworks with publish/subscribe (pub/sub) tools to address the integration of LLMs with current CEP systems.
• Score: 8.729059187561761
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This paper presents the development and evaluation of a Large Language Model (LLM), also known as foundation models, based multi-agent system framework for complex event processing (CEP) with a focus on video query processing use cases. The primary goal is to create a proof-of-concept (POC) that integrates state-of-the-art LLM orchestration frameworks with publish/subscribe (pub/sub) tools to address the integration of LLMs with current CEP systems. Utilizing the Autogen framework in conjunction with Kafka message brokers, the system demonstrates an autonomous CEP pipeline capable of handling complex workflows. Extensive experiments evaluate the system's performance across varying configurations, complexities, and video resolutions, revealing the trade-offs between functionality and latency. The results show that while higher agent count and video complexities increase latency, the system maintains high consistency in narrative coherence. This research builds upon and contributes to, existing novel approaches to distributed AI systems, offering detailed insights into integrating such systems into existing infrastructures.

Related papers

• Towards Conversational AI for Human-Machine Collaborative MLOps [0.17152709285783643]
  This paper presents a Large Language Model (LLM) based conversational agent system designed to enhance human-machine collaboration in Machine Learning Operations (MLOps) We introduce the Swarm Agent, an architecture that integrates specialized agents to create and manage ML through natural language interactions. The paper describes the architecture, implementation details, and demonstrates how this conversational MLOps assistant reduces complexity and lowers to entry for users across diverse technical skill levels.
  arXiv  Detail & Related papers  (2025-04-16T20:28:50Z)
• Agent-Centric Projection of Prompting Techniques and Implications for Synthetic Training Data for Large Language Models [0.8879149917735942]
  This paper introduces and explains the concepts of linear contexts (a single, continuous sequence of interactions) and non-linear contexts (branching or multi-path) in Large Language Models (LLMs) These concepts enable the development of an agent-centric projection of prompting techniques, a framework that can reveal deep connections between prompting strategies and multi-agent systems.
  arXiv  Detail & Related papers  (2025-01-14T03:26:43Z)
• LatteReview: A Multi-Agent Framework for Systematic Review Automation Using Large Language Models [0.0]
  LatteReview is a Python-based framework that leverages large language models (LLMs) and multi-agent systems to automate key elements of the systematic review process. The framework supports features such as Retrieval-Augmented Generation (RAG) for incorporating external context, multimodal reviews, Pydantic-based validation for structured inputs and outputs, and asynchronous programming for handling large-scale datasets.
  arXiv  Detail & Related papers  (2025-01-05T17:53:00Z)
• AgentPS: Agentic Process Supervision for Multi-modal Content Quality Assurance through Multi-round QA [9.450927573476822]
  textitAgentPS is a novel framework that integrates Agentic Process Supervision into MLLMs via multi-round question answering during fine-tuning.<n>textitAgentPS demonstrates significant performance improvements over baseline MLLMs on proprietary TikTok datasets.
  arXiv  Detail & Related papers  (2024-12-15T04:58:00Z)
• VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation [100.06122876025063]
  This paper introduces VisDoMBench, the first comprehensive benchmark designed to evaluate QA systems in multi-document settings.<n>We propose VisDoMRAG, a novel multimodal Retrieval Augmented Generation (RAG) approach that simultaneously utilizes visual and textual RAG.
  arXiv  Detail & Related papers  (2024-12-14T06:24:55Z)
• Iris: Breaking GUI Complexity with Adaptive Focus and Self-Refining [67.87810796668981]
  Information-Sensitive Cropping (ISC) and Self-Refining Dual Learning (SRDL) Iris achieves state-of-the-art performance across multiple benchmarks with only 850K GUI annotations. These improvements translate to significant gains in both web and OS agent downstream tasks.
  arXiv  Detail & Related papers  (2024-12-13T18:40:10Z)
• CUE-M: Contextual Understanding and Enhanced Search with Multimodal Large Language Model [9.224965304457708]
  This paper introduces Contextual Understanding and Enhanced Search with MLLM (CUE-M), a novel multimodal search framework.<n> Evaluations on a multimodal Q&A dataset and a public safety benchmark demonstrate that CUE-M outperforms baselines in accuracy, knowledge integration, and safety.
  arXiv  Detail & Related papers  (2024-11-19T07:16:48Z)
• VisionCoder: Empowering Multi-Agent Auto-Programming for Image Processing with Hybrid LLMs [8.380216582290025]
  This paper presents a multi-agent framework that collaboratively completes auto-programming tasks. Each agent plays a distinct role in the software development cycle, collectively forming a virtual organisation. By establishing a tree-structured thought distribution and development mechanism across project, module, and function levels, this framework offers a cost-effective and efficient solution.
  arXiv  Detail & Related papers  (2024-10-25T01:52:15Z)
• The Compressor-Retriever Architecture for Language Model OS [20.56093501980724]
  This paper explores the concept of using a language model as the core component of an operating system (OS) A key challenge in realizing such an LM OS is managing the life-long context and ensuring statefulness across sessions. We introduce compressor-retriever, a model-agnostic architecture designed for life-long context management.
  arXiv  Detail & Related papers  (2024-09-02T23:28:15Z)
• ComfyBench: Benchmarking LLM-based Agents in ComfyUI for Autonomously Designing Collaborative AI Systems [80.69865295743149]
  This work attempts to study using LLM-based agents to design collaborative AI systems autonomously.<n>Based on ComfyBench, we develop ComfyAgent, a framework that empowers agents to autonomously design collaborative AI systems by generating.<n>While ComfyAgent achieves a comparable resolve rate to o1-preview and significantly surpasses other agents on ComfyBench, ComfyAgent has resolved only 15% of creative tasks.
  arXiv  Detail & Related papers  (2024-09-02T17:44:10Z)
• Very Large-Scale Multi-Agent Simulation in AgentScope [112.98986800070581]
  We develop new features and components for AgentScope, a user-friendly multi-agent platform. We propose an actor-based distributed mechanism towards great scalability and high efficiency. We also provide a web-based interface for conveniently monitoring and managing a large number of agents.
  arXiv  Detail & Related papers  (2024-07-25T05:50:46Z)
• Towards More Unified In-context Visual Understanding [74.55332581979292]
  We present a new ICL framework for visual understanding with multi-modal output enabled. First, we quantize and embed both text and visual prompt into a unified representational space. Then a decoder-only sparse transformer architecture is employed to perform generative modeling on them.
  arXiv  Detail & Related papers  (2023-12-05T06:02:21Z)
• Concepts and Algorithms for Agent-based Decentralized and Integrated Scheduling of Production and Auxiliary Processes [78.120734120667]
  This paper describes an agent-based decentralized and integrated scheduling approach. Part of the requirements is to develop a linearly scaling communication architecture. The approach is explained using an example based on industrial requirements.
  arXiv  Detail & Related papers  (2022-05-06T18:44:29Z)
• A Data-Centric Framework for Composable NLP Workflows [109.51144493023533]
  Empirical natural language processing systems in application domains (e.g., healthcare, finance, education) involve interoperation among multiple components. We establish a unified open-source framework to support fast development of such sophisticated NLP in a composable manner.
  arXiv  Detail & Related papers  (2021-03-02T16:19:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.