Towards a Middleware for Large Language Models

• URL: http://arxiv.org/abs/2411.14513v1
• Date: Thu, 21 Nov 2024 13:55:24 GMT
• Title: Towards a Middleware for Large Language Models
• Authors: Narcisa Guran, Florian Knauf, Man Ngo, Stefan Petrescu, Jan S. Rellermeyer,
• Abstract summary: Large language models have gained widespread popularity for their ability to process natural language inputs. This advancement has prompted enterprises worldwide to integrate LLMs into their services. As the technology matures, there is a strong incentive for independence from major cloud providers through self-hosting "LLM as a Service"
• Score: 3.491539485256461
• License:
• Abstract: Large language models have gained widespread popularity for their ability to process natural language inputs and generate insights derived from their training data, nearing the qualities of true artificial intelligence. This advancement has prompted enterprises worldwide to integrate LLMs into their services. So far, this effort is dominated by commercial cloud-based solutions like OpenAI's ChatGPT and Microsoft Azure. As the technology matures, however, there is a strong incentive for independence from major cloud providers through self-hosting "LLM as a Service", driven by privacy, cost, and customization needs. In practice, hosting LLMs independently presents significant challenges due to their complexity and integration issues with existing systems. In this paper, we discuss our vision for a forward-looking middleware system architecture that facilitates the deployment and adoption of LLMs in enterprises, even for advanced use cases in which we foresee LLMs to serve as gateways to a complete application ecosystem and, to some degree, absorb functionality traditionally attributed to the middleware.

Related papers

• Large Action Models: From Inception to Implementation [51.81485642442344]
  Large Action Models (LAMs) are designed for action generation and execution within dynamic environments. LAMs hold the potential to transform AI from passive language understanding to active task completion. We present a comprehensive framework for developing LAMs, offering a systematic approach to their creation, from inception to deployment.
  arXiv  Detail & Related papers  (2024-12-13T11:19:56Z)
• LLM-based Multi-Agent Systems: Techniques and Business Perspectives [26.74974842247119]
  In the era of (multi-modal) large language models, most operational processes can be reformulated and reproduced using LLM agents. As a natural trend of development, the tools for calling are becoming autonomous agents, thus the full intelligent system turns out to be a LLM-based Multi-Agent System (LaMAS) Compared to the previous single-LLM-agent system, LaMAS has the advantages of i) dynamic task decomposition and organic specialization, ii) higher flexibility for system changing, and iv) feasibility of monetization for each entity.
  arXiv  Detail & Related papers  (2024-11-21T11:36:29Z)
• When IoT Meet LLMs: Applications and Challenges [0.5461938536945723]
  We show how Large Language Models (LLMs) can facilitate advanced decision making and contextual understanding in the Internet of Things (IoT) This is the first comprehensive study covering IoT-LLM integration between edge, fog, and cloud systems. We propose a novel system model for industrial IoT applications that leverages LLM-based collective intelligence to enable predictive maintenance and condition monitoring.
  arXiv  Detail & Related papers  (2024-11-20T23:44:51Z)
• Dynamic Fog Computing for Enhanced LLM Execution in Medical Applications [1.0500536774309863]
  Large language models (LLMs) can transform, interpret, and comprehend vast quantities of heterogeneous data. The sensitive nature of protected health information (PHI) raises valid concerns about data privacy and trust in remote LLM platforms. We propose a shift in the LLM execution environment from opaque, centralized cloud providers to a decentralized and dynamic fog computing architecture.
  arXiv  Detail & Related papers  (2024-08-08T04:49:21Z)
• Large Language Model as a Catalyst: A Paradigm Shift in Base Station Siting Optimization [62.16747639440893]
  Large language models (LLMs) and their associated technologies advance, particularly in the realms of prompt engineering and agent engineering. Our proposed framework incorporates retrieval-augmented generation (RAG) to enhance the system's ability to acquire domain-specific knowledge and generate solutions.
  arXiv  Detail & Related papers  (2024-08-07T08:43:32Z)
• A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Tasks [74.52259252807191]
  Multimodal Large Language Models (MLLMs) address the complexities of real-world applications far beyond the capabilities of single-modality systems. This paper systematically sorts out the applications of MLLM in multimodal tasks such as natural language, vision, and audio.
  arXiv  Detail & Related papers  (2024-08-02T15:14:53Z)
• LEGENT: Open Platform for Embodied Agents [60.71847900126832]
  We introduce LEGENT, an open, scalable platform for developing embodied agents using Large Language Models (LLMs) and Large Multimodal Models (LMMs) LEGENT offers a rich, interactive 3D environment with communicable and actionable agents, paired with a user-friendly interface. In experiments, an embryonic vision-language-action model trained on LEGENT-generated data surpasses GPT-4V in embodied tasks.
  arXiv  Detail & Related papers  (2024-04-28T16:50:12Z)
• LLMs as On-demand Customizable Service [8.440060524215378]
  We introduce a concept of hierarchical, distributed Large Language Models (LLMs) By introducing a "layered" approach, the proposed architecture enables on-demand accessibility to LLMs as a customizable service. We envision that the concept of hierarchical LLM will empower extensive, crowd-sourced user bases to harness the capabilities of LLMs.
  arXiv  Detail & Related papers  (2024-01-29T21:24:10Z)
• FATE-LLM: A Industrial Grade Federated Learning Framework for Large Language Models [18.65547577691255]
  Large Language Models (LLMs) have exhibited remarkable performances across various tasks in recent years. FATE-LLM is an industrial-grade federated learning framework for large language models. We release the code of FATE-LLM to facilitate the research of FedLLM and enable a broad range of industrial applications.
  arXiv  Detail & Related papers  (2023-10-16T04:17:13Z)
• Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly [62.473245910234304]
  This paper takes a hardware-centric approach to explore how Large Language Models can be brought to modern edge computing systems. We provide a micro-level hardware benchmark, compare the model FLOP utilization to a state-of-the-art data center GPU, and study the network utilization in realistic conditions.
  arXiv  Detail & Related papers  (2023-10-04T20:27:20Z)
• Augmented Large Language Models with Parametric Knowledge Guiding [72.71468058502228]
  Large Language Models (LLMs) have significantly advanced natural language processing (NLP) with their impressive language understanding and generation capabilities. Their performance may be suboptimal for domain-specific tasks that require specialized knowledge due to limited exposure to the related data. We propose the novel Parametric Knowledge Guiding (PKG) framework, which equips LLMs with a knowledge-guiding module to access relevant knowledge.
  arXiv  Detail & Related papers  (2023-05-08T15:05:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.