Porting an LLM based Application from ChatGPT to an On-Premise Environment

• URL: http://arxiv.org/abs/2504.07907v1
• Date: Thu, 10 Apr 2025 16:29:26 GMT
• Title: Porting an LLM based Application from ChatGPT to an On-Premise Environment
• Authors: Teemu Paloniemi, Manu Setälä, Tommi Mikkonen,
• Abstract summary: We study the porting process of a real-life application using ChatGPT to an on-premise environment.<n>The main considerations in the porting process include transparency of open source models and cost of hardware.
• Score: 2.4742581572364126
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Given the data-intensive nature of Machine Learning (ML) systems in general, and Large Language Models (LLM) in particular, using them in cloud based environments can become a challenge due to legislation related to privacy and security of data. Taking such aspects into consideration implies porting the LLMs to an on-premise environment, where privacy and security can be controlled. In this paper, we study this porting process of a real-life application using ChatGPT, which runs in a public cloud, to an on-premise environment. The application being ported is AIPA, a system that leverages Large Language Models (LLMs) and sophisticated data analytics to enhance the assessment of procurement call bids. The main considerations in the porting process include transparency of open source models and cost of hardware, which are central design choices of the on-premise environment. In addition to presenting the porting process, we evaluate downsides and benefits associated with porting.

Related papers

• A General Pseudonymization Framework for Cloud-Based LLMs: Replacing Privacy Information in Controlled Text Generation [0.6699777383856287]
  ChatGPT services leverage cloud-based large language models (LLMs)<n>Privacy concerns arise as prompts are transmitted and processed by the model providers.<n>We propose a general pseudonymization framework applicable to cloud-based LLMs.
  arXiv  Detail & Related papers  (2025-02-21T06:15:53Z)
• PRIV-QA: Privacy-Preserving Question Answering for Cloud Large Language Models [10.050972891318324]
  We propose a privacy preservation pipeline for protecting privacy and sensitive information during interactions between users and large language models. We construct SensitiveQA, the first privacy open-ended question-answering dataset. Our proposed solution employs a multi-stage strategy aimed at preemptively securing user information while simultaneously preserving the response quality of cloud-based LLMs.
  arXiv  Detail & Related papers  (2025-02-19T09:17:07Z)
• LLM-Lasso: A Robust Framework for Domain-Informed Feature Selection and Regularization [59.75242204923353]
  We introduce LLM-Lasso, a framework that leverages large language models (LLMs) to guide feature selection in Lasso regression.<n>LLMs generate penalty factors for each feature, which are converted into weights for the Lasso penalty using a simple, tunable model.<n>Features identified as more relevant by the LLM receive lower penalties, increasing their likelihood of being retained in the final model.
  arXiv  Detail & Related papers  (2025-02-15T02:55:22Z)
• FedDTPT: Federated Discrete and Transferable Prompt Tuning for Black-Box Large Language Models [14.719919025265224]
  Fine-tuning large language models (LLMs) with data from specific scenarios poses privacy leakage risks. We propose for the first time a federated discrete and transferable prompt tuning, namely FedDTPT, for black-box large language models. Our approach achieves higher accuracy, reduced communication overhead, and robustness to non-iid data in a black-box setting.
  arXiv  Detail & Related papers  (2024-11-01T19:19:23Z)
• LLM-PBE: Assessing Data Privacy in Large Language Models [111.58198436835036]
  Large Language Models (LLMs) have become integral to numerous domains, significantly advancing applications in data management, mining, and analysis. Despite the critical nature of this issue, there has been no existing literature to offer a comprehensive assessment of data privacy risks in LLMs. Our paper introduces LLM-PBE, a toolkit crafted specifically for the systematic evaluation of data privacy risks in LLMs.
  arXiv  Detail & Related papers  (2024-08-23T01:37:29Z)
• Dynamic Fog Computing for Enhanced LLM Execution in Medical Applications [1.0500536774309863]
  Large language models (LLMs) can transform, interpret, and comprehend vast quantities of heterogeneous data. The sensitive nature of protected health information (PHI) raises valid concerns about data privacy and trust in remote LLM platforms. We propose a shift in the LLM execution environment from opaque, centralized cloud providers to a decentralized and dynamic fog computing architecture.
  arXiv  Detail & Related papers  (2024-08-08T04:49:21Z)
• Robust Utility-Preserving Text Anonymization Based on Large Language Models [80.5266278002083]
  Text anonymization is crucial for sharing sensitive data while maintaining privacy. Existing techniques face the emerging challenges of re-identification attack ability of Large Language Models. This paper proposes a framework composed of three LLM-based components -- a privacy evaluator, a utility evaluator, and an optimization component.
  arXiv  Detail & Related papers  (2024-07-16T14:28:56Z)
• Federated Domain-Specific Knowledge Transfer on Large Language Models Using Synthetic Data [53.70870879858533]
  We introduce a Federated Domain-specific Knowledge Transfer framework. It enables domain-specific knowledge transfer from LLMs to SLMs while preserving clients' data privacy. The proposed FDKT framework consistently and greatly improves SLMs' task performance by around 5% with a privacy budget of less than 10.
  arXiv  Detail & Related papers  (2024-05-23T06:14:35Z)
• LEARN: Knowledge Adaptation from Large Language Model to Recommendation for Practical Industrial Application [54.984348122105516]
  Llm-driven knowlEdge Adaptive RecommeNdation (LEARN) framework synergizes open-world knowledge with collaborative knowledge.<n>We propose an Llm-driven knowlEdge Adaptive RecommeNdation (LEARN) framework that synergizes open-world knowledge with collaborative knowledge.
  arXiv  Detail & Related papers  (2024-05-07T04:00:30Z)
• IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems [34.92643238651264]
  Large language models (LLMs) have begun supporting third-party applications.<n>These apps leverage the de facto natural language-based automated execution paradigm of LLMs.<n>Because third-party apps may not be trustworthy, and exacerbated by the imprecision of natural language interfaces, the current designs pose security and privacy risks for users.
  arXiv  Detail & Related papers  (2024-03-08T00:02:30Z)
• ChatSOS: LLM-based knowledge Q&A system for safety engineering [0.0]
  This study introduces an LLM-based Q&A system for safety engineering, enhancing the comprehension and response accuracy of the model. We employ prompt engineering to incorporate external knowledge databases, thus enriching the LLM with up-to-date and reliable information. Our findings indicate that the integration of external knowledge significantly augments the capabilities of LLM for in-depth problem analysis and autonomous task assignment.
  arXiv  Detail & Related papers  (2023-12-14T03:25:23Z)
• Federated Learning: A Cutting-Edge Survey of the Latest Advancements and Applications [6.042202852003457]
  Federated learning (FL) is a technique for developing robust machine learning (ML) models. To protect user privacy, FL requires users to send model updates rather than transmitting large quantities of raw and potentially confidential data. This survey provides a comprehensive analysis and comparison of the most recent FL algorithms.
  arXiv  Detail & Related papers  (2023-10-08T19:54:26Z)
• Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly [62.473245910234304]
  This paper takes a hardware-centric approach to explore how Large Language Models can be brought to modern edge computing systems. We provide a micro-level hardware benchmark, compare the model FLOP utilization to a state-of-the-art data center GPU, and study the network utilization in realistic conditions.
  arXiv  Detail & Related papers  (2023-10-04T20:27:20Z)
• Augmented Large Language Models with Parametric Knowledge Guiding [72.71468058502228]
  Large Language Models (LLMs) have significantly advanced natural language processing (NLP) with their impressive language understanding and generation capabilities. Their performance may be suboptimal for domain-specific tasks that require specialized knowledge due to limited exposure to the related data. We propose the novel Parametric Knowledge Guiding (PKG) framework, which equips LLMs with a knowledge-guiding module to access relevant knowledge.
  arXiv  Detail & Related papers  (2023-05-08T15:05:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.