No Size Fits All: The Perils and Pitfalls of Leveraging LLMs Vary with Company Size

• URL: http://arxiv.org/abs/2408.01444v2
• Date: Sun, 01 Dec 2024 16:11:18 GMT
• Title: No Size Fits All: The Perils and Pitfalls of Leveraging LLMs Vary with Company Size
• Authors: Ashok Urlana, Charaka Vinayak Kumar, Bala Mallikarjunarao Garlapati, Ajeet Kumar Singh, Rahul Mishra,
• Abstract summary: Large language models (LLMs) are playing a pivotal role in deploying strategic use cases across a range of organizations.<n>The issues and challenges involved in the successful utilization of LLMs can vary significantly depending on the size of the organization.
• Score: 7.660415388174536
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Large language models (LLMs) are playing a pivotal role in deploying strategic use cases across a range of organizations, from large pan-continental companies to emerging startups. The issues and challenges involved in the successful utilization of LLMs can vary significantly depending on the size of the organization. It is important to study and discuss these pertinent issues of LLM adaptation with a focus on the scale of the industrial concerns and brainstorm possible solutions and prospective directions. Such a study has not been prominently featured in the current research literature. In this study, we adopt a threefold strategy: first, we conduct a case study with industry practitioners to formulate the key research questions; second, we examine existing industrial publications to address these questions; and finally, we provide a practical guide for industries to utilize LLMs more efficiently. We release the GitHub\footnote{\url{https://github.com/vinayakcse/IndustrialLLMsPapers}} repository with the most recent papers in the field.

Related papers

• A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment [291.03029298928857]
  This paper introduces the concept of "full-stack" safety to systematically consider safety issues throughout the entire process of LLM training, deployment, and commercialization. Our research is grounded in an exhaustive review of over 800+ papers, ensuring comprehensive coverage and systematic organization of security issues. Our work identifies promising research directions, including safety in data generation, alignment techniques, model editing, and LLM-based agent systems.
  arXiv  Detail & Related papers  (2025-04-22T05:02:49Z)
• Should You Use Your Large Language Model to Explore or Exploit? [55.562545113247666]
  We evaluate the ability of large language models to help a decision-making agent facing an exploration-exploitation tradeoff. We find that while the current LLMs often struggle to exploit, in-context mitigations may be used to substantially improve performance for small-scale tasks.
  arXiv  Detail & Related papers  (2025-01-31T23:42:53Z)
• Large language models for automated scholarly paper review: A survey [2.41072532750517]
  This paper aims to provide a holistic view of automated scholarly paper review (ASPR) in the era of large language models (LLMs) We review what ASPR-related technological bottlenecks have been solved with the incorporation of LLM technology. We summarize the performance and issues of LLMs in ASPR, and investigate the attitudes and reactions of publishers and academia to ASPR.
  arXiv  Detail & Related papers  (2025-01-17T17:56:58Z)
• EVOLvE: Evaluating and Optimizing LLMs For Exploration [76.66831821738927]
  Large language models (LLMs) remain under-studied in scenarios requiring optimal decision-making under uncertainty. We measure LLMs' (in)ability to make optimal decisions in bandits, a state-less reinforcement learning setting relevant to many applications. Motivated by the existence of optimal exploration algorithms, we propose efficient ways to integrate this algorithmic knowledge into LLMs.
  arXiv  Detail & Related papers  (2024-10-08T17:54:03Z)
• Demystifying Issues, Causes and Solutions in LLM Open-Source Projects [15.881912703104376]
  We conducted an empirical study to understand the issues that practitioners encounter when developing and using LLM open-source software. We collected all closed issues from 15 LLM open-source projects and labelled issues that met our requirements. Our study results show that Model Issue is the most common issue faced by practitioners.
  arXiv  Detail & Related papers  (2024-09-25T02:16:45Z)
• A Reality check of the benefits of LLM in business [1.9181612035055007]
  Large language models (LLMs) have achieved remarkable performance in language understanding and generation tasks. This paper thoroughly examines the usefulness and readiness of LLMs for business processes.
  arXiv  Detail & Related papers  (2024-06-09T02:36:00Z)
• Can LLMs Solve longer Math Word Problems Better? [47.227621867242]
  Math Word Problems (MWPs) are crucial for evaluating the capability of Large Language Models (LLMs) This study pioneers the exploration of Context Length Generalizability (CoLeG) Two novel metrics are proposed to assess the efficacy and resilience of LLMs in solving these problems.
  arXiv  Detail & Related papers  (2024-05-23T17:13:50Z)
• A Survey on the Memory Mechanism of Large Language Model based Agents [66.4963345269611]
  Large language model (LLM) based agents have recently attracted much attention from the research and industry communities. LLM-based agents are featured in their self-evolving capability, which is the basis for solving real-world problems. The key component to support agent-environment interactions is the memory of the agents.
  arXiv  Detail & Related papers  (2024-04-21T01:49:46Z)
• Benchmarking LLMs on the Semantic Overlap Summarization Task [9.656095701778975]
  This paper comprehensively evaluates Large Language Models (LLMs) on the Semantic Overlap Summarization (SOS) task. We report well-established metrics like ROUGE, BERTscore, and SEM-F1$ on two different datasets of alternative narratives.
  arXiv  Detail & Related papers  (2024-02-26T20:33:50Z)
• LLMs with Industrial Lens: Deciphering the Challenges and Prospects -- A Survey [8.149749907267054]
  Large language models (LLMs) have become the secret ingredient driving numerous industrial applications. This paper unravels and evaluate the obstacles and opportunities inherent in leveraging LLMs within an industrial context.
  arXiv  Detail & Related papers  (2024-02-22T13:52:02Z)
• The Efficiency Spectrum of Large Language Models: An Algorithmic Survey [54.19942426544731]
  The rapid growth of Large Language Models (LLMs) has been a driving force in transforming various domains. This paper examines the multi-faceted dimensions of efficiency essential for the end-to-end algorithmic development of LLMs.
  arXiv  Detail & Related papers  (2023-12-01T16:00:25Z)
• A Comprehensive Overview of Large Language Models [68.22178313875618]
  Large Language Models (LLMs) have recently demonstrated remarkable capabilities in natural language processing tasks. This article provides an overview of the existing literature on a broad range of LLM-related concepts.
  arXiv  Detail & Related papers  (2023-07-12T20:01:52Z)
• How Can Recommender Systems Benefit from Large Language Models: A Survey [82.06729592294322]
  Large language models (LLM) have shown impressive general intelligence and human-like capabilities. We conduct a comprehensive survey on this research direction from the perspective of the whole pipeline in real-world recommender systems.
  arXiv  Detail & Related papers  (2023-06-09T11:31:50Z)
• Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey [100.24095818099522]
  Large language models (LLMs) have significantly advanced the field of natural language processing (NLP) They provide a highly useful, task-agnostic foundation for a wide range of applications. However, directly applying LLMs to solve sophisticated problems in specific domains meets many hurdles.
  arXiv  Detail & Related papers  (2023-05-30T03:00:30Z)
• Sentiment Analysis in the Era of Large Language Models: A Reality Check [69.97942065617664]
  This paper investigates the capabilities of large language models (LLMs) in performing various sentiment analysis tasks. We evaluate performance across 13 tasks on 26 datasets and compare the results against small language models (SLMs) trained on domain-specific datasets.
  arXiv  Detail & Related papers  (2023-05-24T10:45:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.