Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Review

• URL: http://arxiv.org/abs/2501.12557v1
• Date: Wed, 22 Jan 2025 00:31:51 GMT
• Title: Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Review
• Authors: Rock Yuren Pang, Hope Schroeder, Kynnedy Simone Smith, Solon Barocas, Ziang Xiao, Emily Tseng, Danielle Bragg,
• Abstract summary: Large language models (LLMs) have been positioned to revolutionize HCI. To-date, there has been little understanding of LLMs' uptake in HCI. We review 153 CHI papers from 2020-24 that engage with LLMs.
• Score: 8.524433537542484
• License:
• Abstract: Large language models (LLMs) have been positioned to revolutionize HCI, by reshaping not only the interfaces, design patterns, and sociotechnical systems that we study, but also the research practices we use. To-date, however, there has been little understanding of LLMs' uptake in HCI. We address this gap via a systematic literature review of 153 CHI papers from 2020-24 that engage with LLMs. We taxonomize: (1) domains where LLMs are applied; (2) roles of LLMs in HCI projects; (3) contribution types; and (4) acknowledged limitations and risks. We find LLM work in 10 diverse domains, primarily via empirical and artifact contributions. Authors use LLMs in five distinct roles, including as research tools or simulated users. Still, authors often raise validity and reproducibility concerns, and overwhelmingly study closed models. We outline opportunities to improve HCI research with and on LLMs, and provide guiding questions for researchers to consider the validity and appropriateness of LLM-related work.

Related papers

• Large language models for automated scholarly paper review: A survey [2.41072532750517]
  This paper aims to provide a holistic view of automated scholarly paper review (ASPR) in the era of large language models (LLMs) We review what ASPR-related technological bottlenecks have been solved with the incorporation of LLM technology. We summarize the performance and issues of LLMs in ASPR, and investigate the attitudes and reactions of publishers and academia to ASPR.
  arXiv  Detail & Related papers  (2025-01-17T17:56:58Z)
• Transforming Scholarly Landscapes: Influence of Large Language Models on Academic Fields beyond Computer Science [77.31665252336157]
  Large Language Models (LLMs) have ushered in a transformative era in Natural Language Processing (NLP) This work empirically examines the influence and use of LLMs in fields beyond NLP.
  arXiv  Detail & Related papers  (2024-09-29T01:32:35Z)
• LLM-based Multi-Agent Reinforcement Learning: Current and Future Directions [8.55917897789612]
  We focus on the cooperative tasks of multiple agents with a common goal and communication among them. We also consider human-in/on-the-loop scenarios enabled by the language component in the framework.
  arXiv  Detail & Related papers  (2024-05-17T22:10:23Z)
• Using Large Language Models for Natural Language Processing Tasks in Requirements Engineering: A Systematic Guideline [2.6644624823848426]
  Large Language Models (LLMs) are the cornerstone in automating Requirements Engineering (RE) tasks. This chapter aims to furnish readers with essential knowledge about LLMs in its initial segment. It provides a comprehensive guideline tailored for students, researchers, and practitioners on harnessing LLMs to address their specific objectives.
  arXiv  Detail & Related papers  (2024-02-21T14:00:52Z)
• Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs [60.40396361115776]
  This paper introduces a novel collaborative approach, namely SlimPLM, that detects missing knowledge in large language models (LLMs) with a slim proxy model. We employ a proxy model which has far fewer parameters, and take its answers as answers. Heuristic answers are then utilized to predict the knowledge required to answer the user question, as well as the known and unknown knowledge within the LLM.
  arXiv  Detail & Related papers  (2024-02-19T11:11:08Z)
• A Survey on Large Language Models for Software Engineering [15.468484685849983]
  Large Language Models (LLMs) are used to automate a broad range of Software Engineering (SE) tasks. This paper summarizes the current state-of-the-art research in the LLM-based SE community.
  arXiv  Detail & Related papers  (2023-12-23T11:09:40Z)
• Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity [61.54815512469125]
  This survey addresses the crucial issue of factuality in Large Language Models (LLMs) As LLMs find applications across diverse domains, the reliability and accuracy of their outputs become vital.
  arXiv  Detail & Related papers  (2023-10-11T14:18:03Z)
• Large Language Models for Software Engineering: A Systematic Literature Review [34.12458948051519]
  Large Language Models (LLMs) have significantly impacted numerous domains, including Software Engineering (SE) We select and analyze 395 research papers from January 2017 to January 2024 to answer four key research questions (RQs) From the answers to these RQs, we discuss the current state-of-the-art and trends, identifying gaps in existing research, and flagging promising areas for future study.
  arXiv  Detail & Related papers  (2023-08-21T10:37:49Z)
• Position: Key Claims in LLM Research Have a Long Tail of Footnotes [81.14898541318198]
  We argue that we do not have a working definition of Large Language Models (LLMs) We critically examine five common claims regarding their properties. We conclude with suggestions for future research directions and their framing.
  arXiv  Detail & Related papers  (2023-08-14T13:00:53Z)
• Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation [109.8527403904657]
  We show that large language models (LLMs) possess unwavering confidence in their knowledge and cannot handle the conflict between internal and external knowledge well. Retrieval augmentation proves to be an effective approach in enhancing LLMs' awareness of knowledge boundaries. We propose a simple method to dynamically utilize supporting documents with our judgement strategy.
  arXiv  Detail & Related papers  (2023-07-20T16:46:10Z)
• A Comprehensive Overview of Large Language Models [68.22178313875618]
  Large Language Models (LLMs) have recently demonstrated remarkable capabilities in natural language processing tasks. This article provides an overview of the existing literature on a broad range of LLM-related concepts.
  arXiv  Detail & Related papers  (2023-07-12T20:01:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.