Sensecape: Enabling Multilevel Exploration and Sensemaking with Large Language Models

• URL: http://arxiv.org/abs/2305.11483v2
• Date: Wed, 30 Aug 2023 03:35:31 GMT
• Title: Sensecape: Enabling Multilevel Exploration and Sensemaking with Large Language Models
• Authors: Sangho Suh, Bryan Min, Srishti Palani, Haijun Xia
• Abstract summary: Sensecape is an interactive system designed to support complex information tasks with a large language model. Our within-subject user study reveals that Sensecape empowers users to explore more topics and structure their knowledge hierarchically.
• Score: 12.141818433363628
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: People are increasingly turning to large language models (LLMs) for complex information tasks like academic research or planning a move to another city. However, while they often require working in a nonlinear manner -- e.g., to arrange information spatially to organize and make sense of it, current interfaces for interacting with LLMs are generally linear to support conversational interaction. To address this limitation and explore how we can support LLM-powered exploration and sensemaking, we developed Sensecape, an interactive system designed to support complex information tasks with an LLM by enabling users to (1) manage the complexity of information through multilevel abstraction and (2) seamlessly switch between foraging and sensemaking. Our within-subject user study reveals that Sensecape empowers users to explore more topics and structure their knowledge hierarchically, thanks to the externalization of levels of abstraction. We contribute implications for LLM-based workflows and interfaces for information tasks.

Related papers

• Navigating the Unknown: A Chat-Based Collaborative Interface for Personalized Exploratory Tasks [35.09558253658275]
  This paper introduces the Collaborative Assistant for Personalized Exploration (CARE) CARE is a system designed to enhance personalization in exploratory tasks by combining a multi-agent LLM framework with a structured user interface. Our findings highlight CARE's potential to transform LLM-based systems from passive information retrievers to proactive partners in personalized problem-solving and exploration.
  arXiv  Detail & Related papers  (2024-10-31T15:30:55Z)
• RA-BLIP: Multimodal Adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-training [55.54020926284334]
  Multimodal Large Language Models (MLLMs) have recently received substantial interest, which shows their emerging potential as general-purpose models for various vision-language tasks. Retrieval augmentation techniques have proven to be effective plugins for both LLMs and MLLMs. In this study, we propose multimodal adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-training (RA-BLIP), a novel retrieval-augmented framework for various MLLMs.
  arXiv  Detail & Related papers  (2024-10-18T03:45:19Z)
• Cognitive LLMs: Towards Integrating Cognitive Architectures and Large Language Models for Manufacturing Decision-making [51.737762570776006]
  LLM-ACTR is a novel neuro-symbolic architecture that provides human-aligned and versatile decision-making. Our framework extracts and embeds knowledge of ACT-R's internal decision-making process as latent neural representations. Our experiments on novel Design for Manufacturing tasks show both improved task performance as well as improved grounded decision-making capability.
  arXiv  Detail & Related papers  (2024-08-17T11:49:53Z)
• Wiki-LLaVA: Hierarchical Retrieval-Augmented Generation for Multimodal LLMs [39.54891426369773]
  We focus on endowing such models with the capability of answering questions that require external knowledge. Our approach, termed Wiki-LLaVA, aims at integrating an external knowledge source of multimodal documents. We conduct extensive experiments on datasets tailored for visual question answering with external data and demonstrate the appropriateness of our approach.
  arXiv  Detail & Related papers  (2024-04-23T18:00:09Z)
• On the Multi-turn Instruction Following for Conversational Web Agents [83.51251174629084]
  We introduce a new task of Conversational Web Navigation, which necessitates sophisticated interactions that span multiple turns with both the users and the environment. We propose a novel framework, named self-reflective memory-augmented planning (Self-MAP), which employs memory utilization and self-reflection techniques.
  arXiv  Detail & Related papers  (2024-02-23T02:18:12Z)
• Exploring Interaction Patterns for Debugging: Enhancing Conversational Capabilities of AI-assistants [18.53732314023887]
  Large Language Models (LLMs) enable programmers to obtain natural language explanations for various software development tasks. LLMs often leap to action without sufficient context, giving rise to implicit assumptions and inaccurate responses. In this paper, we draw inspiration from interaction patterns and conversation analysis -- to design Robin, an enhanced conversational AI-assistant for debug.
  arXiv  Detail & Related papers  (2024-02-09T07:44:27Z)
• Empowering Language Models with Active Inquiry for Deeper Understanding [31.11672018840381]
  We introduce LaMAI (Language Model with Active Inquiry), designed to endow large language models with interactive engagement. LaMAI uses active learning techniques to raise the most informative questions, fostering a dynamic bidirectional dialogue. Our empirical studies, across a variety of complex datasets, demonstrate the effectiveness of LaMAI.
  arXiv  Detail & Related papers  (2024-02-06T05:24:16Z)
• INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning [59.07490387145391]
  Large language models (LLMs) have demonstrated impressive capabilities in various natural language processing tasks. Their application to information retrieval (IR) tasks is still challenging due to the infrequent occurrence of many IR-specific concepts in natural language. We introduce a novel instruction tuning dataset, INTERS, encompassing 20 tasks across three fundamental IR categories.
  arXiv  Detail & Related papers  (2024-01-12T12:10:28Z)
• LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving [87.1164964709168]
  This work employs Large Language Models (LLMs) as a decision-making component for complex autonomous driving scenarios. Extensive experiments demonstrate that our proposed method not only consistently surpasses baseline approaches in single-vehicle tasks, but also helps handle complex driving behaviors even multi-vehicle coordination.
  arXiv  Detail & Related papers  (2023-10-04T17:59:49Z)
• Low-code LLM: Graphical User Interface over Large Language Models [115.08718239772107]
  This paper introduces a novel human-LLM interaction framework, Low-code LLM. It incorporates six types of simple low-code visual programming interactions to achieve more controllable and stable responses. We highlight three advantages of the low-code LLM: user-friendly interaction, controllable generation, and wide applicability.
  arXiv  Detail & Related papers  (2023-04-17T09:27:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.