Blending Queries and Conversations: Understanding Tactics, Trust, Verification, and System Choice in Web Search and Chat Interactions

• URL: http://arxiv.org/abs/2504.05156v1
• Date: Mon, 07 Apr 2025 14:59:55 GMT
• Title: Blending Queries and Conversations: Understanding Tactics, Trust, Verification, and System Choice in Web Search and Chat Interactions
• Authors: Kerstin Mayerhofer, Rob Capra, David Elsweiler,
• Abstract summary: This paper presents a user study where participants used an interface combining Web Search and a Generative AI-Chat feature to solve health-related information tasks.<n>We study how people behaved with the interface, why they behaved in certain ways, and what the outcomes of these behaviours were.
• Score: 0.8397730500554048
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents a user study (N=22) where participants used an interface combining Web Search and a Generative AI-Chat feature to solve health-related information tasks. We study how people behaved with the interface, why they behaved in certain ways, and what the outcomes of these behaviours were. A think-aloud protocol captured their thought processes during searches. Our findings suggest that GenAI is neither a search panacea nor a major regression compared to standard Web Search interfaces. Qualitative and quantitative analyses identified 78 tactics across five categories and provided insight into how and why different interface features were used. We find evidence that pre-task confidence and trust both influenced which interface feature was used. In both systems, but particularly when using the chat feature, trust was often misplaced in favour of ease-of-use and seemingly perfect answers, leading to increased confidence post-search despite having incorrect results. We discuss what our findings mean in the context of our defined research questions and outline several open questions for future research.

Related papers

• Agentic Conversational Search with Contextualized Reasoning via Reinforcement Learning [66.52010873968383]
  We introduce a conversational agent that interleaves search and reasoning across turns, enabling exploratory and adaptive behaviors learned through reinforcement learning (RL) training.<n>The experimental results across four widely used conversational benchmarks demonstrate the effectiveness of our methods.
  arXiv  Detail & Related papers  (2026-01-19T14:55:54Z)
• Personality over Precision: Exploring the Influence of Human-Likeness on ChatGPT Use for Search [8.544772506500188]
  We examined user perceptions regarding trust, human-likeness (anthropomorphism), and design preferences between ChatGPT and Google.<n>Our analysis identified two distinct user groups: those who use both ChatGPT and Google daily (DUB), and those who primarily rely on Google (DUG)<n>The DUB group exhibited higher trust in ChatGPT, perceiving it as more human-like, and expressed greater willingness to trade factual accuracy for enhanced personalization and conversational flow.
  arXiv  Detail & Related papers  (2025-11-09T16:28:55Z)
• Generative Interfaces for Language Models [70.25765232527762]
  We propose a paradigm in which large language models (LLMs) respond to user queries by proactively generating user interfaces (UIs)<n>Our framework leverages structured interface-specific representations and iterative refinements to translate user queries into task-specific UIs.<n>Results show that generative interfaces consistently outperform conversational ones, with up to a 72% improvement in human preference.
  arXiv  Detail & Related papers  (2025-08-26T17:43:20Z)
• Understanding Mental Models of Generative Conversational Search and The Effect of Interface Transparency [9.630893350528709]
  Mental models are internal frameworks for understanding and predicting system behaviour.<n>Most user mental models were too abstract to support users in explaining individual search instances.<n>Findings suggest that mental models may pose a barrier to appropriate trust in conversational search.
  arXiv  Detail & Related papers  (2025-06-04T10:27:22Z)
• Quriosity: Analyzing Human Questioning Behavior and Causal Inquiry through Curiosity-Driven Queries [91.70689724416698]
  We present Quriosity, a collection of 13.5K naturally occurring questions from three diverse sources.<n>Our analysis reveals a significant presence of causal questions (up to 42%) in the dataset.
  arXiv  Detail & Related papers  (2024-05-30T17:55:28Z)
• Towards Detecting and Mitigating Cognitive Bias in Spoken Conversational Search [14.916529791823868]
  This paper draws upon insights from information seeking, psychology, cognitive science, and wearable sensors to provoke novel conversations in the community. We propose a framework including multimodal instruments and methods for experimental designs and settings.
  arXiv  Detail & Related papers  (2024-05-21T03:50:32Z)
• Social Commonsense-Guided Search Query Generation for Open-Domain Knowledge-Powered Conversations [66.16863141262506]
  We present a novel approach that focuses on generating internet search queries guided by social commonsense. Our proposed framework addresses passive user interactions by integrating topic tracking, commonsense response generation and instruction-driven query generation.
  arXiv  Detail & Related papers  (2023-10-22T16:14:56Z)
• From Data to Dialogue: Leveraging the Structure of Knowledge Graphs for Conversational Exploratory Search [4.861125297881693]
  We propose a knowledge-driven dialogue system for exploring news articles by asking natural language questions. Based on a user study with 54 participants, we empirically evaluate the effectiveness of the graph-based exploratory search.
  arXiv  Detail & Related papers  (2023-10-08T12:52:09Z)
• How does AI chat change search behaviors? [7.601937548486356]
  Generative AI tools such as chatGPT are poised to change the way people engage with online information. Microsoft announced their "new Bing" search system which incorporates chat and generative AI technology from OpenAI. Google has announced plans to deploy search interfaces that incorporate similar types of technology.
  arXiv  Detail & Related papers  (2023-07-07T20:41:26Z)
• WebCPM: Interactive Web Search for Chinese Long-form Question Answering [104.676752359777]
  Long-form question answering (LFQA) aims at answering complex, open-ended questions with detailed, paragraph-length responses. We introduce WebCPM, the first Chinese LFQA dataset. We collect 5,500 high-quality question-answer pairs, together with 14,315 supporting facts and 121,330 web search actions.
  arXiv  Detail & Related papers  (2023-05-11T14:47:29Z)
• Integrity and Junkiness Failure Handling for Embedding-based Retrieval: A Case Study in Social Network Search [26.705196461992845]
  Embedding based retrieval has seen its usage in a variety of search applications like e-commerce, social networking search etc. In this paper, we conduct an analysis of embedding-based retrieval launched in early 2021 on our social network search engine. We define two main categories of failures introduced by it, integrity and junkiness.
  arXiv  Detail & Related papers  (2023-04-18T20:53:47Z)
• Investigating Conversational Search Behavior For Domain Exploration [0.5512295869673147]
  This study investigates open-ended search behavior for navigation through unknown information landscapes. We apply statistical analyses and process mining techniques to uncover general information-seeking patterns across five different domains.
  arXiv  Detail & Related papers  (2023-01-10T17:43:03Z)
• Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Survey [71.43956423427397]
  We aim to identify the nonverbal cues and computational methodologies resulting in effective performance. This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings. Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3-4 persons equipped with microphones and cameras, respectively.
  arXiv  Detail & Related papers  (2022-07-20T13:37:57Z)
• First Contact: Unsupervised Human-Machine Co-Adaptation via Mutual Information Maximization [112.40598205054994]
  We formalize this idea as a completely unsupervised objective for optimizing interfaces. We conduct an observational study on 540K examples of users operating various keyboard and eye gaze interfaces for typing, controlling simulated robots, and playing video games. The results show that our mutual information scores are predictive of the ground-truth task completion metrics in a variety of domains.
  arXiv  Detail & Related papers  (2022-05-24T21:57:18Z)
• Mining Implicit Relevance Feedback from User Behavior for Web Question Answering [92.45607094299181]
  We make the first study to explore the correlation between user behavior and passage relevance. Our approach significantly improves the accuracy of passage ranking without extra human labeled data. In practice, this work has proved effective to substantially reduce the human labeling cost for the QA service in a global commercial search engine.
  arXiv  Detail & Related papers  (2020-06-13T07:02:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.