Interactive Reasoning: Visualizing and Controlling Chain-of-Thought Reasoning in Large Language Models

• URL: http://arxiv.org/abs/2506.23678v1
• Date: Mon, 30 Jun 2025 10:00:43 GMT
• Title: Interactive Reasoning: Visualizing and Controlling Chain-of-Thought Reasoning in Large Language Models
• Authors: Rock Yuren Pang, K. J. Kevin Feng, Shangbin Feng, Chu Li, Weijia Shi, Yulia Tsvetkov, Jeffrey Heer, Katharina Reinecke,
• Abstract summary: We introduce Interactive Reasoning, an interaction design that visualizes chain-of-thought outputs as a hierarchy of topics.<n>We implement interactive reasoning in Hippo, a prototype for AI-assisted decision making in the face of uncertain trade-offs.
• Score: 54.85405423240165
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The output quality of large language models (LLMs) can be improved via "reasoning": generating segments of chain-of-thought (CoT) content to further condition the model prior to producing user-facing output. While these chains contain valuable information, they are verbose and lack explicit organization, making them tedious to review. Moreover, they lack opportunities for user feedback, such as to remove unwanted considerations, add desired ones, or clarify unclear assumptions. We introduce Interactive Reasoning, an interaction design that visualizes chain-of-thought outputs as a hierarchy of topics and enables user review and modification. We implement interactive reasoning in Hippo, a prototype for AI-assisted decision making in the face of uncertain trade-offs. In a user study with 16 participants, we find that interactive reasoning in Hippo allows users to quickly identify and interrupt erroneous generations, efficiently steer the model towards customized responses, and better understand both model reasoning and model outputs. Our work contributes to a new paradigm that incorporates user oversight into LLM reasoning processes.

Related papers

• Predicting thinking time in Reasoning models [42.58699486487709]
  Reasoning models produce long, hidden chains of thought.<n>Users have little insight into how much time the model will spend reasoning before returning an answer.
  arXiv  Detail & Related papers  (2025-06-29T15:01:01Z)
• PixelThink: Towards Efficient Chain-of-Pixel Reasoning [70.32510083790069]
  PixelThink is a simple yet effective scheme that integrates externally estimated task difficulty and internally measured model uncertainty.<n>It learns to compress reasoning length in accordance with scene complexity and predictive confidence.<n> Experimental results demonstrate that the proposed approach improves both reasoning efficiency and overall segmentation performance.
  arXiv  Detail & Related papers  (2025-05-29T17:55:49Z)
• VAGUE: Visual Contexts Clarify Ambiguous Expressions [15.140825578254324]
  We introduce VAGUE, a benchmark evaluating multimodal AI systems' ability to integrate visual context for intent.<n>VAGUE consists of 1.6K ambiguous textual expressions, each paired with an image and multiple-choice interpretations.<n>Our experiments reveal that existing multimodal AI models struggle to infer the speaker's true intent.
  arXiv  Detail & Related papers  (2024-11-21T14:01:42Z)
• Unsupervised Model Diagnosis [49.36194740479798]
  This paper proposes Unsupervised Model Diagnosis (UMO) to produce semantic counterfactual explanations without any user guidance. Our approach identifies and visualizes changes in semantics, and then matches these changes to attributes from wide-ranging text sources.
  arXiv  Detail & Related papers  (2024-10-08T17:59:03Z)
• From Feature Importance to Natural Language Explanations Using LLMs with RAG [4.204990010424084]
  We introduce traceable question-answering, leveraging an external knowledge repository to inform responses of Large Language Models (LLMs) This knowledge repository comprises contextual details regarding the model's output, containing high-level features, feature importance, and alternative probabilities. We integrate four key characteristics - social, causal, selective, and contrastive - drawn from social science research on human explanations into a single-shot prompt, guiding the response generation process.
  arXiv  Detail & Related papers  (2024-07-30T17:27:20Z)
• Unlocking the Potential of User Feedback: Leveraging Large Language Model as User Simulator to Enhance Dialogue System [65.93577256431125]
  We propose an alternative approach called User-Guided Response Optimization (UGRO) to combine it with a smaller task-oriented dialogue model. This approach uses LLM as annotation-free user simulator to assess dialogue responses, combining them with smaller fine-tuned end-to-end TOD models. Our approach outperforms previous state-of-the-art (SOTA) results.
  arXiv  Detail & Related papers  (2023-06-16T13:04:56Z)
• Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillings [61.04460792203266]
  We introduce VCoT, a novel method that leverages chain-of-thought prompting with vision-language grounding to bridge the logical gaps within sequential data. Our method uses visual guidance to generate synthetic multimodal infillings that add consistent and novel information to reduce the logical gaps for downstream tasks.
  arXiv  Detail & Related papers  (2023-05-03T17:58:29Z)
• Leveraging Explanations in Interactive Machine Learning: An Overview [10.284830265068793]
  Explanations have gained an increasing level of interest in the AI and Machine Learning (ML) communities. This paper presents an overview of research where explanations are combined with interactive capabilities.
  arXiv  Detail & Related papers  (2022-07-29T07:46:11Z)
• Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations [97.91630330328815]
  We conduct a crowdsourcing study, where participants interact with deception detection models that have been trained to distinguish between genuine and fake hotel reviews. We observe that for a linear bag-of-words model, participants with access to the feature coefficients during training are able to cause a larger reduction in model confidence in the testing phase when compared to the no-explanation control.
  arXiv  Detail & Related papers  (2021-12-17T18:29:56Z)
• Intuitively Assessing ML Model Reliability through Example-Based Explanations and Editing Model Inputs [19.09848738521126]
  Interpretability methods aim to help users build trust in and understand the capabilities of machine learning models. We present two interface modules to facilitate a more intuitive assessment of model reliability.
  arXiv  Detail & Related papers  (2021-02-17T02:41:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.