A User-Centered, Interactive, Human-in-the-Loop Topic Modelling System

• URL: http://arxiv.org/abs/2304.01774v1
• Date: Tue, 4 Apr 2023 13:05:10 GMT
• Title: A User-Centered, Interactive, Human-in-the-Loop Topic Modelling System
• Authors: Zheng Fang, Lama Alqazlan, Du Liu, Yulan He, and Rob Procter
• Abstract summary: Human-in-the-loop topic modelling incorporates users' knowledge into the modelling process, enabling them to refine the model iteratively. Recent research has demonstrated the value of user feedback, but there are still issues to consider. We developed a novel, interactive human-in-the-loop topic modeling system with a user-friendly interface.
• Score: 32.065158970382036
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human-in-the-loop topic modelling incorporates users' knowledge into the modelling process, enabling them to refine the model iteratively. Recent research has demonstrated the value of user feedback, but there are still issues to consider, such as the difficulty in tracking changes, comparing different models and the lack of evaluation based on real-world examples of use. We developed a novel, interactive human-in-the-loop topic modeling system with a user-friendly interface that enables users compare and record every step they take, and a novel topic words suggestion feature to help users provide feedback that is faithful to the ground truth. Our system also supports not only what traditional topic models can do, i.e., learning the topics from the whole corpus, but also targeted topic modelling, i.e., learning topics for specific aspects of the corpus. In this article, we provide an overview of the system and present the results of a series of user studies designed to assess the value of the system in progressively more realistic applications of topic modelling.

Related papers

• Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and Algorithms [91.19304518033144]
  We aim to align vision models with human aesthetic standards in a retrieval system. We propose a preference-based reinforcement learning method that fine-tunes the vision models to better align the vision models with human aesthetics.
  arXiv  Detail & Related papers  (2024-06-13T17:59:20Z)
• Towards Personalized Evaluation of Large Language Models with An Anonymous Crowd-Sourcing Platform [64.76104135495576]
  We propose a novel anonymous crowd-sourcing evaluation platform, BingJian, for large language models. Through this platform, users have the opportunity to submit their questions, testing the models on a personalized and potentially broader range of capabilities.
  arXiv  Detail & Related papers  (2024-03-13T07:31:20Z)
• Improving the TENOR of Labeling: Re-evaluating Topic Models for Content Analysis [5.757610495733924]
  We conduct the first evaluation of neural, supervised and classical topic models in an interactive task based setting. We show that current automated metrics do not provide a complete picture of topic modeling capabilities.
  arXiv  Detail & Related papers  (2024-01-29T17:54:04Z)
• Labeled Interactive Topic Models [10.555664965166232]
  We introduce a user-friendly interaction for neural topic models. This interaction permits users to assign a word label to a topic. We evaluate our method through a human study, where users can relabel topics to find relevant documents.
  arXiv  Detail & Related papers  (2023-11-15T23:18:01Z)
• Understanding User Intent Modeling for Conversational Recommender Systems: A Systematic Literature Review [1.3630870408844922]
  We conducted a systematic literature review to gather data on models typically employed in designing conversational recommender systems. We developed a decision model to assist researchers in selecting the most suitable models for their systems. Our study contributes practical insights and a comprehensive understanding of user intent modeling, empowering the development of more effective and personalized conversational recommender systems.
  arXiv  Detail & Related papers  (2023-08-05T22:50:21Z)
• Foundational Models Defining a New Era in Vision: A Survey and Outlook [151.49434496615427]
  Vision systems to see and reason about the compositional nature of visual scenes are fundamental to understanding our world. The models learned to bridge the gap between such modalities coupled with large-scale training data facilitate contextual reasoning, generalization, and prompt capabilities at test time. The output of such models can be modified through human-provided prompts without retraining, e.g., segmenting a particular object by providing a bounding box, having interactive dialogues by asking questions about an image or video scene or manipulating the robot's behavior through language instructions.
  arXiv  Detail & Related papers  (2023-07-25T17:59:18Z)
• Are Neural Topic Models Broken? [81.15470302729638]
  We study the relationship between automated and human evaluation of topic models. We find that neural topic models fare worse in both respects compared to an established classical method.
  arXiv  Detail & Related papers  (2022-10-28T14:38:50Z)
• Modeling User Behaviour in Research Paper Recommendation System [8.980876474818153]
  A user intention model is proposed based on deep sequential topic analysis. The model predicts a user's intention in terms of the topic of interest. The proposed approach introduces a new road map to model a user activity suitable for the design of a research paper recommendation system.
  arXiv  Detail & Related papers  (2021-07-16T11:31:03Z)
• Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence [62.826466543958624]
  We look at the standardization gap and the validation gap in topic model evaluation. Recent models relying on neural components surpass classical topic models according to these metrics. We use automatic coherence along with the two most widely accepted human judgment tasks, namely, topic rating and word intrusion.
  arXiv  Detail & Related papers  (2021-07-05T17:58:52Z)
• User Ex Machina : Simulation as a Design Probe in Human-in-the-Loop Text Analytics [29.552736183006672]
  We conduct a simulation-based analysis of human-centered interactions with topic models. We find that user interactions have impacts that differ in magnitude but often negatively affect the quality of the resulting modelling.
  arXiv  Detail & Related papers  (2021-01-06T19:44:11Z)
• Improving Neural Topic Models using Knowledge Distillation [84.66983329587073]
  We use knowledge distillation to combine the best attributes of probabilistic topic models and pretrained transformers. Our modular method can be straightforwardly applied with any neural topic model to improve topic quality.
  arXiv  Detail & Related papers  (2020-10-05T22:49:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.