A Topic Coverage Approach to Evaluation of Topic Models

• URL: http://arxiv.org/abs/2012.06274v1
• Date: Fri, 11 Dec 2020 12:08:27 GMT
• Title: A Topic Coverage Approach to Evaluation of Topic Models
• Authors: Damir Koren\v{c}i\'c (1), Strahil Ristov (1), Jelena Repar (1), Jan \v{S}najder (2) ((1) Rudjer Bo\v{s}kovi\'c Institute, Croatia, (2) University of Zagreb, Faculty of Electrical Engineering and Computing, Croatia)
• Abstract summary: We investigate an approach to topic model evaluation based on measuring topic coverage. We demonstrate the benefits of the approach by evaluating, in a series of experiments, different types of topic models. The contributions of the paper include the measures of coverage and the recommendations for the use of topic models for topic discovery.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: When topic models are used for discovery of topics in text collections, a question that arises naturally is how well the model-induced topics correspond to topics of interest to the analyst. We investigate an approach to topic model evaluation based on measuring topic coverage, and propose measures of coverage based on matching between model topics and reference topics. We demonstrate the benefits of the approach by evaluating, in a series of experiments, different types of topic models on two distinct text domains. The experiments include evaluation of model quality, analysis of coverage of distinct topic categories, and the relation between coverage and other topic model evaluation methods. The contributions of the paper include the measures of coverage and the recommendations for the use of topic models for topic discovery.

Related papers

• Embedded Topic Models Enhanced by Wikification [3.082729239227955]
  We incorporate the Wikipedia knowledge into a neural topic model to make it aware of named entities. Our experiments show that our method improves the performance of neural topic models in generalizability.
  arXiv  Detail & Related papers  (2024-10-03T12:39:14Z)
• Interactive Topic Models with Optimal Transport [75.26555710661908]
  We present EdTM, as an approach for label name supervised topic modeling. EdTM models topic modeling as an assignment problem while leveraging LM/LLM based document-topic affinities.
  arXiv  Detail & Related papers  (2024-06-28T13:57:27Z)
• GINopic: Topic Modeling with Graph Isomorphism Network [0.8962460460173959]
  We introduce GINopic, a topic modeling framework based on graph isomorphism networks to capture the correlation between words. We demonstrate the effectiveness of GINopic compared to existing topic models and highlight its potential for advancing topic modeling.
  arXiv  Detail & Related papers  (2024-04-02T17:18:48Z)
• Neural Multimodal Topic Modeling: A Comprehensive Evaluation [18.660262940980477]
  This paper presents the first systematic and comprehensive evaluation of multimodal topic modeling. We propose two novel topic modeling solutions and two novel evaluation metrics. Overall, our evaluation on an unprecedented rich and diverse collection of datasets indicates that both of our models generate coherent and diverse topics.
  arXiv  Detail & Related papers  (2024-03-26T01:29:46Z)
• The Geometric Structure of Topic Models [0.0]
  Despite their widespread use in research and application, an in-depth analysis of topic models is still an open research topic. We propose an incidence-geometric method for deriving an ordinal structure from flat topic models. We present a new visualization paradigm for concept hierarchies based on ordinal motifs.
  arXiv  Detail & Related papers  (2024-03-06T10:53:51Z)
• TopicAdapt- An Inter-Corpora Topics Adaptation Approach [27.450275637652418]
  This paper proposes a neural topic model, TopicAdapt, that can adapt relevant topics from a related source corpus and also discover new topics in a target corpus that are absent in the source corpus. Experiments over multiple datasets from diverse domains show the superiority of the proposed model against the state-of-the-art topic models.
  arXiv  Detail & Related papers  (2023-10-08T02:56:44Z)
• Are Neural Topic Models Broken? [81.15470302729638]
  We study the relationship between automated and human evaluation of topic models. We find that neural topic models fare worse in both respects compared to an established classical method.
  arXiv  Detail & Related papers  (2022-10-28T14:38:50Z)
• Knowledge-Aware Bayesian Deep Topic Model [50.58975785318575]
  We propose a Bayesian generative model for incorporating prior domain knowledge into hierarchical topic modeling. Our proposed model efficiently integrates the prior knowledge and improves both hierarchical topic discovery and document representation.
  arXiv  Detail & Related papers  (2022-09-20T09:16:05Z)
• Author Clustering and Topic Estimation for Short Texts [69.54017251622211]
  We propose a novel model that expands on the Latent Dirichlet Allocation by modeling strong dependence among the words in the same document. We also simultaneously cluster users, removing the need for post-hoc cluster estimation. Our method performs as well as -- or better -- than traditional approaches to problems arising in short text.
  arXiv  Detail & Related papers  (2021-06-15T20:55:55Z)
• ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining [61.82562838486632]
  We crowdsource four new datasets on diverse online conversation forms of news comments, discussion forums, community question answering forums, and email threads. We benchmark state-of-the-art models on our datasets and analyze characteristics associated with the data.
  arXiv  Detail & Related papers  (2021-06-01T22:17:13Z)
• Improving Neural Topic Models using Knowledge Distillation [84.66983329587073]
  We use knowledge distillation to combine the best attributes of probabilistic topic models and pretrained transformers. Our modular method can be straightforwardly applied with any neural topic model to improve topic quality.
  arXiv  Detail & Related papers  (2020-10-05T22:49:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.