Computational thematics: Comparing algorithms for clustering the genres of literary fiction

• URL: http://arxiv.org/abs/2305.11251v1
• Date: Thu, 18 May 2023 18:32:03 GMT
• Title: Computational thematics: Comparing algorithms for clustering the genres of literary fiction
• Abstract summary: This paper compares a variety of algorithms for unsupervised learning of thematic similarities between texts. Every combination of algorithms is given a task to cluster a corpus of books belonging to four pre-tagged genres of fiction. This clustering is validated against the "ground truth" genre labels.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: What are the best methods of capturing thematic similarity between literary texts? Knowing the answer to this question would be useful for automatic clustering of book genres, or any other thematic grouping. This paper compares a variety of algorithms for unsupervised learning of thematic similarities between texts, which we call "computational thematics". These algorithms belong to three steps of analysis: text preprocessing, extraction of text features, and measuring distances between the lists of features. Each of these steps includes a variety of options. We test all the possible combinations of these options: every combination of algorithms is given a task to cluster a corpus of books belonging to four pre-tagged genres of fiction. This clustering is then validated against the "ground truth" genre labels. Such comparison of algorithms allows us to learn the best and the worst combinations for computational thematic analysis. To illustrate the sharp difference between the best and the worst methods, we then cluster 5000 random novels from the HathiTrust corpus of fiction.

Related papers

• A Comparison of Document Similarity Algorithms [0.0]
  It addresses the most effective document similarity algorithm by categorizing them into 3 types of document similarity algorithms. The most effective algorithms in each category are also compared in our work using a series of benchmark datasets and evaluations.
  arXiv  Detail & Related papers  (2023-04-03T19:50:55Z)
• A Gold Standard Dataset for the Reviewer Assignment Problem [117.59690218507565]
  "Similarity score" is a numerical estimate of the expertise of a reviewer in reviewing a paper. Our dataset consists of 477 self-reported expertise scores provided by 58 researchers. For the task of ordering two papers in terms of their relevance for a reviewer, the error rates range from 12%-30% in easy cases to 36%-43% in hard cases.
  arXiv  Detail & Related papers  (2023-03-23T16:15:03Z)
• Graph-based Semantical Extractive Text Analysis [0.0]
  In this work, we improve the results of the TextRank algorithm by incorporating the semantic similarity between parts of the text. Aside from keyword extraction and text summarization, we develop a topic clustering algorithm based on our framework.
  arXiv  Detail & Related papers  (2022-12-19T18:30:26Z)
• Word Embeddings and Validity Indexes in Fuzzy Clustering [5.063728016437489]
  fuzzy-based analysis of various vector representations of words, i.e., word embeddings. We use two popular fuzzy clustering algorithms on count-based word embeddings, with different methods and dimensionality. We evaluate results of experiments with various clustering validity indexes to compare different algorithm variation with different embeddings accuracy.
  arXiv  Detail & Related papers  (2022-04-26T18:08:19Z)
• LeQua@CLEF2022: Learning to Quantify [76.22817970624875]
  LeQua 2022 is a new lab for the evaluation of methods for learning to quantify'' in textual datasets. The goal of this lab is to provide a setting for the comparative evaluation of methods for learning to quantify, both in the binary setting and in the single-label multiclass setting.
  arXiv  Detail & Related papers  (2021-11-22T14:54:20Z)
• Evaluating Various Tokenizers for Arabic Text Classification [4.110108749051656]
  We introduce three new tokenization algorithms for Arabic and compare them to three other baselines using unsupervised evaluations. Our experiments show that the performance of such tokenization algorithms depends on the size of the dataset, type of the task, and the amount of morphology that exists in the dataset.
  arXiv  Detail & Related papers  (2021-06-14T16:05:58Z)
• Determinantal Beam Search [75.84501052642361]
  Beam search is a go-to strategy for decoding neural sequence models. In use-cases that call for multiple solutions, a diverse or representative set is often desired. By posing iterations in beam search as a series of subdeterminant problems, we can turn the algorithm into a diverse subset selection process.
  arXiv  Detail & Related papers  (2021-06-14T13:01:46Z)
• Match-Ignition: Plugging PageRank into Transformer for Long-form Text Matching [66.71886789848472]
  We propose a novel hierarchical noise filtering model, namely Match-Ignition, to tackle the effectiveness and efficiency problem. The basic idea is to plug the well-known PageRank algorithm into the Transformer, to identify and filter both sentence and word level noisy information. Noisy sentences are usually easy to detect because the sentence is the basic unit of a long-form text, so we directly use PageRank to filter such information.
  arXiv  Detail & Related papers  (2021-01-16T10:34:03Z)
• Classification and Clustering of arXiv Documents, Sections, and Abstracts, Comparing Encodings of Natural and Mathematical Language [8.522576207528017]
  We show how selecting and combining encodings of natural and mathematical language affect classification and clustering of documents with mathematical content. Our encodings achieve classification accuracies up to $82.8%$ and cluster purities up to $69.4%$. We show that the computer outperforms a human expert when classifying documents.
  arXiv  Detail & Related papers  (2020-05-22T06:16:32Z)
• IMRAM: Iterative Matching with Recurrent Attention Memory for Cross-Modal Image-Text Retrieval [105.77562776008459]
  Existing methods leverage the attention mechanism to explore such correspondence in a fine-grained manner. It may be difficult to optimally capture such sophisticated correspondences in existing methods. We propose an Iterative Matching with Recurrent Attention Memory (IMRAM) method, in which correspondences are captured with multiple steps of alignments.
  arXiv  Detail & Related papers  (2020-03-08T12:24:41Z)
• Optimal Clustering from Noisy Binary Feedback [75.17453757892152]
  We study the problem of clustering a set of items from binary user feedback. We devise an algorithm with a minimal cluster recovery error rate. For adaptive selection, we develop an algorithm inspired by the derivation of the information-theoretical error lower bounds.
  arXiv  Detail & Related papers  (2019-10-14T09:18:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.