Results of a Single Blind Literary Taste Test with Short Anonymized Novel Fragments

• URL: http://arxiv.org/abs/2011.01624v1
• Date: Tue, 3 Nov 2020 11:10:17 GMT
• Title: Results of a Single Blind Literary Taste Test with Short Anonymized Novel Fragments
• Authors: Andreas van Cranenburgh, Corina Koolen
• Abstract summary: It is an open question to what extent perceptions of literary quality are derived from text-intrinsic versus social factors. We report the results of a pilot study to gauge the effect of textual features on literary ratings of Dutch-language novels. We find moderate to strong correlations of questionnaire ratings with the survey ratings, but the predictions are closer to the survey ratings.
• Score: 4.695687634290403
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: It is an open question to what extent perceptions of literary quality are derived from text-intrinsic versus social factors. While supervised models can predict literary quality ratings from textual factors quite successfully, as shown in the Riddle of Literary Quality project (Koolen et al., 2020), this does not prove that social factors are not important, nor can we assume that readers make judgments on literary quality in the same way and based on the same information as machine learning models. We report the results of a pilot study to gauge the effect of textual features on literary ratings of Dutch-language novels by participants in a controlled experiment with 48 participants. In an exploratory analysis, we compare the ratings to those from the large reader survey of the Riddle in which social factors were not excluded, and to machine learning predictions of those literary ratings. We find moderate to strong correlations of questionnaire ratings with the survey ratings, but the predictions are closer to the survey ratings. Code and data: https://github.com/andreasvc/litquest

Related papers

• LFED: A Literary Fiction Evaluation Dataset for Large Language Models [58.85989777743013]
  We collect 95 literary fictions that are either originally written in Chinese or translated into Chinese, covering a wide range of topics across several centuries. We define a question taxonomy with 8 question categories to guide the creation of 1,304 questions. We conduct an in-depth analysis to ascertain how specific attributes of literary fictions (e.g., novel types, character numbers, the year of publication) impact LLM performance in evaluations.
  arXiv  Detail & Related papers  (2024-05-16T15:02:24Z)
• A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [58.6354685593418]
  This paper proposes several article-level, field-normalized, and large language model-empowered bibliometric indicators to evaluate reviews. The newly emerging AI-generated literature reviews are also appraised. This work offers insights into the current challenges of literature reviews and envisions future directions for their development.
  arXiv  Detail & Related papers  (2024-02-20T11:28:50Z)
• Semantic Properties of cosine based bias scores for word embeddings [48.0753688775574]
  We propose requirements for bias scores to be considered meaningful for quantifying biases. We analyze cosine based scores from the literature with regard to these requirements. We underline these findings with experiments to show that the bias scores' limitations have an impact in the application case.
  arXiv  Detail & Related papers  (2024-01-27T20:31:10Z)
• What factors influence the popularity of user-generated text in the creative domain? A case study of book reviews [0.0]
  This study investigates a range of psychological, lexical, semantic, and readability features of book reviews to elucidate the factors underlying their perceived popularity. We employ traditional machine learning classifiers and transformer-based fine-tuned language models with n-gram features to automatically determine review popularity. Our findings indicate that, with the exception of a few features, most attributes do not exhibit significant differences between popular and non-popular review groups.
  arXiv  Detail & Related papers  (2023-11-12T02:54:11Z)
• An Analysis of Reader Engagement in Literary Fiction through Eye Tracking and Linguistic Features [11.805980147608178]
  We analyzed the significance of various qualities of the text in predicting how engaging a reader is likely to find it. Furthering our understanding of what captivates readers in fiction will help better inform models used in creative narrative generation.
  arXiv  Detail & Related papers  (2023-06-06T22:14:59Z)
• Computational analyses of the topics, sentiments, literariness, creativity and beauty of texts in a large Corpus of English Literature [0.0]
  The Gutenberg Literary English Corpus (GLEC) provides a rich source of textual data for research in digital humanities, computational linguistics or neurocognitive poetics. We report the results of three studies providing i) topic and sentiment analyses for six text categories of GLEC and its >100 authors, ii) novel measures of semantic complexity as indices of the literariness, creativity and book beauty of the works in GLEC.
  arXiv  Detail & Related papers  (2022-01-12T08:16:52Z)
• Modeling Social Readers: Novel Tools for Addressing Reception from Online Book Reviews [0.0]
  We study the readers' distillation of the main storylines in a novel using a corpus of reviews of five popular novels. We make three important contributions to the study of infinite vocabulary networks. We present a new sequencing algorithm, REV2SEQ, that generates a consensus sequence of events based on partial trajectories aggregated from the reviews.
  arXiv  Detail & Related papers  (2021-05-03T20:10:14Z)
• My Teacher Thinks The World Is Flat! Interpreting Automatic Essay Scoring Mechanism [71.34160809068996]
  Recent work shows that automated scoring systems are prone to even common-sense adversarial samples. We utilize recent advances in interpretability to find the extent to which features such as coherence, content and relevance are important for automated scoring mechanisms. We also find that since the models are not semantically grounded with world-knowledge and common sense, adding false facts such as the world is flat'' actually increases the score instead of decreasing it.
  arXiv  Detail & Related papers  (2020-12-27T06:19:20Z)
• Curious Case of Language Generation Evaluation Metrics: A Cautionary Tale [52.663117551150954]
  A few popular metrics remain as the de facto metrics to evaluate tasks such as image captioning and machine translation. This is partly due to ease of use, and partly because researchers expect to see them and know how to interpret them. In this paper, we urge the community for more careful consideration of how they automatically evaluate their models.
  arXiv  Detail & Related papers  (2020-10-26T13:57:20Z)
• Quasi Error-free Text Classification and Authorship Recognition in a large Corpus of English Literature based on a Novel Feature Set [0.0]
  We show that in the entire GLEC quasi error-free text classification and authorship recognition is possible with a method using the same set of five style and five content features. Our data pave the way for many future computational and empirical studies of literature or experiments in reading psychology.
  arXiv  Detail & Related papers  (2020-10-21T07:39:55Z)
• Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition [46.57105755981092]
  We publish a multilingual Twitter corpus for the task of hate speech detection. The corpus covers five languages: English, Italian, Polish, Portuguese and Spanish. We evaluate the inferred demographic labels with a crowdsourcing platform.
  arXiv  Detail & Related papers  (2020-02-24T16:45:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.