Automatic Quality Assessment of Wikipedia Articles -- A Systematic Literature Review

• URL: http://arxiv.org/abs/2310.02235v1
• Date: Tue, 3 Oct 2023 17:45:39 GMT
• Title: Automatic Quality Assessment of Wikipedia Articles -- A Systematic Literature Review
• Authors: Pedro Miguel Mo\'as, Carla Teixeira Lopes
• Abstract summary: We review existing methods for automatically measuring the quality of Wikipedia articles. We identify and comparing machine learning algorithms, article features, quality metrics, and used datasets. We hope that our analysis helps future researchers change that reality.
• Score: 0.8158530638728501
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Wikipedia is the world's largest online encyclopedia, but maintaining article quality through collaboration is challenging. Wikipedia designed a quality scale, but with such a manual assessment process, many articles remain unassessed. We review existing methods for automatically measuring the quality of Wikipedia articles, identifying and comparing machine learning algorithms, article features, quality metrics, and used datasets, examining 149 distinct studies, and exploring commonalities and gaps in them. The literature is extensive, and the approaches follow past technological trends. However, machine learning is still not widely used by Wikipedia, and we hope that our analysis helps future researchers change that reality.

Related papers

• A Test of Time: Predicting the Sustainable Success of Online Collaboration in Wikipedia [17.051622145253855]
  We develop machine learning models to predict the sustainable success of Wikipedia articles. We find that the longer an article takes to be recognized as high-quality, the more likely it is to maintain that status over time. Our analysis provides insights into broader collective actions beyond Wikipedia.
  arXiv  Detail & Related papers  (2024-10-24T20:42:53Z)
• Language-Agnostic Modeling of Wikipedia Articles for Content Quality Assessment across Languages [0.19698344608599344]
  We propose a novel computational framework for modeling the quality of Wikipedia articles. Our framework is based on language-agnostic structural features extracted from the articles. We have built datasets with the feature values and quality scores of all revisions of all articles in the existing language versions of Wikipedia.
  arXiv  Detail & Related papers  (2024-04-15T13:07:31Z)
• A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [58.6354685593418]
  This paper proposes several article-level, field-normalized, and large language model-empowered bibliometric indicators to evaluate reviews. The newly emerging AI-generated literature reviews are also appraised. This work offers insights into the current challenges of literature reviews and envisions future directions for their development.
  arXiv  Detail & Related papers  (2024-02-20T11:28:50Z)
• WikiSQE: A Large-Scale Dataset for Sentence Quality Estimation in Wikipedia [14.325320851640084]
  We propose WikiSQE, the first large-scale dataset for sentence quality estimation in Wikipedia. Each sentence is extracted from the entire revision history of English Wikipedia. WikiSQE has about 3.4 M sentences with 153 quality labels.
  arXiv  Detail & Related papers  (2023-05-10T06:45:13Z)
• The Glass Ceiling of Automatic Evaluation in Natural Language Generation [60.59732704936083]
  We take a step back and analyze recent progress by comparing the body of existing automatic metrics and human metrics. Our extensive statistical analysis reveals surprising findings: automatic metrics -- old and new -- are much more similar to each other than to humans.
  arXiv  Detail & Related papers  (2022-08-31T01:13:46Z)
• Surfer100: Generating Surveys From Web Resources on Wikipedia-style [49.23675182917996]
  We show that recent advances in pretrained language modeling can be combined for a two-stage extractive and abstractive approach for Wikipedia lead paragraph generation. We extend this approach to generate longer Wikipedia-style summaries with sections and examine how such methods struggle in this application through detailed studies with 100 reference human-collected surveys.
  arXiv  Detail & Related papers  (2021-12-13T02:18:01Z)
• Human-in-the-Loop Disinformation Detection: Stance, Sentiment, or Something Else? [93.91375268580806]
  Both politics and pandemics have recently provided ample motivation for the development of machine learning-enabled disinformation (a.k.a. fake news) detection algorithms. Existing literature has focused primarily on the fully-automated case, but the resulting techniques cannot reliably detect disinformation on the varied topics, sources, and time scales required for military applications. By leveraging an already-available analyst as a human-in-the-loop, canonical machine learning techniques of sentiment analysis, aspect-based sentiment analysis, and stance detection become plausible methods to use for a partially-automated disinformation detection system.
  arXiv  Detail & Related papers  (2021-11-09T13:30:34Z)
• Assessing the quality of sources in Wikidata across languages: a hybrid approach [64.05097584373979]
  We run a series of microtasks experiments to evaluate a large corpus of references, sampled from Wikidata triples with labels in several languages. We use a consolidated, curated version of the crowdsourced assessments to train several machine learning models to scale up the analysis to the whole of Wikidata. The findings help us ascertain the quality of references in Wikidata, and identify common challenges in defining and capturing the quality of user-generated multilingual structured data on the web.
  arXiv  Detail & Related papers  (2021-09-20T10:06:46Z)
• Measuring Wikipedia Article Quality in One Dimension by Extending ORES with Ordinal Regression [1.52292571922932]
  Article quality ratings on English language Wikipedia have been widely used by both Wikipedia community members and academic researchers. measuring quality presents many methodological challenges. The most widely used systems use labels on discrete ordinal scales when assessing quality, but such labels can be inconvenient for statistics and machine learning.
  arXiv  Detail & Related papers  (2021-08-15T23:05:28Z)
• Wiki-Reliability: A Large Scale Dataset for Content Reliability on Wikipedia [4.148821165759295]
  We build the first dataset of English Wikipedia articles annotated with a wide set of content reliability issues. To build this dataset, we rely on Wikipedia "templates" We select the 10 most popular reliability-related templates on Wikipedia, and propose an effective method to label almost 1M samples of Wikipedia article revisions as positive or negative.
  arXiv  Detail & Related papers  (2021-05-10T05:07:03Z)
• Multiple Texts as a Limiting Factor in Online Learning: Quantifying (Dis-)similarities of Knowledge Networks across Languages [60.00219873112454]
  We investigate the hypothesis that the extent to which one obtains information on a given topic through Wikipedia depends on the language in which it is consulted. Since Wikipedia is a central part of the web-based information landscape, this indicates a language-related, linguistic bias. The article builds a bridge between reading research, educational science, Wikipedia research and computational linguistics.
  arXiv  Detail & Related papers  (2020-08-05T11:11:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.