A Test of Time: Predicting the Sustainable Success of Online Collaboration in Wikipedia

• URL: http://arxiv.org/abs/2410.19150v1
• Date: Thu, 24 Oct 2024 20:42:53 GMT
• Title: A Test of Time: Predicting the Sustainable Success of Online Collaboration in Wikipedia
• Authors: Abraham Israeli, David Jurgens, Daniel Romero,
• Abstract summary: We develop machine learning models to predict the sustainable success of Wikipedia articles. We find that the longer an article takes to be recognized as high-quality, the more likely it is to maintain that status over time. Our analysis provides insights into broader collective actions beyond Wikipedia.
• Score: 17.051622145253855
• License:
• Abstract: The Internet has significantly expanded the potential for global collaboration, allowing millions of users to contribute to collective projects like Wikipedia. While prior work has assessed the success of online collaborations, most approaches are time-agnostic, evaluating success without considering its longevity. Research on the factors that ensure the long-term preservation of high-quality standards in online collaboration is scarce. In this study, we address this gap. We propose a novel metric, `Sustainable Success,' which measures the ability of collaborative efforts to maintain their quality over time. Using Wikipedia as a case study, we introduce the SustainPedia dataset, which compiles data from over 40K Wikipedia articles, including each article's sustainable success label and more than 300 explanatory features such as edit history, user experience, and team composition. Using this dataset, we develop machine learning models to predict the sustainable success of Wikipedia articles. Our best-performing model achieves a high AU-ROC score of 0.88 on average. Our analysis reveals important insights. For example, we find that the longer an article takes to be recognized as high-quality, the more likely it is to maintain that status over time (i.e., be sustainable). Additionally, user experience emerged as the most critical predictor of sustainability. Our analysis provides insights into broader collective actions beyond Wikipedia (e.g., online activism, crowdsourced open-source software), where the same social dynamics that drive success on Wikipedia might play a role. We make all data and code used for this study publicly available for further research.

Related papers

• HelloFresh: LLM Evaluations on Streams of Real-World Human Editorial Actions across X Community Notes and Wikipedia edits [92.62157408704594]
  HelloFresh is based on continuous streams of real-world data generated by intrinsically motivated human labelers. It covers recent events from X (formerly Twitter) community notes and edits of Wikipedia pages. It mitigates the risk of test data contamination and benchmark overfitting.
  arXiv  Detail & Related papers  (2024-06-05T16:25:57Z)
• Automatic Quality Assessment of Wikipedia Articles -- A Systematic Literature Review [0.8158530638728501]
  We review existing methods for automatically measuring the quality of Wikipedia articles. We identify and comparing machine learning algorithms, article features, quality metrics, and used datasets. We hope that our analysis helps future researchers change that reality.
  arXiv  Detail & Related papers  (2023-10-03T17:45:39Z)
• Publishing Wikipedia usage data with strong privacy guarantees [6.410779699541235]
  For almost 20 years, the Wikimedia Foundation has been publishing statistics about how many people visited each Wikipedia page on each day. In June 2023, the Wikimedia Foundation started publishing these statistics with finer granularity, including the country of origin in the daily counts of page views. This paper describes this data publication: its goals, the process followed from its inception to its deployment, and the outcomes of the data release.
  arXiv  Detail & Related papers  (2023-08-30T19:58:56Z)
• NEVIS'22: A Stream of 100 Tasks Sampled from 30 Years of Computer Vision Research [96.53307645791179]
  We introduce the Never-Ending VIsual-classification Stream (NEVIS'22), a benchmark consisting of a stream of over 100 visual classification tasks. Despite being limited to classification, the resulting stream has a rich diversity of tasks from OCR, to texture analysis, scene recognition, and so forth. Overall, NEVIS'22 poses an unprecedented challenge for current sequential learning approaches due to the scale and diversity of tasks.
  arXiv  Detail & Related papers  (2022-11-15T18:57:46Z)
• Mapping Process for the Task: Wikidata Statements to Text as Wikipedia Sentences [68.8204255655161]
  We propose our mapping process for the task of converting Wikidata statements to natural language text (WS2T) for Wikipedia projects at the sentence level. The main step is to organize statements, represented as a group of quadruples and triples, and then to map them to corresponding sentences in English Wikipedia. We evaluate the output corpus in various aspects: sentence structure analysis, noise filtering, and relationships between sentence components based on word embedding models.
  arXiv  Detail & Related papers  (2022-10-23T08:34:33Z)
• Algorithmic Fairness Datasets: the Story so Far [68.45921483094705]
  Data-driven algorithms are studied in diverse domains to support critical decisions, directly impacting people's well-being. A growing community of researchers has been investigating the equity of existing algorithms and proposing novel ones, advancing the understanding of risks and opportunities of automated decision-making for historically disadvantaged populations. Progress in fair Machine Learning hinges on data, which can be appropriately used only if adequately documented. Unfortunately, the algorithmic fairness community suffers from a collective data documentation debt caused by a lack of information on specific resources (opacity) and scatteredness of available information (sparsity)
  arXiv  Detail & Related papers  (2022-02-03T17:25:46Z)
• Surfer100: Generating Surveys From Web Resources on Wikipedia-style [49.23675182917996]
  We show that recent advances in pretrained language modeling can be combined for a two-stage extractive and abstractive approach for Wikipedia lead paragraph generation. We extend this approach to generate longer Wikipedia-style summaries with sections and examine how such methods struggle in this application through detailed studies with 100 reference human-collected surveys.
  arXiv  Detail & Related papers  (2021-12-13T02:18:01Z)
• Assessing the quality of sources in Wikidata across languages: a hybrid approach [64.05097584373979]
  We run a series of microtasks experiments to evaluate a large corpus of references, sampled from Wikidata triples with labels in several languages. We use a consolidated, curated version of the crowdsourced assessments to train several machine learning models to scale up the analysis to the whole of Wikidata. The findings help us ascertain the quality of references in Wikidata, and identify common challenges in defining and capturing the quality of user-generated multilingual structured data on the web.
  arXiv  Detail & Related papers  (2021-09-20T10:06:46Z)
• Measuring Wikipedia Article Quality in One Dimension by Extending ORES with Ordinal Regression [1.52292571922932]
  Article quality ratings on English language Wikipedia have been widely used by both Wikipedia community members and academic researchers. measuring quality presents many methodological challenges. The most widely used systems use labels on discrete ordinal scales when assessing quality, but such labels can be inconvenient for statistics and machine learning.
  arXiv  Detail & Related papers  (2021-08-15T23:05:28Z)
• Wiki-Reliability: A Large Scale Dataset for Content Reliability on Wikipedia [4.148821165759295]
  We build the first dataset of English Wikipedia articles annotated with a wide set of content reliability issues. To build this dataset, we rely on Wikipedia "templates" We select the 10 most popular reliability-related templates on Wikipedia, and propose an effective method to label almost 1M samples of Wikipedia article revisions as positive or negative.
  arXiv  Detail & Related papers  (2021-05-10T05:07:03Z)
• Tracking Knowledge Propagation Across Wikipedia Languages [1.8447697408534176]
  We present a dataset of inter-language knowledge propagation in Wikipedia. The dataset covers the entire 309 language editions and 33M articles. We find that the size of language editions is associated with the speed of propagation.
  arXiv  Detail & Related papers  (2021-03-30T18:36:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.