On the effective transfer of knowledge from English to Hindi Wikipedia

• URL: http://arxiv.org/abs/2412.05708v1
• Date: Sat, 07 Dec 2024 17:43:21 GMT
• Title: On the effective transfer of knowledge from English to Hindi Wikipedia
• Authors: Paramita Das, Amartya Roy, Ritabrata Chakraborty, Animesh Mukherjee,
• Abstract summary: We propose a lightweight framework to enhance knowledge equity between English and Hindi. In case the English Wikipedia page is not up-to-date, our framework adapts it to align with Wikipedia's distinctive style. Our framework effectively generates new content for Hindi Wikipedia sections, enhancing Hindi Wikipedia articles respectively by 65% and 62% according to automatic and human judgment-based evaluations.
• Score: 4.427603894929721
• License:
• Abstract: Although Wikipedia is the largest multilingual encyclopedia, it remains inherently incomplete. There is a significant disparity in the quality of content between high-resource languages (HRLs, e.g., English) and low-resource languages (LRLs, e.g., Hindi), with many LRL articles lacking adequate information. To bridge these content gaps, we propose a lightweight framework to enhance knowledge equity between English and Hindi. In case the English Wikipedia page is not up-to-date, our framework extracts relevant information from external resources readily available (such as English books) and adapts it to align with Wikipedia's distinctive style, including its \textit{neutral point of view} (NPOV) policy, using in-context learning capabilities of large language models. The adapted content is then machine-translated into Hindi for integration into the corresponding Wikipedia articles. On the other hand, if the English version is comprehensive and up-to-date, the framework directly transfers knowledge from English to Hindi. Our framework effectively generates new content for Hindi Wikipedia sections, enhancing Hindi Wikipedia articles respectively by 65% and 62% according to automatic and human judgment-based evaluations.

Related papers

• How Good is Your Wikipedia? [13.814955569390207]
  This paper critically examines the data quality of Wikipedia in a non-English setting by subjecting it to various quality filtering techniques. We find that data quality pruning is an effective means for resource-efficient training without hurting performance.
  arXiv  Detail & Related papers  (2024-11-08T12:35:58Z)
• An Open Multilingual System for Scoring Readability of Wikipedia [3.992677070507323]
  We develop a multilingual model to score the readability of Wikipedia articles. We create a novel multilingual dataset spanning 14 languages, by matching articles from Wikipedia to simplified Wikipedia and online childrens. We show that our model performs well in a zero-shot scenario, yielding a ranking accuracy of more than 80% across 14 languages.
  arXiv  Detail & Related papers  (2024-06-03T23:07:18Z)
• Language-Agnostic Modeling of Wikipedia Articles for Content Quality Assessment across Languages [0.19698344608599344]
  We propose a novel computational framework for modeling the quality of Wikipedia articles. Our framework is based on language-agnostic structural features extracted from the articles. We have built datasets with the feature values and quality scores of all revisions of all articles in the existing language versions of Wikipedia.
  arXiv  Detail & Related papers  (2024-04-15T13:07:31Z)
• NusaWrites: Constructing High-Quality Corpora for Underrepresented and Extremely Low-Resource Languages [54.808217147579036]
  We conduct a case study on Indonesian local languages. We compare the effectiveness of online scraping, human translation, and paragraph writing by native speakers in constructing datasets. Our findings demonstrate that datasets generated through paragraph writing by native speakers exhibit superior quality in terms of lexical diversity and cultural content.
  arXiv  Detail & Related papers  (2023-09-19T14:42:33Z)
• Mapping Process for the Task: Wikidata Statements to Text as Wikipedia Sentences [68.8204255655161]
  We propose our mapping process for the task of converting Wikidata statements to natural language text (WS2T) for Wikipedia projects at the sentence level. The main step is to organize statements, represented as a group of quadruples and triples, and then to map them to corresponding sentences in English Wikipedia. We evaluate the output corpus in various aspects: sentence structure analysis, noise filtering, and relationships between sentence components based on word embedding models.
  arXiv  Detail & Related papers  (2022-10-23T08:34:33Z)
• WikiDes: A Wikipedia-Based Dataset for Generating Short Descriptions from Paragraphs [66.88232442007062]
  We introduce WikiDes, a dataset to generate short descriptions of Wikipedia articles. The dataset consists of over 80k English samples on 6987 topics. Our paper shows a practical impact on Wikipedia and Wikidata since there are thousands of missing descriptions.
  arXiv  Detail & Related papers  (2022-09-27T01:28:02Z)
• Assessing the quality of sources in Wikidata across languages: a hybrid approach [64.05097584373979]
  We run a series of microtasks experiments to evaluate a large corpus of references, sampled from Wikidata triples with labels in several languages. We use a consolidated, curated version of the crowdsourced assessments to train several machine learning models to scale up the analysis to the whole of Wikidata. The findings help us ascertain the quality of references in Wikidata, and identify common challenges in defining and capturing the quality of user-generated multilingual structured data on the web.
  arXiv  Detail & Related papers  (2021-09-20T10:06:46Z)
• Crosslingual Topic Modeling with WikiPDA [15.198979978589476]
  We present Wikipedia-based Polyglot Dirichlet Allocation (WikiPDA) It learns to represent Wikipedia articles written in any language as distributions over a common set of language-independent topics. We show its utility in two applications: a study of topical biases in 28 Wikipedia editions, and crosslingual supervised classification.
  arXiv  Detail & Related papers  (2020-09-23T15:19:27Z)
• Multiple Texts as a Limiting Factor in Online Learning: Quantifying (Dis-)similarities of Knowledge Networks across Languages [60.00219873112454]
  We investigate the hypothesis that the extent to which one obtains information on a given topic through Wikipedia depends on the language in which it is consulted. Since Wikipedia is a central part of the web-based information landscape, this indicates a language-related, linguistic bias. The article builds a bridge between reading research, educational science, Wikipedia research and computational linguistics.
  arXiv  Detail & Related papers  (2020-08-05T11:11:55Z)
• Design Challenges in Low-resource Cross-lingual Entity Linking [56.18957576362098]
  Cross-lingual Entity Linking (XEL) is the problem of grounding mentions of entities in a foreign language text into an English knowledge base such as Wikipedia. This paper focuses on the key step of identifying candidate English Wikipedia titles that correspond to a given foreign language mention. We present a simple yet effective zero-shot XEL system, QuEL, that utilizes search engines query logs.
  arXiv  Detail & Related papers  (2020-05-02T04:00:26Z)
• Architecture for a multilingual Wikipedia [0.0]
  We argue that we need a new approach to tackle this problem more effectively. This paper proposes an architecture for a system that fulfills this goal. It separates the goal in two parts: creating and maintaining content in an abstract notation within a project called Abstract Wikipedia, and creating an infrastructure called Wikilambda that can translate this notation to natural language.
  arXiv  Detail & Related papers  (2020-04-08T22:25:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.