SiTSE: Sinhala Text Simplification Dataset and Evaluation

• URL: http://arxiv.org/abs/2412.01293v1
• Date: Mon, 02 Dec 2024 09:08:06 GMT
• Title: SiTSE: Sinhala Text Simplification Dataset and Evaluation
• Authors: Surangika Ranathunga, Rumesh Sirithunga, Himashi Rathnayake, Lahiru De Silva, Thamindu Aluthwala, Saman Peramuna, Ravi Shekhar,
• Abstract summary: This paper presents a human curated sentence-level text simplification dataset for the Sinhala language. We model the text simplification task as a zero-shot and zero resource sequence-to-sequence (seq-seq) task on the multilingual language models mT5 and mBART. Our analysis shows that ITTL outperforms the previously proposed zero-resource methods for text simplification.
• Score: 1.7806363928929385
• License:
• Abstract: Text Simplification is a task that has been minimally explored for low-resource languages. Consequently, there are only a few manually curated datasets. In this paper, we present a human curated sentence-level text simplification dataset for the Sinhala language. Our evaluation dataset contains 1,000 complex sentences and corresponding 3,000 simplified sentences produced by three different human annotators. We model the text simplification task as a zero-shot and zero resource sequence-to-sequence (seq-seq) task on the multilingual language models mT5 and mBART. We exploit auxiliary data from related seq-seq tasks and explore the possibility of using intermediate task transfer learning (ITTL). Our analysis shows that ITTL outperforms the previously proposed zero-resource methods for text simplification. Our findings also highlight the challenges in evaluating text simplification systems, and support the calls for improved metrics for measuring the quality of automated text simplification systems that would suit low-resource languages as well. Our code and data are publicly available: https://github.com/brainsharks-fyp17/Sinhala-Text-Simplification-Dataset-and-Evaluation

Related papers

• Improving Text Embeddings with Large Language Models [59.930513259982725]
  We introduce a novel and simple method for obtaining high-quality text embeddings using only synthetic data and less than 1k training steps. We leverage proprietary LLMs to generate diverse synthetic data for hundreds of thousands of text embedding tasks across 93 languages. Experiments demonstrate that our method achieves strong performance on highly competitive text embedding benchmarks without using any labeled data.
  arXiv  Detail & Related papers  (2023-12-31T02:13:18Z)
• A Novel Dataset for Financial Education Text Simplification in Spanish [4.475176409401273]
  In Spanish, there are few datasets that can be used to create text simplification systems. We created a dataset with 5,314 complex and simplified sentence pairs using established simplification rules.
  arXiv  Detail & Related papers  (2023-12-15T15:47:08Z)
• SEMQA: Semi-Extractive Multi-Source Question Answering [94.04430035121136]
  We introduce a new QA task for answering multi-answer questions by summarizing multiple diverse sources in a semi-extractive fashion. We create the first dataset of this kind, QuoteSum, with human-written semi-extractive answers to natural and generated questions.
  arXiv  Detail & Related papers  (2023-11-08T18:46:32Z)
• MCTS: A Multi-Reference Chinese Text Simplification Dataset [15.080614581458091]
  There has been very little research on Chinese text simplification for a long time. We introduce MCTS, a multi-reference Chinese text simplification dataset. We evaluate the performance of several unsupervised methods and advanced large language models.
  arXiv  Detail & Related papers  (2023-06-05T11:46:36Z)
• SCROLLS: Standardized CompaRison Over Long Language Sequences [62.574959194373264]
  We introduce SCROLLS, a suite of tasks that require reasoning over long texts. SCROLLS contains summarization, question answering, and natural language inference tasks. We make all datasets available in a unified text-to-text format and host a live leaderboard to facilitate research on model architecture and pretraining methods.
  arXiv  Detail & Related papers  (2022-01-10T18:47:15Z)
• SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation on Natural Speech [44.68649535280397]
  We propose a suite of benchmark tasks for Spoken Language Understanding Evaluation (SLUE) SLUE consists of limited-size labeled training sets and corresponding evaluation sets. We present the first phase of the SLUE benchmark suite, consisting of named entity recognition, sentiment analysis, and ASR on the corresponding datasets. We provide new transcriptions and annotations on subsets of the VoxCeleb and VoxPopuli datasets, evaluation metrics and results for baseline models, and an open-source toolkit to reproduce the baselines and evaluate new models.
  arXiv  Detail & Related papers  (2021-11-19T18:59:23Z)
• Document-Level Text Simplification: Dataset, Criteria and Baseline [75.58761130635824]
  We define and investigate a new task of document-level text simplification. Based on Wikipedia dumps, we first construct a large-scale dataset named D-Wikipedia. We propose a new automatic evaluation metric called D-SARI that is more suitable for the document-level simplification task.
  arXiv  Detail & Related papers  (2021-10-11T08:15:31Z)
• Zero-Shot Information Extraction as a Unified Text-to-Triple Translation [56.01830747416606]
  We cast a suite of information extraction tasks into a text-to-triple translation framework. We formalize the task as a translation between task-specific input text and output triples. We study the zero-shot performance of this framework on open information extraction.
  arXiv  Detail & Related papers  (2021-09-23T06:54:19Z)
• ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations [97.27005783856285]
  This paper introduces ASSET, a new dataset for assessing sentence simplification in English. We show that simplifications in ASSET are better at capturing characteristics of simplicity when compared to other standard evaluation datasets for the task.
  arXiv  Detail & Related papers  (2020-05-01T16:44:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.