ELQA: A Corpus of Metalinguistic Questions and Answers about English

• URL: http://arxiv.org/abs/2205.00395v2
• Date: Mon, 3 Jul 2023 17:42:36 GMT
• Title: ELQA: A Corpus of Metalinguistic Questions and Answers about English
• Authors: Shabnam Behzad, Keisuke Sakaguchi, Nathan Schneider, Amir Zeldes
• Abstract summary: Collected from two online forums, the >70k questions cover wide-ranging topics including grammar, meaning, fluency, and etymology. Unlike most NLP datasets, this corpus is metalinguistic -- it consists of language about language.
• Score: 24.006858451437534
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We present ELQA, a corpus of questions and answers in and about the English language. Collected from two online forums, the >70k questions (from English learners and others) cover wide-ranging topics including grammar, meaning, fluency, and etymology. The answers include descriptions of general properties of English vocabulary and grammar as well as explanations about specific (correct and incorrect) usage examples. Unlike most NLP datasets, this corpus is metalinguistic -- it consists of language about language. As such, it can facilitate investigations of the metalinguistic capabilities of NLU models, as well as educational applications in the language learning domain. To study this, we define a free-form question answering task on our dataset and conduct evaluations on multiple LLMs (Large Language Models) to analyze their capacity to generate metalinguistic answers.

Related papers

• How to Engage Your Readers? Generating Guiding Questions to Promote Active Reading [60.19226384241482]
  We introduce GuidingQ, a dataset of 10K in-text questions from textbooks and scientific articles. We explore various approaches to generate such questions using language models. We conduct a human study to understand the implication of such questions on reading comprehension.
  arXiv  Detail & Related papers  (2024-07-19T13:42:56Z)
• INDIC QA BENCHMARK: A Multilingual Benchmark to Evaluate Question Answering capability of LLMs for Indic Languages [26.13077589552484]
  Indic-QA is the largest publicly available context-grounded question-answering dataset for 11 major Indian languages from two language families. We generate a synthetic dataset using the Gemini model to create question-answer pairs given a passage, which is then manually verified for quality assurance. We evaluate various multilingual Large Language Models and their instruction-fine-tuned variants on the benchmark and observe that their performance is subpar, particularly for low-resource languages.
  arXiv  Detail & Related papers  (2024-07-18T13:57:16Z)
• CaLMQA: Exploring culturally specific long-form question answering across 23 languages [58.18984409715615]
  CaLMQA is a collection of 1.5K culturally specific questions spanning 23 languages and 51 culturally translated questions from English into 22 other languages. We collect naturally-occurring questions from community web forums and hire native speakers to write questions to cover under-studied languages such as Fijian and Kirundi. Our dataset contains diverse, complex questions that reflect cultural topics (e.g. traditions, laws, news) and the language usage of native speakers.
  arXiv  Detail & Related papers  (2024-06-25T17:45:26Z)
• Evaluating the Elementary Multilingual Capabilities of Large Language Models with MultiQ [16.637598165238934]
  Large language models (LLMs) need to serve everyone, including a global majority of non-English speakers. Recent research shows that, despite limits in their intended use, people prompt LLMs in many different languages. We introduce MultiQ, a new silver standard benchmark for basic open-ended question answering with 27.4k test questions.
  arXiv  Detail & Related papers  (2024-03-06T16:01:44Z)
• Decomposed Prompting: Unveiling Multilingual Linguistic Structure Knowledge in English-Centric Large Language Models [12.700783525558721]
  English-centric Large Language Models (LLMs) like GPT-3 and LLaMA display a remarkable ability to perform multilingual tasks. This paper introduces the decomposed prompting approach to probe the linguistic structure understanding of these LLMs in sequence labeling tasks.
  arXiv  Detail & Related papers  (2024-02-28T15:15:39Z)
• Natural Language Processing for Dialects of a Language: A Survey [56.93337350526933]
  State-of-the-art natural language processing (NLP) models are trained on massive training corpora, and report a superlative performance on evaluation datasets. This survey delves into an important attribute of these datasets: the dialect of a language. Motivated by the performance degradation of NLP models for dialectic datasets and its implications for the equity of language technologies, we survey past research in NLP for dialects in terms of datasets, and approaches.
  arXiv  Detail & Related papers  (2024-01-11T03:04:38Z)
• The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants [80.4837840962273]
  We present Belebele, a dataset spanning 122 language variants. This dataset enables the evaluation of text models in high-, medium-, and low-resource languages.
  arXiv  Detail & Related papers  (2023-08-31T17:43:08Z)
• How Do We Answer Complex Questions: Discourse Structure of Long-form Answers [51.973363804064704]
  We study the functional structure of long-form answers collected from three datasets. Our main goal is to understand how humans organize information to craft complex answers. Our work can inspire future research on discourse-level modeling and evaluation of long-form QA systems.
  arXiv  Detail & Related papers  (2022-03-21T15:14:10Z)
• Multilingual Answer Sentence Reranking via Automatically Translated Data [97.98885151955467]
  We present a study on the design of multilingual Answer Sentence Selection (AS2) models, which are a core component of modern Question Answering (QA) systems. The main idea is to transfer data, created from one resource rich language, e.g., English, to other languages, less rich in terms of resources.
  arXiv  Detail & Related papers  (2021-02-20T03:52:08Z)
• TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages [27.588857710802113]
  TyDi QA is a question answering dataset covering 11 typologically diverse languages with 204K question-answer pairs. We present a quantitative analysis of the data quality and example-level qualitative linguistic analyses of observed language phenomena.
  arXiv  Detail & Related papers  (2020-03-10T21:11:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.