I Wish I Would Have Loved This One, But I Didn't -- A Multilingual Dataset for Counterfactual Detection in Product Reviews

• URL: http://arxiv.org/abs/2104.06893v1
• Date: Wed, 14 Apr 2021 14:38:36 GMT
• Title: I Wish I Would Have Loved This One, But I Didn't -- A Multilingual Dataset for Counterfactual Detection in Product Reviews
• Authors: James O'Neill and Polina Rozenshtein and Ryuichi Kiryo and Motoko Kubota and Danushka Bollegala
• Abstract summary: We consider the problem of counterfactual detection (CFD) in product reviews. For this purpose, we annotate a multilingual CFD dataset from Amazon product reviews. The dataset is unique as it contains counterfactuals in multiple languages.
• Score: 19.533526638034047
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Counterfactual statements describe events that did not or cannot take place. We consider the problem of counterfactual detection (CFD) in product reviews. For this purpose, we annotate a multilingual CFD dataset from Amazon product reviews covering counterfactual statements written in English, German, and Japanese languages. The dataset is unique as it contains counterfactuals in multiple languages, covers a new application area of e-commerce reviews, and provides high quality professional annotations. We train CFD models using different text representation methods and classifiers. We find that these models are robust against the selectional biases introduced due to cue phrase-based sentence selection. Moreover, our CFD dataset is compatible with prior datasets and can be merged to learn accurate CFD models. Applying machine translation on English counterfactual examples to create multilingual data performs poorly, demonstrating the language-specificity of this problem, which has been ignored so far.

Related papers

• Is It Good Data for Multilingual Instruction Tuning or Just Bad Multilingual Evaluation for Large Language Models? [17.011882550422452]
  It is unknown whether the nature of the instruction data has an impact on the model output. It is questionable whether translated test sets can capture such nuances. We show that native or generation benchmarks reveal a notable difference between native and translated instruction data.
  arXiv  Detail & Related papers  (2024-06-18T17:43:47Z)
• Do We Need Language-Specific Fact-Checking Models? The Case of Chinese [15.619421104102516]
  This paper investigates the potential benefits of language-specific fact-checking models, focusing on the case of Chinese. We first demonstrate the limitations of translation-based methods and multilingual large language models, highlighting the need for language-specific systems. We propose a Chinese fact-checking system that can better retrieve evidence from a document by incorporating context information.
  arXiv  Detail & Related papers  (2024-01-27T20:26:03Z)
• Question Translation Training for Better Multilingual Reasoning [108.10066378240879]
  Large language models show compelling performance on reasoning tasks but they tend to perform much worse in languages other than English. A typical solution is to translate instruction data into all languages of interest, and then train on the resulting multilingual data, which is called translate-training. In this paper we explore the benefits of question alignment, where we train the model to translate reasoning questions into English by finetuning on X-English parallel question data.
  arXiv  Detail & Related papers  (2024-01-15T16:39:10Z)
• XFEVER: Exploring Fact Verification across Languages [40.1637899493061]
  This paper introduces the Cross-lingual Fact Extraction and VERification dataset designed for benchmarking the fact verification models across different languages. We constructed it by translating the claim and evidence texts of the Fact Extraction and VERification dataset into six languages. The training and development sets were translated using machine translation, whereas the test set includes texts translated by professional translators and machine-translated texts.
  arXiv  Detail & Related papers  (2023-10-25T01:20:17Z)
• Towards Fine-Grained Information: Identifying the Type and Location of Translation Errors [80.22825549235556]
  Existing approaches can not synchronously consider error position and type. We build an FG-TED model to predict the textbf addition and textbfomission errors. Experiments show that our model can identify both error type and position concurrently, and gives state-of-the-art results.
  arXiv  Detail & Related papers  (2023-02-17T16:20:33Z)
• Ensemble Transfer Learning for Multilingual Coreference Resolution [60.409789753164944]
  A problem that frequently occurs when working with a non-English language is the scarcity of annotated training data. We design a simple but effective ensemble-based framework that combines various transfer learning techniques. We also propose a low-cost TL method that bootstraps coreference resolution models by utilizing Wikipedia anchor texts.
  arXiv  Detail & Related papers  (2023-01-22T18:22:55Z)
• mFACE: Multilingual Summarization with Factual Consistency Evaluation [79.60172087719356]
  Abstractive summarization has enjoyed renewed interest in recent years, thanks to pre-trained language models and the availability of large-scale datasets. Despite promising results, current models still suffer from generating factually inconsistent summaries. We leverage factual consistency evaluation models to improve multilingual summarization.
  arXiv  Detail & Related papers  (2022-12-20T19:52:41Z)
• Linguistic Cues of Deception in a Multilingual April Fools' Day Context [0.8487852486413651]
  We introduce a corpus that includes diachronic AFD and normal articles from Greek newspapers and news websites. We build a rich linguistic feature set, and analyze and compare its deception cues with the only AFD collection currently available.
  arXiv  Detail & Related papers  (2021-11-06T16:28:12Z)
• X-FACT: A New Benchmark Dataset for Multilingual Fact Checking [21.2633064526968]
  We introduce X-FACT: the largest publicly available multilingual dataset for factual verification of naturally existing real-world claims. The dataset contains short statements in 25 languages and is labeled for veracity by expert fact-checkers.
  arXiv  Detail & Related papers  (2021-06-17T05:09:54Z)
• Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation [105.41167108465085]
  Cross-lingual Machine Reading (CLMRC) remains a challenging problem due to the lack of large-scale datasets in low-source languages. We propose a novel augmentation approach named Language Branch Machine Reading (LBMRC) LBMRC trains multiple machine reading comprehension (MRC) models proficient in individual language. We devise a multilingual distillation approach to amalgamate knowledge from multiple language branch models to a single model for all target languages.
  arXiv  Detail & Related papers  (2020-10-27T13:12:17Z)
• XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization [98.61159823343036]
  We present the Word-in-Context dataset (WiC) for assessing the ability to correctly model distinct meanings of a word. We put forward a large multilingual benchmark, XL-WiC, featuring gold standards in 12 new languages. Experimental results show that even when no tagged instances are available for a target language, models trained solely on the English data can attain competitive performance.
  arXiv  Detail & Related papers  (2020-10-13T15:32:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.