Related papers: Evaluation of Automatically Constructed Word Meaning Explanations

Evaluation of Automatically Constructed Word Meaning Explanations

URL: http://arxiv.org/abs/2302.13625v1
Date: Mon, 27 Feb 2023 09:47:55 GMT
Title: Evaluation of Automatically Constructed Word Meaning Explanations
Authors: Marie Star\'a and Pavel Rychl\'y and Ale\v{s} Hor\'ak
Abstract summary: We present a new tool that derives explanations automatically based on collective information from very large corpora. We show that the presented approach allows to create explanations that contain data useful for understanding the word meaning in approximately 90% of cases.
Score: 0.0
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Abstract: Preparing exact and comprehensive word meaning explanations is one of the key steps in the process of monolingual dictionary writing. In standard methodology, the explanations need an expert lexicographer who spends a substantial amount of time checking the consistency between the descriptive text and corpus evidence. In the following text, we present a new tool that derives explanations automatically based on collective information from very large corpora, particularly on word sketches. We also propose a quantitative evaluation of the constructed explanations, concentrating on explanations of nouns. The methodology is to a certain extent language independent; however, the presented verification is limited to Czech and English. We show that the presented approach allows to create explanations that contain data useful for understanding the word meaning in approximately 90% of cases. However, in many cases, the result requires post-editing to remove redundant information.

Related papers

Evaluation of Machine Translation Based on Semantic Dependencies and Keywords [7.240399904675839]
This paper proposes a computational method for evaluating the semantic correctness of machine translations based on reference translations. Use the language technology platform developed by the Social Computing and Information Retrieval Research Center of Harbin Institute of Technology.
arXiv Detail & Related papers (2024-04-20T04:14:28Z)
Quantifying the redundancy between prosody and text [67.07817268372743]
We use large language models to estimate how much information is redundant between prosody and the words themselves. We find a high degree of redundancy between the information carried by the words and prosodic information across several prosodic features. Still, we observe that prosodic features can not be fully predicted from text, suggesting that prosody carries information above and beyond the words.
arXiv Detail & Related papers (2023-11-28T21:15:24Z)
Measuring Information in Text Explanations [23.929076318334047]
We argue that placing the explanations on an information-theoretic framework could unify the evaluations of two popular text explanation methods. We quantify the information flow through these channels, thereby facilitating the assessment of explanation characteristics. Our work contributes to the ongoing efforts in establishing rigorous and standardized evaluation criteria in the rapidly evolving field of explainable AI.
arXiv Detail & Related papers (2023-10-06T19:46:51Z)
Elaborative Simplification as Implicit Questions Under Discussion [51.17933943734872]
This paper proposes to view elaborative simplification through the lens of the Question Under Discussion (QUD) framework. We show that explicitly modeling QUD provides essential understanding of elaborative simplification and how the elaborations connect with the rest of the discourse.
arXiv Detail & Related papers (2023-05-17T17:26:16Z)
Explanation Selection Using Unlabeled Data for Chain-of-Thought Prompting [80.9896041501715]
Explanations that have not been "tuned" for a task, such as off-the-shelf explanations written by nonexperts, may lead to mediocre performance. This paper tackles the problem of how to optimize explanation-infused prompts in a blackbox fashion.
arXiv Detail & Related papers (2023-02-09T18:02:34Z)
AUTOLEX: An Automatic Framework for Linguistic Exploration [93.89709486642666]
We propose an automatic framework that aims to ease linguists' discovery and extraction of concise descriptions of linguistic phenomena. Specifically, we apply this framework to extract descriptions for three phenomena: morphological agreement, case marking, and word order. We evaluate the descriptions with the help of language experts and propose a method for automated evaluation when human evaluation is infeasible.
arXiv Detail & Related papers (2022-03-25T20:37:30Z)
Human Interpretation of Saliency-based Explanation Over Text [65.29015910991261]
We study saliency-based explanations over textual data. We find that people often mis-interpret the explanations. We propose a method to adjust saliencies based on model estimates of over- and under-perception.
arXiv Detail & Related papers (2022-01-27T15:20:32Z)
Explaining Latent Representations with a Corpus of Examples [72.50996504722293]
We propose SimplEx: a user-centred method that provides example-based explanations with reference to a freely selected set of examples. SimplEx uses the corpus to improve the user's understanding of the latent space with post-hoc explanations. We show that SimplEx empowers the user by highlighting relevant patterns in the corpus that explain model representations.
arXiv Detail & Related papers (2021-10-28T17:59:06Z)
ReSCo-CC: Unsupervised Identification of Key Disinformation Sentences [3.7405995078130148]
We propose a novel unsupervised task of identifying sentences containing key disinformation within a document that is known to be untrustworthy. We design a three-phase statistical NLP solution for the task which starts with embedding sentences within a bespoke feature space designed for the task. We show that our method is able to identify core disinformation effectively.
arXiv Detail & Related papers (2020-10-21T08:53:36Z)
Extending Text Informativeness Measures to Passage Interestingness Evaluation (Language Model vs. Word Embedding) [1.2998637003026272]
This paper defines the concept of Interestingness as a generalization of Informativeness. We then study the ability of state of the art Informativeness measures to cope with this generalization. We prove that the CLEF-INEX Tweet Contextualization 2012 Logarithm Similarity measure provides best results.
arXiv Detail & Related papers (2020-04-14T18:22:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.