The Paradigm Discovery Problem

• URL: http://arxiv.org/abs/2005.01630v1
• Date: Mon, 4 May 2020 16:38:54 GMT
• Title: The Paradigm Discovery Problem
• Authors: Alexander Erdmann, Micha Elsner, Shijie Wu, Ryan Cotterell and Nizar Habash
• Abstract summary: We formalize the paradigm discovery problem and develop metrics for judging systems. We report empirical results on five diverse languages. Our code and data are available for public use.
• Score: 121.79963594279893
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work treats the paradigm discovery problem (PDP), the task of learning an inflectional morphological system from unannotated sentences. We formalize the PDP and develop evaluation metrics for judging systems. Using currently available resources, we construct datasets for the task. We also devise a heuristic benchmark for the PDP and report empirical results on five diverse languages. Our benchmark system first makes use of word embeddings and string similarity to cluster forms by cell and by paradigm. Then, we bootstrap a neural transducer on top of the clustered data to predict words to realize the empty paradigm slots. An error analysis of our system suggests clustering by cell across different inflection classes is the most pressing challenge for future work. Our code and data are available for public use.

Related papers

• DISCERN: Decoding Systematic Errors in Natural Language for Text Classifiers [18.279429202248632]
  We introduce DISCERN, a framework for interpreting systematic biases in text classifiers using language explanations. DISCERN iteratively generates precise natural language descriptions of systematic errors by employing an interactive loop between two large language models. We show that users can interpret systematic biases more effectively (by over 25% relative) and efficiently when described through language explanations as opposed to cluster exemplars.
  arXiv  Detail & Related papers  (2024-10-29T17:04:55Z)
• Explaining Datasets in Words: Statistical Models with Natural Language Parameters [66.69456696878842]
  We introduce a family of statistical models -- including clustering, time series, and classification models -- parameterized by natural language predicates. We apply our framework to a wide range of problems: taxonomizing user chat dialogues, characterizing how they evolve across time, finding categories where one language model is better than the other.
  arXiv  Detail & Related papers  (2024-09-13T01:40:20Z)
• Vocabulary-Defined Semantics: Latent Space Clustering for Improving In-Context Learning [32.178931149612644]
  In-context learning enables language models to adapt to downstream data or incorporate tasks by few samples as demonstrations within the prompts. However, the performance of in-context learning can be unstable depending on the quality, format, or order of demonstrations. We propose a novel approach "vocabulary-defined semantics"
  arXiv  Detail & Related papers  (2024-01-29T14:29:48Z)
• On the Blind Spots of Model-Based Evaluation Metrics for Text Generation [79.01422521024834]
  We explore a useful but often neglected methodology for robustness analysis of text generation evaluation metrics. We design and synthesize a wide range of potential errors and check whether they result in a commensurate drop in the metric scores. Our experiments reveal interesting insensitivities, biases, or even loopholes in existing metrics.
  arXiv  Detail & Related papers  (2022-12-20T06:24:25Z)
• Meta-learning Pathologies from Radiology Reports using Variance Aware Prototypical Networks [3.464871689508835]
  We propose a simple extension of the Prototypical Networks for few-shot text classification. Our main idea is to replace the class prototypes by Gaussians and introduce a regularization term that encourages the examples to be clustered near the appropriate class centroids.
  arXiv  Detail & Related papers  (2022-10-22T05:22:29Z)
• Cross-Domain Evaluation of a Deep Learning-Based Type Inference System [0.44098366957385177]
  We investigate Type4Py as a representative of state-of-the-art deep learning-based type inference systems. We address the following problems: class imbalances, out-of-vocabulary words, dataset shifts, and unknown classes. Our dataset enables the evaluation of type inference systems in different domains of software projects.
  arXiv  Detail & Related papers  (2022-08-19T07:28:31Z)
• Vector Representations of Idioms in Conversational Systems [1.6507910904669727]
  We utilize the Potentialatic Expression (PIE)-English idioms corpus for the two tasks that we investigate. We achieve state-of-the-art (SoTA) result of 98% macro F1 score on the classification task by using the SoTA T5 model. The results show that the model trained on the idiom corpus generates more fitting responses to prompts containing idioms 71.9% of the time.
  arXiv  Detail & Related papers  (2022-05-07T14:50:05Z)
• Infusing Finetuning with Semantic Dependencies [62.37697048781823]
  We show that, unlike syntax, semantics is not brought to the surface by today's pretrained models. We then use convolutional graph encoders to explicitly incorporate semantic parses into task-specific finetuning.
  arXiv  Detail & Related papers  (2020-12-10T01:27:24Z)
• Unsupervised Paraphrasing with Pretrained Language Models [85.03373221588707]
  We propose a training pipeline that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting. Our recipe consists of task-adaptation, self-supervision, and a novel decoding algorithm named Dynamic Blocking. We show with automatic and human evaluations that our approach achieves state-of-the-art performance on both the Quora Question Pair and the ParaNMT datasets.
  arXiv  Detail & Related papers  (2020-10-24T11:55:28Z)
• Comparison of Interactive Knowledge Base Spelling Correction Models for Low-Resource Languages [81.90356787324481]
  Spelling normalization for low resource languages is a challenging task because the patterns are hard to predict. This work shows a comparison of a neural model and character language models with varying amounts on target language data. Our usage scenario is interactive correction with nearly zero amounts of training examples, improving models as more data is collected.
  arXiv  Detail & Related papers  (2020-10-20T17:31:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.