Corpus Frequencies in Morphological Inflection: Do They Matter?

• URL: http://arxiv.org/abs/2510.23131v1
• Date: Mon, 27 Oct 2025 09:12:04 GMT
• Title: Corpus Frequencies in Morphological Inflection: Do They Matter?
• Abstract summary: We explore the incorporation of corpus frequency information into the task of morphological inflection along three key dimensions during system development.<n>We show that frequency-aware training outperforms uniform sampling in 26 out of 43 languages.
• Score: 0.5934049163128877
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The traditional approach to morphological inflection (the task of modifying a base word (lemma) to express grammatical categories) has been, for decades, to consider lexical entries of lemma-tag-form triples uniformly, lacking any information about their frequency distribution. However, in production deployment, one might expect the user inputs to reflect a real-world distribution of frequencies in natural texts. With future deployment in mind, we explore the incorporation of corpus frequency information into the task of morphological inflection along three key dimensions during system development: (i) for train-dev-test split, we combine a lemma-disjoint approach, which evaluates the model's generalization capabilities, with a frequency-weighted strategy to better reflect the realistic distribution of items across different frequency bands in training and test sets; (ii) for evaluation, we complement the standard type accuracy (often referred to simply as accuracy), which treats all items equally regardless of frequency, with token accuracy, which assigns greater weight to frequent words and better approximates performance on running text; (iii) for training data sampling, we introduce a method novel in the context of inflection, frequency-aware training, which explicitly incorporates word frequency into the sampling process. We show that frequency-aware training outperforms uniform sampling in 26 out of 43 languages.

Related papers

• How Data Shapes RoPE Frequency Usage: From Positional Scale Matching to Length Generalization [19.24080183297062]
  Rotary Position Embeddings (RoPE) provide transformers with a fixed grid of positional frequencies.<n>We study what determines this frequency usage and propose a data-centered explanation.
  arXiv  Detail & Related papers  (2026-07-08T17:38:14Z)
• Learning to Align: Addressing Character Frequency Distribution Shifts in Handwritten Text Recognition [7.913119666154837]
  Handwritten text recognition aims to convert visual input into machine-readable text.<n>Character sets change over time, and character frequency distributions shift across historical periods or regions.<n>We propose a novel loss function that incorporates the Wasserstein distance between the character frequency distribution of the predicted text and a target distribution.
  arXiv  Detail & Related papers  (2025-06-11T15:20:30Z)
• The Harmonic Structure of Information Contours [54.38365999922221]
  We find consistent evidence of periodic patterns in information rate in texts in English, Spanish, German, Dutch, Basque, and Brazilian Portuguese.<n>Many dominant frequencies align with discourse structure, suggesting these oscillations reflect meaningful linguistic organization.
  arXiv  Detail & Related papers  (2025-06-04T12:56:30Z)
• Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization [74.3339999119713]
  We develop a wavelet-based tokenizer that allows models to learn complex representations directly in the space of time-localized frequencies.<n>Our method first scales and decomposes the input time series, then thresholds and quantizes the wavelet coefficients, and finally pre-trains an autoregressive model to forecast coefficients for the forecast horizon.
  arXiv  Detail & Related papers  (2024-12-06T18:22:59Z)
• Collar-aware Training for Streaming Speaker Change Detection in Broadcast Speech [0.0]
  We present a novel training method for speaker change detection models. The proposed method uses an objective function which encourages the model to predict a single positive label within a specified collar.
  arXiv  Detail & Related papers  (2022-05-14T15:35:43Z)
• Frequency-Aware Contrastive Learning for Neural Machine Translation [24.336356651877388]
  Low-frequency word prediction remains a challenge in modern neural machine translation (NMT) systems. Inspired by the observation that low-frequency words form a more compact embedding space, we tackle this challenge from a representation learning perspective. We propose a frequency-aware token-level contrastive learning method, in which the hidden state of each decoding step is pushed away from the counterparts of other target words.
  arXiv  Detail & Related papers  (2021-12-29T10:10:10Z)
• Self-Normalized Importance Sampling for Neural Language Modeling [97.96857871187052]
  In this work, we propose self-normalized importance sampling. Compared to our previous work, the criteria considered in this work are self-normalized and there is no need to further conduct a correction step. We show that our proposed self-normalized importance sampling is competitive in both research-oriented and production-oriented automatic speech recognition tasks.
  arXiv  Detail & Related papers  (2021-11-11T16:57:53Z)
• Voice2Series: Reprogramming Acoustic Models for Time Series Classification [65.94154001167608]
  Voice2Series is a novel end-to-end approach that reprograms acoustic models for time series classification. We show that V2S either outperforms or is tied with state-of-the-art methods on 20 tasks, and improves their average accuracy by 1.84%.
  arXiv  Detail & Related papers  (2021-06-17T07:59:15Z)
• Consistency Regularization for Cross-Lingual Fine-Tuning [61.08704789561351]
  We propose to improve cross-lingual fine-tuning with consistency regularization. Specifically, we use example consistency regularization to penalize the prediction sensitivity to four types of data augmentations. Experimental results on the XTREME benchmark show that our method significantly improves cross-lingual fine-tuning across various tasks.
  arXiv  Detail & Related papers  (2021-06-15T15:35:44Z)
• On Sampling-Based Training Criteria for Neural Language Modeling [97.35284042981675]
  We consider Monte Carlo sampling, importance sampling, a novel method we call compensated partial summation, and noise contrastive estimation. We show that all these sampling methods can perform equally well, as long as we correct for the intended class posterior probabilities. Experimental results in language modeling and automatic speech recognition on Switchboard and LibriSpeech support our claim.
  arXiv  Detail & Related papers  (2021-04-21T12:55:52Z)
• Fast and Robust Unsupervised Contextual Biasing for Speech Recognition [16.557586847398778]
  We propose an alternative approach that does not entail explicit contextual language model. We derive the bias score for every word in the system vocabulary from the training corpus. We show significant improvement in recognition accuracy when the relevant context is available.
  arXiv  Detail & Related papers  (2020-05-04T17:29:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.