When Irregularity Helps: A Subclass Analysis of Inductive Bias in Neural Morphology

• URL: http://arxiv.org/abs/2605.20558v2
• Date: Thu, 21 May 2026 23:18:49 GMT
• Title: When Irregularity Helps: A Subclass Analysis of Inductive Bias in Neural Morphology
• Abstract summary: We show that a very small, structurally specific irregular subtype accounts for a disproportionate share of model errors.<n>These findings suggest that error concentration is driven by the interaction between extreme low-frequency morphological patterns and specific morphophonological processes.
• Score: 6.1031222024562695
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural morphological generation systems often achieve high aggregate accuracy on benchmark datasets, yet such performance can conceal systematic errors concentrated in rare morphological subclasses. We examine Japanese past-tense verb inflection and show that a very small, structurally specific irregular subtype (<1% of data) accounts for a disproportionate share of model errors. Controlled ablation experiments demonstrate that removing this subtype yields larger improvements in generalization than removing all irregular verbs, indicating that not all irregularity contributes equally to model instability. These findings suggest that error concentration is driven by the interaction between extreme low-frequency morphological patterns and specific morphophonological processes, particularly gemination. We argue that morphological evaluation should incorporate finer-grained subclass analysis beyond standard conjugation categories.

Related papers

• Benign Overfitting Does Not Occur in Diffusion Models [33.8340803958907]
  Diffusion models share much of the machinery of standard deep learning.<n>We show that, unless the sample size grows exponentially with the data dimension, overfitting and good generalization cannot occur simultaneously.<n>We identify implicit regularization stemming from time-smoothness of the score and early stopping during training as mechanisms that prevent such overfitting.
  arXiv  Detail & Related papers  (2026-07-02T18:05:23Z)
• Mind Your Moras: Orthography-Aware Error Analysis of Neural Japanese Morphological Generation [6.1031222024562695]
  We present an orthography-aware error analysis of Japanese past-tense morphological inflection.<n>We evaluate two character-level sequence-to-sequence architectures on past-tense formation using datasets formatted according to the SIGMORPHON 2020 and 2023 shared task conventions.<n>Despite high aggregate accuracy, models exhibit systematic, linguistically interpretable errors that cluster around specific orthographic properties of hiragana.
  arXiv  Detail & Related papers  (2026-05-19T16:04:33Z)
• Generative Classifiers Avoid Shortcut Solutions [84.23247217037134]
  Discriminative approaches to classification often learn shortcuts that hold in-distribution but fail under minor distribution shift.<n>We show that generative classifiers can avoid this issue by modeling all features, both core and spurious, instead of mainly spurious ones.<n>We find that diffusion-based and autorerimigressive generative classifiers achieve state-of-the-art performance on five standard image and text distribution shift benchmarks.
  arXiv  Detail & Related papers  (2025-12-31T18:31:46Z)
• Memorization and Regularization in Generative Diffusion Models [5.128303432235475]
  Diffusion models have emerged as a powerful framework for generative modeling.<n>The analysis highlights the need for regularization to avoid reproducing the analytically tractable minimizer.<n>Experiments are evaluated in the context of memorization, and directions for future development of regularization are highlighted.
  arXiv  Detail & Related papers  (2025-01-27T05:17:06Z)
• SLA$^2$P: Self-supervised Anomaly Detection with Adversarial Perturbation [77.71161225100927]
  Anomaly detection is a fundamental yet challenging problem in machine learning. We propose a novel and powerful framework, dubbed as SLA$2$P, for unsupervised anomaly detection.
  arXiv  Detail & Related papers  (2021-11-25T03:53:43Z)
• Explainable Deep Few-shot Anomaly Detection with Deviation Networks [123.46611927225963]
  We introduce a novel weakly-supervised anomaly detection framework to train detection models. The proposed approach learns discriminative normality by leveraging the labeled anomalies and a prior probability. Our model is substantially more sample-efficient and robust, and performs significantly better than state-of-the-art competing methods in both closed-set and open-set settings.
  arXiv  Detail & Related papers  (2021-08-01T14:33:17Z)
• Predicting Unreliable Predictions by Shattering a Neural Network [145.3823991041987]
  Piecewise linear neural networks can be split into subfunctions. Subfunctions have their own activation pattern, domain, and empirical error. Empirical error for the full network can be written as an expectation over subfunctions.
  arXiv  Detail & Related papers  (2021-06-15T18:34:41Z)
• Searching for Search Errors in Neural Morphological Inflection [40.011530606430895]
  On word-level tasks, exact inference of neural sequence-to-sequence models reveals the empty string is often the global optimum. These observations suggest that the poor calibration of many neural models may stem from characteristics of a specific subset of tasks.
  arXiv  Detail & Related papers  (2021-02-16T19:42:42Z)
• Understanding Classifier Mistakes with Generative Models [88.20470690631372]
  Deep neural networks are effective on supervised learning tasks, but have been shown to be brittle. In this paper, we leverage generative models to identify and characterize instances where classifiers fail to generalize. Our approach is agnostic to class labels from the training set which makes it applicable to models trained in a semi-supervised way.
  arXiv  Detail & Related papers  (2020-10-05T22:13:21Z)
• Good Classifiers are Abundant in the Interpolating Regime [64.72044662855612]
  We develop a methodology to compute precisely the full distribution of test errors among interpolating classifiers. We find that test errors tend to concentrate around a small typical value $varepsilon*$, which deviates substantially from the test error of worst-case interpolating model. Our results show that the usual style of analysis in statistical learning theory may not be fine-grained enough to capture the good generalization performance observed in practice.
  arXiv  Detail & Related papers  (2020-06-22T21:12:31Z)
• Generalization Error of Generalized Linear Models in High Dimensions [25.635225717360466]
  We provide a framework to characterize neural networks with arbitrary non-linearities. We analyze the effect of regular logistic regression on learning. Our model also captures examples between training and distributions special cases.
  arXiv  Detail & Related papers  (2020-05-01T02:17:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.