Why Code, Why Now: Learnability, Computability, and the Real Limits of Machine Learning

• URL: http://arxiv.org/abs/2602.13934v1
• Date: Sun, 15 Feb 2026 00:14:31 GMT
• Title: Why Code, Why Now: Learnability, Computability, and the Real Limits of Machine Learning
• Abstract summary: We propose a five-level hierarchy of learnability based on information structure.<n>We argue that the ceiling on ML progress depends less on model size than on whether a task is learnable at all.
• Score: 0.38677478583601776
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Code generation has progressed more reliably than reinforcement learning, largely because code has an information structure that makes it learnable. Code provides dense, local, verifiable feedback at every token, whereas most reinforcement learning problems do not. This difference in feedback quality is not binary but graded. We propose a five-level hierarchy of learnability based on information structure and argue that the ceiling on ML progress depends less on model size than on whether a task is learnable at all. The hierarchy rests on a formal distinction among three properties of computational problems (expressibility, computability, and learnability). We establish their pairwise relationships, including where implications hold and where they fail, and present a unified template that makes the structural differences explicit. The analysis suggests why supervised learning on code scales predictably while reinforcement learning does not, and why the common assumption that scaling alone will solve remaining ML challenges warrants scrutiny.

Related papers

• When Does Continual Learning Require Learning [68.55508958298942]
  We show that continual learning is not a single capability.<n>Different patterns of environmental change require fundamentally different update behaviors.<n>We hope that understanding where each method succeeds and fails will guide the design of stronger continual learning systems.
  arXiv  Detail & Related papers  (2026-07-08T18:27:40Z)
• A Mathematical Theory of Understanding [0.0]
  Generative AI has transformed the economics of information production, making explanations, proofs, examples, and analyses available at very low cost.<n>Yet the value of information still depends on whether downstream users can absorb and act on it.<n>This paper develops a mathematical model of that learner-side bottleneck.<n>We model the learner as a mind, an abstract learning system characterized by a prerequisite structure over concepts.
  arXiv  Detail & Related papers  (2026-03-19T17:00:11Z)
• Readability-Robust Code Summarization via Meta Curriculum Learning [53.44612630063336]
  In the real world, code is often poorly structured or obfuscated, significantly degrading model performance.<n>We propose RoFTCodeSum, a novel fine-tuning method that enhances the robustness of code summarization against poorly readable code.
  arXiv  Detail & Related papers  (2026-01-09T02:38:24Z)
• Nested Learning: The Illusion of Deep Learning Architectures [57.41377373511876]
  We present a new learning paradigm, called Nested Learning (NL), that coherently represents a machine learning model with a set of nested, multi-level, and/or parallel problems.<n>We show three core contributions: Expressive generalizations are in fact as generalizations with deep memory and/or more powerful learning rules.<n>We present a new continuum for memory system that generalizes the traditional viewpoint of long/short-term memory.
  arXiv  Detail & Related papers  (2025-12-31T07:59:43Z)
• Understanding the Dilemma of Unlearning for Large Language Models [50.54260066313032]
  Unlearning seeks to remove specific knowledge from large language models (LLMs)<n>We propose unPact, an interpretable framework for unlearning via prompt attribution and contribution tracking.
  arXiv  Detail & Related papers  (2025-09-29T12:15:19Z)
• On Code-Induced Reasoning in LLMs [21.875805779552564]
  We construct parallel instruction datasets in ten programming languages.<n>We apply controlled perturbations that selectively disrupt structural or semantic properties of code.<n>Across 3,331 experiments, our results show that LLMs are more vulnerable to structural perturbations than semantic ones.
  arXiv  Detail & Related papers  (2025-09-25T19:57:36Z)
• Learning-Time Encoding Shapes Unlearning in LLMs [25.48677069802298]
  We investigate how learning-time choices in knowledge encoding impact the effectiveness of unlearning factual knowledge.<n>Our results suggest that learning-time knowledge encoding may play a central role in enabling reliable post-hoc unlearning.
  arXiv  Detail & Related papers  (2025-06-18T02:42:02Z)
• The Strawberry Problem: Emergence of Character-level Understanding in Tokenized Language Models [2.531076482407163]
  Large Language Models (LLMs) consistently fail at simple character-level tasks, such as counting letters in words, due to a fundamental limitation: tokenization.<n>We propose a lightweight architectural modification that significantly improves character-level reasoning while preserving the inductive advantages of subword models.<n>Our results bridge low-level perceptual gaps in tokenized LMs and provide a principled framework for understanding and mitigating their structural blind spots.
  arXiv  Detail & Related papers  (2025-05-20T10:25:17Z)
• LLM Library Learning Fails: A LEGO-Prover Case Study [20.25809428140996]
  We investigate LEGO-Prover, which purports to learn reusable lemmas for mathematical reasoning.<n>We find no evidence of the direct reuse of learned lemmas, and find evidence against the soft reuse of learned lemmas.<n>Our findings suggest that serious misconceptions exist as to the effectiveness of these techniques.
  arXiv  Detail & Related papers  (2025-04-03T21:53:51Z)
• General Intelligence Requires Reward-based Pretraining [14.057301560895505]
  Large Language Models (LLMs) have demonstrated impressive real-world utility.<n>But their ability to reason adaptively and robustly remains fragile.<n>We propose disangling knowledge and reasoning through three key directions.
  arXiv  Detail & Related papers  (2025-02-26T18:51:12Z)
• CLOSER: Towards Better Representation Learning for Few-Shot Class-Incremental Learning [52.63674911541416]
  Few-shot class-incremental learning (FSCIL) faces several challenges, such as overfitting and forgetting. Our primary focus is representation learning on base classes to tackle the unique challenge of FSCIL. We find that trying to secure the spread of features within a more confined feature space enables the learned representation to strike a better balance between transferability and discriminability.
  arXiv  Detail & Related papers  (2024-10-08T02:23:16Z)
• Knowledge Crosswords: Geometric Knowledge Reasoning with Large Language Models [49.23348672822087]
  We propose Knowledge Crosswords, a benchmark consisting of incomplete knowledge networks bounded by structured factual constraints. The novel setting of geometric knowledge reasoning necessitates new LM abilities beyond existing atomic/linear multi-hop QA. We conduct extensive experiments to evaluate existing LLMs and approaches on Knowledge Crosswords.
  arXiv  Detail & Related papers  (2023-10-02T15:43:53Z)
• RECKONING: Reasoning through Dynamic Knowledge Encoding [51.076603338764706]
  We show that language models can answer questions by reasoning over knowledge provided as part of the context. In these situations, the model fails to distinguish the knowledge that is necessary to answer the question. We propose teaching the model to reason more robustly by folding the provided contextual knowledge into the model's parameters.
  arXiv  Detail & Related papers  (2023-05-10T17:54:51Z)
• Online Structured Meta-learning [137.48138166279313]
  Current online meta-learning algorithms are limited to learn a globally-shared meta-learner. We propose an online structured meta-learning (OSML) framework to overcome this limitation. Experiments on three datasets demonstrate the effectiveness and interpretability of our proposed framework.
  arXiv  Detail & Related papers  (2020-10-22T09:10:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.