The FIL Hypothesis: Inductive Biases Help with Kernel Engineering

• URL: http://arxiv.org/abs/2606.30442v1
• Date: Mon, 29 Jun 2026 15:16:33 GMT
• Title: The FIL Hypothesis: Inductive Biases Help with Kernel Engineering
• Abstract summary: General-purpose methods that scale with computation and data ultimately outperform those with built-in human knowledge.<n>We revisit this principle by observing a new and critical scaling dimension: the duration of the Feedback Information Loop (FIL)<n>We argue that future AI applications in science and the physical world will inherently involve FILs ranging from hours to weeks.
• Score: 88.25617363280156
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The Bitter Lesson, which posits that general-purpose methods that scale with computation and data ultimately outperform those with built-in human knowledge, has become a dominant paradigm in the era of Large Language Models. We revisit this principle by observing a new and critical scaling dimension: the duration of the Feedback Information Loop (FIL), the time required for a system to receive a verification signal after generating a prediction. Most historic successes in Artificial Intelligence (AI) have benefited from near instantaneous feedback (e.g., games or classification tasks), but we argue that future AI applications in science and the physical world will inherently involve FILs ranging from hours to weeks. This trend poses a fundamental scaling limit, as obtaining enough verification steps required by purely data-driven methods becomes practically impossible. Additionally, we propose a method that is orthogonal to purely data-driven approaches, based on human-inspired expert knowledge. The method relies on inductive biases and constraining the solution space. We provide an initial validation of the hypothesis and the method, by studying the real-world GPU programming task, a domain with non-trivial FIL, and demonstrate that incorporating inductive biases yields superior performance over data-driven approaches. The code is released under: https://github.com/ai-nikolai/robust_kernelbench

Related papers

• Can In-Context Learning Support Intrinsic Curiosity? [25.43664318918942]
  We investigate whether the emergent in-context learning capabilities of sequence models can eliminate this bottleneck.<n>Specifically, we evaluate whether an exploration policy can be trained to maximize learning progress.<n>We prove a positive result for a broad subclass of non-temporal settings.
  arXiv  Detail & Related papers  (2026-06-17T18:11:06Z)
• AI Agents as Universal Task Solvers [94.49762121230042]
  We show that the optimal speed-up that a universal solver can achieve using past data is tightly related to their algorithmic information.<n>We argue that the key quantity to optimize when scaling reasoning models is time, whose critical role in learning has so far only been indirectly considered.
  arXiv  Detail & Related papers  (2025-10-14T02:17:54Z)
• Embedding Domain Knowledge for Large Language Models via Reinforcement Learning from Augmented Generation [18.99847259801634]
  We propose Reinforcement Learning from Augmented Generation (RLAG) to embed domain knowledge into large language models.<n>Our approach iteratively cycles between sampling generations and optimize the model through calculated rewards.<n> Experimental results across medical, legal, astronomy, and current events datasets demonstrate that our proposed method significantly outperforms baseline approaches.
  arXiv  Detail & Related papers  (2025-09-24T14:30:16Z)
• Reasoning to Learn from Latent Thoughts [61.2395150828168]
  We show that explicitly modeling and inferring the emphlatent thoughts that underlie the text generation process can significantly improve pretraining data efficiency.<n>We show that a 1B LM can bootstrap its performance across at least three iterations and significantly outperform baselines trained on raw data.
  arXiv  Detail & Related papers  (2025-03-24T16:41:23Z)
• Evaluation of Artificial Intelligence Methods for Lead Time Prediction in Non-Cycled Areas of Automotive Production [1.3499500088995464]
  The present study examines the effectiveness of applying Artificial Intelligence methods in an automotive production environment.<n>Data structures are analyzed to identify contextual features and then preprocessed using one-hot encoding.<n>The research demonstrates that AI methods can be effectively applied to highly variable production data, adding business value.
  arXiv  Detail & Related papers  (2025-01-13T13:28:03Z)
• Informed Spectral Normalized Gaussian Processes for Trajectory Prediction [0.0]
  We propose a novel regularization-based continual learning method for SNGPs. Our proposal builds upon well-established methods and requires no rehearsal memory or parameter expansion. We apply our informed SNGP model to the trajectory prediction problem in autonomous driving by integrating prior drivability knowledge.
  arXiv  Detail & Related papers  (2024-03-18T17:05:24Z)
• Adversarial Auto-Augment with Label Preservation: A Representation Learning Principle Guided Approach [95.74102207187545]
  We show that a prior-free autonomous data augmentation's objective can be derived from a representation learning principle. We then propose a practical surrogate to the objective that can be efficiently optimized and integrated seamlessly into existing methods.
  arXiv  Detail & Related papers  (2022-11-02T02:02:51Z)
• Principled Knowledge Extrapolation with GANs [92.62635018136476]
  We study counterfactual synthesis from a new perspective of knowledge extrapolation. We show that an adversarial game with a closed-form discriminator can be used to address the knowledge extrapolation problem. Our method enjoys both elegant theoretical guarantees and superior performance in many scenarios.
  arXiv  Detail & Related papers  (2022-05-21T08:39:42Z)
• Value-driven Hindsight Modelling [68.658900923595]
  Value estimation is a critical component of the reinforcement learning (RL) paradigm. Model learning can make use of the rich transition structure present in sequences of observations, but this approach is usually not sensitive to the reward function. We develop an approach for representation learning in RL that sits in between these two extremes. This provides tractable prediction targets that are directly relevant for a task, and can thus accelerate learning the value function.
  arXiv  Detail & Related papers  (2020-02-19T18:10:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.