The FIL Hypothesis: Inductive Biases Help with Kernel Engineering
Abstract Overview
This paper argues that the duration of the Feedback Information Loop (FIL)—the time required to verify an AI system’s output—is an important scaling dimension that has been underemphasized relative to data and compute. The authors contend that many emerging AI applications in science and engineering involve verification times from minutes to weeks, which makes the large numbers of feedback-driven iterations used by purely data-driven methods increasingly impractical. To address this, they propose an inductive-bias-based test-time search method that decomposes a task into ordered intermediate stages and constrains the search space using human-inspired guidance. They study this idea on GPU kernel engineering with KernelBench, where outputs must be compiled, run, checked for correctness, and profiled for speed.
Novelty
The paper’s main novelty is the FIL hypothesis itself: that longer verification delays impose a fundamental limit on purely data-driven scaling and reopen a role for explicit inductive biases. Methodologically, it introduces an ordered decomposition of verification metrics with stage-specific inductive-bias prompting for slow-FIL tasks such as kernel engineering.
Results
On KernelBench, the inductive-bias variant consistently improves over iterative refinement across compilation, runtime, and correctness for all tested model families, with no reported degradation on these core metrics. The largest absolute gains reported are up to +0.31 in compilation success, +0.17 in runtime success, and +0.12 in correctness, while absolute correctness still remains limited, topping out at 0.33 for the best setting. A control study further suggests that the benefit of inductive biases shrinks when models have additional coding-focused pretraining.
Key Points
- The paper proposes Feedback Information Loop (FIL) duration as a third scaling dimension, arguing that long verification times make many future scientific AI tasks poorly suited to purely data-driven methods.
- Its proposed solution is an inductive-bias search procedure that breaks verification into ordered binary stages—such as compilation, runtime, correctness, and speed—and uses stage-specific prompts to constrain search.
- Experiments on KernelBench show broad improvements from inductive biases over iterative refinement, but also indicate that the task remains difficult and that gains are smaller for models with more coding-specialized pretraining.