Verifiable, Articulable, and Tacit Components of Preference
Abstract Overview
This paper investigates how much human preference can be captured by explicit rules versus what remains difficult to articulate or verify. It introduces CreativePreferences, a large-scale dataset spanning 2.8 million texts and 317 million human preference judgments across 42 tasks in seven domains: coding, mathematics, law, academia, journalism, creative writing, and humor. The authors evaluate three modeling regimes—verifiable metric models, articulated rubric-based models, and dense full-text preference models—and use metric discovery, deconfounding, and capture-recapture bounds to quantify articulability and verifiability gaps. Across domains, they find substantial articulability and verifiability gaps, even in domains traditionally treated as verifiable, and examine the drivers and consequences of these gaps.
Novelty
The paper introduces a systematic cross-domain measurement framework for decomposing preference into verifiable, articulable, and residual components at scale using the novel CreativePreferences dataset. It also develops a measurement pipeline integrating automated metric discovery, spurious-feature auditing, and capture-recapture estimation to bound the performance contributions of undiscovered criteria.
Results
Articulability and verifiability gaps are observed across all evaluated domains, with peer review and creative writing exhibiting the largest articulability gaps, while humor is relatively articulable but less verifiable. These gaps remain robust against metric scaling, prompt optimization, stronger judge models, and alternative aggregation architectures. Dense full preference models more reliably reflect human choices than articulated models, and optimizing strictly on articulated criteria leads to shedding tacit preference components.
Key Points
- CreativePreferences aggregates 317 million human preference judgments across 2.8 million texts and 42 tasks spanning seven creative and technical domains.
- The authors decompose preference into verifiable metrics, articulable rubrics, and dense full models, using missing-metric bounds and deconfounding audits to robustly estimate gap bounds.
- Articulated rubrics and verifiable checks consistently fail to fully recover human preference, leading to measurable selection regret when ranking candidate outputs.