Position: Every Ground Truth is a Human Construction, not an Objective Truth

• URL: http://arxiv.org/abs/2607.09668v1
• Date: Thu, 28 May 2026 20:10:50 GMT
• Title: Position: Every Ground Truth is a Human Construction, not an Objective Truth
• Abstract summary: Ground truth datasets play a fundamental role as reference values in the training and evaluation of machine learning models.<n>We argue that ground truths are not neutral objective measurements that are naturally given, but instead that they are constructed by arrangements of humans and technologies.
• Score: 1.20053409042067
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Ground truth datasets play a fundamental role as reference values in the training and evaluation of machine learning models. This position paper argues that ground truths are not neutral objective measurements that are naturally given, but instead that they are constructed by arrangements of humans and technologies. We argue that the ML community will benefit from articulating and discussing these often invisible or unreported choices and acknowledging that reference data sets are contingent, not universal. Focusing on the situated and context-dependent nature of ground truths can improve reliability by enabling a better informed perspective on where, when, and how the datasets, and the models they have shaped, can best be used. We argue for increasing `situated reliability' which includes articulating the limits and strengths of models and their truth claims. Finally, paying more attention to the construction of ground truths can support transparency, accountability, and interdisciplinary work.

Related papers

• Information Discernment in Large Language Models [4.277461007196155]
  Learn2Discern is an experimental framework and benchmark grounded in three normative axioms with interpretable metrics.<n>We find consistent failures across both dimensions: models perform near chance on source and truth discernment, rely on source popularity twice as much as source reliability, and update roughly equally whether a claim improves or worsens their position relative to the ground truth.
  arXiv  Detail & Related papers  (2026-05-22T12:13:16Z)
• Exploring the Generalizability of Factual Hallucination Mitigation via Enhancing Precise Knowledge Utilization [49.95746521480879]
  We introduce PKUE (Precise Knowledge Utilization Enhancement), which fine-tunes the model on self-generated responses to precise and simple factual questions.<n>Extensive experiments demonstrate that PKUE significantly improves LLM overall performance.
  arXiv  Detail & Related papers  (2025-02-26T13:34:52Z)
• Look Ma, No Ground Truth! Ground-Truth-Free Tuning of Structure from Motion and Visual SLAM [64.57742015099531]
  Evaluation is critical to both developing and tuning Structure from Motion (SfM) and Visual SLAM (VSLAM) systems.<n>This dependency on ground truth restricts SfM and SLAM applications across diverse environments.<n>We propose a novel ground-truth-free (GTF) evaluation methodology that eliminates the need for geometric ground truth.
  arXiv  Detail & Related papers  (2024-12-02T04:40:03Z)
• FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows" [74.7488607599921]
  FaithEval is a benchmark to evaluate the faithfulness of large language models (LLMs) in contextual scenarios.<n>FaithEval comprises 4.9K high-quality problems in total, validated through a rigorous four-stage context construction and validation framework.<n>Our study reveals that even state-of-the-art models often struggle to remain faithful to the given context, and that larger models do not necessarily exhibit improved faithfulness.
  arXiv  Detail & Related papers  (2024-09-30T06:27:53Z)
• AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agents [27.10147264744531]
  We study how Large Language Models (LLM)-based agents navigate scenarios in a multi-turn interactive setting.<n>We develop a truthfulness detector inspired by psychological literature to assess the agents' responses.<n>Our experiment demonstrates that all models are truthful less than 50% of the time, though truthfulness and goal achievement (utility) rates vary across models.
  arXiv  Detail & Related papers  (2024-09-13T17:41:12Z)
• LLMs' Reading Comprehension Is Affected by Parametric Knowledge and Struggles with Hypothetical Statements [59.71218039095155]
  Task of reading comprehension (RC) provides a primary means to assess language models' natural language understanding (NLU) capabilities.<n>If the context aligns with the models' internal knowledge, it is hard to discern whether the models' answers stem from context comprehension or from internal information.<n>To address this issue, we suggest to use RC on imaginary data, based on fictitious facts and entities.
  arXiv  Detail & Related papers  (2024-04-09T13:08:56Z)
• Alignment for Honesty [105.72465407518325]
  Recent research has made significant strides in aligning large language models (LLMs) with helpfulness and harmlessness. In this paper, we argue for the importance of alignment for emphhonesty, ensuring that LLMs proactively refuse to answer questions when they lack knowledge. We address these challenges by first establishing a precise problem definition and defining honesty'' inspired by the Analects of Confucius.
  arXiv  Detail & Related papers  (2023-12-12T06:10:42Z)
• A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia [57.31074448586854]
  Large language models (LLMs) have an impressive ability to draw on novel information supplied in their context. Yet the mechanisms underlying this contextual grounding remain unknown. We present a novel method to study grounding abilities using Fakepedia.
  arXiv  Detail & Related papers  (2023-12-04T17:35:42Z)
• Personas as a Way to Model Truthfulness in Language Models [23.86655844340011]
  Large language models (LLMs) are trained on vast amounts of text from the internet. This paper presents an explanation for why LMs appear to know the truth despite not being trained with truth labels.
  arXiv  Detail & Related papers  (2023-10-27T14:27:43Z)
• Truth Machines: Synthesizing Veracity in AI Language Models [0.0]
  We discuss the struggle for truth in AI systems and the general responses to date. It then investigates the production of truth in InstructGPT, a large language model. We argue that these same logics and inconsistencies play out in ChatGPT, reiterating truth as a non-trivial problem.
  arXiv  Detail & Related papers  (2023-01-28T02:47:50Z)
• The Inconvenient Truths of Ground Truth for Binary Analysis [3.198144010381572]
  We show that not all ground truths are created equal. This paper challenges the binary analysis community to take a long look at the concept of ground truth.
  arXiv  Detail & Related papers  (2022-10-26T23:27:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.