Identification and Learning of Semantic Observation Kernels: Partial Observation, Uniform Recovery, & Minimax Limits

• URL: http://arxiv.org/abs/2607.23130v1
• Date: Sat, 25 Jul 2026 10:14:58 GMT
• Title: Identification and Learning of Semantic Observation Kernels: Partial Observation, Uniform Recovery, & Minimax Limits
• Abstract summary: We ask when an observable language law can support a reproducible posterior over declared states.<n>A semantic map groups meaning-equivalent continuations; held-out cases with reference posteriors identify a semiparametric inverse from grouped language probabilities to state probabilities.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Probabilistic text generators supply conditional distributions over tokens and complete verbal continuations, whereas scientific use often requires a posterior over a finite state. Large language models are the leading example: phrase probabilities depend on prompt wording, and model-printed percentages are generated text rather than state posteriors. More generally, we ask when an observable language law can support a reproducible posterior over declared states. A semantic map groups meaning-equivalent continuations; held-out cases with reference posteriors identify a semiparametric inverse from grouped language probabilities to state probabilities. The language law remains nonparametric and no hidden model quantity is used. This is principally a theory and methods paper and makes several contributions. On the theoretical side, we derive conditions for existence, identification, stable recovery, and sequential updating; concentration, asymptotic, and nonparametric rates; identified sets under truncated probabilities; and a minimax boundary for uniform stability. On the empirical side, theorem- directed simulations verify recovery rates, compatible-set coverage, and stability gates, while two frozen language-model studies illustrate held-out recovery and conformal coverage. The results specify when observable language probabilities can provide an auditable state measurement without being interpreted as internal belief.

Related papers

• How to Verify Consistency of Probabilistic Claims [49.47758039235095]
  We construct an interactive PCP to prove a predictive model's consistency.<n>We show how a small additive-soundness gap removes dependence on B.<n>We view our interactive PCP as a first step toward predictive models to prove their own consistency.
  arXiv  Detail & Related papers  (2026-08-11T17:41:39Z)
• Calibrating Semantic Uncertainty from Observable Language-Model Probabilities [0.0]
  Language models produce probabilities over words, but professional decisions require uncertainty over meaningful states.<n>We introduce a semantic map: a prespecified, testable bridge from probabilities over verbal responses to probabilities over declared states.<n>We derive posterior-error bounds and conditions for existence, uniqueness, stability and sequential Bayesian updating.
  arXiv  Detail & Related papers  (2026-07-20T00:29:04Z)
• Finite Certificates for In-Context Determinacy and a Threshold Theory of Emergence in Language Models [0.2864713389096699]
  This paper develops a model-theoretic framework for verifying context-conditioned language-model behavior.<n>It provides finite context certificates, pair-separator hitting sets, query teaching dimension, prompt-preservation criteria, and scale-limit witnesses.
  arXiv  Detail & Related papers  (2026-05-30T14:07:58Z)
• The Need for an External Observer Formalizing the Sufficiency Gap: A Mathematical Extension of Mixture Identifiability and Contextual Grounding in Sequence Models [0.0]
  We construct a binary mixed-regime process with one deterministic textual regime and one random regime governed by an unobserved latent state.<n>The resulting entropy difference is not an ordinary optimization error; it is a sufficiency gap caused by marginalization over an unobserved state.<n>A corrective signal reverses the posterior odds induced by the textual history exactly when its fidelity exceeds the text-only posterior weight assigned to the misleading regime.
  arXiv  Detail & Related papers  (2026-05-26T08:53:11Z)
• When Is Next-Token Prediction Useful? Marginalization, Ergodicity, Mixture Identifiability, Local Sufficiency, RAG, Tools, and Programming [0.0]
  The paper argues that interpreting model training as estimating the marginal text-only law requires strong assumptions of stationarity, representativeness, and ergodicity.<n>The paper interprets Retrieval Augmented Generation (RAG) and tool use as conditional sufficiency devices.
  arXiv  Detail & Related papers  (2026-05-22T06:34:17Z)
• Agent-BRACE: Decoupling Beliefs from Actions in Long-Horizon Tasks via Verbalized State Uncertainty [70.43119366710778]
  We introduce Agent-BRACE: Agent Belief state Representation via Abstraction and Confidence Estimation.<n>It is a method that decouples an LLM agent into a belief state model and a policy model, jointly optimized via reinforcement learning.<n>It achieves an average absolute improvement of +14.5% across long-horizon, partially observable language environments.
  arXiv  Detail & Related papers  (2026-05-12T02:37:04Z)
• Diverse Dictionary Learning [47.381311762862985]
  Given only observational data $X = g(Z)$, where both the latent variables $Z$ and the generating process $g$ are unknown, recovering $Z$ is ill-posed without additional assumptions.<n>We show that intersections, complements, and symmetric differences of latent variables linked to arbitrary observations, along with the latent-to-observed dependency structure, are still identifiable up to appropriate indeterminacies.
  arXiv  Detail & Related papers  (2026-04-19T18:18:34Z)
• FactReasoner: A Probabilistic Approach to Long-Form Factuality Assessment for Large Language Models [59.171510592986735]
  We propose FactReasoner, a new factuality assessor that relies on probabilistic reasoning to assess the factuality of a long-form generated response.<n>Our experiments on labeled and unlabeled benchmark datasets demonstrate clearly that FactReasoner improves considerably over state-of-the-art prompt-based approaches.
  arXiv  Detail & Related papers  (2025-02-25T19:01:48Z)
• General detectability measure [45.85698554568285]
  Distinguishing resource states from resource-free states is a fundamental task in quantum information.<n>We derived the optimal exponential decay rate of the failure probability for detecting a given $n$-tensor product state.
  arXiv  Detail & Related papers  (2025-01-16T05:39:22Z)
• Detecting Renewal States in Chains of Variable Length via Intrinsic Bayes Factors [0.0]
  We propose the use of Intrinsic Bayes Factor to evaluate the plausibility of each set of renewal states. To show the strength of our method, we analyzed artificial datasets generated from two binary models models and one example coming from the field of Linguistics.
  arXiv  Detail & Related papers  (2021-10-14T14:57:44Z)
• Nested Counterfactual Identification from Arbitrary Surrogate Experiments [95.48089725859298]
  We study the identification of nested counterfactuals from an arbitrary combination of observations and experiments. Specifically, we prove the counterfactual unnesting theorem (CUT), which allows one to map arbitrary nested counterfactuals to unnested ones.
  arXiv  Detail & Related papers  (2021-07-07T12:51:04Z)
• Tractable Inference in Credal Sentential Decision Diagrams [116.6516175350871]
  Probabilistic sentential decision diagrams are logic circuits where the inputs of disjunctive gates are annotated by probability values. We develop the credal sentential decision diagrams, a generalisation of their probabilistic counterpart that allows for replacing the local probabilities with credal sets of mass functions. For a first empirical validation, we consider a simple application based on noisy seven-segment display images.
  arXiv  Detail & Related papers  (2020-08-19T16:04:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.