Beyond QA Matching: Perturbation-Response Fingerprinting via Probability Distributions for Large Language Models

• URL: http://arxiv.org/abs/2609.06330v1
• Date: Sun, 06 Sep 2026 01:35:07 GMT
• Title: Beyond QA Matching: Perturbation-Response Fingerprinting via Probability Distributions for Large Language Models
• Abstract summary: We introduce BReF, a training-free fingerprint that compares how probability distributions over four answer-option labels A/B/C/D move under controlled perturbations.<n>On a unified benchmark with 34 checkpoints, 22 documented direct-parent relations, and 411 suspect-candidate pairs, BReF retrieves the documented parent in 22/22 cases.<n>Same-family discrimination is harder, and paired tests show a significant exact-retrieval gain over a magnitude-only Top-25 control.
• Score: 12.84004924761875
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models are often instruction-tuned, specialized, quantized, or otherwise transformed, making fine-grained provenance difficult. In this paper, we introduce BReF, a training-free fingerprint that compares how probability distributions over four answer-option labels A/B/C/D move under controlled textual perturbations. For each pair of models, BReF selects 25 jointly responsive probes and compares their perturbation log-ratio (PLR) response directions by global cosine similarity. On a unified benchmark with 34 checkpoints, 22 documented direct-parent relations, and 411 suspect-candidate pairs, BReF retrieves the documented parent in 22/22 cases (MRR=1.0000), with DP-DF AUC 1.0000. Same-family discrimination is harder (DP-SF AUC 0.8969), and paired tests show a significant exact-retrieval gain over a magnitude-only Top-25 control. Together with static, random-probe, permutation, calibration, and transformation-level controls, the results show that strong pooled separation does not guarantee correct parent ranking among closely related checkpoints, verifying the superiority of our work.

Related papers

• Coding Agents Have Converged: Why the SWE-bench Leaderboard Can No Longer Order Its Top Entries, and What to Measure Instead [9.25138311822007]
  Small differences on coding-agent leaderboards are often read as an ordering of systems.<n>We audit whether the published verdicts support this reading, using 254 SWE-bench submissions across four splits without running models.<n>We release the partition and a five-step audit protocol that profiles shared outcomes, tests paired differences, reports grouping sensitivity, and estimates the instance budget needed for resolution.
  arXiv  Detail & Related papers  (2026-09-15T16:29:12Z)
• Visual Credit Audit for Multimodal Spatial Reasoning [70.16915309526443]
  Visual Credit Audit separates two Estimands: whether the benchmark image gives the model's declared decision more support than text-only and blank controls, and whether the model responds to relation-specific visual evidence.<n>Applying labels yields dependence-credited correctness (D-CC)<n>Across four open MLLMs and two spatial benchmarks, 12.73-26.25% of decisions are correct yet uncredited.
  arXiv  Detail & Related papers  (2026-07-29T15:55:31Z)
• EvidentialRAG: Quantifying and Mitigating Information Conflict in Multi-Source Retrieval-Augmented Generation via Evidential Deep Learning [0.2730969268472861]
  This paper introduces ERAG, an uncertainty-aware RAG framework that converts retrieved chunks into probabilistic evidence before generation.<n>Experiments on CRAG, ConflictQA, and MuSiQue show that ERAG remains competitive with the strongest matched baseline on standard question answering.
  arXiv  Detail & Related papers  (2026-07-11T21:56:15Z)
• When LLMs Agree, Are They Right? Auditing Self-Consistency and Cross-Model Agreement as Confidence Signals [0.0]
  LLM-as-judge is increasingly the default for evaluating AI systems in enterprise pipelines.<n>We show that consistency -- agreement among judges, or a model's own samples -- indicates correctness.<n>We ask when agreement is nonetheless a usable proxy, in a large-scale cross-runner study.
  arXiv  Detail & Related papers  (2026-07-09T02:46:51Z)
• Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search [50.16356451328644]
  Proving Shannon-type entropy inequalities is a fundamental task in information theory.<n>We investigate whether small-scale large language models, fine-tuned on atomic proof steps, can automate this process.<n>We show that GPT-5.5 solves 1.7% samples under zero-shot prompting while Psitip solves 33.3% samples.
  arXiv  Detail & Related papers  (2026-06-04T05:43:12Z)
• CAPS: Cascaded Adaptive Pairwise Selection for Efficient Parallel Reasoning [24.596125996494717]
  CAPS is an inference-only framework that allocates verifier compute non-uniformly along two axes.<n> CAPS outperforms the leading pairwise verifier on 14 of 20 suites while using 25.4% of its verifier-token budget on code.
  arXiv  Detail & Related papers  (2026-05-15T01:16:12Z)
• Confidence-Aware Alignment Makes Reasoning LLMs More Reliable [65.44962502963378]
  We introduce CASPO, a framework that aligns token-level confidence with step-wise logical correctness without training a separate reward model.<n>During inference, we propose Confidence-aware Thought (CaT), which dynamically prunes uncertain reasoning branches with negligible O(V) latency.<n>Experiments across ten benchmarks and multiple model families show that CASPO consistently improves reasoning reliability and inference efficiency.
  arXiv  Detail & Related papers  (2026-05-08T07:08:25Z)
• Heterogeneous Ordinal Structure Learning with Bayesian Nonparametric Complexity Discovery [1.089614199781423]
  Existing ordinal structure learners assume a shared directed acyclic graph (DAG) across all respondents.<n>Recent ordinal graphical-model approaches focus on subgroup discovery rather than confirmatory cluster-specific DAG estimation.
  arXiv  Detail & Related papers  (2026-05-05T18:32:49Z)
• SpatialBench-UC: Uncertainty-Aware Evaluation of Spatial Prompt Following in Text-to-Image Generation [0.0]
  SpatialBench-UC is a small, reproducible benchmark for pairwise spatial relations.<n>We release a benchmark package, versioned prompts, pinned configs, per-sample checker outputs, and report tables.<n>We evaluate three baselines, Stable Diffusion 1.5, SD 1.5 BoxDiff, and SD 1.4 GLIGEN.
  arXiv  Detail & Related papers  (2026-01-19T23:37:10Z)
• CLUE: Non-parametric Verification from Experience via Hidden-State Clustering [64.50919789875233]
  We show that correctness of a solution is encoded as a geometrically separable signature within the trajectory of hidden activations.<n>ClUE consistently outperforms LLM-as-a-judge baselines and matches or exceeds modern confidence-based methods in reranking candidates.
  arXiv  Detail & Related papers  (2025-10-02T02:14:33Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• Two-Sample Testing on Ranked Preference Data and the Role of Modeling Assumptions [57.77347280992548]
  In this paper, we design two-sample tests for pairwise comparison data and ranking data. Our test requires essentially no assumptions on the distributions. By applying our two-sample test on real-world pairwise comparison data, we conclude that ratings and rankings provided by people are indeed distributed differently.
  arXiv  Detail & Related papers  (2020-06-21T20:51:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.