Symbolic Augmentation Closes a Canonical-Equivalence Blind Spot in Neural Fact-Checkers

• URL: http://arxiv.org/abs/2607.16212v1
• Date: Tue, 19 May 2026 15:58:00 GMT
• Title: Symbolic Augmentation Closes a Canonical-Equivalence Blind Spot in Neural Fact-Checkers
• Abstract summary: We introduce a five-class typed-quantity error taxonomy and a 1500-item benchmark.<n>A ModernBERT encoder fine-tuned on this benchmark reaches macro-F1 = 0.899, far above any off-the-shelf neural fact-checker.<n>We propose Symbolic Augmentation, a training-time framework that runs the modules of a symbolic verifier in reverse to generate label-preserving augmented training data.
• Score: 1.7844382164707184
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models hallucinate numbers and units when summarizing scientific text, a failure mode that can silently invert a scientific claim. We recast the detection of such errors as typed verification: we introduce a five-class typed-quantity error taxonomy and a 1500-item benchmark, rewritten from PMC and arXiv sources and labeled by two independent LLM annotators with adjudication (Krippendorff's alpha = 0.882). A ModernBERT encoder fine-tuned on this benchmark reaches macro-F1 = 0.899, far above any off-the-shelf neural fact-checker, yet four probes expose a sharp structural blind spot: on canonical-equivalent rewrites of physically equivalent quantities (e.g., 95°C and 368.15 K) its accuracy collapses to 36.5%. We propose Symbolic Augmentation, a training-time framework that runs the modules of a symbolic verifier in reverse to generate label-preserving augmented training data. The augmentation lifts canonical-equivalence robustness to 98.2% while slightly improving in-distribution accuracy (macro-F1: 0.899 to 0.902); the augmented encoder matches a closed-frontier LLM at no inference cost and transfers to an external benchmark (SciFact-Open binary macro-F1: 0.791 to 0.828). Two negative results sharpen the claim: symbolic features as auxiliary encoder inputs add nothing, and symbolic silver labels scale negatively under teacher noise. Together these results identify training-time augmentation as the right integration point between symbolic and learned components.

Related papers

• Empirical Evaluation of Membership Inference Attacks on NLP Text Classifiers: A Baseline Study on SST-2 [0.0]
  Membership inference attacks (MIAs) try to determine whether a specific record was used to train a model.<n>This paper presents a benchmark of membership inference vulnerability for text classification on the GLUE SST-2 sentiment dataset.
  arXiv  Detail & Related papers  (2026-09-10T00:53:20Z)
• Support Operation Factorization: Compositional Readout of Frozen Vision Encoders under Controlled Interventions [87.95757610455173]
  We introduce an injectively aligned leave-one-cell-out protocol over support x operation grids and SO-OPF.<n>We separate two questions that aggregate scores conflate: whether the carrier composes held-out bindings when the grid is known, and whether that grid can be recovered from flat cell labels.
  arXiv  Detail & Related papers  (2026-08-06T15:38:55Z)
• Mapping CVEs to MITRE ATT&CK Techniques: A Curated Gold-Set Classifier and the Limits of LLM-Assisted Label Expansion [46.262619407930266]
  We train a multi-label classifier on a curated gold dataset of 1,207 CVEs from expert MITRE Center for Threat-Informed Defense mappings.<n>The resulting model approximately doubles recall@5 compared with a zero-shot embedding-similarity baseline.<n>We then investigate whether LLM-assisted labeling can extend the gold dataset.
  arXiv  Detail & Related papers  (2026-07-28T10:59:04Z)
• Closing the Modality Gap in Zero-Shot HAR: Contrastive Training and Separability-Optimized Prototypes on IMU Data [0.0]
  We evaluate seven configurations combining three inference methods with two training pipelines on the PAMAP2 dataset.<n>We find that the modality gap is a training-time phenomenon governed by the objective.<n>We also demonstrate that overall accuracy is a misleading primary metric when test-set class distributions are imbalanced.
  arXiv  Detail & Related papers  (2026-06-09T12:39:41Z)
• From Theory to Decision Rule: Calibrating the Noisy-Label Crossover for Vision-Language Model Weak Supervision Across Three Medical-Imaging Benchmarks [41.99844472131922]
  We provide a benchmark calibration for foundation-model weak labels.<n>The crossover predicted by theory appears at ng100 on PCAM, 20-50 on ISIC, and 250-500 on NIH-CXR.<n>A structured-vs-random noise sign flip on NIH-CXR shows that the rate-only formulation of the bound is incomplete.
  arXiv  Detail & Related papers  (2026-05-23T23:15:07Z)
• Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction [51.56484100374058]
  Text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary.<n>On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures.<n>On RoBERTa-base, raw projection exceeds fine-tuning; on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested.
  arXiv  Detail & Related papers  (2026-05-20T19:08:38Z)
• SERSEM: Selective Entropy-Weighted Scoring for Membership Inference in Code Language Models [1.327416973220814]
  We propose SERSEM, a novel white-box attack framework that suppresses uninformative syntactical boilerplate to amplify memorization signals.<n>Our findings demonstrate that focusing on human-centric coding anomalies provides a significantly more robust indicator of verbatim memorization than sequence-level probability averages.
  arXiv  Detail & Related papers  (2026-04-01T17:03:58Z)
• BiGain: Unified Token Compression for Joint Generation and Classification [47.040577759493004]
  BiGain is a training-free, plug-and-play framework that preserves generation quality while improving classification in accelerated diffusion models.<n>Our key insight is frequency separation, which disentangles fine detail from global semantics, enabling compression that respects both generative fidelity and discriminative utility.<n>Our analyses indicate that balanced spectral retention, preserving high-frequency detail and low/mid-frequency semantics, is a reliable design rule for token compression in diffusion models.
  arXiv  Detail & Related papers  (2026-03-12T17:55:53Z)
• Probing for Knowledge Attribution in Large Language Models [45.47366023067617]
  Large language models (LLMs) often generate fluent but unfounded claims, or hallucinations.<n> Proper mitigation depends on knowing whether a model's answer is based on the prompt or its internal weights.<n>We show that a probe, a simple linear classifier trained on model hidden representations, can reliably predict contributive attribution.
  arXiv  Detail & Related papers  (2026-02-26T09:21:12Z)
• Predictive Coding and Information Bottleneck for Hallucination Detection in Large Language Models [0.8552050317027305]
  Hallucinations in Large Language Models (LLMs) remain a critical barrier to high-stakes deployment.<n>We introduce [Model Name], a hybrid detection framework that combines neuroscience-inspired signal design with supervised machine learning.
  arXiv  Detail & Related papers  (2026-01-22T05:00:21Z)
• SEER: Spectral Entropy Encoding of Roles for Context-Aware Attention-Based Design Pattern Detection [0.0]
  This paper presents an upgraded version of our prior method Context Is All You Need for detecting Gang of Four (GoF) design patterns from source code.<n> SEER addresses these limitations with two principled additions: (i) a spectral-entropy role encoder that derives per-member role embeddings from the Laplacian spectrum of each class's interaction graph, and (ii) a time-weighted calling context that assigns empirically calibrated duration priors to method categories.<n>We evaluate SEER on PyDesignNet (1,832 files, 35,000 sequences, 23 GoF patterns) and observe consistent gains over our previous system
  arXiv  Detail & Related papers  (2026-01-19T19:13:40Z)
• Synergistic Feature Fusion for Latent Lyrical Classification: A Gated Deep Learning Architecture [0.0]
  This study addresses the challenge of integrating complex, high-dimensional deep semantic features with simple, interpretable structural cues for lyrical content classification.<n>We introduce a novel Synergistic Fusion Layer (SFL) architecture, a deep learning model utilizing a gated mechanism to modulate Sentence-BERT embeddings (Fdeep) using low-dimensional auxiliary features (Fstruct)<n>The SFL model achieved an accuracy of 0.9894 and a Macro F1 score of 0.9894, outperforming a comprehensive Random Forest (RF) baseline that used feature concatenation.
  arXiv  Detail & Related papers  (2025-11-11T21:12:52Z)
• Mint: A Simple Test-Time Adaptation of Vision-Language Models against Common Corruptions [44.25678062208464]
  We investigate how corruptions affect CLIP's image embeddings and uncover a consistent phenomenon we term as embedding variance collapse.<n>We find that this collapse is closely tied to performance degradation, with inter-class variance strongly correlated with classification accuracy.<n>We propose Mint, a simple test-time adaptation method that maximizes pseudo-label-based inter-class variance on the fly.
  arXiv  Detail & Related papers  (2025-10-25T02:55:08Z)
• Co-training for Low Resource Scientific Natural Language Inference [65.37685198688538]
  We propose a novel co-training method that assigns weights based on the training dynamics of the classifiers to the distantly supervised labels. By assigning importance weights instead of filtering out examples based on an arbitrary threshold on the predicted confidence, we maximize the usage of automatically labeled data. The proposed method obtains an improvement of 1.5% in Macro F1 over the distant supervision baseline, and substantial improvements over several other strong SSL baselines.
  arXiv  Detail & Related papers  (2024-06-20T18:35:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.