FregeLogic at SemEval 2026 Task 11: A Hybrid Neuro-Symbolic Architecture for Content-Robust Syllogistic Validity Prediction

• URL: http://arxiv.org/abs/2604.18328v1
• Date: Mon, 20 Apr 2026 14:31:00 GMT
• Title: FregeLogic at SemEval 2026 Task 11: A Hybrid Neuro-Symbolic Architecture for Content-Robust Syllogistic Validity Prediction
• Abstract summary: We present FregeLogic, a hybrid neuro-symbolic system for SemEval-2026 Task 11 (Subtask 1)<n>Our system achieves 94.3% accuracy with a content effect of 2.85 and a combined score of 41.88 in nested 5-fold cross-validation on the dataset (N=960)<n>Our results suggest that targeted neuro-symbolic integration, applying formal methods precisely where ensemble consensus is lowest, can improve the combined accuracy-plus-content-effect metric used by this task.
• Score: 0.0017904458681854366
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present FregeLogic, a hybrid neuro-symbolic system for SemEval-2026 Task 11 (Subtask 1), which addresses syllogistic validity prediction while reducing content effects on predictions. Our approach combines an ensemble of five LLM classifiers, spanning three open-weights models (Llama 4 Maverick, Llama 4 Scout, and Qwen3-32B) paired with varied prompting strategies, with a Z3 SMT solver that serves as a formal logic tiebreaker. The central hypothesis is that LLM disagreement within the ensemble signals likely content-biased errors, where real-world believability interferes with logical judgment. By deferring to Z3's structurally-grounded formal verification on these disputed cases, our system achieves 94.3% accuracy with a content effect of 2.85 and a combined score of 41.88 in nested 5-fold cross-validation on the dataset (N=960). This represents a 2.76-point improvement in combined score over the pure ensemble (39.12), with a 0.9% accuracy gain, driven by a 16% reduction in content effect (3.39 to 2.85). Adopting structured-output API calls for Z3 extraction reduced failure rates from ~22% to near zero, and an Aristotelian encoding with existence axioms was validated against task annotations. Our results suggest that targeted neuro-symbolic integration, applying formal methods precisely where ensemble consensus is lowest, can improve the combined accuracy-plus-content-effect metric used by this task.

Related papers

• State of Thought Enables Endogenous Reasoning [48.902117018115256]
  Test-time reasoning has emerged as a major approach to improving the capabilities of Large Language Models (LLMs)<n>We propose State of Thought, a new reasoning paradigm that enables endogenous reasoning in LLMs.<n>We show that SoT consistently improves mean-baseline accuracy while reducing generated tokens by 62.6% and end-to-end latency by 44.6%.
  arXiv  Detail & Related papers  (2026-09-13T05:55:03Z)
• Nyström Attention Matches Full Attention for Cross-Sectional Stock Prediction [0.0]
  MASTER's inter-stock multi-head attention accounts for 42.5% of model parameters and 25% of predictive value.<n>We uncover a surprising structure: the learned attention is near-uniform (perplexity 278/300), yet forcing exact uniformity eliminates all cross-sectional discrimination.
  arXiv  Detail & Related papers  (2026-09-08T01:34:53Z)
• Improving O-RADS Risk Stratification from Ultrasound Reports: A Comparative Evaluation of Hybrid versus End-to-End LLM Reasoning Strategies [17.85382282659117]
  We compared the performance of large language models (LLMs) and reasoning strategies for automated OvarianAd Reportingal and Data Fable System (ORADS) classification.<n>The feature-based hybrid architecture demonstrated almost the best performance, achieving an accuracy of 99.2% (387 of 390) and agreement with the reference standard.
  arXiv  Detail & Related papers  (2026-08-24T10:04:17Z)
• Beyond Activation Alignment:The Alignment-Diversity Tradeoff in Task-Aware LLM Quantization [36.286359836094086]
  We propose TASA (Task-Aware Sensitivity Analysis) to jointly optimize calibration-data composition and mixed-precision bit allocation.<n>Experiments on LLaMA-3-8B and Qwen2.5-7B reveal a precision inversion: appropriately allocated 3.5-bit models can match or surpass less task-aware 4-bit baselines.
  arXiv  Detail & Related papers  (2026-07-01T13:12:21Z)
• ReacTOD: Bounded Neuro-Symbolic Agentic NLU for Zero-Shot Dialogue State Tracking [2.2573512203799626]
  We propose ReacTOD, a bounded neuro-symbolic architecture that reformulates NLU as discrete tool calls within a self-correcting ReAct loop governed by deterministic validation.<n>A bounded ReAct loop enables iterative self-correction, improving accuracy by up to 9.3 percentage points over single-pass inference on MultiWOZ.
  arXiv  Detail & Related papers  (2026-05-18T20:06:04Z)
• Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning [40.342912574072024]
  Large Language Models produce structured outputs such as travel plans or code solutions.<n>Individual reasoning steps may appear correct while the output as a whole violates budgets, fails test cases, or contradicts earlier deductions.<n>We propose a learned quality scorer with deterministic analytical constraint penalties for verifying structured LLM outputs.
  arXiv  Detail & Related papers  (2026-05-15T17:08:27Z)
• WISV: Wireless-Informed Semantic Verification for Distributed Speculative Decoding in Device-Edge LLM Inference [56.297697169678095]
  WISV (Wireless-Informed Semantic Verification) is a novel distributed speculative decoding framework.<n>We show that WISV achieves up to a 60.8% increase in accepted length, a 37.3% reduction in interaction rounds, and a 31.4% improvement in end-to-end latency.<n>We validate WISV on a hardware testbed comprising an NVIDIA Jetson AGX Orin and an A40-equipped server.
  arXiv  Detail & Related papers  (2026-04-20T01:29:56Z)
• THEIA: Learning Complete Kleene Three-Valued Logic in a Pure-Neural Modular Architecture [0.0]
  THEIA learns complete Kleene three-valued logic (K3) from task data without external symbolic inference or hand-encoded K3 gate primitives.<n> Transformer baselines also reach >99% on all 39 rules, and flats match THEIA on Phase-1 accuracy within 0.04pp.
  arXiv  Detail & Related papers  (2026-04-13T10:44:15Z)
• Fragile Reasoning: A Mechanistic Analysis of LLM Sensitivity to Meaning-Preserving Perturbations [2.7946918847372277]
  Large language models demonstrate strong performance on mathematical reasoning benchmarks, yet remain surprisingly fragile to meaning-preserving surface perturbations.<n>We systematically evaluate three open-weight LLMs, Mistral-7B, Llama-3-8B, and Qwen2.5-7B, on 677 GSM8K problems paired with semantically equivalent variants.<n>All three models exhibit substantial answer-flip rates (28.8%-45.1%), with number paraphrasing consistently more disruptive than name swaps.
  arXiv  Detail & Related papers  (2026-04-02T05:30:20Z)
• LongCat-Flash-Prover: Advancing Native Formal Reasoning via Agentic Tool-Integrated Reinforcement Learning [46.294745464571456]
  LongCat-Flash-Prover is an open-source MoE model for agentic tool-integrated reasoning.<n>It sets a new state-of-the-art for open-weights models in both auto-formalization and theorem proving.<n>It achieves a 97.1% pass rate on MiniF2F-Test using only 72 inference budget per problem.
  arXiv  Detail & Related papers  (2026-03-22T05:16:09Z)
• From Black Box to Glass Box: Cross-Model ASR Disagreement to Prioto Review in Ambient AI Scribe Documentation [43.148402136307716]
  Cross-model disagreement among heterogeneous ASR systems can act as a reference-free uncertainty signal.<n>We transcribed 50 publicly available medical audio clips with eight ASR systems spanning commercial APIs and open-source engines.<n>Low-agreement regions were enriched for content disagreements, with the content fraction increasing from 53.9% to 73.9% across quintiles of high-risk mass.
  arXiv  Detail & Related papers  (2026-03-02T13:02:13Z)
• Scalable Multi-Task Low-Rank Model Adaptation [43.22544779625565]
  Scaling multi-task low-rank adaptation (LoRA) to a large number of tasks induces catastrophic performance degradation.<n>We find that existing solutions, like regularization and dynamic routing, fail at scale because they are constrained by a fundamental trade-off.<n>We propose mtLoRA, a scalable solution with three novel designs.
  arXiv  Detail & Related papers  (2026-03-02T06:57:11Z)
• Differentiable Symbolic Planning: A Neural Architecture for Constraint Reasoning with Learned Feasibility [0.0]
  Differentiable Symbolic Planning (DSP) is a neural architecture that performs discrete symbolic reasoning.<n>We integrate DSP into a Universal Cognitive Kernel (UCK) that combines graph attention with iterative constraint propagation.<n>UCK+DSP achieves 97.4% accuracy on planning under 4x size generalization.
  arXiv  Detail & Related papers  (2026-02-19T03:38:03Z)
• ACAR: Adaptive Complexity Routing for Multi-Model Ensembles with Auditable Decision Traces [3.151184728006369]
  We present ACAR, a measurement framework for studying multi-model orchestration under auditable conditions.<n>ACAR uses self-consistency variance (sigma) computed from N=3 probe samples to route tasks across single-model, two-model, and three-model execution modes.<n>We evaluate ACAR on 1,510 tasks spanning four benchmarks, producing more than 7,550 auditable runs.
  arXiv  Detail & Related papers  (2026-02-06T23:27:17Z)
• JTok: On Token Embedding as another Axis of Scaling Law via Joint Token Self-modulation [46.64215658042213]
  We introduce Joint-Token (JTok) and Mixture of Joint-Token (JTok-M), which augment Transformer layers with modulation vectors retrieved from auxiliary embedding tables.<n>These vectors modulate the backbone via lightweight, element-wise operations, incurring negligible FLOPs overhead.<n>Our approach consistently reduces validation loss and significantly improves downstream task performance.
  arXiv  Detail & Related papers  (2026-01-31T16:15:18Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• Accurate and Reliable Predictions with Mutual-Transport Ensemble [46.368395985214875]
  We propose a co-trained auxiliary model and adaptively regularizes the cross-entropy loss using Kullback-Leibler (KL) We show that MTE can simultaneously enhance both accuracy and uncertainty calibration. For example, on the CIFAR-100 dataset, our MTE method on ResNet34/50 achieved significant improvements compared to previous state-of-the-art method.
  arXiv  Detail & Related papers  (2024-05-30T03:15:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.