PerCoR: Evaluating Commonsense Reasoning in Persian via Multiple-Choice Sentence Completion

• URL: http://arxiv.org/abs/2510.22616v1
• Date: Sun, 26 Oct 2025 10:25:02 GMT
• Title: PerCoR: Evaluating Commonsense Reasoning in Persian via Multiple-Choice Sentence Completion
• Abstract summary: We introduce PerCoR, the first large-scale Persian benchmark for commonsense reasoning.<n>PerCoR contains 106K multiple-choice sentence-completion problems drawn from more than forty news, cultural, and other web sources.<n>We introduce a novel conjunction-based segmentation strategy to generate coherent sentence-completion pairs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduced PerCoR (Persian Commonsense Reasoning), the first large-scale Persian benchmark for commonsense reasoning. PerCoR contains 106K multiple-choice sentence-completion problems drawn from more than forty news, cultural, and other web sources. We introduce a novel conjunction-based segmentation strategy to generate coherent sentence-completion pairs, enabling broad topical and structural diversity. To create challenging distractors, we propose DRESS-AF (Distractor Ranking via Embedding Similarity Scoring and Adversarial Filtering), a generation-free adversarial filtering method that selects distractors from the pool of gold continuations while maximising model confusion. Human annotators score 89% on PerCoR, while OpenAI-o3 achieves the highest performance at 92.18%, followed closely by Claude-Sonnet-3.7 (91.17%). The strongest open-source model, DeepSeek-R1, reaches 82.51%, underscoring both the dataset's difficulty and the remaining performance gap in Persian commonsense reasoning. We further show that DRESS-AF transfers to the English HellaSwag benchmark, increasing its difficulty without hurting human solvability. The dataset is available at https://huggingface.co/datasets/MCINext/PerCoR.

Related papers

• Efficient Visual Pointing for Embodied AI:Agent-Driven Data Synthesis, Cross-Block Attention, and Iterative Correction [55.11480729304395]
  We describe our PointArena 2026 solution, which achieves 77.2% overall accuracy and ranks second on the benchmark.<n>The ap proach targets three failure modes. First, agent-driven syn thesis builds large semantic and anchor-relative candidate pools.<n>Second, a determinis tic steerable-data pipeline creates a verified 10,000-sample main set, plus reserve samples, using masks, templates, and path verification.
  arXiv  Detail & Related papers  (2026-06-29T06:39:03Z)
• Scheduling Thoughts: Learning the Order of Thought in Diffusion Language Models [51.21958617138889]
  Self-Aware Scheduling learns a lightweight order policy and applies seamlessly to both any-order and semi-autoregressive decoding.<n>On Sudoku with 1B MDM, SAS improves puzzle accuracy from 82.0% (best schedule) to 91.8%, and reaches 97.5% with second-stage fine-tuning.<n>On mathematical reasoning with LLaDA-8B, SAS improves pass@1 on GSM8K from 64% to 76% and on MBPP from 39.5% to 41%.
  arXiv  Detail & Related papers  (2026-06-22T16:32:25Z)
• Byzantine-Resilient Federated Learning via QUBO-Based Client Selection on Quantum Annealers [0.0]
  Federated Learning (FL) trains a global model across decentralized clients while preserving data privacy, but at scale it is vulnerable to malicious updates.<n>We propose a quantum annealing approach that reformulates client selection as a Quadratic Unconstrained Binary Optimization problem.<n>At small scale (15 clients), QUBO outperforms MultiKrum on the most challenging Byzantine attacks.<n>At 100 clients on MNIST, MultiSignal achieves 95.3% average detection accuracy versus 91.8% for classical MultiKrum.
  arXiv  Detail & Related papers  (2026-05-14T22:42:14Z)
• Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching [66.39914384073145]
  We propose a self-consistency framework that turns cheap diffusion-sampled reasoning into a reusable pool of step-level candidates.<n>We find that step-level recombination is most beneficial on harder problems.<n>Our training-free framework improves average accuracy by up to 2 across six math and coding tasks.
  arXiv  Detail & Related papers  (2026-02-26T11:08:39Z)
• ALPS: A Diagnostic Challenge Set for Arabic Linguistic & Pragmatic Reasoning [0.0]
  ALPS (Arabic Linguistic & Pragmatic Suite) is a native, expert-curated diagnostic challenge set probing Deep Semantics and Pragmatics.<n> ALPS targets the depth of linguistic understanding through 531 rigorously crafted questions across 15 tasks and 47 subtasks.<n>We developed the dataset with deep expertise in Arabic linguistics, guaranteeing cultural authenticity and eliminating translation artifacts.
  arXiv  Detail & Related papers  (2026-02-19T03:51:37Z)
• Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space [110.80564213032729]
  Diffusion Language Models (DLMs) offer order-agnostic generation that can explore many possible decoding trajectories.<n>We introduce Order-Token Search to explore this space through jointly searching over generation order and token values.
  arXiv  Detail & Related papers  (2026-01-28T07:55:07Z)
• Autonomous Knowledge Graph Exploration with Adaptive Breadth-Depth Retrieval [26.815539094904278]
  ARK: Adaptive Retriever of Knowledge, gives a language model control over this breadth-depth tradeoff.<n>ARK alternates between breadth-oriented discovery and depth-oriented expansion without depending on a fragile seed selection.<n>ARK reaches 59.1% average Hit@1 and 67.4 average MRR, improving average Hit@1 by up to 31.4% and average MRR by up to 28.0% over retrieval-based and agentic training-free methods.
  arXiv  Detail & Related papers  (2026-01-20T13:46:37Z)
• SpeechJudge: Towards Human-Level Judgment for Speech Naturalness [25.566620976752443]
  SpeechJudge is a suite comprising a dataset, a benchmark, and a reward model.<n>We present SpeechJudge-Data, a large-scale human feedback corpus of 99K speech pairs.<n> SpeechJudge-Eval is a challenging benchmark for speech naturalness judgment.<n>We develop SpeechJudge-GRM, a generative reward model based on Qwen2.5- Omni-7B.
  arXiv  Detail & Related papers  (2025-11-11T07:27:20Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• Automated Knowledge Graph Construction using Large Language Models and Sentence Complexity Modelling [0.39089069256361736]
  We introduce CoDe-KG, an end-to-end pipeline for extracting sentence-level knowledge graphs.<n>We contribute a dataset of over 150,000 knowledge triples, which is open source.<n>We show that hybrid chain-of-thought and few-shot prompting yields up to 99.8% exact-match accuracy on sentence simplification.
  arXiv  Detail & Related papers  (2025-09-22T00:01:50Z)
• Optimized Text Embedding Models and Benchmarks for Amharic Passage Retrieval [49.1574468325115]
  We introduce Amharic-specific dense retrieval models based on pre-trained Amharic BERT and RoBERTa backbones.<n>Our proposed RoBERTa-Base-Amharic-Embed model (110M parameters) achieves a 17.6% relative improvement in MRR@10.<n>More compact variants, such as RoBERTa-Medium-Amharic-Embed (42M) remain competitive while being over 13x smaller.
  arXiv  Detail & Related papers  (2025-05-25T23:06:20Z)
• ProgRoCC: A Progressive Approach to Rough Crowd Counting [66.09510514180593]
  We label Rough Crowd Counting that delivers better accuracy on the basis of training data that is easier to acquire.<n>We propose an approach to the rough crowd counting problem based on CLIP, termed ProgRoCC.<n>Specifically, we introduce a progressive estimation learning strategy that determines the object count through a coarse-to-fine approach.
  arXiv  Detail & Related papers  (2025-04-18T01:57:42Z)
• CoKe: Customizable Fine-Grained Story Evaluation via Chain-of-Keyword Rationalization [90.15027447565427]
  Chain of thought (CoT) generates free-text explanations that help guide a model's predictions.<n>Self-Consistency (SC) marginalizes predictions over multiple generated explanations.<n>We propose $textbfC$hain-$textbfo$f-$textbfKe$ywords (CoKe)
  arXiv  Detail & Related papers  (2025-03-21T13:37:46Z)
• STOP! Benchmarking Large Language Models with Sensitivity Testing on Offensive Progressions [6.19084217044276]
  Mitigating explicit and implicit biases in Large Language Models (LLMs) has become a critical focus in the field of natural language processing.<n>We introduce the Sensitivity Testing on Offensive Progressions dataset, which includes 450 offensive progressions containing 2,700 unique sentences.<n>Our findings reveal that even the best-performing models detect bias inconsistently, with success rates ranging from 19.3% to 69.8%.
  arXiv  Detail & Related papers  (2024-09-20T18:34:38Z)
• DCR: Divide-and-Conquer Reasoning for Multi-choice Question Answering with LLMs [9.561022942046279]
  We propose Divide and Conquer Reasoning (DCR) to enhance the reasoning capability of large language models (LLMs) We first categorize questions into two subsets based on confidence score ($mathcalCS$), which is estimated by statistical frequency of generated answers. In particular, we first categorize questions into two subsets based on confidence score ($mathcalCS$), which is estimated by statistical frequency of generated answers.
  arXiv  Detail & Related papers  (2024-01-10T14:38:46Z)
• CTC-based Non-autoregressive Speech Translation [51.37920141751813]
  We investigate the potential of connectionist temporal classification for non-autoregressive speech translation. We develop a model consisting of two encoders that are guided by CTC to predict the source and target texts. Experiments on the MuST-C benchmarks show that our NAST model achieves an average BLEU score of 29.5 with a speed-up of 5.67$times$.
  arXiv  Detail & Related papers  (2023-05-27T03:54:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.