PA-CDM: Position-Aware Character Detection Matching for Evaluating Handwritten Mathematical Expression Recognition

• URL: http://arxiv.org/abs/2609.12917v1
• Date: Fri, 11 Sep 2026 14:43:05 GMT
• Title: PA-CDM: Position-Aware Character Detection Matching for Evaluating Handwritten Mathematical Expression Recognition
• Abstract summary: Handwritten mathematical expression recognition is scored by exact-match rates and string-similarity metrics that are blind to where an error occurs.<n>We propose a position-aware metric that couples character detection matching with position-forest encoding and divergence-level weighting.<n>In a six-annotator study, PA-CDM attains the highest correlation with human judgments among seven automatic metrics.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Handwritten mathematical expression recognition (HMER) is conventionally scored by exact-match rates and string-similarity metrics that are blind to where an error occurs: two predictions with identical token-error counts receive identical scores whether they misplace a subscript or swap the operands of a fraction. Render-based character detection matching (CDM) aligns glyphs robustly but remains position-blind---on controlled fraction-operand swaps it scores 0.8595 where position-aware scoring yields 0.6253. Tree-edit metrics exhibit a complementary blind spot: rewrites outside the parser's normalization coverage are penalized as structural errors (0.8552 where render-based metrics score 1.0). We propose PA-CDM, a position-aware metric that couples character detection matching with position-forest encoding and divergence-level weighting; StructPerturb v2.0, a frozen benchmark of 1,340 controlled perturbation pairs across 15 type--intensity cells; and a cross-metric consistency protocol combining a sensitivity matrix, a human study, and LLM-judge calibration. In a six-annotator study, PA-CDM attains the highest correlation with human judgments among seven automatic metrics (Spearman rho=0.9535, n=990). A frontier LLM judge correlates slightly higher (rho=0.9613) but is costly, nondeterministic, and API-dependent; PA-CDM approaches it at zero marginal cost with deterministic, diagnosable behavior.

Related papers

• SHIFT-M3: Pre-fusion Alignment-based Consistency Screening for Multimodal ECG Record Integrity [0.0]
  Multimodal clinical AI typically assumes that the waveform, report, metadata, and downstream predictions attached to a record belong to the same patient.<n>We study this problem as multimodal record integrity triage: given an assembled record, should its modalities be trusted to belong together?<n>We introduce SHIFT-M3, a lightweight text-based pre-fusion screen that measures alignment-based consistency between two separately produced ECG text views.
  arXiv  Detail & Related papers  (2026-09-12T10:58:02Z)
• Relational Over-Regularization: Graph-Based AI-Generated Text Detection via Sentence Transition Deviation [4.4331439696271415]
  CrossSource Stylometric Fingerprint (CSFG) is a graph-based framework that encodes positional, sequential, semantic and transition deviation signals as learnable GNN edge features.<n>CSFG achieves 97.14% accuracy under binary detection, outperforming the strongest graph-based baseline by 11.14 pp.
  arXiv  Detail & Related papers  (2026-08-27T06:51:40Z)
• Towards Human-Like Interactive Speech Recognition With Agentic Correction and Semantic Evaluation [53.844308305341166]
  We propose textbfAgentic ASR, a closed-loop framework that combines a single-pass ASR front-end with semantic correction, intent routing, and reasoning-based editing.<n> Experiments on multilingual, named-entity-intensive, and code-switching benchmarks show that iterative interaction consistently reduces semantic errors.
  arXiv  Detail & Related papers  (2026-05-28T06:23:31Z)
• Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction [51.56484100374058]
  Text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary.<n>On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures.<n>On RoBERTa-base, raw projection exceeds fine-tuning; on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested.
  arXiv  Detail & Related papers  (2026-05-20T19:08:38Z)
• Symbolic Augmentation Closes a Canonical-Equivalence Blind Spot in Neural Fact-Checkers [1.7844382164707184]
  We introduce a five-class typed-quantity error taxonomy and a 1500-item benchmark.<n>A ModernBERT encoder fine-tuned on this benchmark reaches macro-F1 = 0.899, far above any off-the-shelf neural fact-checker.<n>We propose Symbolic Augmentation, a training-time framework that runs the modules of a symbolic verifier in reverse to generate label-preserving augmented training data.
  arXiv  Detail & Related papers  (2026-05-19T15:58:00Z)
• Representation Before Training: A Fixed-Budget Benchmark for Generative Medical Event Models [0.3250525349446657]
  We evaluate how representation decisions affect downstream prediction after a shared one-epoch pretraining budget.<n>We train 28 matched transformers on MIMIC-IV and evaluate them on 30 clinical outcomes.
  arXiv  Detail & Related papers  (2026-04-18T01:38:47Z)
• SERSEM: Selective Entropy-Weighted Scoring for Membership Inference in Code Language Models [1.327416973220814]
  We propose SERSEM, a novel white-box attack framework that suppresses uninformative syntactical boilerplate to amplify memorization signals.<n>Our findings demonstrate that focusing on human-centric coding anomalies provides a significantly more robust indicator of verbatim memorization than sequence-level probability averages.
  arXiv  Detail & Related papers  (2026-04-01T17:03:58Z)
• Beyond Raw Detection Scores: Markov-Informed Calibration for Boosting Machine-Generated Text Detection [105.14032334647932]
  Machine-generated texts (MGTs) pose risks such as disinformation and phishing, highlighting the need for reliable detection.<n> Metric-based methods, which extract statistically distinguishable features of MGTs, are often more practical than complex model-based methods that are prone to overfitting.<n>We propose a Markov-informed score calibration strategy that models two relationships of context detection scores that may aid calibration.
  arXiv  Detail & Related papers  (2026-02-08T16:06:12Z)
• Semiparametric conformal prediction [79.6147286161434]
  We construct a conformal prediction set accounting for the joint correlation structure of the vector-valued non-conformity scores.<n>We flexibly estimate the joint cumulative distribution function (CDF) of the scores.<n>Our method yields desired coverage and competitive efficiency on a range of real-world regression problems.
  arXiv  Detail & Related papers  (2024-11-04T14:29:02Z)
• End-to-End Page-Level Assessment of Handwritten Text Recognition [69.55992406968495]
  HTR systems increasingly face the end-to-end page-level transcription of a document. Standard metrics do not take into account the inconsistencies that might appear. We propose a two-fold evaluation, where the transcription accuracy and the RO goodness are considered separately.
  arXiv  Detail & Related papers  (2023-01-14T15:43:07Z)
• Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metrics [64.81682222169113]
  How reliably an automatic summarization evaluation metric replicates human judgments of summary quality is quantified by system-level correlations. We identify two ways in which the definition of the system-level correlation is inconsistent with how metrics are used to evaluate systems in practice.
  arXiv  Detail & Related papers  (2022-04-21T15:52:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.