metasignal: A Python Package for Comprehensive Metacognitive Analysis and Decision-Making

• URL: http://arxiv.org/abs/2607.29093v1
• Date: Fri, 31 Jul 2026 07:18:19 GMT
• Title: metasignal: A Python Package for Comprehensive Metacognitive Analysis and Decision-Making
• Abstract summary: Metasignal is an open-source Python package for signal detection theory and metacognitive measurement.<n>It implements the 17 metacognitive measures evaluated by Rahnev (2025)<n>The package also provides a command-line interface, group summaries, bootstrap confidence intervals, permutation tests, optional hierarchical Bayesian models, and information-theoretic measures.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Metasignal is an open-source Python package for signal detection theory (SDT) and metacognitive measurement. It implements the 17 metacognitive measures evaluated by Rahnev (2025), together with the reference variables d' (perceptual sensitivity), response criterion c (response bias), and mean confidence. The 17 measures comprise three meta-d' family estimates, meta-d', M-ratio, and M-difference; four nonparametric Type-2 measures, the Type-2 area under the receiver-operating-characteristic curve (AUC2), Gamma, Phi, and delta confidence, together with their eight SDT-normalized ratio and difference forms; and two model-based measures, meta-noise and meta-uncertainty. A single function computes the complete set from trial-level stimulus, response, and confidence arrays. `metasignal` currently supports binary (two-alternative) discrimination tasks, in which each trial's stimulus and response are coded with exactly two categories. The package also provides a command-line interface, group summaries, bootstrap confidence intervals, permutation tests, optional hierarchical Bayesian models, and information-theoretic measures. `metasignal` unifies these measures in a single platform to encourage broader metacognition research and adoption in decision-making studies.

Related papers

• DynaBridge: Dynamic Summary-Guided Cross-Task Multimodal Fusion for DASS-Structured Mental Health Assessment [21.51648903357327]
  DynaBridge is a dynamic summary-guided cross-task multimodal framework for DASS-structured mental health assessment.<n>It encodes acoustic, visual, and textual cues across multiple sessions and augments them with frozen-LLM-generated DASS-aware summaries as participant-level semantic evidence.<n>It predicts ordinal item distributions, reconstructs depression, anxiety, and stress risk evidence from item-level soft scores, and fuses this evidence with direct multimodal risk predictions.
  arXiv  Detail & Related papers  (2026-07-28T12:55:29Z)
• Semantic Drift in Bug Resolution: How Behavioral Signals Propagate from Reports to Tests and Patches [18.727291516223115]
  Desc2Fix is a framework for measuring semantic alignment between bug reports, triggering tests, and developer-written fixes.<n>We analyze 2,857 report-test-patch triplets from Defects4J and SWT-Bench using GPT-4o and DeepSeek-Chat.
  arXiv  Detail & Related papers  (2026-07-20T22:32:23Z)
• Traits Run Deeper: Trait-Specific Asymmetric Fusion for Personality Assessment [53.50824432699408]
  Traits Run Deeper is a novel personality assessment framework.<n>It consists of three components: the Multimodal Foundation Representation (MFR), the Trait-Specific Modality Fusion (TSMF) and the Distribution-Calibrated Personality Regression (DCPR)
  arXiv  Detail & Related papers  (2026-06-09T06:38:36Z)
• Do LLMs Know What They Know? Measuring Metacognitive Efficiency with Signal Detection Theory [0.0]
  We introduce an evaluation framework based on Type-2 Signal Detection Theory that decomposes capacities using meta-calibrated' and the metacognitive efficiency ratio M-ratio.<n> Applied to four LLMs across 224,000 factual QA trials, we find: (1) metacognitive efficiency varies substantially across models even when Type-1 sensitivity is similar -- Mistral achieves the highest d' but the lowest M-ratio; (2) metacognitive efficiency is domain-specific, with different models showing different weakest domains, invisible to aggregate metrics;.<n>The meta-calibrated' framework reveals which models "know what they don't know" versus which merely
  arXiv  Detail & Related papers  (2026-03-26T07:38:28Z)
• Variational Rectification Inference for Learning with Noisy Labels [74.85528327499662]
  We propose variational rectification inference (VRI) to formulate the adaptive rectification for loss functions.<n>VRI is constructed as a hierarchical Bayes by treating the rectifying vector as a latent variable.<n>By introducing the variational term in VRI, the conditional posterior is estimated accurately and avoids collapsing to a Dirac delta function.
  arXiv  Detail & Related papers  (2026-03-18T01:25:08Z)
• Episode-specific Fine-tuning for Metric-based Few-shot Learners with Optimization-based Training [13.5196633635749]
  Metric-based models operate by computing similarities between query and support embeddings within a learned metric space.<n>A small set of labeled support samples is provided during inference to aid the classification of unlabeled query samples.<n>We propose a series of simple yet effective episode-specific, during-inference fine-tuning methods for metric-based models.
  arXiv  Detail & Related papers  (2025-06-20T22:24:38Z)
• MetaScale: Test-Time Scaling with Evolving Meta-Thoughts [51.35594569020857]
  Experimental results demonstrate that MetaScale consistently outperforms standard inference approaches.<n> METASCALE scales more effectively with increasing sampling budgets and produces more structured, expert-level responses.
  arXiv  Detail & Related papers  (2025-03-17T17:59:54Z)
• Machine Learning Evaluation Metric Discrepancies across Programming Languages and Their Components: Need for Standardization [2.4125217894669184]
  This study evaluates metrics for tasks such as classification, regression, clustering, correlation analysis, statistical tests, segmentation, and image-to-image (I2I) translation.<n> Metrics were compared across Python libraries, R packages, and Matlab functions to assess consistency and highlight discrepancies.
  arXiv  Detail & Related papers  (2024-11-18T20:07:31Z)
• Improving Bias Correction Standards by Quantifying its Effects on Treatment Outcomes [54.18828236350544]
  Propensity score matching (PSM) addresses selection biases by selecting comparable populations for analysis. Different matching methods can produce significantly different Average Treatment Effects (ATE) for the same task, even when meeting all validation criteria. To address this issue, we introduce a novel metric, A2A, to reduce the number of valid matches.
  arXiv  Detail & Related papers  (2024-07-20T12:42:24Z)
• A Statistical Analysis of Summarization Evaluation Metrics using Resampling Methods [60.04142561088524]
  We find that the confidence intervals are rather wide, demonstrating high uncertainty in how reliable automatic metrics truly are. Although many metrics fail to show statistical improvements over ROUGE, two recent works, QAEval and BERTScore, do in some evaluation settings.
  arXiv  Detail & Related papers  (2021-03-31T18:28:14Z)
• Meta-Learned Confidence for Few-shot Learning [60.6086305523402]
  A popular transductive inference technique for few-shot metric-based approaches, is to update the prototype of each class with the mean of the most confident query examples. We propose to meta-learn the confidence for each query sample, to assign optimal weights to unlabeled queries. We validate our few-shot learning model with meta-learned confidence on four benchmark datasets.
  arXiv  Detail & Related papers  (2020-02-27T10:22:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.