Martingale Doppelgänger-Eval: An Identification Framework for Auditing Candlestick Understanding in Vision-Language Models

• URL: http://arxiv.org/abs/2606.17423v1
• Date: Tue, 16 Jun 2026 02:10:37 GMT
• Title: Martingale Doppelgänger-Eval: An Identification Framework for Auditing Candlestick Understanding in Vision-Language Models
• Abstract summary: Martingale Doppelgnger-Eval is a public shadow-market benchmark for auditing whether vision-language models (VLMs) use candlestick evidence rather than extrapolate past trends.<n>A structural behavioral model identifies null-market bias, trend sensitivity, evidence sensitivity, prompt/renderer fragility, and evidence faithfulness.
• Score: 7.600410227599106
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce Martingale Doppelgänger-Eval, a public shadow-market benchmark for auditing whether vision-language models (VLMs) use candlestick evidence rather than extrapolate past trends. The central difficulty is identification: on real market histories, chart evidence and trend are strongly coupled, so an observational score cannot determine whether a fluent technical-analysis narrative is grounded in local visual evidence. We prove this limitation formally: no evaluation functional computed from observational chart--label data can distinguish a grounded responder from a trend-shortcut responder under strong coupling, whereas matched evidence interventions separate the same responders at an exponential rate and trend--label swaps provide an independent shortcut stress test. The benchmark therefore evaluates frozen VLMs on rendered OHLCV charts under four controlled mechanisms: a martingale-null market, injected-alpha counterfactual pairs, trend-confounder swaps, and regime shifts. A structural behavioral model identifies null-market bias, trend sensitivity, evidence sensitivity, prompt/renderer fragility, and evidence faithfulness; the accompanying statistical toolkit provides minimum detectable effects, block-aware sequential testing for metered APIs, and an overlap-weighted artifact check. Across frozen commercial and open VLMs, the identified regression assigns large positive coefficients to past trend but evidence coefficients that are zero or opposite to the rule-implied sign. Matched-pair analyses show that models either ignore injected candlestick semantics or move opposite to the rule-implied direction conditional on responding. The benchmark isolates a failure mode that standard observational chart benchmarks cannot detect and gives a reusable audit template for time-series imagery with controllable label mechanisms.

Related papers

• DiffImaginE: Imagine to Verify Entity Types with Diffusion [17.922112345279643]
  We introduce DiffImaginE, which formulates MNER type verification as conditional latent diffusion inference.<n>Given span-localised visual evidence, a type-conditioned denoiser predicts noise injected into its standardised latent.<n>The resulting denoising error provides an ELBO-consistent surrogate for type-conditional negative log-likelihood.
  arXiv  Detail & Related papers  (2026-08-04T02:13:45Z)
• PerceptionRubrics: Calibrating Multimodal Evaluation to Human Perception [75.70273182182397]
  PerceptionRubrics is a rubric-based evaluation framework.<n>It addresses the gap between saturated benchmark scores and real-world brittleness.<n>It pairs 1,038 information-dense images with over 10,000 instance-specific rubrics.
  arXiv  Detail & Related papers  (2026-06-26T17:59:15Z)
• Scaling Pretrained Representations Enables Label-Free Out-of-Distribution Detection Without Fine-Tuning [5.008779702997125]
  We show that modern pretrained models encode sufficient geometric structure for accurate label-free out-of-distribution detection.<n>Our results suggest that label-free OOD detection depends strongly on the geometry exposed by frozen pretrained backbones.
  arXiv  Detail & Related papers  (2026-05-07T03:45:52Z)
• Deployment-Relevant Alignment Cannot Be Inferred from Model-Level Evaluation Alone [11.663456969895462]
  Alignment evaluation in machine learning has largely become evaluation of models.<n>This paper argues that deployment-relevant alignment cannot be inferred from model-level evaluation alone.
  arXiv  Detail & Related papers  (2026-05-06T03:28:30Z)
• Psychologically Potent, Computationally Invisible: LLMs Generate Social-Comparison Triggers They Fail to Detect [0.5735035463793009]
  Task targets a socially meaningful relational signal that is behaviorally real yet not reducible to sentiment.<n>XHS-SCoRE contributes both a benchmark for reader-grounded comparison detection and a diagnostic framework for studying when socially meaningful relational cues remain only partially visible to prompt-based inference.
  arXiv  Detail & Related papers  (2026-05-01T18:28:10Z)
• SAAG: Structured Agent Assessment and Grounding [13.62148061055744]
  We propose a cascaded diagnostic framework that decomposes agent-calling evaluation into three sequential stages.<n>We evaluate this framework on a benchmark derived from Glaive's function-calling dataset across registry sizes of 5, 10, and 15 agents.
  arXiv  Detail & Related papers  (2026-04-30T19:33:58Z)
• Grounding the Score: Explicit Visual Premise Verification for Reliable Vision-Language Process Reward Models [8.630726904040781]
  Explicit Visual Premise Verification (EVPV) is a lightweight verification interface that conditions step scoring on the reliability of the visual premises a step depends on.<n>EVPV improves step-level verification and consistently boosts Best-of-N reranking accuracy over strong baselines.
  arXiv  Detail & Related papers  (2026-03-17T08:40:26Z)
• Same Answer, Different Representations: Hidden instability in VLMs [65.36933543377346]
  We introduce a representation-aware and frequency-aware evaluation framework that measures internal embedding drift, spectral sensitivity, and structural smoothness.<n>We apply this framework to modern Vision Language Models (VLMs) across the SEEDBench, MMMU, and POPE datasets.
  arXiv  Detail & Related papers  (2026-02-06T12:24:26Z)
• RISE: Interactive Visual Diagnosis of Fairness in Machine Learning Models [3.409008933025072]
  We introduce textitRISE (Residual Inspection through Sorted Evaluation), an interactive visualization tool that converts sorted residuals into interpretable patterns.<n>By connecting residual curve structures to formal fairness notions, RISE enables localized disparity diagnosis, subgroup comparison across environments, and the detection of hidden fairness issues.
  arXiv  Detail & Related papers  (2026-02-04T09:01:21Z)
• Cross-modal Proxy Evolving for OOD Detection with Vision-Language Models [59.242742594156546]
  CoEvo is a test-time framework that performs bidirectional, sample-conditioned adaptation of both textual and visual proxies.<n>CoEvo achieves state-of-the-art performance, improving AUROC by 1.33% and reducing FPR95 by 45.98% on ImageNet-1K compared to strong negative-label baselines.
  arXiv  Detail & Related papers  (2026-01-13T12:08:26Z)
• MM-CoT:A Benchmark for Probing Visual Chain-of-Thought Reasoning in Multimodal Models [49.32415342913976]
  We introduce MM-CoT, a diagnostic benchmark designed to probe the visual grounding and logical coherence of CoT reasoning in multimodal models.<n>We evaluate leading vision-language models on MM-CoT and find that even the most advanced systems struggle, revealing a sharp discrepancy between generative fluency and true reasoning fidelity.
  arXiv  Detail & Related papers  (2025-12-09T04:13:31Z)
• Reference-Specific Unlearning Metrics Can Hide the Truth: A Reality Check [60.77691669644931]
  We propose Functional Alignment for Distributional Equivalence (FADE), a novel metric that measures distributional similarity between unlearned and reference models.<n>We show that FADE captures functional alignment across the entire output distribution, providing a principled assessment of genuine unlearning.<n>These findings expose fundamental gaps in current evaluation practices and demonstrate that FADE provides a more robust foundation for developing and assessing truly effective unlearning methods.
  arXiv  Detail & Related papers  (2025-10-14T20:50:30Z)
• Reference-Free Rating of LLM Responses via Latent Information [53.463883683503106]
  We study the common practice of asking a judge model to assign Likert-scale scores to free-text responses.<n>We then propose and evaluate Latent Judges, which derive scalar ratings from internal model signals.<n>Across a broad suite of pairwise and single-rating benchmarks, latent methods match or surpass standard prompting.
  arXiv  Detail & Related papers  (2025-09-29T12:15:52Z)
• TrustLoRA: Low-Rank Adaptation for Failure Detection under Out-of-distribution Data [62.22804234013273]
  We propose a simple failure detection framework to unify and facilitate classification with rejection under both covariate and semantic shifts.<n>Our key insight is that by separating and consolidating failure-specific reliability knowledge with low-rank adapters, we can enhance the failure detection ability effectively and flexibly.
  arXiv  Detail & Related papers  (2025-04-20T09:20:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.