Tree-of-Evidence: Efficient "System 2" Search for Faithful Multimodal Grounding

• URL: http://arxiv.org/abs/2604.07692v1
• Date: Thu, 09 Apr 2026 01:28:14 GMT
• Title: Tree-of-Evidence: Efficient "System 2" Search for Faithful Multimodal Grounding
• Abstract summary: We introduce Tree-of-Evidence (ToE), an inference-time search algorithm that frames interpretability as a discrete optimization problem.<n>ToE employs lightweight Evidence Bottlenecks that score coarse groups or units of data and performs a beam search to identify the compact evidence set required to reproduce the model's prediction.<n>Under sparse evidence budgets, ToE achieves higher decision agreement and lower probability fidelity error than other approaches.
• Score: 1.6365003324232734
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Multimodal Models (LMMs) achieve state-of-the-art performance in high-stakes domains like healthcare, yet their reasoning remains opaque. Current interpretability methods, such as attention mechanisms or post-hoc saliency, often fail to faithfully represent the model's decision-making process, particularly when integrating heterogeneous modalities like time-series and text. We introduce Tree-of-Evidence (ToE), an inference-time search algorithm that frames interpretability as a discrete optimization problem. Rather than relying on soft attention weights, ToE employs lightweight Evidence Bottlenecks that score coarse groups or units of data (e.g., vital-sign windows, report sentences) and performs a beam search to identify the compact evidence set required to reproduce the model's prediction. We evaluate ToE across six tasks spanning three datasets and two domains: four clinical prediction tasks on MIMIC-IV, cross-center validation on eICU, and non-clinical fault detection on LEMMA-RCA. ToE produces auditable evidence traces while maintaining predictive performance, retaining over 0.98 of full-model AUROC with as few as five evidence units across all settings. Under sparse evidence budgets, ToE achieves higher decision agreement and lower probability fidelity error than other approaches. Qualitative analyses show that ToE adapts its search strategy: it often resolves straightforward cases using only vitals, while selectively incorporating text when physiological signals are ambiguous. ToE therefore provides a practical mechanism for auditing multimodal models by revealing which discrete evidence units support each prediction.

Related papers

• MedBench v5: A Dynamic, Process-Oriented, and Hallucination-Aware Benchmark for Clinical Multimodal Models [9.503660471458295]
  Existing medical AI benchmarks lack process visibility, atomic skill evaluation, and integrated hallucination detection.<n>We introduce MedBench v5, a redesigned benchmark for clinical multimodal models.
  arXiv  Detail & Related papers  (2026-06-23T05:23:46Z)
• Probing, Fusion, and Trustworthiness: A Systematic Evaluation of Foundation Model Representations for Multimodal Cancer Analysis [3.009018931689546]
  Foundation models (FMs) have emerged as powerful representation extractors for medical data, yet their generalizability to datasets under distribution shift remains underexplored.<n>This work systematically evaluates FM-based representations on a suite of computational pathology tasks across two real-world commercial cohorts, IH-BC and IH-NSCLC.<n>We first benchmark unimodal probing performance across five FMs on eight downstream classification tasks, and find that image and omics representations carry complementary predictive signals.
  arXiv  Detail & Related papers  (2026-06-15T09:50:58Z)
• Struct-Searcher: Agentic Structural Thinking Advances Multimodal Deep Information Seeking [57.25141826641528]
  Struct-Searcher is a structural agentic workflow grounded in belief theory.<n>It explicitly maintains an evolving multimodal structural graph throughout the reasoning process.
  arXiv  Detail & Related papers  (2026-06-05T06:25:11Z)
• PRA-PoE: Robust Multimodal Alzheimer's Diagnosis with Arbitrary Missing Modalities [9.014577470296013]
  Missing modalities are prevalent in real-world Alzheimer's disease (AD) assessment.<n>Existing approaches that rely on implicit imputation or modality synthesis often fail to explicitly model modality availability and uncertainty.<n>We propose Prototype-anchored Representation Alignment (PRA) and an Uncertainty-aware Product of Experts (UA-PoE) fusion mechanism.
  arXiv  Detail & Related papers  (2026-05-13T06:54:21Z)
• The Cost of Context: Mitigating Textual Bias in Multimodal Retrieval-Augmented Generation [18.83395586542378]
  Multimodal Large Language Models (MLLMs) are increasingly integrated with Retrieval-Augmented Generation (RAG)<n>We identify and formalize the phenomenon of recorruption, where the introduction of even perfectly accurate "oracle" context causes a capable model to abandon an initially correct prediction.<n>We propose Bottleneck Attention Intervention for Recovery (BAIR), a parameter-free, inference-time framework that restores visual saliency and applies position-aware penalties to textual distractors.
  arXiv  Detail & Related papers  (2026-05-07T02:27:04Z)
• MMEB-V3: Measuring the Performance Gaps of Omni-Modality Embedding Models [70.34265674686516]
  Multimodal embedding models aim to map heterogeneous inputs, such as text, images, videos, and audio, into a shared semantic space.<n>We introduce MMEB-V3, a benchmark that evaluates embeddings across text, image, video, audio, as well as agent-centric scenarios.<n>We conduct a systematic analysis of full-modality embeddings and identify three key findings.
  arXiv  Detail & Related papers  (2026-04-25T14:15:05Z)
• DecepGPT: Schema-Driven Deception Detection with Multicultural Datasets and Robust Multimodal Learning [64.33887406863899]
  Multimodal deception detection aims to identify deceptive behavior by analyzing audiovisual cues for forensics and security.<n>Existing benchmarks provide only binary labels without intermediate reasoning cues.<n>We construct reasoning datasets with structured cue-level descriptions and reasoning chains.<n>With 1695 samples, it is the largest non-laboratory deception detection dataset.
  arXiv  Detail & Related papers  (2026-03-25T04:06:36Z)
• ShortcutBreaker: Low-Rank Noisy Bottleneck with Global Perturbation Attention for Multi-Class Unsupervised Anomaly Detection [59.89803740308262]
  ShortcutBreaker is a novel unified feature-reconstruction framework for MUAD tasks.<n>It features two key innovations to address the issue of shortcuts.<n>The proposed method achieves a remarkable image-level AUROC of 99.8%, 98.9%, 90.6%, and 87.8% on four datasets.
  arXiv  Detail & Related papers  (2025-10-21T06:51:30Z)
• When Does Multimodality Lead to Better Time Series Forecasting? [96.26052272121615]
  We investigate whether and under what conditions such multimodal integration consistently yields gains.<n>Our findings reveal that the benefits of multimodality are highly condition-dependent.<n>Our study offers a rigorous, quantitative foundation for understanding when multimodality can be expected to aid forecasting tasks.
  arXiv  Detail & Related papers  (2025-06-20T23:55:56Z)
• Zero-Shot Event Causality Identification via Multi-source Evidence Fuzzy Aggregation with Large Language Models [11.541829239773643]
  Event Causality Identification (ECI) aims to detect causal relationships between events in textual contexts.<n>Existing ECI models predominantly rely on supervised methodologies, suffering from dependence on large-scale annotated data.<n>We propose MEFA, a novel zero-shot framework based on Multi-source Evidence Fuzzy Aggregation.
  arXiv  Detail & Related papers  (2025-06-06T01:56:05Z)
• A Cooperative Multi-Agent Framework for Zero-Shot Named Entity Recognition [71.61103962200666]
  Zero-shot named entity recognition (NER) aims to develop entity recognition systems from unannotated text corpora.<n>Recent work has adapted large language models (LLMs) for zero-shot NER by crafting specialized prompt templates.<n>We introduce the cooperative multi-agent system (CMAS), a novel framework for zero-shot NER.
  arXiv  Detail & Related papers  (2025-02-25T23:30:43Z)
• A NotSo Simple Way to Beat Simple Bench [0.0]
  This paper presents a novel framework for enhancing reasoning capabilities in large language models (LLMs)<n>We propose a multi-step prompting strategy coupled with global consistency checks to improve model accuracy and robustness.<n>Our results reveal model-specific strengths: Claude excels in maintaining logical consistency, while GPT-4o exhibits exploratory creativity but struggles with ambiguous prompts.
  arXiv  Detail & Related papers  (2024-12-12T16:04:31Z)
• A Reliable Framework for Human-in-the-Loop Anomaly Detection in Time Series [17.08674819906415]
  We introduce HILAD, a novel framework designed to foster a dynamic and bidirectional collaboration between humans and AI.<n>Through our visual interface, HILAD empowers domain experts to detect, interpret, and correct unexpected model behaviors at scale.
  arXiv  Detail & Related papers  (2024-05-06T07:44:07Z)
• Cross-Attention is Not Enough: Incongruity-Aware Dynamic Hierarchical Fusion for Multimodal Affect Recognition [69.32305810128994]
  Incongruity between modalities poses a challenge for multimodal fusion, especially in affect recognition. We propose the Hierarchical Crossmodal Transformer with Dynamic Modality Gating (HCT-DMG), a lightweight incongruity-aware model. HCT-DMG: 1) outperforms previous multimodal models with a reduced size of approximately 0.8M parameters; 2) recognizes hard samples where incongruity makes affect recognition difficult; 3) mitigates the incongruity at the latent level in crossmodal attention.
  arXiv  Detail & Related papers  (2023-05-23T01:24:15Z)
• SUOD: Accelerating Large-Scale Unsupervised Heterogeneous Outlier Detection [63.253850875265115]
  Outlier detection (OD) is a key machine learning (ML) task for identifying abnormal objects from general samples. We propose a modular acceleration system, called SUOD, to address it.
  arXiv  Detail & Related papers  (2020-03-11T00:22:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.