Related papers: JARVIS: An Evidence-Grounded Retrieval System for Interpretable Deceptive Reviews Adjudication

Related papers

Measurement Without Validity: The Compounding Reliability Problem in Agentic AI Evaluation [0.0]
Agentic AI evaluation pipelines produce benchmark scores that justify deployment decisions, safety certifications, and regulatory compliance claims.<n>No formal framework has yet characterized how validity degrades across the stages of these pipelines.<n>We present a three-layer compounding validity model that captures multiplicative degradation across task generation, human-simulator calibration, and automated judgment.
arXiv Detail & Related papers (2026-08-01T17:50:12Z)
Before the Action: Benchmarking LLMs on Prospective Hypothesis Discovery [95.44432473367836]
Large language models (LLMs) excel at answering pre-specified questions, yet their ability to navigate the open-ended, pre-conclusion stage of discovery remains largely unmeasured.<n>We introduce Prospective Hypothesis Discovery (PHD), which asks models to autonomously construct grounded, discnative, and testable hypothesis spaces from inconclusive evidence.<n>We introduce HypoArena, comprising HypoData, a benchmark of 988 cases across six scientific and analytical domains, and HypoEval, an evaluation framework for open-ended hypothesis sets.
arXiv Detail & Related papers (2026-07-17T08:56:43Z)
Skill-Aligned Annotation for Reliable Evaluation in Text-to-Image Generation [50.85337196571056]
Text-to-image (T2I) generation has advanced rapidly, making reliable evaluation critical as performance differences between models narrow.<n>Existing evaluation practices typically apply uniform annotation mechanisms, such as Likert-scale or binary question answering (BQA)<n>We revisit T2I evaluation through the lens of skill-aligned annotation, where annotation strategies reflect the underlying characteristics of each evaluation skill.
arXiv Detail & Related papers (2026-05-13T09:14:31Z)
FRESCO: Benchmarking and Optimizing Re-rankers for Evolving Semantic Conflict in Retrieval-Augmented Generation [73.22935457705057]
We introduce FRESCO, a benchmark for evaluating re-rankers in temporally dynamic contexts.<n>By pairing recency-seeking queries with historical Wikipedia revisions, FRESCO tests whether re-rankers can prioritize factually recent evidence while maintaining semantic relevance.<n>Our evaluation reveals a consistent failure mode across existing re-rankers: a strong bias toward older, semantically rich documents, even when they are factually obsolete.
arXiv Detail & Related papers (2026-04-14T17:04:25Z)
Are we still able to recognize pearls? Machine-driven peer review and the risk to creativity: An explainable RAG-XAI detection framework with markers extraction [7.723181091241251]
We introduce an explainable framework (RAG-XAI) for assessing review quality and detecting automated patterns.<n>The framework achieves near-perfect detection performance, with XGBoost, Random Forest and LightGBM reaching 99.61% accuracy, AUC-ROC above 0.999 and F1-scores of 0.9925 on the test set.
arXiv Detail & Related papers (2026-04-09T08:25:49Z)
Reason and Verify: A Framework for Faithful Retrieval-Augmented Generation [0.0]
We propose a domain-specific RAG framework that integrates explicit rea- soning and faithfulness verification.<n>Our architecture augments standard retrieval with neural query rewriting, BGE-based cross-encoder reranking, and a rationale generation module.<n>We evaluate this framework on the BioASQ and PubMedQA benchmarks, specifically analyzing the impact of dynamic in-context learning.
arXiv Detail & Related papers (2026-03-10T18:25:07Z)
Reviewing the Reviewer: Graph-Enhanced LLMs for E-commerce Appeal Adjudication [0.7136933021609076]
We propose a minimal representation for adjudication reasoning that prevents hallucination through operational grounding.<n>We develop a conflict-aware graph reasoning framework that performs top-down deductive reasoning for new cases.<n>We evaluate the framework in large-scale e-commerce seller appeal adjudication.
arXiv Detail & Related papers (2026-02-27T00:43:59Z)
Towards Faithful Industrial RAG: A Reinforced Co-adaptation Framework for Advertising QA [11.136092421166097]
Industrial advertising question answering (QA) is a high-stakes task in which hallucinated content can lead to financial loss, compliance violations, and legal risk.<n>We propose a reinforced co-adaptation framework that jointly optimize retrieval and generation through two components.<n> Experiments on an internal advertising QA dataset show consistent gains across expert-judged dimensions including accuracy, completeness, and safety.<n>A two-week online A/B test demonstrates a 28.6% increase in like rate, a 46.2% decrease in dislike rate, and a 92.7% reduction in URL hallucination.
arXiv Detail & Related papers (2026-02-26T03:35:09Z)
Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generation [50.87199039334856]
Retrieval-Augmented Generation (RAG) has become a cornerstone of knowledge-intensive applications.<n>Recent studies show that knowledge-extraction attacks can recover sensitive knowledge-base content through maliciously crafted queries.<n>We introduce the first systematic benchmark for knowledge-extraction attacks on RAG systems.
arXiv Detail & Related papers (2026-02-10T01:27:46Z)
Chunking, Retrieval, and Re-ranking: An Empirical Evaluation of RAG Architectures for Policy Document Question Answering [0.0]
The integration of Large Language Models (LLMs) into the public health policy sector offers a transformative approach to navigating the vast repositories of regulatory guidance maintained by agencies such as the Centers for Disease Control and Prevention (CDC)<n>The propensity for LLMs to generate hallucinations, defined as plausible but factually incorrect assertions, presents a critical barrier to the adoption of these technologies in high-stakes environments where information integrity is non-negotiable.<n>This empirical evaluation explores the effectiveness of Retrieval-Augmented Generation (RAG) architectures in mitigating these risks by grounding generative outputs in authoritative document context.
arXiv Detail & Related papers (2026-01-21T20:52:48Z)
From Evidence-Based Medicine to Knowledge Graph: Retrieval-Augmented Generation for Sports Rehabilitation and a Domain Benchmark [12.595335483488052]
In medicine, large language models increasingly rely on retrieval-augmented generation (RAG) to ground outputs in up-to-date external evidence.<n>This study addresses two key gaps: (1) the lack of PICO alignment between queries and retrieved evidence, and (2) the absence of evidence hierarchy considerations during reranking.<n>We present a generalizable strategy for adapting EBM to graph-based RAG, integrating the PICO framework into knowledge graph construction and retrieval.
arXiv Detail & Related papers (2026-01-01T05:20:54Z)
Factuality and Transparency Are All RAG Needs! Self-Explaining Contrastive Evidence Re-ranking [0.2864713389096699]
This extended abstract introduces Self-Explaining Contrastive Evidence Re-Ranking (CER)<n>CER restructures retrieval around factual evidence by fine-tuning embeddings with contrastive learning and generating token-level attribution rationales for each retrieved passage.<n>We evaluated our method on clinical trial reports, and initial experimental results show that CER improves retrieval accuracy, mitigates the potential for hallucinations in RAG systems, and provides transparent, evidence-based retrieval that enhances reliability, especially in safety-critical domains.
arXiv Detail & Related papers (2025-12-04T17:24:35Z)
Towards Real-Time Fake News Detection under Evidence Scarcity [66.58597356379907]
We propose Evaluation-Aware Selection of Experts (EASE), a novel framework for real-time fake news detection.<n>EASE adapts its decision-making process according to the assessed sufficiency of available evidence.<n>We introduce RealTimeNews-25, a new benchmark for evaluating model generalization on emerging news with limited evidence.
arXiv Detail & Related papers (2025-10-13T11:11:46Z)
Demystifying deep search: a holistic evaluation with hint-free multi-hop questions and factorised metrics [89.1999907891494]
We present WebDetective, a benchmark of hint-free multi-hop questions paired with a controlled Wikipedia sandbox.<n>Our evaluation of 25 state-of-the-art models reveals systematic weaknesses across all architectures.<n>We develop an agentic workflow, EvidenceLoop, that explicitly targets the challenges our benchmark identifies.
arXiv Detail & Related papers (2025-10-01T07:59:03Z)
Innovative Deep Learning Architecture for Enhanced Altered Fingerprint Recognition [0.0]
We present DeepAFRNet, a deep learning recognition model that matches and recognizes distorted fingerprint samples.<n>The approach uses a VGG16 backbone to extract high-dimensional features and cosine similarity to compare embeddings.<n>With strict thresholds, DeepAFRNet achieves accuracies of 96.7 percent, 98.76 percent, and 99.54 percent for the three levels.
arXiv Detail & Related papers (2025-09-24T20:12:37Z)
EviNote-RAG: Enhancing RAG Models via Answer-Supportive Evidence Notes [39.61443457073034]
We introduce EviNote-RAG, a framework that follows a retrieve-note-answer workflow.<n>Instead of reasoning directly over raw external information, the model first produces Supportive-Evidence Notes.<n>EviNote-RAG achieves state-of-the-art performance, improving answer accuracy, training stability, robustness, and efficiency.
arXiv Detail & Related papers (2025-08-31T14:44:45Z)
Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimization [58.390885294401066]
Retrieval-augmented generation (RAG) has become a widely adopted paradigm for enabling knowledge-grounded large language models (LLMs)<n>RAG pipelines often fail to ensure that model reasoning remains consistent with the evidence retrieved, leading to factual inconsistencies or unsupported conclusions.<n>We propose AlignRAG, a novel iterative framework grounded in Critique-Driven Alignment (CDA)<n>We introduce AlignRAG-auto, an autonomous variant that dynamically terminates refinement, removing the need to pre-specify the number of critique iterations.
arXiv Detail & Related papers (2025-04-21T04:56:47Z)
TrendFact: A Benchmark for Explainable Hotspot Perception in Fact-Checking with Natural Language Explanation [9.221637941193606]
We introduce TrendFact, the first benchmark capable of evaluating hotspot perception ability (HPA) and all fact-checking tasks.<n> TrendFact consists of 7,643 curated samples sourced from trending platforms and professional fact-checking datasets.<n>We also propose FactISR, a reasoning framework that integrates dynamic evidence augmentation with influence score-based iterative self-reflection.
arXiv Detail & Related papers (2024-10-19T15:25:19Z)
Controlling Risk of Retrieval-augmented Generation: A Counterfactual Prompting Framework [77.45983464131977]
We focus on how likely it is that a RAG model's prediction is incorrect, resulting in uncontrollable risks in real-world applications.<n>Our research identifies two critical latent factors affecting RAG's confidence in its predictions.<n>We develop a counterfactual prompting framework that induces the models to alter these factors and analyzes the effect on their answers.
arXiv Detail & Related papers (2024-09-24T14:52:14Z)
Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Driving [55.93813178692077]
We present RoboBEV, an extensive benchmark suite designed to evaluate the resilience of BEV algorithms.<n>We assess 33 state-of-the-art BEV-based perception models spanning tasks like detection, map segmentation, depth estimation, and occupancy prediction.<n>Our experimental results also underline the efficacy of strategies like pre-training and depth-free BEV transformations in enhancing robustness against out-of-distribution data.
arXiv Detail & Related papers (2024-05-27T17:59:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.