Cost-Pragmatic Quality Gating and Selection-Fusion Multi-Model Combiners for BioASQ Phases A+ and B

• URL: http://arxiv.org/abs/2607.13551v1
• Date: Wed, 15 Jul 2026 07:56:33 GMT
• Title: Cost-Pragmatic Quality Gating and Selection-Fusion Multi-Model Combiners for BioASQ Phases A+ and B
• Abstract summary: Work centers on two design decisions: how aggressively to re-retrieve when first-stage retrieval is weak, and how to combine multiple language-model answers.<n>On Task 12B 2024 validation, a cost-pragmatic re-retrieval policy beats a skill-strict baseline significantly on list F1 and list precision, at 12% lower re-retrieval cost.<n>For Phase B answering we decompose multi-model ensemble lift into a selection component bounded by the per-question oracle and a fusion component that aggregators can exceed.
• Score: 1.348563827917271
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We describe our BioASQ Task 14B 2026 system. The work centers on two design decisions: how aggressively to re-retrieve when first-stage retrieval is weak, and how to combine multiple language-model answers. Retrieval unions two parallel pipelines - a hybrid first stage (dense BGE + BM25 + RRF, reaching R@200 = 99.3% on the BioASQ-13b historical archive) and an agent-driven pipeline that decomposes the question over PubMed, Europe PMC, and iCite - with a BGE cross-encoder quality gate flagging weakly-supported questions for selective re-retrieval. On Task 12B 2024 validation, a cost-pragmatic re-retrieval policy beats a skill-strict baseline significantly on list F1 and list precision, at 12% lower re-retrieval cost. Holding prompt and model fixed across val and test 13B (different question sets), list F1 rises by +0.132 absolute on the BioASQ-released gold-input pool, consistent with substantial retrieval-side headroom. For Phase B answering we decompose multi-model ensemble lift into a selection component bounded by the per-question oracle and a fusion component that aggregators can exceed. The decomposition predicts before any experiment that LLM-as-judge wins on selection-dominated metrics (yes/no, multi-reference ROUGE) but is structurally insufficient on the recall component of fusion-friendly metrics (factoid rank-1, list recall). On Task 13B 2025 our synonym-union resolver wins list recall on every head, while GPT-5.5 solo retains the list-F1 lead because the resolver's wider item set costs precision. On the Task 14B 2026 preliminary leaderboard our team places first on the combined-exact aggregate on three of the eight (phase x batch) leaderboards, wins four individual question-type cells, and takes #1 on Phase B b3 ideal.

Related papers

• IGT @ FinMMEval 2026 Task 2: Question-Type Prompting with Targeted Extraction for Multilingual Financial QA [0.0]
  PolyFiQA Task 2 was a multilingual financial question answering task over English SEC filings and multilingual news articles.<n>Our central observation is that the 344 development questions divide into two families requiring fundamentally different approaches.<n>A dataset analysis reveals that 17-18 of 19 ground-truth reference answers per synthesis type share an exact evidence label prefix.<n>The final system achieves development ROUGE-1 approximately 0.395, a 60% relative improvement over a generic RAG baseline.
  arXiv  Detail & Related papers  (2026-09-08T02:23:31Z)
• Retrieval-Augmented Multi-Prompt Ensemble for Minor-Grain Breeding Information Extraction [45.40972888157319]
  RAME (Retrieval-Augmented Multi-Prompt Ensemble) is a training-free framework that elicits multiple LLM outputs under controlled diversity.<n> RAME achieves a Total Score of 0.499 (NER 0.730, RE 0.346) on the leaderboard, ranking 1st and surpassing the official Track-A baseline powered by GPT-5.5 (0.448)
  arXiv  Detail & Related papers  (2026-09-07T07:30:49Z)
• BIT.UA at BioASQ 14B: Modular Retrieval with pg_textsearch and Qdrant, and Agent-Based Answer Generation [37.19059223604783]
  This paper describes the participation of the BIT.UA team in the 14th edition of the BioASQ Task B challenge on biomedical question answering.<n>We introduce a substantiallyed and modular approach to both the retrieval and generation components of the pipeline.<n>We discuss the impact of these architectural changes, lessons learned, and outline directions for future work including SPLADE and ColBERT integration.
  arXiv  Detail & Related papers  (2026-09-04T11:10:07Z)
• Overview of FinMMEval 2026 Task 2: Multilingual Financial Short-Answer Question Answering [59.6753725406784]
  FinMMEval 2026 Task 2 evaluates short-answer financial question answering over multilingual evidence.<n>The final-test set contains 256 items, split evenly between easy and expert tiers.<n>The strongest systems are closely clustered, with the top four separated by less than one percentage point in ROUGE-1 F1.
  arXiv  Detail & Related papers  (2026-07-22T07:54:17Z)
• Analyzing Quality-Latency-Resource Trade-offs in a Technical Documentation RAG Assistant Using LoRA Adaptation [0.0]
  We study quality-latency-resource trade-offs in a retrieval-augmented generation (RAG) system that uses Low-Rank Adaptation (LoRA) of the generator.<n>We build a benchmark of 5,144 question-answer pairs over the official documentation and combine it with a fixed hybrid-retrieval pipeline (BGE-M3 dense, BGE-M3 native, Reciprocal Rank Fusion, cross-encoder reranking)
  arXiv  Detail & Related papers  (2026-05-27T09:37:55Z)
• GridProbe: Posterior-Probing for Adaptive Test-Time Compute in Long-Video VLMs [3.9266376632068485]
  GridProbe is an efficient training-free posterior-probing inference paradigm.<n>It scores evidence in answer space using a frozen VLM's own reasoning.<n>It selects question-relevant frames adaptively, resulting in sub-quadratic attention cost with little to no accuracy loss.
  arXiv  Detail & Related papers  (2026-05-11T15:57:46Z)
• QU-NLP at ArchEHR-QA 2026: Two-Stage QLoRA Fine-Tuning of Qwen3-4B for Patient-Oriented Clinical Question Answering and Evidence Sentence Alignment [1.0152838128195467]
  We present a unified system addressing both Subtask 3 (answer generation) and Subtask 4 (evidence sentence alignment) of the ArchEHR-QA Shared Task.<n>For Subtask 3, we apply two-stage Quantised Low-Rank Adaptation (QLoRA) to Qwen3-4B loaded in 4-bit NF4 quantisation.<n>For Subtask 4, we develop a weighted ensemble of three retrieval methods - BM25 with relative thresholding, TF-IDF cosine similarity, and a fine-tuned cross-encoder.
  arXiv  Detail & Related papers  (2026-03-26T11:24:13Z)
• Test-Time Strategies for More Efficient and Accurate Agentic RAG [58.44913384057518]
  Retrieval-Augmented Generation (RAG) systems face challenges with complex, multihop questions.<n>Such approaches can introduce inefficiencies, including repetitive retrieval of previously processed information.<n>We investigate test-time modifications to the Search-R1 pipeline to mitigate these identified shortcomings.
  arXiv  Detail & Related papers  (2026-03-12T19:18:59Z)
• AILS-NTUA at SemEval-2026 Task 8: Evaluating Multi-Turn RAG Conversations [14.20669481486209]
  We present the AILS-NTUA system for SemEval-2026 Task 8 (MTRAGEval)<n>We address all three subtasks of multi-turn retrieval-augmented generation: passage retrieval (A), reference-grounded response generation (B), and end-to-end RAG (C)<n>Our unified architecture is built on two principles: (i) a query-diversity-over-retriever-diversity strategy, and (ii) a multistage generation pipeline.
  arXiv  Detail & Related papers  (2026-03-11T08:28:31Z)
• HLTCOE Evaluation Team at TREC 2025: VQA Track [76.85337417923331]
  HLT Evaluation team participated in TREC VQA's Answer Generation (AG) task.<n>We developed a listwise learning framework that aims to improve semantic precision and ranking consistency in answer generation.
  arXiv  Detail & Related papers  (2025-12-08T17:25:13Z)
• RefineBench: Evaluating Refinement Capability of Language Models via Checklists [71.02281792867531]
  We evaluate two refinement modes: guided refinement and self-refinement.<n>In guided refinement, both proprietary LMs and large open-weight LMs can leverage targeted feedback to refine responses to near-perfect levels within five turns.<n>These findings suggest that frontier LMs require breakthroughs to self-refine their incorrect responses.
  arXiv  Detail & Related papers  (2025-11-27T07:20:52Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• Question Decomposition for Retrieval-Augmented Generation [2.6409776648054764]
  We propose a RAG pipeline that incorporates question decomposition into sub-questions.<n>We show that question decomposition effectively assembles complementary documents, while reranking reduces noise.<n>Although reranking itself is standard, we show that pairing an off-the-shelf cross-encoder reranker with LLM-driven question decomposition bridges the retrieval gap on multi-hop questions.
  arXiv  Detail & Related papers  (2025-07-01T01:01:54Z)
• TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance [71.76018597965378]
  We build a new large-scale Question Answering dataset containing both Tabular And Textual data, named TAT-QA. We propose a novel QA model termed TAGOP, which is capable of reasoning over both tables and text.
  arXiv  Detail & Related papers  (2021-05-17T06:12:06Z)
• Pre-training Tasks for Embedding-based Large-scale Retrieval [68.01167604281578]
  We consider the large-scale query-document retrieval problem. Given a query (e.g., a question), return the set of relevant documents from a large document corpus. We show that the key ingredient of learning a strong embedding-based Transformer model is the set of pre-training tasks.
  arXiv  Detail & Related papers  (2020-02-10T16:44:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.