5ting at SemEval-2026 Task 8: Strong End-to-End Multi-Turn RAG via LLM-Based Reranking and Faithfulness Control

• URL: http://arxiv.org/abs/2606.28737v1
• Date: Sat, 27 Jun 2026 05:13:49 GMT
• Title: 5ting at SemEval-2026 Task 8: Strong End-to-End Multi-Turn RAG via LLM-Based Reranking and Faithfulness Control
• Abstract summary: We introduce 5ting, our system for the SemEval2026 Task 8 (MTRAGEval)<n>We evaluate multi-turn Retrieval Augmented Generation (RAG) systems.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We introduce 5ting, our system for the SemEval2026 Task 8 (MTRAGEval), which evaluates multi-turn Retrieval Augmented Generation (RAG) systems. Multi turn RAG involves context drift, under specification, and hallucination risk. Our system combines BGE-M3 dense retrieval with FAISS indexing, dual-query merged retrieval, and LLM based reranking, followed by role separated generation constrained to retrieved evidence. The retriever achieved nDCG@5 = 0.4719 in Task A, while the end to end system ranked in Task C with a harmonic score of 0.5597 and RL_F = 0.7692.

Related papers

• A-SR: Self-Evolving Agentic LLMs for Symbolic Regression via Hierarchical Coordination [51.06539604709775]
  We propose A-SR, a self-evolving agentic framework that shifts the control unit from expression edits to role-conditioned evidence views.<n>A-SR coordinates formula discovery through routing among coordination protocols, an online evaluator-reward role policy, and state-routed process memory.
  arXiv  Detail & Related papers  (2026-08-05T14:01:10Z)
• Sifei at SemEval-2026 Task 8: Hybrid Retrieval and Query Rewriting for Multi-Turn RAG [2.9721401411878254]
  Multi-turn retrieval-augmented generation (RAG) is challenging due to evolving user intent, conversational noise, and strict context limits.<n>We propose a training-free hybrid retrieval pipeline for SemEval-2026 Task 8 that combines dense and sparse retrieval with controlled query rewriting and cross-encoder reranking.
  arXiv  Detail & Related papers  (2026-06-05T07:28:01Z)
• What and When to Distill: Selective Hindsight Distillation for Multi-Turn Agents [70.6980022118038]
  Reinforcement learning can train LLM agents from sparse task rewards, but long-horizon credit assignment remains challenging.<n>Existing methods rely on trajectory-level rewards or proxy signals, without fully leveraging per-step environmental feedback.<n>We introduce SERL, a selective environment-reweighted learning framework.
  arXiv  Detail & Related papers  (2026-05-19T07:00:55Z)
• MARQUIS: A Three-Stage Pipeline for Video Retrieval-Augmented Generation [64.33427079838602]
  Retrieval-augmented generation from videos requires systems to retrieve relevant audiovisual evidence from large corpora and synthesize it into coherent, attributed text.<n>We present MARQUIS: a three-stage pipeline that addresses these limitations through query expansion, fusion, and reranking.<n>For article generation, ITER-QA-BASE improves average human score from 3.09 to 3.83 over the CAG baseline, while MARQUIS-RLM achieves a human score of 3.30 and the strongest citation recall among non-QA systems.
  arXiv  Detail & Related papers  (2026-05-17T20:19:04Z)
• Caraman at SemEval-2026 Task 8: Three-Stage Multi-Turn Retrieval with Query Rewriting, Hybrid Search, and Cross-Encoder Reranking [0.0]
  We describe our system for SemEval-2026 Task 8 (MTRAGEval), participating in Task A (Retrieval) across four English-language domains.<n>Our approach employs a three-stage pipeline: (1) query rewriting via a LoRA-fine-tuned Qwen 2.5 7B model that transforms context-dependent follow-up questions into standalone queries, (2) hybrid BM25 and dense retrieval combined through Reciprocal Rank Fusion, and (3) cross-encoder reranking with BGE-reranker-v2-m3.
  arXiv  Detail & Related papers  (2026-05-12T12:13:45Z)
• H-RAG at SemEval-2026 Task 8: Hierarchical Parent-Child Retrieval for Multi-Turn RAG Conversations [6.221953137049602]
  H-RAG addresses both Task A (Retrieval) and Task C (Generation with Retrieved Passages) in SemEval-2026 Task 8 (MTRAGEval)<n>Our approach implements a hierarchical parent-child RAG pipeline that separates fine-grained child-level retrieval from parent-level context reconstruction.<n>H-RAG achieves an nDCG@5 score of 0.4271 on Task A and a harmonic mean score of 0.3241 on Task C.
  arXiv  Detail & Related papers  (2026-05-01T13:07:19Z)
• AILS-NTUA at SemEval-2026 Task 8: Evaluating Multi-Turn RAG Conversations [14.20669481486209]
  We present the AILS-NTUA system for SemEval-2026 Task 8 (MTRAGEval)<n>We address all three subtasks of multi-turn retrieval-augmented generation: passage retrieval (A), reference-grounded response generation (B), and end-to-end RAG (C)<n>Our unified architecture is built on two principles: (i) a query-diversity-over-retriever-diversity strategy, and (ii) a multistage generation pipeline.
  arXiv  Detail & Related papers  (2026-03-11T08:28:31Z)
• Beyond Relevance: On the Relationship Between Retrieval and RAG Information Coverage [89.58253972744531]
  Retrieval-augmented generation (RAG) systems combine document retrieval with a generative model to address complex information seeking tasks like report generation.<n>We investigate whether upstream retrieval metrics can serve as reliable early indicators of the final generated response's information coverage.<n>Our findings demonstrate strong correlations between coverage-based retrieval metrics and nugget coverage in generated responses at both topic and system levels.
  arXiv  Detail & Related papers  (2026-03-09T18:20:20Z)
• DS@GT at TREC TOT 2025: Bridging Vague Recollection with Fusion Retrieval and Learned Reranking [0.5352699766206809]
  We develop a two-stage retrieval system to address the TREC Tip-of-the-Tongue (ToT) task.<n>In the first stage, we employ hybrid retrieval that merges LLM-based retrieval, sparse (BM25), and dense (BGE-M3) retrieval methods.<n>We also introduce topic-aware multi-index dense retrieval that partitions the Wikipedia corpus into 24 topical domains.
  arXiv  Detail & Related papers  (2026-01-21T23:09:17Z)
• Q-RAG: Long Context Multi-step Retrieval via Value-based Embedder Training [50.37345200692884]
  We propose Q-RAG, a novel approach that fine-tunes Embedder model for multi-step retrieval using reinforcement learning (RL)<n>Q-RAG offers a competitive, resource-efficient alternative to existing multi-step retrieval methods for open-domain question answering.
  arXiv  Detail & Related papers  (2025-11-10T17:31:02Z)
• MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems [21.554151870748104]
  We present MTRAG: an end-to-end human-generated multi-turn RAG benchmark.<n>It reflects several real-world properties across diverse dimensions for evaluating the full RAG pipeline.<n>Our evaluations show that even state-of-the-art LLM RAG systems struggle on MTRAG.
  arXiv  Detail & Related papers  (2025-01-07T01:52:56Z)
• Overcoming Catastrophic Forgetting with Gaussian Mixture Replay [79.0660895390689]
  We present a rehearsal-based approach for continual learning (CL) based on Gaussian Mixture Models (GMM) We mitigate catastrophic forgetting (CF) by generating samples from previous tasks and merging them with current training data. We evaluate GMR on multiple image datasets, which are divided into class-disjoint sub-tasks.
  arXiv  Detail & Related papers  (2021-04-19T11:41:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.