SIGIR 2025 -- LiveRAG Challenge Report

• URL: http://arxiv.org/abs/2507.04942v2
• Date: Tue, 08 Jul 2025 06:37:05 GMT
• Title: SIGIR 2025 -- LiveRAG Challenge Report
• Authors: David Carmel, Simone Filice, Guy Horowitz, Yoelle Maarek, Oren Somekh, Ran Tavory, Mehdi Ghissassi, Edo Liberty, Roy Miara,
• Abstract summary: The LiveRAG Challenge at SIGIR 2025 provided a competitive platform for advancing Retrieval-Augmented Generation technologies.<n>70 teams from 27 different countries provided answers and supportive information to 500 unseen questions within a strict two-hour time window.<n>The finalists were announced on June 12, 2025, with prizes awarded during the LiveRAG Workshop at SIGIR 2025 in Padua, Italy.
• Score: 10.33475098016246
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The LiveRAG Challenge at SIGIR 2025, held between March and May 2025, provided a competitive platform for advancing Retrieval-Augmented Generation (RAG) technologies. Participants from academia and industry were invited to develop a RAG-based question-answering system using a fixed corpus (Fineweb-10BT) and a common open-source LLM (Falcon3-10B-Instruct). The goal was to facilitate challenging comparisons of retrieval and prompting strategies. During the Live Challenge Day, 70 teams from 27 different countries provided answers and supportive information to 500 unseen questions within a strict two-hour time window. Evaluation was conducted in two stages: first an automated LLM-as-a-judge approach was used to compute correctness and faithfulness score, then a manual review of top ranked submissions was conducted. The finalists were announced on June 12, 2025, with prizes awarded during the LiveRAG Workshop at SIGIR 2025 in Padua, Italy.

Related papers

• Benchmarking Generalizable Bimanual Manipulation: RoboTwin Dual-Arm Collaboration Challenge at CVPR 2025 MEIS Workshop [120.2806035123366]
  RoboTwin Dual-Arm Collaboration Challenge was held at the 2nd MEIS Workshop, CVPR 2025.<n>Competitors totally tackled 17 dual-arm manipulation tasks, covering rigid, deformable, and tactile-based scenarios.<n>Report outlines the competition setup, task design, evaluation methodology, key findings and future direction.
  arXiv  Detail & Related papers  (2025-06-29T17:56:41Z)
• Knowledge-Aware Diverse Reranking for Cross-Source Question Answering [9.788039182463768]
  This paper presents Team Marikarp's solution for the SIGIR 2025 LiveRAG competition.<n>The competition's evaluation set, automatically generated by DataMorgana from internet corpora, encompassed a wide range of target topics.<n>Our proposed knowledge-aware diverse reranking RAG pipeline achieved first place in the competition.
  arXiv  Detail & Related papers  (2025-06-25T14:23:21Z)
• RMIT-ADM+S at the SIGIR 2025 LiveRAG Challenge [4.364909807482374]
  This paper presents the RMIT--ADM+S participation in the SIGIR 2025 LiveRAG Challenge.<n>Our Generation-Retrieval-Augmented Generation (GRAG) approach relies on generating a hypothetical answer that is used in the retrieval phase, alongside the original question.<n>We describe the system architecture and the rationale behind our design choices.
  arXiv  Detail & Related papers  (2025-06-17T13:41:12Z)
• NTIRE 2025 Challenge on UGC Video Enhancement: Methods and Results [73.23764765210825]
  This paper presents an overview of the NTIRE 2025 Challenge on Video Enhancement.<n>The challenge constructed a set of 150 user-generated content videos without reference ground truth.<n>The goal of the participants was to develop an algorithm capable of improving the visual quality of such videos.
  arXiv  Detail & Related papers  (2025-05-05T20:06:11Z)
• NTIRE 2025 Challenge on Image Super-Resolution ($\times$4): Methods and Results [159.15538432295656]
  The NTIRE 2025 image super-resolution ($times$4) challenge is one of the associated competitions of the 10th NTIRE Workshop at CVPR 2025.<n>The challenge aims to recover high-resolution (HR) images from low-resolution (LR) counterparts generated through bicubic downsampling with a $times$4 scaling factor.<n>A total of 286 participants registered for the competition, with 25 teams submitting valid entries.
  arXiv  Detail & Related papers  (2025-04-20T12:08:22Z)
• [Call for Papers] The 2nd BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus [81.34965784440176]
  This CfP provides rules for the BabyLM Challenge 2024-2025. The overarching goals of the challenge remain the same. We replace the loose track with a paper track. We relax the rules around pretraining data. We introduce a multimodal vision-and-language track.
  arXiv  Detail & Related papers  (2024-04-09T11:04:50Z)
• The NeurIPS 2022 Neural MMO Challenge: A Massively Multiagent Competition with Specialization and Trade [41.639843908635875]
  The NeurIPS-2022 Neural MMO Challenge attracted 500 participants and received over 1,600 submissions. This year's competition runs on the latest v1.6 Neural MMO, which introduces new equipment, combat, trading, and a better scoring system. This paper summarizes the design and results of the challenge, explores the potential of this environment as a benchmark for learning methods, and presents some practical reinforcement learning approaches for complex tasks with sparse rewards.
  arXiv  Detail & Related papers  (2023-11-07T04:14:45Z)
• ICDAR 2023 Competition on Hierarchical Text Detection and Recognition [60.68100769639923]
  The competition is aimed to promote research into deep learning models and systems that can jointly perform text detection and recognition. We present details of the proposed competition organization, including tasks, datasets, evaluations, and schedule. During the competition period (from January 2nd 2023 to April 1st 2023), at least 50 submissions from more than 20 teams were made in the 2 proposed tasks.
  arXiv  Detail & Related papers  (2023-05-16T18:56:12Z)
• RangL: A Reinforcement Learning Competition Platform [82.1944886411643]
  RangL aims to encourage the wider uptake of reinforcement learning by supporting competitions relating to real-world dynamic decision problems. This article describes the reusable code repository developed by the RangL team and deployed for the 2022 Pathways to Net Zero Challenge. The winning solutions to this particular Challenge seek to optimize the UK's energy transition policy to net zero carbon emissions by 2050.
  arXiv  Detail & Related papers  (2022-07-28T09:44:21Z)
• 2nd Place Solution to ECCV 2020 VIPriors Object Detection Challenge [24.368684444351068]
  We show that by using state-of-the-art data augmentation strategies, model designs, and post-processing ensemble methods, it is possible to overcome the difficulty of data shortage and obtain competitive results. Our overall detection system achieves 36.6$%$ AP on the COCO 2017 validation set using only 10K training images without any pre-training or transfer learning weights ranking us 2nd place in the challenge.
  arXiv  Detail & Related papers  (2020-07-17T09:21:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.