Winning Solution For Meta KDD Cup' 24

• URL: http://arxiv.org/abs/2410.00005v1
• Date: Fri, 13 Sep 2024 06:10:42 GMT
• Title: Winning Solution For Meta KDD Cup' 24
• Authors: Yikuan Xia, Jiazun Chen, Jun Gao,
• Abstract summary: This paper describes the winning solutions of all tasks in Meta KDD Cup 24 from db3 team. The challenge is to build a RAG system from web sources and knowledge graphs. Our solution achieves 1st place on all three tasks, achieving a score of 28.4%, 42.7%, and 47.8%, respectively.
• Score: 6.471894753117029
• License:
• Abstract: This paper describes the winning solutions of all tasks in Meta KDD Cup 24 from db3 team. The challenge is to build a RAG system from web sources and knowledge graphs. We are given multiple sources for each query to help us answer the question. The CRAG challenge involves three tasks: (1) condensing information from web pages into accurate answers, (2) integrating structured data from mock knowledge graphs, and (3) selecting and integrating critical data from extensive web pages and APIs to reflect real-world retrieval challenges. Our solution for Task #1 is a framework of web or open-data retrieval and answering. The large language model (LLM) is tuned for better RAG performance and less hallucination. Task #2 and Task #3 solutions are based on a regularized API set for domain questions and the API generation method using tuned LLM. Our knowledge graph API interface extracts directly relevant information to help LLMs answer correctly. Our solution achieves 1st place on all three tasks, achieving a score of 28.4%, 42.7%, and 47.8%, respectively.

Related papers

• GraphTeam: Facilitating Large Language Model-based Graph Analysis via Multi-Agent Collaboration [46.663380413396226]
  GraphTeam consists of five LLM-based agents from three modules, and the agents with different specialities can collaborate to address complex problems. Experiments on six graph analysis benchmarks demonstrate that GraphTeam achieves state-of-the-art performance with an average 25.85% improvement over the best baseline in terms of accuracy.
  arXiv  Detail & Related papers  (2024-10-23T17:02:59Z)
• Contri(e)ve: Context + Retrieve for Scholarly Question Answering [0.0]
  We present a two step solution using open source Large Language Model(LLM): Llama3.1 for Scholarly-QALD dataset. Firstly, we extract the context pertaining to the question from different structured and unstructured data sources. Secondly, we implement prompt engineering to improve the information retrieval performance of the LLM.
  arXiv  Detail & Related papers  (2024-09-13T17:38:47Z)
• MARAGS: A Multi-Adapter System for Multi-Task Retrieval Augmented Generation Question Answering [0.43512163406552007]
  We present a multi-adapter retrieval augmented generation system (MARAGS) for Meta's Comprehensive RAG (CRAG) competition for KDD CUP 2024. Our system achieved 2nd place for Task 1 as well as 3rd place on Task 2.
  arXiv  Detail & Related papers  (2024-09-05T01:58:29Z)
• EWEK-QA: Enhanced Web and Efficient Knowledge Graph Retrieval for Citation-based Question Answering Systems [103.91826112815384]
  citation-based QA systems are suffering from two shortcomings. They usually rely only on web as a source of extracted knowledge and adding other external knowledge sources can hamper the efficiency of the system. We propose our enhanced web and efficient knowledge graph (KG) retrieval solution (EWEK-QA) to enrich the content of the extracted knowledge fed to the system.
  arXiv  Detail & Related papers  (2024-06-14T19:40:38Z)
• A Solution-based LLM API-using Methodology for Academic Information Seeking [49.096714812902576]
  SoAy is a solution-based LLM API-using methodology for academic information seeking. It uses code with a solution as the reasoning method, where a solution is a pre-constructed API calling sequence. Results show a 34.58-75.99% performance improvement compared to state-of-the-art LLM API-based baselines.
  arXiv  Detail & Related papers  (2024-05-24T02:44:14Z)
• From Local to Global: A Graph RAG Approach to Query-Focused Summarization [3.9676927113698626]
  We propose a Graph RAG approach to question answering over private text corpora. Our approach uses an entity knowledge graph from the source documents, then to pregenerate community summaries for all groups of closely-related entities. For a class of global sensemaking questions over datasets in the 1 million token range, we show that Graph RAG leads to substantial improvements over a na"ive RAG baseline.
  arXiv  Detail & Related papers  (2024-04-24T18:38:11Z)
• API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMs [28.840207102132286]
  We focus on the task of identifying, curating, and transforming existing datasets. We introduce API-BLEND, a large corpora for training and systematic testing of tool-augmented LLMs. We demonstrate the utility of the API-BLEND dataset for both training and benchmarking purposes.
  arXiv  Detail & Related papers  (2024-02-23T18:30:49Z)
• DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text [73.68051228972024]
  Large Language Models (LLMs) have exhibited impressive generation capabilities, but they suffer from hallucinations when relying on their internal knowledge. Retrieval-augmented LLMs have emerged as a potential solution to ground LLMs in external knowledge.
  arXiv  Detail & Related papers  (2023-10-31T04:37:57Z)
• AVIS: Autonomous Visual Information Seeking with Large Language Model Agent [123.75169211547149]
  We propose an autonomous information seeking visual question answering framework, AVIS. Our method leverages a Large Language Model (LLM) to dynamically strategize the utilization of external tools. AVIS achieves state-of-the-art results on knowledge-intensive visual question answering benchmarks such as Infoseek and OK-VQA.
  arXiv  Detail & Related papers  (2023-06-13T20:50:22Z)
• Learning to Learn from APIs: Black-Box Data-Free Meta-Learning [95.41441357931397]
  Data-free meta-learning (DFML) aims to enable efficient learning of new tasks by meta-learning from a collection of pre-trained models without access to the training data. Existing DFML work can only meta-learn from (i) white-box and (ii) small-scale pre-trained models. We propose a Bi-level Data-free Meta Knowledge Distillation (BiDf-MKD) framework to transfer more general meta knowledge from a collection of black-box APIs to one single model.
  arXiv  Detail & Related papers  (2023-05-28T18:00:12Z)
• IIRC: A Dataset of Incomplete Information Reading Comprehension Questions [53.3193258414806]
  We present a dataset, IIRC, with more than 13K questions over paragraphs from English Wikipedia. The questions were written by crowd workers who did not have access to any of the linked documents. We follow recent modeling work on various reading comprehension datasets to construct a baseline model for this dataset.
  arXiv  Detail & Related papers  (2020-11-13T20:59:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.