DSEBench: A Test Collection for Explainable Dataset Search with Examples

• URL: http://arxiv.org/abs/2510.17228v1
• Date: Mon, 20 Oct 2025 07:19:47 GMT
• Title: DSEBench: A Test Collection for Explainable Dataset Search with Examples
• Authors: Qing Shi, Jing He, Qiaosheng Chen, Gong Cheng,
• Abstract summary: We develop a test collection that provides high-quality dataset- and field-level annotations to enable the evaluation of explainable DSE.<n>We establish extensive baselines on DSEBench by adapting and evaluating a variety of sparse, dense, and LLM-based retrieval, reranking, and explanation methods.
• Score: 14.572501479346442
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dataset search has been an established information retrieval task. Current paradigms either retrieve datasets that are relevant to a keyword query or find datasets that are similar to an input target dataset. To allow for their combined specification of information needs, in this article, we investigate the more generalized task of Dataset Search with Examples (DSE) and further extend it to Explainable DSE that requires identifying the metadata and content fields of a dataset that indicate its relevance to the query and similarity to the target datasets. To facilitate this research, we construct DSEBench, a test collection that provides high-quality dataset- and field-level annotations to enable the evaluation of explainable DSE. We also employ a large language model to generate numerous annotations to be used for training. We establish extensive baselines on DSEBench by adapting and evaluating a variety of sparse, dense, and LLM-based retrieval, reranking, and explanation methods.

Related papers

• Keywords are not always the key: A metadata field analysis for natural language search on open data portals [3.974422712382188]
  We examine how individual metadata fields affect the success of conversational dataset retrieval.<n>We compare existing content of the metadata field 'description' with LLM-generated content.<n>Our findings suggest that dataset descriptions play a central role in aligning with user intent.
  arXiv  Detail & Related papers  (2025-09-17T22:14:27Z)
• A Survey on Open Dataset Search in the LLM Era: Retrospectives and Perspectives [13.669798235894064]
  We focus on advances in open dataset search beyond traditional approaches that rely on metadata and keywords.<n>LLMs help address complex challenges in query understanding, semantic modeling, and interactive guidance within open dataset search.<n>This work aims to offer a structured reference for researchers and practitioners in the field of open dataset search.
  arXiv  Detail & Related papers  (2025-08-31T07:45:40Z)
• TARGET: Benchmarking Table Retrieval for Generative Tasks [7.379012456053551]
  TARGET is a benchmark for evaluating TAble Retrieval for GEnerative Tasks.<n>We analyze the retrieval performance of different retrievers in isolation, as well as their impact on downstream tasks.<n>We find that dense embedding-based retrievers far outperform a BM25 baseline which is less effective than it is for retrieval over unstructured text.
  arXiv  Detail & Related papers  (2025-05-14T19:39:46Z)
• Generative Retrieval for Book search [106.67655212825025]
  We propose an effective Generative retrieval framework for Book Search.<n>It features two main components: data augmentation and outline-oriented book encoding.<n>Experiments on a proprietary Baidu dataset demonstrate that GBS outperforms strong baselines.
  arXiv  Detail & Related papers  (2025-01-19T12:57:13Z)
• Metadata-based Data Exploration with Retrieval-Augmented Generation for Large Language Models [3.7685718201378746]
  This research introduces a new architecture for data exploration which employs a form of Retrieval-Augmented Generation (RAG) to enhance metadata-based data discovery. The proposed framework offers a new method for evaluating semantic similarity among heterogeneous data sources.
  arXiv  Detail & Related papers  (2024-10-05T17:11:37Z)
• BabelBench: An Omni Benchmark for Code-Driven Analysis of Multimodal and Multistructured Data [61.936320820180875]
  Large language models (LLMs) have become increasingly pivotal across various domains. BabelBench is an innovative benchmark framework that evaluates the proficiency of LLMs in managing multimodal multistructured data with code execution. Our experimental findings on BabelBench indicate that even cutting-edge models like ChatGPT 4 exhibit substantial room for improvement.
  arXiv  Detail & Related papers  (2024-10-01T15:11:24Z)
• DiscoveryBench: Towards Data-Driven Discovery with Large Language Models [50.36636396660163]
  We present DiscoveryBench, the first comprehensive benchmark that formalizes the multi-step process of data-driven discovery. Our benchmark contains 264 tasks collected across 6 diverse domains, such as sociology and engineering. Our benchmark, thus, illustrates the challenges in autonomous data-driven discovery and serves as a valuable resource for the community to make progress.
  arXiv  Detail & Related papers  (2024-07-01T18:58:22Z)
• UQE: A Query Engine for Unstructured Databases [71.49289088592842]
  We investigate the potential of Large Language Models to enable unstructured data analytics. We propose a new Universal Query Engine (UQE) that directly interrogates and draws insights from unstructured data collections.
  arXiv  Detail & Related papers  (2024-06-23T06:58:55Z)
• DataFinder: Scientific Dataset Recommendation from Natural Language Descriptions [100.52917027038369]
  We operationalize the task of recommending datasets given a short natural language description. To facilitate this task, we build the DataFinder dataset which consists of a larger automatically-constructed training set and a smaller expert-annotated evaluation set. This system, trained on the DataFinder dataset, finds more relevant search results than existing third-party dataset search engines.
  arXiv  Detail & Related papers  (2023-05-26T05:22:36Z)
• Detection Hub: Unifying Object Detection Datasets via Query Adaptation on Language Embedding [137.3719377780593]
  A new design (named Detection Hub) is dataset-aware and category-aligned. It mitigates the dataset inconsistency and provides coherent guidance for the detector to learn across multiple datasets. The categories across datasets are semantically aligned into a unified space by replacing one-hot category representations with word embedding.
  arXiv  Detail & Related papers  (2022-06-07T17:59:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.