The Effect of Text Chunk Size on Retrieval-Augmented Generation Performance

• URL: http://arxiv.org/abs/2607.24767v1
• Date: Mon, 08 Jun 2026 13:55:14 GMT
• Title: The Effect of Text Chunk Size on Retrieval-Augmented Generation Performance
• Abstract summary: This paper evaluates how chunk size, along with the number of retrieved segments, influences generation quality and retrieval effectiveness.<n>By comparing these configurations, this study seeks to better understand how document segmentation affects the performance and efficiency of Retrieval-Augmented Generation systems.
• Score: 0.0
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Retrieval-Augmented Generation (RAG) systems have emerged as a powerful process for allowing large language models (LLMs) to retrieve relevant information to use as source material during text generation. A critical yet under-explored component of these systems is the granularity at which source documents are segmented into retrievable chunks. The size of these chunks has the potential to significantly influence generation quality, contextual correctness, retrieval precision, and computational efficiency. Despite its importance, chunk size is often selected without proper evaluation of its impact on generation quality. Smaller chunks, such as individual sentences, may allow for precise retrieval by narrowing the focus of each chunk. However, they contain less information, which may limit the model's ability to generate coherent responses. Larger chunks, such as entire chapters, contain lots of broad information that may improve correctness, but also introduce additional noise and increase computational cost. Because larger chunks contain more information, the number of chunks returned to the model must also be considered. This paper evaluates how chunk size, along with the number of retrieved segments, influences generation quality and retrieval effectiveness. By comparing these configurations, this study seeks to better understand how document segmentation affects the performance and efficiency of Retrieval-Augmented Generation systems. segmentation affects the performance and efficiency of Retrieval-Augmented Generation systems.

Related papers

• Evaluating Chunking Strategies for Retrieval-Augmented Generation on Academic Texts [0.38597698994416435]
  Cluster-based semantic chunking improves retrieval and answer quality compared to fixed-size and recursion.<n>RAGAs based faithfulness shows limited reliability in this setup.
  arXiv  Detail & Related papers  (2026-07-02T08:12:35Z)
• Information Capacity: Evaluating the Efficiency of Large Language Models via Text Compression [53.39128997308138]
  We introduce information capacity, a measure of model efficiency based on text compression performance.<n> Empirical evaluations on mainstream open-source models show that models of varying sizes within a series exhibit consistent information capacity.<n>A distinctive feature of information capacity is that it incorporates tokenizer efficiency, which affects both input and output token counts.
  arXiv  Detail & Related papers  (2025-11-11T10:07:32Z)
• SitEmb-v1.5: Improved Context-Aware Dense Retrieval for Semantic Association and Long Story Comprehension [77.93156509994994]
  We show how to represent short chunks in a way that is conditioned on a broader context window to enhance retrieval performance.<n>Existing embedding models are not well-equipped to encode such situated context effectively.<n>Our method substantially outperforms state-of-the-art embedding models.
  arXiv  Detail & Related papers  (2025-08-03T23:59:31Z)
• Rethinking Chunk Size For Long-Document Retrieval: A Multi-Dataset Analysis [0.0]
  We evaluate fixed-size chunking strategies and their influence on retrieval performance using multiple embedding models.<n>Our experiments, conducted on both short-form and long-form datasets, reveal that chunk size plays a critical role in retrieval effectiveness.
  arXiv  Detail & Related papers  (2025-05-27T19:39:16Z)
• Quantifying Memory Utilization with Effective State-Size [73.52115209375343]
  We develop a measure of textitmemory utilization'<n>This metric is tailored to the fundamental class of systems with textitinput-invariant and textitinput-varying linear operators
  arXiv  Detail & Related papers  (2025-04-28T08:12:30Z)
• Exploring Training and Inference Scaling Laws in Generative Retrieval [50.82554729023865]
  Generative retrieval reformulates retrieval as an autoregressive generation task, where large language models generate target documents directly from a query.<n>We systematically investigate training and inference scaling laws in generative retrieval, exploring how model size, training data scale, and inference-time compute jointly influence performance.
  arXiv  Detail & Related papers  (2025-03-24T17:59:03Z)
• Improving Factuality with Explicit Working Memory [68.39261790277615]
  Large language models can generate factually inaccurate content, a problem known as hallucination.<n>We introduce EWE (Explicit Working Memory), a novel approach that enhances factuality in long-form text generation by integrating a working memory that receives real-time feedback from external resources.
  arXiv  Detail & Related papers  (2024-12-24T00:55:59Z)
• Is Semantic Chunking Worth the Computational Cost? [0.0]
  This study systematically evaluates the effectiveness of semantic chunking using three common retrieval-related tasks. The results show that the computational costs associated with semantic chunking are not justified by consistent performance gains.
  arXiv  Detail & Related papers  (2024-10-16T21:53:48Z)
• Introducing a new hyper-parameter for RAG: Context Window Utilization [0.0]
  RAG systems enhance generative models by incorporating relevant information retrieved from external knowledge bases. The size of the text chunks retrieved and processed is a critical factor influencing RAG performance. This study aims to identify the optimal chunk size that maximizes answer generation quality.
  arXiv  Detail & Related papers  (2024-07-29T08:38:14Z)
• Financial Report Chunking for Effective Retrieval Augmented Generation [2.6680382112425374]
  Chunking information is a key step in Retrieval Augmented Generation (RAG) Current research primarily centers on paragraph-level chunking. We propose an expanded approach to chunk documents by moving beyond mere paragraph-level chunking to chunk primary by structural element components of documents.
  arXiv  Detail & Related papers  (2024-02-05T22:35:42Z)
• An Empirical Investigation of Commonsense Self-Supervision with Knowledge Graphs [67.23285413610243]
  Self-supervision based on the information extracted from large knowledge graphs has been shown to improve the generalization of language models. We study the effect of knowledge sampling strategies and sizes that can be used to generate synthetic data for adapting language models.
  arXiv  Detail & Related papers  (2022-05-21T19:49:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.