Markov-Enhanced Clustering for Long Document Summarization: Tackling the 'Lost in the Middle' Challenge with Large Language Models

• URL: http://arxiv.org/abs/2506.18036v1
• Date: Sun, 22 Jun 2025 13:34:58 GMT
• Title: Markov-Enhanced Clustering for Long Document Summarization: Tackling the 'Lost in the Middle' Challenge with Large Language Models
• Authors: Aziz Amari, Mohamed Achref Ben Ammar,
• Abstract summary: We propose a hybrid summarization approach that combines extractive and abstractive techniques.<n>Our method splits the document into smaller text chunks, clusters their vector embeddings, and generates a summary for each cluster.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid expansion of information from diverse sources has heightened the need for effective automatic text summarization, which condenses documents into shorter, coherent texts. Summarization methods generally fall into two categories: extractive, which selects key segments from the original text, and abstractive, which generates summaries by rephrasing the content coherently. Large language models have advanced the field of abstractive summarization, but they are resourceintensive and face significant challenges in retaining key information across lengthy documents, which we call being "lost in the middle". To address these issues, we propose a hybrid summarization approach that combines extractive and abstractive techniques. Our method splits the document into smaller text chunks, clusters their vector embeddings, generates a summary for each cluster that represents a key idea in the document, and constructs the final summary by relying on a Markov chain graph when selecting the semantic order of ideas.

Related papers

• Context-Aware Hierarchical Merging for Long Document Summarization [56.96619074316232]
  We propose different approaches to enrich hierarchical merging with context from the source document.<n> Experimental results on datasets representing legal and narrative domains show that contextual augmentation consistently outperforms zero-shot and hierarchical merging baselines.
  arXiv  Detail & Related papers  (2025-02-03T01:14:31Z)
• JADS: A Framework for Self-supervised Joint Aspect Discovery and Summarization [3.992091862806936]
  Our solution integrates topic discovery and summarization into a single step. Given text data, our Joint Aspect Discovery and Summarization algorithm (JADS) discovers aspects from the input. Our proposed method achieves higher semantic alignment with ground truth and is factual.
  arXiv  Detail & Related papers  (2024-05-28T23:01:57Z)
• TRIE++: Towards End-to-End Information Extraction from Visually Rich Documents [51.744527199305445]
  This paper proposes a unified end-to-end information extraction framework from visually rich documents. Text reading and information extraction can reinforce each other via a well-designed multi-modal context block. The framework can be trained in an end-to-end trainable manner, achieving global optimization.
  arXiv  Detail & Related papers  (2022-07-14T08:52:07Z)
• A General Contextualized Rewriting Framework for Text Summarization [15.311467109946571]
  Exiting rewriting systems take each extractive sentence as the only input, which is relatively focused but can lose necessary background knowledge and discourse context. We formalize contextualized rewriting as a seq2seq with group-tag alignments, identifying extractive sentences through content-based addressing. Results show that our approach significantly outperforms non-contextualized rewriting systems without requiring reinforcement learning.
  arXiv  Detail & Related papers  (2022-07-13T03:55:57Z)
• A Survey on Neural Abstractive Summarization Methods and Factual Consistency of Summarization [18.763290930749235]
  summarization is the process of shortening a set of textual data computationally, to create a subset (a summary) Existing summarization methods can be roughly divided into two types: extractive and abstractive. An extractive summarizer explicitly selects text snippets from the source document, while an abstractive summarizer generates novel text snippets to convey the most salient concepts prevalent in the source.
  arXiv  Detail & Related papers  (2022-04-20T14:56:36Z)
• Unsupervised Summarization with Customized Granularities [76.26899748972423]
  We propose the first unsupervised multi-granularity summarization framework, GranuSum. By inputting different numbers of events, GranuSum is capable of producing multi-granular summaries in an unsupervised manner.
  arXiv  Detail & Related papers  (2022-01-29T05:56:35Z)
• Topic Modeling Based Extractive Text Summarization [0.0]
  We propose a novel method to summarize a text document by clustering its contents based on latent topics. We utilize the lesser used and challenging WikiHow dataset in our approach to text summarization.
  arXiv  Detail & Related papers  (2021-06-29T12:28:19Z)
• Summarize, Outline, and Elaborate: Long-Text Generation via Hierarchical Supervision from Extractive Summaries [46.183289748907804]
  We propose SOE, a pipelined system that outlines, outlining and elaborating for long text generation. SOE produces long texts with significantly better quality, along with faster convergence speed.
  arXiv  Detail & Related papers  (2020-10-14T13:22:20Z)
• SummPip: Unsupervised Multi-Document Summarization with Sentence Graph Compression [61.97200991151141]
  SummPip is an unsupervised method for multi-document summarization. We convert the original documents to a sentence graph, taking both linguistic and deep representation into account. We then apply spectral clustering to obtain multiple clusters of sentences, and finally compress each cluster to generate the final summary.
  arXiv  Detail & Related papers  (2020-07-17T13:01:15Z)
• Leveraging Graph to Improve Abstractive Multi-Document Summarization [50.62418656177642]
  We develop a neural abstractive multi-document summarization (MDS) model which can leverage well-known graph representations of documents. Our model utilizes graphs to encode documents in order to capture cross-document relations, which is crucial to summarizing long documents. Our model can also take advantage of graphs to guide the summary generation process, which is beneficial for generating coherent and concise summaries.
  arXiv  Detail & Related papers  (2020-05-20T13:39:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.