CNTLS: A Benchmark Dataset for Abstractive or Extractive Chinese Timeline Summarization

• URL: http://arxiv.org/abs/2105.14201v2
• Date: Wed, 15 Nov 2023 09:52:12 GMT
• Title: CNTLS: A Benchmark Dataset for Abstractive or Extractive Chinese Timeline Summarization
• Authors: Qianren Mao, Jiazheng Wang, Zheng Wang, Xi Li, Bo Li, Jianxin Li
• Abstract summary: We introduce the CNTLS dataset, a versatile resource for Chinese timeline summarization. CNTLS encompasses 77 real-life topics, each with 2524 documents and summarizes nearly 60% days duration compression. We evaluate the performance of various extractive and generative summarization systems on the CNTLS corpus.
• Score: 22.813746290856916
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Timeline summarization (TLS) involves creating summaries of long-running events using dated summaries from numerous news articles. However, limited data availability has significantly slowed down the development of timeline summarization. In this paper, we introduce the CNTLS dataset, a versatile resource for Chinese timeline summarization. CNTLS encompasses 77 real-life topics, each with 2524 documents and summarizes nearly 60\% days duration compression on average all topics. We meticulously analyze the corpus using well-known metrics, focusing on the style of the summaries and the complexity of the summarization task. Specifically, we evaluate the performance of various extractive and generative summarization systems on the CNTLS corpus to provide benchmarks and support further research. To the best of our knowledge, CNTLS is the first Chinese timeline summarization dataset. The dataset and source code are released\footnote{Code and data available at: \emph{\url{https://github.com/OpenSUM/CNTLS}}.}.

Related papers

• Write Summary Step-by-Step: A Pilot Study of Stepwise Summarization [48.57273563299046]
  We propose the task of Stepwise Summarization, which aims to generate a new appended summary each time a new document is proposed. The appended summary should not only summarize the newly added content but also be coherent with the previous summary. We show that SSG achieves state-of-the-art performance in terms of both automatic metrics and human evaluations.
  arXiv  Detail & Related papers  (2024-06-08T05:37:26Z)
• Analyzing Temporal Complex Events with Large Language Models? A Benchmark towards Temporal, Long Context Understanding [57.62275091656578]
  We refer to the complex events composed of many news articles over an extended period as Temporal Complex Event (TCE) This paper proposes a novel approach using Large Language Models (LLMs) to systematically extract and analyze the event chain within TCE.
  arXiv  Detail & Related papers  (2024-06-04T16:42:17Z)
• On Context Utilization in Summarization with Large Language Models [83.84459732796302]
  Large language models (LLMs) excel in abstractive summarization tasks, delivering fluent and pertinent summaries. Recent advancements have extended their capabilities to handle long-input contexts, exceeding 100k tokens. We conduct the first comprehensive study on context utilization and position bias in summarization.
  arXiv  Detail & Related papers  (2023-10-16T16:45:12Z)
• VCSUM: A Versatile Chinese Meeting Summarization Dataset [25.695308276427166]
  We introduce a versatile Chinese meeting summarization dataset, dubbed VCSum, consisting of 239 real-life meetings. We provide the annotations of topic segmentation, headlines, segmentation summaries, overall meeting summaries, and salient sentences for each meeting transcript. Our analysis confirms the effectiveness and robustness of VCSum.
  arXiv  Detail & Related papers  (2023-05-09T09:07:15Z)
• Follow the Timeline! Generating Abstractive and Extractive Timeline Summary in Chronological Order [78.46986998674181]
  We propose a Unified Timeline Summarizer (UTS) that can generate abstractive and extractive timeline summaries in time order. We augment the previous Chinese large-scale timeline summarization dataset and collect a new English timeline dataset. UTS achieves state-of-the-art performance in terms of both automatic and human evaluations.
  arXiv  Detail & Related papers  (2023-01-02T20:29:40Z)
• CLTS+: A New Chinese Long Text Summarization Dataset with Abstractive Summaries [10.113673549224256]
  Abstractive methods lack of creative ability is particularly a problem in automatic text summarization. We propose the first Chinese Long Text Summarization dataset with a high level of abstractiveness, CLTS+. We analyze the extraction strategies used in CLTS+ summaries against other datasets to quantify the abstractiveness and difficulty of our new data.
  arXiv  Detail & Related papers  (2022-06-09T03:53:52Z)
• SQuALITY: Building a Long-Document Summarization Dataset the Hard Way [31.832673451018543]
  We hire highly-qualified contractors to read stories and write original summaries from scratch. To amortize reading time, we collect five summaries per document, with the first giving an overview and the subsequent four addressing specific questions. Experiments with state-of-the-art summarization systems show that our dataset is challenging and that existing automatic evaluation metrics are weak indicators of quality.
  arXiv  Detail & Related papers  (2022-05-23T17:02:07Z)
• CNewSum: A Large-scale Chinese News Summarization Dataset with Human-annotated Adequacy and Deducibility Level [15.969302324314516]
  We present a large-scale Chinese news summarization dataset CNewSum. It consists of 304,307 documents and human-written summaries for the news feed. Its test set contains adequacy and deducibility annotations for the summaries.
  arXiv  Detail & Related papers  (2021-10-21T03:37:46Z)
• Text Summarization with Latent Queries [60.468323530248945]
  We introduce LaQSum, the first unified text summarization system that learns Latent Queries from documents for abstractive summarization with any existing query forms. Under a deep generative framework, our system jointly optimize a latent query model and a conditional language model, allowing users to plug-and-play queries of any type at test time. Our system robustly outperforms strong comparison systems across summarization benchmarks with different query types, document settings, and target domains.
  arXiv  Detail & Related papers  (2021-05-31T21:14:58Z)
• The Shmoop Corpus: A Dataset of Stories with Loosely Aligned Summaries [72.48439126769627]
  We introduce the Shmoop Corpus: a dataset of 231 stories paired with detailed multi-paragraph summaries for each individual chapter. From the corpus, we construct a set of common NLP tasks, including Cloze-form question answering and a simplified form of abstractive summarization. We believe that the unique structure of this corpus provides an important foothold towards making machine story comprehension more approachable.
  arXiv  Detail & Related papers  (2019-12-30T21:03:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.