GLIMPSE: Pragmatically Informative Multi-Document Summarization for Scholarly Reviews

• URL: http://arxiv.org/abs/2406.07359v1
• Date: Tue, 11 Jun 2024 15:27:01 GMT
• Title: GLIMPSE: Pragmatically Informative Multi-Document Summarization for Scholarly Reviews
• Authors: Maxime Darrin, Ines Arous, Pablo Piantanida, Jackie CK Cheung,
• Abstract summary: We introduce sys, a summarization method designed to offer a concise yet comprehensive overview of scholarly reviews. Unlike traditional consensus-based methods, sys extracts both common and unique opinions from the reviews.
• Score: 25.291384842659397
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Scientific peer review is essential for the quality of academic publications. However, the increasing number of paper submissions to conferences has strained the reviewing process. This surge poses a burden on area chairs who have to carefully read an ever-growing volume of reviews and discern each reviewer's main arguments as part of their decision process. In this paper, we introduce \sys, a summarization method designed to offer a concise yet comprehensive overview of scholarly reviews. Unlike traditional consensus-based methods, \sys extracts both common and unique opinions from the reviews. We introduce novel uniqueness scores based on the Rational Speech Act framework to identify relevant sentences in the reviews. Our method aims to provide a pragmatic glimpse into all reviews, offering a balanced perspective on their opinions. Our experimental results with both automatic metrics and human evaluation show that \sys generates more discriminative summaries than baseline methods in terms of human evaluation while achieving comparable performance with these methods in terms of automatic metrics.

Related papers

• A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [58.6354685593418]
  This paper proposes several article-level, field-normalized, and large language model-empowered bibliometric indicators to evaluate reviews. The newly emerging AI-generated literature reviews are also appraised. This work offers insights into the current challenges of literature reviews and envisions future directions for their development.
  arXiv  Detail & Related papers  (2024-02-20T11:28:50Z)
• Incremental Extractive Opinion Summarization Using Cover Trees [81.59625423421355]
  In online marketplaces user reviews accumulate over time, and opinion summaries need to be updated periodically. In this work, we study the task of extractive opinion summarization in an incremental setting. We present an efficient algorithm for accurately computing the CentroidRank summaries in an incremental setting.
  arXiv  Detail & Related papers  (2024-01-16T02:00:17Z)
• Automatically Evaluating Opinion Prevalence in Opinion Summarization [0.9971537447334835]
  We propose an automatic metric to test the prevalence of the opinions that a summary expresses. We consider several existing methods to score the factual consistency of a summary statement. We show that a human authored summary has only slightly better opinion prevalence than randomly selected extracts from the source reviews.
  arXiv  Detail & Related papers  (2023-07-26T17:13:00Z)
• Scientific Opinion Summarization: Paper Meta-review Generation Dataset, Methods, and Evaluation [55.00687185394986]
  We propose the task of scientific opinion summarization, where research paper reviews are synthesized into meta-reviews. We introduce the ORSUM dataset covering 15,062 paper meta-reviews and 57,536 paper reviews from 47 conferences. Our experiments show that (1) human-written summaries do not always satisfy all necessary criteria such as depth of discussion, and identifying consensus and controversy for the specific domain, and (2) the combination of task decomposition and iterative self-refinement shows strong potential for enhancing the opinions.
  arXiv  Detail & Related papers  (2023-05-24T02:33:35Z)
• OpineSum: Entailment-based self-training for abstractive opinion summarization [6.584115526134759]
  We present a novel self-training approach, OpineSum, for abstractive opinion summarization. The summaries in this approach are built using a novel application of textual entailment. OpineSum achieves state-of-the-art performance in both settings.
  arXiv  Detail & Related papers  (2022-12-21T06:20:28Z)
• Submission-Aware Reviewer Profiling for Reviewer Recommender System [26.382772998002523]
  We propose an approach that learns from each abstract published by a potential reviewer the topics studied and the explicit context in which the reviewer studied the topics. Our experiments show a significant, consistent improvement in precision when compared with the existing methods. The new approach has been deployed successfully at top-tier conferences in the last two years.
  arXiv  Detail & Related papers  (2022-11-08T12:18:02Z)
• Learning Opinion Summarizers by Selecting Informative Reviews [81.47506952645564]
  We collect a large dataset of summaries paired with user reviews for over 31,000 products, enabling supervised training. The content of many reviews is not reflected in the human-written summaries, and, thus, the summarizer trained on random review subsets hallucinates. We formulate the task as jointly learning to select informative subsets of reviews and summarizing the opinions expressed in these subsets.
  arXiv  Detail & Related papers  (2021-09-09T15:01:43Z)
• Ranking Scientific Papers Using Preference Learning [48.78161994501516]
  We cast it as a paper ranking problem based on peer review texts and reviewer scores. We introduce a novel, multi-faceted generic evaluation framework for making final decisions based on peer reviews.
  arXiv  Detail & Related papers  (2021-09-02T19:41:47Z)
• Hierarchical Bi-Directional Self-Attention Networks for Paper Review Rating Recommendation [81.55533657694016]
  We propose a Hierarchical bi-directional self-attention Network framework (HabNet) for paper review rating prediction and recommendation. Specifically, we leverage the hierarchical structure of the paper reviews with three levels of encoders: sentence encoder (level one), intra-review encoder (level two) and inter-review encoder (level three) We are able to identify useful predictors to make the final acceptance decision, as well as to help discover the inconsistency between numerical review ratings and text sentiment conveyed by reviewers.
  arXiv  Detail & Related papers  (2020-11-02T08:07:50Z)
• Topic Detection and Summarization of User Reviews [6.779855791259679]
  We propose an effective new summarization method by analyzing both reviews and summaries. A new dataset comprising product reviews and summaries about 1028 products are collected from Amazon and CNET.
  arXiv  Detail & Related papers  (2020-05-30T02:19:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.