Uncertainty-Aware Abstractive Summarization

• URL: http://arxiv.org/abs/2105.10155v1
• Date: Fri, 21 May 2021 06:36:40 GMT
• Title: Uncertainty-Aware Abstractive Summarization
• Authors: Alexios Gidiotis and Grigorios Tsoumakas
• Abstract summary: We propose a novel approach to summarization based on Bayesian deep learning. We show that our variational equivalents of BART and PEG can outperform their deterministic counterparts on multiple benchmark datasets. Having a reliable uncertainty measure, we can improve the experience of the end user by filtering generated summaries of high uncertainty.
• Score: 3.1423034006764965
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a novel approach to summarization based on Bayesian deep learning. We approximate Bayesian summary generation by first extending state-of-the-art summarization models with Monte Carlo dropout and then using them to perform multiple stochastic forward passes. This method allows us to improve summarization performance by simply using the median of multiple stochastic summaries. We show that our variational equivalents of BART and PEGASUS can outperform their deterministic counterparts on multiple benchmark datasets. In addition, we rely on Bayesian inference to measure the uncertainty of the model when generating summaries. Having a reliable uncertainty measure, we can improve the experience of the end user by filtering out generated summaries of high uncertainty. Furthermore, our proposed metric could be used as a criterion for selecting samples for annotation, and can be paired nicely with active learning and human-in-the-loop approaches.

Related papers

• Balancing Diversity and Risk in LLM Sampling: How to Select Your Method and Parameter for Open-Ended Text Generation [60.493180081319785]
  We propose a systematic way to estimate the intrinsic capacity of a truncation sampling method by considering the trade-off between diversity and risk at each decoding step. Our work provides a comprehensive comparison between existing truncation sampling methods, as well as their recommended parameters as a guideline for users.
  arXiv  Detail & Related papers  (2024-08-24T14:14:32Z)
• On diffusion-based generative models and their error bounds: The log-concave case with full convergence estimates [5.13323375365494]
  We provide theoretical guarantees for the convergence behaviour of diffusion-based generative models under strongly log-concave data. Our class of functions used for score estimation is made of Lipschitz continuous functions avoiding any Lipschitzness assumption on the score function. This approach yields the best known convergence rate for our sampling algorithm.
  arXiv  Detail & Related papers  (2023-11-22T18:40:45Z)
• Ensemble Modeling for Multimodal Visual Action Recognition [50.38638300332429]
  We propose an ensemble modeling approach for multimodal action recognition. We independently train individual modality models using a variant of focal loss tailored to handle the long-tailed distribution of the MECCANO [21] dataset.
  arXiv  Detail & Related papers  (2023-08-10T08:43:20Z)
• Sample and Predict Your Latent: Modality-free Sequential Disentanglement via Contrastive Estimation [2.7759072740347017]
  We introduce a self-supervised sequential disentanglement framework based on contrastive estimation with no external signals. In practice, we propose a unified, efficient, and easy-to-code sampling strategy for semantically similar and dissimilar views of the data. Our method presents state-of-the-art results in comparison to existing techniques.
  arXiv  Detail & Related papers  (2023-05-25T10:50:30Z)
• Robust probabilistic inference via a constrained transport metric [8.85031165304586]
  We offer a novel alternative by constructing an exponentially tilted empirical likelihood carefully designed to concentrate near a parametric family of distributions. The proposed approach finds applications in a wide variety of robust inference problems, where we intend to perform inference on the parameters associated with the centering distribution. We demonstrate superior performance of our methodology when compared against state-of-the-art robust Bayesian inference methods.
  arXiv  Detail & Related papers  (2023-03-17T16:10:06Z)
• Bayesian Hierarchical Models for Counterfactual Estimation [12.159830463756341]
  We propose a probabilistic paradigm to estimate a diverse set of counterfactuals. We treat the perturbations as random variables endowed with prior distribution functions. A gradient based sampler with superior convergence characteristics efficiently computes the posterior samples.
  arXiv  Detail & Related papers  (2023-01-21T00:21:11Z)
• BRIO: Bringing Order to Abstractive Summarization [107.97378285293507]
  We propose a novel training paradigm which assumes a non-deterministic distribution. Our method achieves a new state-of-the-art result on the CNN/DailyMail (47.78 ROUGE-1) and XSum (49.07 ROUGE-1) datasets.
  arXiv  Detail & Related papers  (2022-03-31T05:19:38Z)
• Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [61.827054365139645]
  Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. We propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space.
  arXiv  Detail & Related papers  (2021-10-24T07:58:13Z)
• Bias-Robust Bayesian Optimization via Dueling Bandit [57.82422045437126]
  We consider Bayesian optimization in settings where observations can be adversarially biased. We propose a novel approach for dueling bandits based on information-directed sampling (IDS) Thereby, we obtain the first efficient kernelized algorithm for dueling bandits that comes with cumulative regret guarantees.
  arXiv  Detail & Related papers  (2021-05-25T10:08:41Z)
• Oops I Took A Gradient: Scalable Sampling for Discrete Distributions [53.3142984019796]
  We show that this approach outperforms generic samplers in a number of difficult settings. We also demonstrate the use of our improved sampler for training deep energy-based models on high dimensional discrete data.
  arXiv  Detail & Related papers  (2021-02-08T20:08:50Z)
• $\beta$-Cores: Robust Large-Scale Bayesian Data Summarization in the Presence of Outliers [14.918826474979587]
  The quality of classic Bayesian inference depends critically on whether observations conform with the assumed data generating model. We propose a variational inference method that, in a principled way, can simultaneously scale to large datasets. We illustrate the applicability of our approach in diverse simulated and real datasets, and various statistical models.
  arXiv  Detail & Related papers  (2020-08-31T13:47:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.