Rhapsody: A Dataset for Highlight Detection in Podcasts

• URL: http://arxiv.org/abs/2505.19429v1
• Date: Mon, 26 May 2025 02:39:34 GMT
• Title: Rhapsody: A Dataset for Highlight Detection in Podcasts
• Authors: Younghan Park, Anuj Diwan, David Harwath, Eunsol Choi,
• Abstract summary: We introduce Rhapsody, a feature paired with segment-level highlight from YouTube's'most replayed' episodes.<n>We frame the podcast highlight detection as a segment-level binary classification task.<n>Models finetuned with in-domain data significantly outperform their zero-shot performance.<n>These findings highlight the challenges for fine-grained information access in long-form spoken media.
• Score: 49.1662517033426
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Podcasts have become daily companions for half a billion users. Given the enormous amount of podcast content available, highlights provide a valuable signal that helps viewers get the gist of an episode and decide if they want to invest in listening to it in its entirety. However, identifying highlights automatically is challenging due to the unstructured and long-form nature of the content. We introduce Rhapsody, a dataset of 13K podcast episodes paired with segment-level highlight scores derived from YouTube's 'most replayed' feature. We frame the podcast highlight detection as a segment-level binary classification task. We explore various baseline approaches, including zero-shot prompting of language models and lightweight finetuned language models using segment-level classification heads. Our experimental results indicate that even state-of-the-art language models like GPT-4o and Gemini struggle with this task, while models finetuned with in-domain data significantly outperform their zero-shot performance. The finetuned model benefits from leveraging both speech signal features and transcripts. These findings highlight the challenges for fine-grained information access in long-form spoken media.

Related papers

• MoonCast: High-Quality Zero-Shot Podcast Generation [81.29927724674602]
  MoonCast is a solution for high-quality zero-shot podcast generation.<n>It aims to synthesize natural podcast-style speech from text-only sources.<n>Experiments demonstrate that MoonCast outperforms baselines.
  arXiv  Detail & Related papers  (2025-03-18T15:25:08Z)
• Annotation Tool and Dataset for Fact-Checking Podcasts [1.6804613362826175]
  podcasts are a popular medium on the web, featuring diverse and multilingual content that often includes unverified claims.<n>Our tool offers a novel approach to tackle these challenges by enabling real-time annotation of contextual during playback.<n>This unique capability allows users to listen to the podcast and annotate key elements, such as check-worthy claims, claim spans, and contextual errors, simultaneously.
  arXiv  Detail & Related papers  (2025-02-03T14:34:17Z)
• PODTILE: Facilitating Podcast Episode Browsing with Auto-generated Chapters [15.856812659691238]
  We introduce PODTILE, a fine-tuned encoder-decoder transformer to segment conversational data. PODTILE simultaneously generates chapter transitions and titles for the input transcript. Our findings indicate that auto-generated chapters serve as a useful tool for engaging with less popular podcasts.
  arXiv  Detail & Related papers  (2024-10-21T16:17:22Z)
• Auto-ACD: A Large-scale Dataset for Audio-Language Representation Learning [50.28566759231076]
  We propose an innovative, automatic approach to establish an audio dataset with high-quality captions. Specifically, we construct a large-scale, high-quality, audio-language dataset, named as Auto-ACD, comprising over 1.5M audio-text pairs. We employ LLM to paraphrase a congruent caption for each audio, guided by the extracted multi-modality clues.
  arXiv  Detail & Related papers  (2023-09-20T17:59:32Z)
• Topic Modeling on Podcast Short-Text Metadata [0.9539495585692009]
  We assess the feasibility to discover relevant topics from podcast metadata, titles and descriptions, using modeling techniques for short text. We propose a new strategy to named entities (NEs), often present in podcast metadata, in a Non-negative Matrix Factorization modeling framework. Our experiments on two existing datasets from Spotify and iTunes and Deezer, show that our proposed document representation, NEiCE, leads to improved coherence over the baselines.
  arXiv  Detail & Related papers  (2022-01-12T11:07:05Z)
• Identifying Introductions in Podcast Episodes from Automatically Generated Transcripts [0.0]
  We build a novel dataset of complete transcriptions of over 400 podcast episodes. These introductions contain information about the episodes' topics, hosts, and guests. We train three Transformer models based on the pre-trained BERT and different augmentation strategies.
  arXiv  Detail & Related papers  (2021-10-14T00:34:51Z)
• QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queries [89.24431389933703]
  We present the Query-based Video Highlights (QVHighlights) dataset. It consists of over 10,000 YouTube videos, covering a wide range of topics. Each video in the dataset is annotated with: (1) a human-written free-form NL query, (2) relevant moments in the video w.r.t. the query, and (3) five-point scale saliency scores for all query-relevant clips.
  arXiv  Detail & Related papers  (2021-07-20T16:42:58Z)
• Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions [75.77044856100349]
  We present the Spoken Moments dataset of 500k spoken captions each attributed to a unique short video depicting a broad range of different events. We show that our AMM approach consistently improves our results and that models trained on our Spoken Moments dataset generalize better than those trained on other video-caption datasets.
  arXiv  Detail & Related papers  (2021-05-10T16:30:46Z)
• QuerYD: A video dataset with high-quality text and audio narrations [85.6468286746623]
  We introduce QuerYD, a new large-scale dataset for retrieval and event localisation in video. A unique feature of our dataset is the availability of two audio tracks for each video: the original audio, and a high-quality spoken description. The dataset is based on YouDescribe, a volunteer project that assists visually-impaired people by attaching voiced narrations to existing YouTube videos.
  arXiv  Detail & Related papers  (2020-11-22T17:33:44Z)
• PodSumm -- Podcast Audio Summarization [0.0]
  We propose a method to automatically construct a podcast summary via guidance from the text-domain. Motivated by a lack of datasets for this task, we curate an internal dataset, find an effective scheme for data augmentation, and design a protocol to gather summaries from annotators. Our method achieves ROUGE-F(1/2/L) scores of 0.63/0.53/0.63 on our dataset.
  arXiv  Detail & Related papers  (2020-09-22T04:49:33Z)
• Abstractive Summarization of Spoken and Written Instructions with BERT [66.14755043607776]
  We present the first application of the BERTSum model to conversational language. We generate abstractive summaries of narrated instructional videos across a wide variety of topics. We envision this integrated as a feature in intelligent virtual assistants, enabling them to summarize both written and spoken instructional content upon request.
  arXiv  Detail & Related papers  (2020-08-21T20:59:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.