Scribby: A Multi-Level LLM Framework for Semantic Video Analysis

• URL: http://arxiv.org/abs/2606.14762v1
• Date: Mon, 08 Jun 2026 14:41:00 GMT
• Title: Scribby: A Multi-Level LLM Framework for Semantic Video Analysis
• Abstract summary: This paper proposes an LLM-based video summarization framework that balances macro-level comprehension with micro-level semantic analysis cite6,12,13.<n>Contextual continuity is retained during sentence-level processing by incorporating both the global transcript analysis and adjacent sentence information into each evaluation prompt.<n>This framework establishes a foundation for video analysis tools that visualize semantic chunking and semantic matching through relevance-based heatmaps.
• Score: 0.0
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: As video content continues to expand across educational platforms, recorded lectures, and live-streamed entertainment, the need for efficient and structured analysis of long-form footage has increased \cite{1}. Although many existing AI programs provide high-level video summaries based on AI-generated transcripts \cite{2,3,4,5}, these approaches are often limited to coarse overviews and lack detailed analysis of a video's structure, thematic progression, and semantic relationships, all of which are required for comprehensive video analysis. This paper proposes an LLM-based video summarization framework that balances macro-level comprehension with micro-level semantic analysis \cite{6,12,13}. The first stage of the process indexes the video at a micro level by (1) analyzing the full transcript, (2) analyzing individual transcript sentences, and (3) grouping these sentences by semantic similarity using an LLM as a judge \cite{6,13}. Contextual continuity is retained during sentence-level processing by incorporating both the global transcript analysis and adjacent sentence information into each evaluation prompt. This framework establishes a foundation for video analysis tools that visualize semantic chunking and semantic matching through relevance-based heatmaps. Limitations and future expansions of the framework are also discussed.

Related papers

• OmniVideo-100K: A Dataset for Audio-Visual Reasoning through Structured Scripts and Evidence Chains [50.186434778589415]
  Current automated pipelines for audio-visual Question Answering (QA) generally adopt a video-caption-QA'' paradigm.<n>We propose an automated data engine featuring two mechanisms: textbfEntity-Anchored Video Scripting' and textbfClue-Guided QA Generation'
  arXiv  Detail & Related papers  (2026-06-12T17:59:55Z)
• Explainable Forensics of Manipulated Segments in Untrimmed Long Videos [50.190474724159465]
  We formulate the task of Temporal AI-Generated Segment localization and Explanation.<n>We introduce TASLE, a large-scale benchmark comprising 12,472 untrimmed videos with diverse manipulation patterns and rich annotation signals.<n>We propose MSLoc, a coarse-to-fine forensic baseline that combines a boundary-sensitive proposal generation module for efficient long-video scanning with an MLLM-based refinement module for precise boundary localization and interpretable reasoning.
  arXiv  Detail & Related papers  (2026-06-01T15:48:38Z)
• Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding [98.3098451637867]
  Video-MME-v2 is a comprehensive benchmark designed to rigorously evaluate the robustness and faithfulness of video understanding.<n>To guarantee data quality, Video-MME-v2 is constructed through a rigorously controlled human annotation pipeline.
  arXiv  Detail & Related papers  (2026-04-06T17:59:56Z)
• Less is More: Label-Guided Summarization of Procedural and Instructional Videos [21.13311741987469]
  We propose a three-stage framework, PRISM: Procedural Representation via Integrated Semantic and Multimodal analysis.<n>We analyze adaptive visual sampling, label-driven anchoring, and contextual validation using a large language model (LLM)<n>Our approach generalizes across procedural and domain-specific video tasks, achieving strong performance with both semantic alignment and precision.
  arXiv  Detail & Related papers  (2026-01-18T03:41:48Z)
• LUST: A Multi-Modal Framework with Hierarchical LLM-based Scoring for Learned Thematic Significance Tracking in Multimedia Content [0.0]
  The Learned User Significance Tracker (LUST) is a framework designed to analyze video content and quantify the thematic relevance of its segments.<n>The core innovation lies in a hierarchical, two-stage relevance scoring mechanism employing Large Language Models (LLMs)<n>The LUST framework aims to provide a nuanced, temporally-aware measure of user-defined significance, outputting an annotated video with visualized relevance scores and comprehensive analytical logs.
  arXiv  Detail & Related papers  (2025-08-06T11:48:51Z)
• Dense Video Captioning using Graph-based Sentence Summarization [80.52481563888459]
  We propose a graph-based partition-and-summarization framework for dense video captioning.<n>We focus on the summarization" stage, and propose a framework that effectively exploits the relationship between semantic words for summarization.
  arXiv  Detail & Related papers  (2025-06-25T16:23:43Z)
• Prompts to Summaries: Zero-Shot Language-Guided Video Summarization [12.200609701777907]
  We introduce Prompts-to-Summaries: the first zero-shot, text-queryable video summarizer.<n>It converts off-the-shelf video-language models (VidLMs) captions into user-guided skims via large language models (LLMs) judging.<n>Our pipeline generates rich scene-level descriptions through a memory-efficient, batch-style VidLM prompting scheme.<n>On SumMe and TVSum, our data-free approach surpasses all prior data-hungry unsupervised methods.
  arXiv  Detail & Related papers  (2025-06-12T15:23:11Z)
• VidText: Towards Comprehensive Evaluation for Video Text Understanding [56.121054697977115]
  VidText is a benchmark for comprehensive and in-depth evaluation of video text understanding.<n>It covers a wide range of real-world scenarios and supports multilingual content.<n>It introduces a hierarchical evaluation framework with video-level, clip-level, and instance-level tasks.
  arXiv  Detail & Related papers  (2025-05-28T19:39:35Z)
• Video Summarization with Large Language Models [41.51242348081083]
  We propose a new video summarization framework that leverages the capabilities of recent Large Language Models (LLMs)<n>Our method, dubbed LLM-based Video Summarization (LLMVS), translates video frames into a sequence of captions using a Muti-modal Large Language Model (MLLM)<n>Our experimental results demonstrate the superiority of the proposed method over existing ones in standard benchmarks.
  arXiv  Detail & Related papers  (2025-04-15T13:56:14Z)
• Language-Guided Self-Supervised Video Summarization Using Text Semantic Matching Considering the Diversity of the Video [22.60291297308379]
  We investigate the feasibility in transforming the video summarization task into a Natural Language Processing (NLP) task. Our method achieves state-of-the-art performance on the SumMe dataset in rank correlation coefficients.
  arXiv  Detail & Related papers  (2024-05-14T18:07:04Z)
• Towards Balanced Alignment: Modal-Enhanced Semantic Modeling for Video Moment Retrieval [31.42856682276394]
  Video Moment Retrieval (VMR) aims to retrieve temporal segments in untrimmed videos corresponding to a given language query. Existing strategies are often sub-optimal since they ignore the modality imbalance problem. We introduce Modal-Enhanced Semantic Modeling (MESM), a novel framework for more balanced alignment.
  arXiv  Detail & Related papers  (2023-12-19T13:38:48Z)
• Hierarchical Video-Moment Retrieval and Step-Captioning [68.4859260853096]
  HiREST consists of 3.4K text-video pairs from an instructional video dataset. Our hierarchical benchmark consists of video retrieval, moment retrieval, and two novel moment segmentation and step captioning tasks.
  arXiv  Detail & Related papers  (2023-03-29T02:33:54Z)
• TL;DW? Summarizing Instructional Videos with Task Relevance & Cross-Modal Saliency [133.75876535332003]
  We focus on summarizing instructional videos, an under-explored area of video summarization. Existing video summarization datasets rely on manual frame-level annotations. We propose an instructional video summarization network that combines a context-aware temporal video encoder and a segment scoring transformer.
  arXiv  Detail & Related papers  (2022-08-14T04:07:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.