From Content to Audience: A Multimodal Annotation Framework for Broadcast Television Analytics

• URL: http://arxiv.org/abs/2603.26772v1
• Date: Tue, 24 Mar 2026 13:46:51 GMT
• Title: From Content to Audience: A Multimodal Annotation Framework for Broadcast Television Analytics
• Abstract summary: multimodal large language models (MLLMs) have demonstrated strong general-purpose video understanding capabilities.<n>This paper presents a systematic evaluation of multimodal annotation pipelines applied to broadcast television news in the Italian setting.
• Score: 1.6639892342953113
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automated semantic annotation of broadcast television content presents distinctive challenges, combining structured audiovisual composition, domain-specific editorial patterns, and strict operational constraints. While multimodal large language models (MLLMs) have demonstrated strong general-purpose video understanding capabilities, their comparative effectiveness across pipeline architectures and input configurations in broadcast-specific settings remains empirically undercharacterized. This paper presents a systematic evaluation of multimodal annotation pipelines applied to broadcast television news in the Italian setting. We construct a domain-specific benchmark of clips labeled across four semantic dimensions: visual environment classification, topic classification, sensitive content detection, and named entity recognition. Two different pipeline architectures are evaluated across nine frontier models, including Gemini 3.0 Pro, LLaMA 4 Maverick, Qwen-VL variants, and Gemma 3, under progressively enriched input strategies combining visual signals, automatic speech recognition, speaker diarization, and metadata. Experimental results demonstrate that gains from video input are strongly model-dependent: larger models effectively leverage temporal continuity, while smaller models show performance degradation under extended multimodal context, likely due to token overload. Beyond benchmarking, the selected pipeline is deployed on 14 full broadcast episodes, with minute-level annotations integrated with normalized audience measurement data provided by an Italian media company. This integration enables correlational analysis of topic-level audience sensitivity and generational engagement divergence, demonstrating the operational viability of the proposed framework for content-based audience analytics.

Related papers

• Hierarchical Semantic-Constrained Heterogeneous Graph for Audio-Visual Event Localization [69.67186845167568]
  Open-vocabulary audio-visual event localization (OV-AVEL) jointly models audio-visual cues to recognize and temporally localize events.<n>Existing methods primarily learn joint audio-visual representations in Euclidean space.<n>We propose a hierarchical semantic constrained heterogeneous graph (HSCHG) for audio-visual event localization framework.
  arXiv  Detail & Related papers  (2026-06-05T08:23:58Z)
• Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding [98.3098451637867]
  Video-MME-v2 is a comprehensive benchmark designed to rigorously evaluate the robustness and faithfulness of video understanding.<n>To guarantee data quality, Video-MME-v2 is constructed through a rigorously controlled human annotation pipeline.
  arXiv  Detail & Related papers  (2026-04-06T17:59:56Z)
• A Benchmarking Methodology to Assess Open-Source Video Large Language Models in Automatic Captioning of News Videos [0.0]
  This work presents a comparative study of eight state-of-the-art open-source VidLLMs for automatic news video captioning.<n>We employ lexical metrics (METEOR, ROUGE-L), semantic metrics (BERTScore, CLIPScore, Text Similarity, Mean Reciprocal Rank), and two novel fidelity metrics proposed in this work.<n>Our analysis reveals that standard metrics exhibit limited discrimi power for news video captioning due to surface-form dependence, static-frame insensitivity, and function-word inflation.
  arXiv  Detail & Related papers  (2026-03-29T12:28:35Z)
• DecepGPT: Schema-Driven Deception Detection with Multicultural Datasets and Robust Multimodal Learning [64.33887406863899]
  Multimodal deception detection aims to identify deceptive behavior by analyzing audiovisual cues for forensics and security.<n>Existing benchmarks provide only binary labels without intermediate reasoning cues.<n>We construct reasoning datasets with structured cue-level descriptions and reasoning chains.<n>With 1695 samples, it is the largest non-laboratory deception detection dataset.
  arXiv  Detail & Related papers  (2026-03-25T04:06:36Z)
• Understanding Virality: A Rubric based Vision-Language Model Framework for Short-Form Edutainment Evaluation [8.15791379444665]
  VideoScore-2 does not capture how specific audiovisual attributes drive real audience engagement.<n>We propose a data-driven evaluation framework that uses Vision-Language Models (VLMs) to extract unsupervised audiovisual features.<n>Our approach advances toward robust and explainable video understanding.
  arXiv  Detail & Related papers  (2025-12-24T19:43:59Z)
• Complementary and Contrastive Learning for Audio-Visual Segmentation [74.11434759171199]
  We present Complementary and Contrastive Transformer (CCFormer), a novel framework adept at processing both local and global information.<n>Our method sets new state-of-the-art benchmarks across the S4, MS3 and AVSS datasets.
  arXiv  Detail & Related papers  (2025-10-11T06:36:59Z)
• BindWeave: Subject-Consistent Video Generation via Cross-Modal Integration [56.98981194478512]
  We propose a unified framework that handles a broad range of subject-to-video scenarios.<n>We introduce an MLLM-DiT framework in which a pretrained multimodal large language model performs deep cross-modal reasoning to ground entities.<n>Experiments on the OpenS2V benchmark demonstrate that our method achieves superior performance across subject consistency, naturalness, and text relevance in generated videos.
  arXiv  Detail & Related papers  (2025-10-01T02:41:11Z)
• PresentAgent: Multimodal Agent for Presentation Video Generation [30.274831875701217]
  We present PresentAgent, a multimodal agent that transforms long-form documents into narrated presentation videos.<n>To achieve this integration, PresentAgent employs a modular pipeline that segments the input document, plans and renders slide-style visual frames.<n>Given the complexity of evaluating such multimodal outputs, we introduce PresentEval, a unified assessment framework powered by Vision-Language Models.
  arXiv  Detail & Related papers  (2025-07-05T13:24:15Z)
• Multi-Modal interpretable automatic video captioning [1.9874264019909988]
  We introduce a novel video captioning method trained with multi-modal contrastive loss. Our approach is designed to capture the dependency between these modalities, resulting in more accurate, thus pertinent captions.
  arXiv  Detail & Related papers  (2024-11-11T11:12:23Z)
• MMTrail: A Multimodal Trailer Video Dataset with Language and Music Descriptions [69.9122231800796]
  We present MMTrail, a large-scale multi-modality video-language dataset incorporating more than 20M trailer clips with visual captions.<n>We propose a systemic captioning framework, achieving various modality annotations with more than 27.1k hours of trailer videos.<n>Our dataset potentially paves the path for fine-grained large multimodal-language model training.
  arXiv  Detail & Related papers  (2024-07-30T16:43:24Z)
• Multimodal Variational Auto-encoder based Audio-Visual Segmentation [46.67599800471001]
  ECMVAE factorizes the representations of each modality with a modality-shared representation and a modality-specific representation. Our approach leads to a new state-of-the-art for audio-visual segmentation, with a 3.84 mIOU performance leap.
  arXiv  Detail & Related papers  (2023-10-12T13:09:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.