Multi-Modal Framing Analysis of News

• URL: http://arxiv.org/abs/2503.20960v2
• Date: Thu, 03 Apr 2025 13:31:11 GMT
• Title: Multi-Modal Framing Analysis of News
• Authors: Arnav Arora, Srishti Yadav, Maria Antoniak, Serge Belongie, Isabelle Augenstein,
• Abstract summary: We present a method for conducting multi-modal, multi-label framing analysis at scale using large (vision-language) models.<n>We identify highly partisan framing of topics with issue-specific frame analysis found in prior qualitative work.<n>We demonstrate a method for doing scalable integrative framing analysis of both text and image in news, providing a more complete picture for understanding media bias.
• Score: 41.39617209053842
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automated frame analysis of political communication is a popular task in computational social science that is used to study how authors select aspects of a topic to frame its reception. So far, such studies have been narrow, in that they use a fixed set of pre-defined frames and focus only on the text, ignoring the visual contexts in which those texts appear. Especially for framing in the news, this leaves out valuable information about editorial choices, which include not just the written article but also accompanying photographs. To overcome such limitations, we present a method for conducting multi-modal, multi-label framing analysis at scale using large (vision-)language models. Grounding our work in framing theory, we extract latent meaning embedded in images used to convey a certain point and contrast that to the text by comparing the respective frames used. We also identify highly partisan framing of topics with issue-specific frame analysis found in prior qualitative work. We demonstrate a method for doing scalable integrative framing analysis of both text and image in news, providing a more complete picture for understanding media bias.

Related papers

• SciMMIR: Benchmarking Scientific Multi-modal Information Retrieval [64.03631654052445]
  Current benchmarks for evaluating MMIR performance in image-text pairing within the scientific domain show a notable gap. We develop a specialised scientific MMIR benchmark by leveraging open-access paper collections. This benchmark comprises 530K meticulously curated image-text pairs, extracted from figures and tables with detailed captions in scientific documents.
  arXiv  Detail & Related papers  (2024-01-24T14:23:12Z)
• Leveraging Open-Vocabulary Diffusion to Camouflaged Instance Segmentation [59.78520153338878]
  Text-to-image diffusion techniques have shown exceptional capability of producing high-quality images from text descriptions. We propose a method built upon a state-of-the-art diffusion model, empowered by open-vocabulary to learn multi-scale textual-visual features for camouflaged object representations.
  arXiv  Detail & Related papers  (2023-12-29T07:59:07Z)
• FrameFinder: Explorative Multi-Perspective Framing Extraction from News Headlines [3.3181276611945263]
  We present FrameFinder, an open tool for extracting and analyzing frames in textual data. By analyzing the well-established gun violence frame corpus, we demonstrate the merits of our proposed solution.
  arXiv  Detail & Related papers  (2023-12-14T14:41:37Z)
• Improving Multimodal Classification of Social Media Posts by Leveraging Image-Text Auxiliary Tasks [38.943074586111564]
  We present an extensive study on the effectiveness of using two auxiliary losses jointly with the main task during fine-tuning multimodal models. First, Image-Text Contrastive (ITC) is designed to minimize the distance between image-text representations within a post. Second, Image-Text Matching (ITM) enhances the model's ability to understand the semantic relationship between images and text.
  arXiv  Detail & Related papers  (2023-09-14T15:30:59Z)
• Image Captioning with Multi-Context Synthetic Data [16.961112970612447]
  Large models have excelled in producing high-quality images and text. We present an innovative pipeline that introduces multi-context data generation. Our model is exclusively trained on synthetic image-text pairs crafted through this process.
  arXiv  Detail & Related papers  (2023-05-29T13:18:59Z)
• Coarse-to-Fine Contrastive Learning in Image-Text-Graph Space for Improved Vision-Language Compositionality [50.48859793121308]
  Contrastively trained vision-language models have achieved remarkable progress in vision and language representation learning. Recent research has highlighted severe limitations in their ability to perform compositional reasoning over objects, attributes, and relations.
  arXiv  Detail & Related papers  (2023-05-23T08:28:38Z)
• Image-Specific Information Suppression and Implicit Local Alignment for Text-based Person Search [61.24539128142504]
  Text-based person search (TBPS) is a challenging task that aims to search pedestrian images with the same identity from an image gallery given a query text. Most existing methods rely on explicitly generated local parts to model fine-grained correspondence between modalities. We propose an efficient joint Multi-level Alignment Network (MANet) for TBPS, which can learn aligned image/text feature representations between modalities at multiple levels.
  arXiv  Detail & Related papers  (2022-08-30T16:14:18Z)
• NewsStories: Illustrating articles with visual summaries [49.924916589209374]
  We introduce a large-scale multimodal dataset containing over 31M articles, 22M images and 1M videos. We show that state-of-the-art image-text alignment methods are not robust to longer narratives with multiple images. We introduce an intuitive baseline that outperforms these methods on zero-shot image-set retrieval by 10% on the GoodNews dataset.
  arXiv  Detail & Related papers  (2022-07-26T17:34:11Z)
• From Show to Tell: A Survey on Image Captioning [48.98681267347662]
  Connecting Vision and Language plays an essential role in Generative Intelligence. Research in image captioning has not reached a conclusive answer yet. This work aims at providing a comprehensive overview and categorization of image captioning approaches.
  arXiv  Detail & Related papers  (2021-07-14T18:00:54Z)
• An AutoML-based Approach to Multimodal Image Sentiment Analysis [1.0499611180329804]
  We propose a method that combines both textual and image individual sentiment analysis into a final fused classification based on AutoML. Our method achieved state-of-the-art performance in the B-T4SA dataset, with 95.19% accuracy.
  arXiv  Detail & Related papers  (2021-02-16T11:28:50Z)
• Multi-Modal Reasoning Graph for Scene-Text Based Fine-Grained Image Classification and Retrieval [8.317191999275536]
  This paper focuses on leveraging multi-modal content in the form of visual and textual cues to tackle the task of fine-grained image classification and retrieval. We employ a Graph Convolutional Network to perform multi-modal reasoning and obtain relationship-enhanced features by learning a common semantic space between salient objects and text found in an image.
  arXiv  Detail & Related papers  (2020-09-21T12:31:42Z)
• FrameAxis: Characterizing Microframe Bias and Intensity with Word Embedding [8.278618225536807]
  We propose FrameAxis, a method for characterizing documents by identifying the most relevant semantic axes ("microframes") FrameAxis is designed to quantitatively tease out two important dimensions of how microframes are used in the text. We demonstrate that microframes with the highest bias and intensity well align with sentiment, topic, and partisan spectrum by applying FrameAxis to multiple datasets from restaurant reviews to political news.
  arXiv  Detail & Related papers  (2020-02-20T08:01:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.