MMCIG: Multimodal Cover Image Generation for Text-only Documents and Its Dataset Construction via Pseudo-labeling

• URL: http://arxiv.org/abs/2508.17199v1
• Date: Sun, 24 Aug 2025 03:24:35 GMT
• Title: MMCIG: Multimodal Cover Image Generation for Text-only Documents and Its Dataset Construction via Pseudo-labeling
• Authors: Hyeyeon Kim, Sungwoo Han, Jingun Kwon, Hidetaka Kamigaito, Manabu Okumura,
• Abstract summary: We introduce a novel cover image generation task that produces both a concise summary and a visually corresponding image from a given text-only document.<n>Because no existing datasets are available for this task, we propose a multimodal pseudo-labeling method to construct high-quality datasets at low cost.
• Score: 41.089727903080885
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this study, we introduce a novel cover image generation task that produces both a concise summary and a visually corresponding image from a given text-only document. Because no existing datasets are available for this task, we propose a multimodal pseudo-labeling method to construct high-quality datasets at low cost. We first collect documents that contain multiple images with their captions, and their summaries by excluding factually inconsistent instances. Our approach selects one image from the multiple images accompanying the documents. Using the gold summary, we independently rank both the images and their captions. Then, we annotate a pseudo-label for an image when both the image and its corresponding caption are ranked first in their respective rankings. Finally, we remove documents that contain direct image references within texts. Experimental results demonstrate that the proposed multimodal pseudo-labeling method constructs more precise datasets and generates higher quality images than text- and image-only pseudo-labeling methods, which consider captions and images separately. We release our code at: https://github.com/HyeyeeonKim/MMCIG

Related papers

• Transform, Contrast and Tell: Coherent Entity-Aware Multi-Image Captioning [0.65268245109828]
  Coherent entity-aware multi-image captioning aims to generate coherent captions for neighboring images in a news document. This paper proposes a coherent entity-aware multi-image captioning model by making use of coherence relationships.
  arXiv  Detail & Related papers  (2023-02-04T07:50:31Z)
• LANIT: Language-Driven Image-to-Image Translation for Unlabeled Data [39.421312439022316]
  We present a LANguage-driven Image-to-image Translation model, dubbed LANIT. We leverage easy-to-obtain candidate attributes given in texts for a dataset: the similarity between images and attributes indicates per-sample domain labels. Experiments on several standard benchmarks demonstrate that LANIT achieves comparable or superior performance to existing models.
  arXiv  Detail & Related papers  (2022-08-31T14:30:00Z)
• Revising Image-Text Retrieval via Multi-Modal Entailment [25.988058843564335]
  Many-to-many matching phenomenon is quite common in the widely-used image-text retrieval datasets. We propose a multi-modal entailment classifier to determine whether a sentence is entailed by an image plus its linked captions.
  arXiv  Detail & Related papers  (2022-08-22T07:58:54Z)
• NewsStories: Illustrating articles with visual summaries [49.924916589209374]
  We introduce a large-scale multimodal dataset containing over 31M articles, 22M images and 1M videos. We show that state-of-the-art image-text alignment methods are not robust to longer narratives with multiple images. We introduce an intuitive baseline that outperforms these methods on zero-shot image-set retrieval by 10% on the GoodNews dataset.
  arXiv  Detail & Related papers  (2022-07-26T17:34:11Z)
• Image-to-image Translation via Hierarchical Style Disentanglement [115.81148219591387]
  We propose Hierarchical Style Disentanglement (HiSD) to address this issue. Specifically, we organize the labels into a hierarchical tree structure, in which independent tags, exclusive attributes, and disentangled styles are allocated from top to bottom. Both qualitative and quantitative results on the CelebA-HQ dataset verify the ability of the proposed HiSD.
  arXiv  Detail & Related papers  (2021-03-02T03:43:18Z)
• MATCH: Metadata-Aware Text Classification in A Large Hierarchy [60.59183151617578]
  MATCH is an end-to-end framework that leverages both metadata and hierarchy information. We propose different ways to regularize the parameters and output probability of each child label by its parents. Experiments on two massive text datasets with large-scale label hierarchies demonstrate the effectiveness of MATCH.
  arXiv  Detail & Related papers  (2021-02-15T05:23:08Z)
• TediGAN: Text-Guided Diverse Face Image Generation and Manipulation [52.83401421019309]
  TediGAN is a framework for multi-modal image generation and manipulation with textual descriptions. StyleGAN inversion module maps real images to the latent space of a well-trained StyleGAN. visual-linguistic similarity learns the text-image matching by mapping the image and text into a common embedding space. instance-level optimization is for identity preservation in manipulation.
  arXiv  Detail & Related papers  (2020-12-06T16:20:19Z)
• Context-Aware Group Captioning via Self-Attention and Contrastive Features [31.94715153491951]
  We introduce a new task, context-aware group captioning, which aims to describe a group of target images in the context of another group of related reference images. To solve this problem, we propose a framework combining self-attention mechanism with contrastive feature construction. Our datasets are constructed on top of the public Conceptual Captions dataset and our new Stock Captions dataset.
  arXiv  Detail & Related papers  (2020-04-07T20:59:53Z)
• Semi-supervised Learning for Few-shot Image-to-Image Translation [89.48165936436183]
  We propose a semi-supervised method for few-shot image translation, called SEMIT. Our method achieves excellent results on four different datasets using as little as 10% of the source labels.
  arXiv  Detail & Related papers  (2020-03-30T22:46:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.