Motif Mining: Finding and Summarizing Remixed Image Content
- URL: http://arxiv.org/abs/2203.08327v2
- Date: Thu, 17 Mar 2022 14:54:55 GMT
- Title: Motif Mining: Finding and Summarizing Remixed Image Content
- Authors: William Theisen, Daniel Gonzalez Cedre, Zachariah Carmichael, Daniel
Moreira, Tim Weninger, and Walter Scheirer
- Abstract summary: We introduce the idea of motif mining - the process of finding and summarizing remixed image content in large collections of unlabeled and unsorted data.
Experiments are conducted on three meme-style data sets, including a newly collected set associated with the information war in the Russo-Ukrainian conflict.
The proposed motif mining approach is able to identify related remixed content that, when compared to similar approaches, more closely aligns with the preferences and expectations of human observers.
- Score: 7.0095206215942785
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: On the internet, images are no longer static; they have become dynamic
content. Thanks to the availability of smartphones with cameras and easy-to-use
editing software, images can be remixed (i.e., redacted, edited, and recombined
with other content) on-the-fly and with a world-wide audience that can repeat
the process. From digital art to memes, the evolution of images through time is
now an important topic of study for digital humanists, social scientists, and
media forensics specialists. However, because typical data sets in computer
vision are composed of static content, the development of automated algorithms
to analyze remixed content has been limited. In this paper, we introduce the
idea of Motif Mining - the process of finding and summarizing remixed image
content in large collections of unlabeled and unsorted data. In this paper,
this idea is formalized and a reference implementation is introduced.
Experiments are conducted on three meme-style data sets, including a newly
collected set associated with the information war in the Russo-Ukrainian
conflict. The proposed motif mining approach is able to identify related
remixed content that, when compared to similar approaches, more closely aligns
with the preferences and expectations of human observers.
Related papers
- Zero-shot Image Editing with Reference Imitation [50.75310094611476]
We present a new form of editing, termed imitative editing, to help users exercise their creativity more conveniently.
We propose a generative training framework, dubbed MimicBrush, which randomly selects two frames from a video clip, masks some regions of one frame, and learns to recover the masked regions using the information from the other frame.
We experimentally show the effectiveness of our method under various test cases as well as its superiority over existing alternatives.
arXiv Detail & Related papers (2024-06-11T17:59:51Z) - StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generation [117.13475564834458]
We propose a new way of self-attention calculation, termed Consistent Self-Attention.
To extend our method to long-range video generation, we introduce a novel semantic space temporal motion prediction module.
By merging these two novel components, our framework, referred to as StoryDiffusion, can describe a text-based story with consistent images or videos.
arXiv Detail & Related papers (2024-05-02T16:25:16Z) - Enhancing Historical Image Retrieval with Compositional Cues [3.2276097734075426]
We introduce a crucial factor from computational aesthetics, namely image composition, into this topic.
By explicitly integrating composition-related information extracted by CNN into the designed retrieval model, our method considers both the image's composition rules and semantic information.
arXiv Detail & Related papers (2024-03-21T10:51:19Z) - Diffusion Model-Based Image Editing: A Survey [46.244266782108234]
Denoising diffusion models have emerged as a powerful tool for various image generation and editing tasks.
We provide an exhaustive overview of existing methods using diffusion models for image editing.
To further evaluate the performance of text-guided image editing algorithms, we propose a systematic benchmark, EditEval.
arXiv Detail & Related papers (2024-02-27T14:07:09Z) - A Dataset and Benchmark for Copyright Infringement Unlearning from Text-to-Image Diffusion Models [52.49582606341111]
Copyright law confers creators the exclusive rights to reproduce, distribute, and monetize their creative works.
Recent progress in text-to-image generation has introduced formidable challenges to copyright enforcement.
We introduce a novel pipeline that harmonizes CLIP, ChatGPT, and diffusion models to curate a dataset.
arXiv Detail & Related papers (2024-01-04T11:14:01Z) - Decoupled Textual Embeddings for Customized Image Generation [62.98933630971543]
Customized text-to-image generation aims to learn user-specified concepts with a few images.
Existing methods usually suffer from overfitting issues and entangle the subject-unrelated information with the learned concept.
We propose the DETEX, a novel approach that learns the disentangled concept embedding for flexible customized text-to-image generation.
arXiv Detail & Related papers (2023-12-19T03:32:10Z) - Stellar: Systematic Evaluation of Human-Centric Personalized
Text-to-Image Methods [52.806258774051216]
We focus on text-to-image systems that input a single image of an individual and ground the generation process along with text describing the desired visual context.
We introduce a standardized dataset (Stellar) that contains personalized prompts coupled with images of individuals that is an order of magnitude larger than existing relevant datasets and where rich semantic ground-truth annotations are readily available.
We derive a simple yet efficient, personalized text-to-image baseline that does not require test-time fine-tuning for each subject and which sets quantitatively and in human trials a new SoTA.
arXiv Detail & Related papers (2023-12-11T04:47:39Z) - Retargeting Visual Data with Deformation Fields [14.716129471469992]
Seam carving is an image editing method that enable content-aware, including operations like removing objects.
We propose to learn a deformation with a neural network that keeps the output plausible while trying to deform it in places with low information content radiance.
Experiments conducted on different visual data show that our method achieves better content-aware compared to previous methods.
arXiv Detail & Related papers (2023-11-22T10:27:19Z) - Unsupervised Compositional Concepts Discovery with Text-to-Image
Generative Models [80.75258849913574]
In this paper, we consider the inverse problem -- given a collection of different images, can we discover the generative concepts that represent each image?
We present an unsupervised approach to discover generative concepts from a collection of images, disentangling different art styles in paintings, objects, and lighting from kitchen scenes, and discovering image classes given ImageNet images.
arXiv Detail & Related papers (2023-06-08T17:02:15Z) - Comprehensive Dataset of Face Manipulations for Development and
Evaluation of Forensic Tools [0.6091702876917281]
We create a challenge dataset of edited facial images to assist the research community in developing novel approaches to address and classify the authenticity of digital media.
The goals of our dataset are to address the following challenge questions: (1) Can we determine the authenticity of a given image (edit detection)?
Our hope is that our prepared evaluation protocol will assist researchers in improving the state-of-the-art in image forensics as they pertain to these challenges.
arXiv Detail & Related papers (2022-08-24T21:17:28Z) - Learning Transformation-Aware Embeddings for Image Forensics [15.484408315588569]
Image Provenance Analysis aims at discovering relationships among different manipulated image versions that share content.
One of the main sub-problems for provenance analysis that has not yet been addressed directly is the edit ordering of images that share full content or are near-duplicates.
This paper introduces a novel deep learning-based approach to provide a plausible ordering to images that have been generated from a single image through transformations.
arXiv Detail & Related papers (2020-01-13T22:01:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.