Salienteye: Maximizing Engagement While Maintaining Artistic Style on Instagram Using Deep Neural Networks

• URL: http://arxiv.org/abs/2006.11403v1
• Date: Sat, 13 Jun 2020 01:58:02 GMT
• Title: Salienteye: Maximizing Engagement While Maintaining Artistic Style on Instagram Using Deep Neural Networks
• Authors: Lili Wang, Ruibo Liu, and Soroush Vosoughi
• Abstract summary: We use transfer learning to adapt Xception, which is a model for object recognition trained on the ImageNet dataset, to the task of engagement prediction. We also use Gram matrices generated from VGG19, another object recognition model trained on ImageNet, for the task of style similarity measurement. Our models can be trained on individual Instagram accounts to create personalized engagement prediction and style similarity models.
• Score: 27.469454386934274
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Instagram has become a great venue for amateur and professional photographers alike to showcase their work. It has, in other words, democratized photography. Generally, photographers take thousands of photos in a session, from which they pick a few to showcase their work on Instagram. Photographers trying to build a reputation on Instagram have to strike a balance between maximizing their followers' engagement with their photos, while also maintaining their artistic style. We used transfer learning to adapt Xception, which is a model for object recognition trained on the ImageNet dataset, to the task of engagement prediction and utilized Gram matrices generated from VGG19, another object recognition model trained on ImageNet, for the task of style similarity measurement on photos posted on Instagram. Our models can be trained on individual Instagram accounts to create personalized engagement prediction and style similarity models. Once trained on their accounts, users can have new photos sorted based on predicted engagement and style similarity to their previous work, thus enabling them to upload photos that not only have the potential to maximize engagement from their followers but also maintain their style of photography. We trained and validated our models on several Instagram accounts, showing it to be adept at both tasks, also outperforming several baseline models and human annotators.

Related papers

• Pro-Pose: Unpaired Full-Body Portrait Synthesis via Canonical UV Maps [30.970209890835793]
  We explore how to create a professional'' version of a person's photograph.<n>A key challenge is to preserve the person's unique identity, face and body features while transforming the photo.<n>Our approach yields high-quality, reposed portraits and achieves strong qualitative and quantitative performance on real-world imagery.
  arXiv  Detail & Related papers  (2025-12-19T00:40:53Z)
• How Many Van Goghs Does It Take to Van Gogh? Finding the Imitation Threshold [50.33428591760124]
  We study the relationship between a concept's frequency in the training dataset and the ability of a model to imitate it. We propose an efficient approach that estimates the imitation threshold without incurring the colossal cost of training multiple models from scratch.
  arXiv  Detail & Related papers  (2024-10-19T06:28:14Z)
• Learning Subject-Aware Cropping by Outpainting Professional Photos [69.0772948657867]
  We propose a weakly-supervised approach to learn what makes a high-quality subject-aware crop from professional stock images. Our insight is to combine a library of stock images with a modern, pre-trained text-to-image diffusion model. We are able to automatically generate a large dataset of cropped-uncropped training pairs to train a cropping model.
  arXiv  Detail & Related papers  (2023-12-19T11:57:54Z)
• Measuring the Success of Diffusion Models at Imitating Human Artists [7.007492782620398]
  We show how to measure a model's ability to imitate specific artists. We use Contrastive Language-Image Pretrained (CLIP) encoders to classify images in a zero-shot fashion. We also show that a sample of the artist's work can be matched to these imitation images with a high degree of statistical reliability.
  arXiv  Detail & Related papers  (2023-07-08T18:31:25Z)
• Identifying Professional Photographers Through Image Quality and Aesthetics in Flickr [0.0]
  This study reveals the lack of suitable data sets in photo and video sharing platforms. We created one of the largest labelled data sets in Flickr with the multimodal data which has been open sourced. We examined the relationship between the aesthetics and technical quality of a picture and the social activity of that picture.
  arXiv  Detail & Related papers  (2023-07-04T14:55:37Z)
• Fashion-model pose recommendation and generation using Machine Learning [0.0]
  This research concentrates on suggesting the fashion personnel a series of similar images based on the input image. The image is segmented into different parts and similar images are suggested for the user. This was achieved by calculating the color histogram of the input image and applying the same for all the images in the dataset.
  arXiv  Detail & Related papers  (2023-02-19T09:12:46Z)
• CtlGAN: Few-shot Artistic Portraits Generation with Contrastive Transfer Learning [77.27821665339492]
  CtlGAN is a new few-shot artistic portraits generation model with a novel contrastive transfer learning strategy. We adapt a pretrained StyleGAN in the source domain to a target artistic domain with no more than 10 artistic faces. We propose a new encoder which embeds real faces into Z+ space and proposes a dual-path training strategy to better cope with the adapted decoder.
  arXiv  Detail & Related papers  (2022-03-16T13:28:17Z)
• InvGAN: Invertible GANs [88.58338626299837]
  InvGAN, short for Invertible GAN, successfully embeds real images to the latent space of a high quality generative model. This allows us to perform image inpainting, merging, and online data augmentation.
  arXiv  Detail & Related papers  (2021-12-08T21:39:00Z)
• Photozilla: A Large-Scale Photography Dataset and Visual Embedding for 20 Photography Styles [0.6308539010172307]
  We introduce a large-scale dataset termed 'Photozilla' that includes over 990k images belonging to 10 different photographic styles. The dataset is then used to train 3 classification models to automatically classify the images into the relevant style. We report an accuracy of over 68% for identifying 10 other distinct types of photography styles.
  arXiv  Detail & Related papers  (2021-06-21T18:45:06Z)
• PhotoApp: Photorealistic Appearance Editing of Head Portraits [97.23638022484153]
  We present an approach for high-quality intuitive editing of the camera viewpoint and scene illumination in a portrait image. Most editing approaches rely on supervised learning using training data captured with setups such as light and camera stages. We design a supervised problem which learns in the latent space of StyleGAN. This combines the best of supervised learning and generative adversarial modeling.
  arXiv  Detail & Related papers  (2021-03-13T08:59:49Z)
• Unselfie: Translating Selfies to Neutral-pose Portraits in the Wild [57.944605468653414]
  In selfies, constraints such as human arm length often make the body pose look unnatural. We introduce $textitunselfie$, a novel photographic transformation that automatically translates a selfie into a neutral-pose portrait. We propose a novel nearest pose search module that makes the reposing task easier and enables the generation of multiple neutral-pose results.
  arXiv  Detail & Related papers  (2020-07-29T19:21:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.