Analyzing Images for Music Recommendation
- URL: http://arxiv.org/abs/2105.07135v1
- Date: Sat, 15 May 2021 04:14:47 GMT
- Title: Analyzing Images for Music Recommendation
- Authors: Anant Baijal, Vivek Agarwal and Danny Hyun
- Abstract summary: The proposed image analysis method treats an artwork image differently from a photograph image.
The Mean Opinion Score (MOS) obtained from subjective assessments of the respective image and recommended music pairs supports the effectiveness of our approach.
- Score: 1.2891210250935146
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Experiencing images with suitable music can greatly enrich the overall user
experience. The proposed image analysis method treats an artwork image
differently from a photograph image. Automatic image classification is
performed using deep-learning based models. An illustrative analysis showcasing
the ability of our deep-models to inherently learn and utilize perceptually
relevant features when classifying artworks is also presented. The Mean Opinion
Score (MOS) obtained from subjective assessments of the respective image and
recommended music pairs supports the effectiveness of our approach.
Related papers
- Impressions: Understanding Visual Semiotics and Aesthetic Impact [66.40617566253404]
We present Impressions, a novel dataset through which to investigate the semiotics of images.
We show that existing multimodal image captioning and conditional generation models struggle to simulate plausible human responses to images.
This dataset significantly improves their ability to model impressions and aesthetic evaluations of images through fine-tuning and few-shot adaptation.
arXiv Detail & Related papers (2023-10-27T04:30:18Z) - Image Aesthetics Assessment via Learnable Queries [59.313054821874864]
We propose the Image Aesthetics Assessment via Learnable Queries (IAA-LQ) approach.
It adapts learnable queries to extract aesthetic features from pre-trained image features obtained from a frozen image encoder.
Experiments on real-world data demonstrate the advantages of IAA-LQ, beating the best state-of-the-art method by 2.2% and 2.1% in terms of SRCC and PLCC, respectively.
arXiv Detail & Related papers (2023-09-06T09:42:16Z) - ARTxAI: Explainable Artificial Intelligence Curates Deep Representation
Learning for Artistic Images using Fuzzy Techniques [11.286457041998569]
We show how the features obtained from different tasks in artistic image classification are suitable to solve other ones of similar nature.
We propose an explainable artificial intelligence method to map known visual traits of an image with the features used by the deep learning model.
arXiv Detail & Related papers (2023-08-29T13:15:13Z) - VILA: Learning Image Aesthetics from User Comments with Vision-Language
Pretraining [53.470662123170555]
We propose learning image aesthetics from user comments, and exploring vision-language pretraining methods to learn multimodal aesthetic representations.
Specifically, we pretrain an image-text encoder-decoder model with image-comment pairs, using contrastive and generative objectives to learn rich and generic aesthetic semantics without human labels.
Our results show that our pretrained aesthetic vision-language model outperforms prior works on image aesthetic captioning over the AVA-Captions dataset.
arXiv Detail & Related papers (2023-03-24T23:57:28Z) - Leveraging Computer Vision Application in Visual Arts: A Case Study on
the Use of Residual Neural Network to Classify and Analyze Baroque Paintings [0.0]
In this case study, we focus on the classification of a selected painting 'Portrait of the Painter Charles Bruni' by Johann Kupetzky.
We show that the features extracted during residual network training can be useful for image retrieval within search systems in online art collections.
arXiv Detail & Related papers (2022-10-27T10:15:36Z) - Exploring CLIP for Assessing the Look and Feel of Images [87.97623543523858]
We introduce Contrastive Language-Image Pre-training (CLIP) models for assessing both the quality perception (look) and abstract perception (feel) of images in a zero-shot manner.
Our results show that CLIP captures meaningful priors that generalize well to different perceptual assessments.
arXiv Detail & Related papers (2022-07-25T17:58:16Z) - Composition and Style Attributes Guided Image Aesthetic Assessment [66.60253358722538]
We propose a method for the automatic prediction of the aesthetics of an image.
The proposed network includes: a pre-trained network for semantic features extraction (the Backbone); a Multi Layer Perceptron (MLP) network that relies on the Backbone features for the prediction of image attributes (the AttributeNet)
Given an image, the proposed multi-network is able to predict: style and composition attributes, and aesthetic score distribution.
arXiv Detail & Related papers (2021-11-08T17:16:38Z) - Learning Conditional Knowledge Distillation for Degraded-Reference Image
Quality Assessment [157.1292674649519]
We propose a practical solution named degraded-reference IQA (DR-IQA)
DR-IQA exploits the inputs of IR models, degraded images, as references.
Our results can even be close to the performance of full-reference settings.
arXiv Detail & Related papers (2021-08-18T02:35:08Z) - A Survey of Hand Crafted and Deep Learning Methods for Image Aesthetic
Assessment [2.9005223064604078]
This paper presents a literature review of the recent techniques of automatic image aesthetics assessment.
A large number of traditional hand crafted and deep learning based approaches are reviewed.
arXiv Detail & Related papers (2021-03-22T07:00:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.