Rethinking movie genre classification with fine-grained semantic
clustering
- URL: http://arxiv.org/abs/2012.02639v3
- Date: Wed, 20 Jan 2021 16:46:09 GMT
- Title: Rethinking movie genre classification with fine-grained semantic
clustering
- Authors: Edward Fish, Jon Weinbren, Andrew Gilbert
- Abstract summary: We find large semantic variations between movies within a single genre definition.
We expand these 'coarse' genre labels by identifying 'fine-grained' semantic information.
Our approach is demonstrated on a newly introduced multi-modal 37,866,450 frame, 8,800 movie trailer dataset.
- Score: 5.54966601302758
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Movie genre classification is an active research area in machine learning.
However, due to the limited labels available, there can be large semantic
variations between movies within a single genre definition. We expand these
'coarse' genre labels by identifying 'fine-grained' semantic information within
the multi-modal content of movies. By leveraging pre-trained 'expert' networks,
we learn the influence of different combinations of modes for multi-label genre
classification. Using a contrastive loss, we continue to fine-tune this
'coarse' genre classification network to identify high-level intertextual
similarities between the movies across all genre labels. This leads to a more
'fine-grained' and detailed clustering, based on semantic similarities while
still retaining some genre information. Our approach is demonstrated on a newly
introduced multi-modal 37,866,450 frame, 8,800 movie trailer dataset,
MMX-Trailer-20, which includes pre-computed audio, location, motion, and image
embeddings.
Related papers
- Tomato, Tomahto, Tomate: Measuring the Role of Shared Semantics among Subwords in Multilingual Language Models [88.07940818022468]
We take an initial step on measuring the role of shared semantics among subwords in the encoder-only multilingual language models (mLMs)
We form "semantic tokens" by merging the semantically similar subwords and their embeddings.
inspections on the grouped subwords show that they exhibit a wide range of semantic similarities.
arXiv Detail & Related papers (2024-11-07T08:38:32Z) - Beyond Labels: Leveraging Deep Learning and LLMs for Content Metadata [1.6574413179773761]
Analyzing the metadata can help understand the user preferences to generate personalized recommendations and item cold starting.
We present some of the challenges associated with using genre label information and propose a new way of examining the genre information.
The Genre Spectrum helps capture the various nuanced genres in a title and our offline and online experiments corroborate the effectiveness of the approach.
arXiv Detail & Related papers (2023-09-15T22:11:29Z) - Description-Enhanced Label Embedding Contrastive Learning for Text
Classification [65.01077813330559]
Self-Supervised Learning (SSL) in model learning process and design a novel self-supervised Relation of Relation (R2) classification task.
Relation of Relation Learning Network (R2-Net) for text classification, in which text classification and R2 classification are treated as optimization targets.
external knowledge from WordNet to obtain multi-aspect descriptions for label semantic learning.
arXiv Detail & Related papers (2023-06-15T02:19:34Z) - Reliable Representations Learning for Incomplete Multi-View Partial Multi-Label Classification [78.15629210659516]
In this paper, we propose an incomplete multi-view partial multi-label classification network named RANK.
We break through the view-level weights inherent in existing methods and propose a quality-aware sub-network to dynamically assign quality scores to each view of each sample.
Our model is not only able to handle complete multi-view multi-label datasets, but also works on datasets with missing instances and labels.
arXiv Detail & Related papers (2023-03-30T03:09:25Z) - Complex Network-Based Approach for Feature Extraction and Classification
of Musical Genres [0.0]
This work presents a feature extraction method for the automatic classification of musical genres.
The proposed method initially converts the musics into sequences of musical notes and then maps the sequences as complex networks.
Topological measurements are extracted to characterize the network topology, which composes a feature vector that applies to the classification of musical genres.
arXiv Detail & Related papers (2021-10-09T22:23:33Z) - Multilevel profiling of situation and dialogue-based deep networks for
movie genre classification using movie trailers [7.904790547594697]
We propose a novel multi-modality: situation, dialogue, and metadata-based movie genre classification framework.
We develop the English movie trailer dataset (EMTD), which contains 2000 Hollywood movie trailers belonging to five popular genres.
arXiv Detail & Related papers (2021-09-14T07:33:56Z) - Multi-layered Semantic Representation Network for Multi-label Image
Classification [8.17894017454724]
Multi-label image classification (MLIC) is a fundamental and practical task, which aims to assign multiple possible labels to an image.
In recent years, many deep convolutional neural network (CNN) based approaches have been proposed which model label correlations.
This paper advances this research direction by improving the modeling of label correlations and the learning of semantic representations.
arXiv Detail & Related papers (2021-06-22T08:04:22Z) - Dynamic Semantic Matching and Aggregation Network for Few-shot Intent
Detection [69.2370349274216]
Few-shot Intent Detection is challenging due to the scarcity of available annotated utterances.
Semantic components are distilled from utterances via multi-head self-attention.
Our method provides a comprehensive matching measure to enhance representations of both labeled and unlabeled instances.
arXiv Detail & Related papers (2020-10-06T05:16:38Z) - A multimodal approach for multi-label movie genre classification [2.1342631813973507]
We created a dataset composed of trailer video clips, subtitles, synopses, and movie posters from 152,622 movie titles from The Movie Database.
The dataset was carefully curated and organized, and it was also made available as a contribution of this work.
arXiv Detail & Related papers (2020-06-01T00:51:39Z) - Hierarchical Image Classification using Entailment Cone Embeddings [68.82490011036263]
We first inject label-hierarchy knowledge into an arbitrary CNN-based classifier.
We empirically show that availability of such external semantic information in conjunction with the visual semantics from images boosts overall performance.
arXiv Detail & Related papers (2020-04-02T10:22:02Z) - Adaptive Semantic-Visual Tree for Hierarchical Embeddings [67.01307058209709]
We propose a hierarchical adaptive semantic-visual tree to depict the architecture of merchandise categories.
The tree evaluates semantic similarities between different semantic levels and visual similarities within the same semantic class simultaneously.
At each level, we set different margins based on the semantic hierarchy and incorporate them as prior information to learn a fine-grained feature embedding.
arXiv Detail & Related papers (2020-03-08T03:36:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.