Harnessing Self-Supervised Features for Art Classification

• URL: http://arxiv.org/abs/2605.18974v1
• Date: Mon, 18 May 2026 18:00:59 GMT
• Title: Harnessing Self-Supervised Features for Art Classification
• Abstract summary: Classifying artworks presents a significant challenge due to the complex interplay of fine-grained details and abstract features.<n>This paper presents a systematic investigation of the effectiveness of supervised and self-supervised backbones for both artwork classification and retrieval.
• Score: 6.193074790121984
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Classifying artworks presents a significant challenge due to the complex interplay of fine-grained details and abstract features that condition the style or genre of an artwork. This paper presents a systematic investigation of the effectiveness of supervised and self-supervised backbones as feature extractors for both artwork classification and retrieval, with a particular focus on paintings. We conduct an extensive experimental evaluation using the DINO family and CLIP models, assessing multiple classification strategies and feature representations. Our results demonstrate that employing a self-supervised backbone leads to consistent improvements in artwork classification performance. Moreover, our work provides insights into the applicability of classification and retrieval modules in real-world applications, such as virtual reality (VR) applications that support museum navigation.

Related papers

• Learning Task-Agnostic Representations through Multi-Teacher Distillation [59.488314181423284]
  We introduce a task-agnostic framework based on a majority vote" objective function.<n>We demonstrate that this function is bounded by the mutual information between student and teachers' embeddings.<n>Our method effectively leverages teacher diversity, resulting in representations enabling better performance for a wide range of downstream tasks.
  arXiv  Detail & Related papers  (2025-10-21T14:36:33Z)
• Instruction-Guided Fusion of Multi-Layer Visual Features in Large Vision-Language Models [50.98559225639266]
  We investigate the contributions of visual features from different encoder layers using 18 benchmarks spanning 6 task categories.<n>Our findings reveal that multilayer features provide complementary strengths with varying task dependencies, and uniform fusion leads to suboptimal performance.<n>We propose the instruction-guided vision aggregator, a module that dynamically integrates multi-layer visual features based on textual instructions.
  arXiv  Detail & Related papers  (2024-12-26T05:41:31Z)
• A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trends [67.43992456058541]
  Image restoration (IR) seeks to recover high-quality images from degraded observations caused by a wide range of factors, including noise, blur, compression, and adverse weather.<n>Traditional IR methods have made notable progress by targeting individual degradation types, but their specialization often comes at the cost of generalization.<n>The all-in-one image restoration paradigm has recently emerged, offering a unified framework that adeptly addresses multiple degradation types.
  arXiv  Detail & Related papers  (2024-10-19T11:11:09Z)
• Style-based Clustering of Visual Artworks and the Play of Neural Style-Representations [2.4374097382908477]
  Clustering artworks based on style can have many potential real-world applications like art recommendations, style-based search and retrieval.<n>We argue that clustering artworks based on style is largely an unaddressed problem.
  arXiv  Detail & Related papers  (2024-09-12T17:44:07Z)
• Zero-Shot Object-Centric Representation Learning [72.43369950684057]
  We study current object-centric methods through the lens of zero-shot generalization. We introduce a benchmark comprising eight different synthetic and real-world datasets. We find that training on diverse real-world images improves transferability to unseen scenarios.
  arXiv  Detail & Related papers  (2024-08-17T10:37:07Z)
• An Information Compensation Framework for Zero-Shot Skeleton-based Action Recognition [49.45660055499103]
  Zero-shot human skeleton-based action recognition aims to construct a model that can recognize actions outside the categories seen during training. Previous research has focused on aligning sequences' visual and semantic spatial distributions. We introduce a new loss function sampling method to obtain a tight and robust representation.
  arXiv  Detail & Related papers  (2024-06-02T06:53:01Z)
• Loss Functions in the Era of Semantic Segmentation: A Survey and Outlook [11.119967679567587]
  Loss functions are crucial for shaping the development of deep learning-based segmentation algorithms. We provide a novel taxonomy and review of how these loss functions are customized and leveraged in image segmentation. We conclude this review by identifying current challenges and unveiling future research opportunities.
  arXiv  Detail & Related papers  (2023-12-08T22:06:05Z)
• Exploiting CLIP-based Multi-modal Approach for Artwork Classification and Retrieval [29.419743866789187]
  We perform exhaustive experiments on the NoisyArt dataset which is a dataset of artwork images crawled from public resources on the web. On such dataset CLIP achieves impressive results on (zero-shot) classification and promising results in both artwork-to-artwork and description-to-artwork domain.
  arXiv  Detail & Related papers  (2023-09-21T14:29:44Z)
• Knowledge-Aware Prompt Tuning for Generalizable Vision-Language Models [64.24227572048075]
  We propose a Knowledge-Aware Prompt Tuning (KAPT) framework for vision-language models. Our approach takes inspiration from human intelligence in which external knowledge is usually incorporated into recognizing novel categories of objects.
  arXiv  Detail & Related papers  (2023-08-22T04:24:45Z)
• Leveraging Computer Vision Application in Visual Arts: A Case Study on the Use of Residual Neural Network to Classify and Analyze Baroque Paintings [0.0]
  In this case study, we focus on the classification of a selected painting 'Portrait of the Painter Charles Bruni' by Johann Kupetzky. We show that the features extracted during residual network training can be useful for image retrieval within search systems in online art collections.
  arXiv  Detail & Related papers  (2022-10-27T10:15:36Z)
• An Empirical Investigation of Representation Learning for Imitation [76.48784376425911]
  Recent work in vision, reinforcement learning, and NLP has shown that auxiliary representation learning objectives can reduce the need for large amounts of expensive, task-specific data. We propose a modular framework for constructing representation learning algorithms, then use our framework to evaluate the utility of representation learning for imitation.
  arXiv  Detail & Related papers  (2022-05-16T11:23:42Z)
• Representation-Centric Survey of Skeletal Action Recognition and the ANUBIS Benchmark [43.00059447663327]
  3D skeleton-based human action recognition has emerged as a powerful alternative to traditional RGB and depth-based approaches.<n>Despite remarkable progress, current research remains fragmented across diverse input representations.<n>ANUBIS is a large-scale, challenging skeleton action dataset designed to address critical gaps in existing benchmarks.
  arXiv  Detail & Related papers  (2022-05-04T14:03:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.