Unlocking ImageNet's Multi-Object Nature: Automated Large-Scale Multilabel Annotation

• URL: http://arxiv.org/abs/2603.05729v1
• Date: Thu, 05 Mar 2026 22:36:48 GMT
• Title: Unlocking ImageNet's Multi-Object Nature: Automated Large-Scale Multilabel Annotation
• Abstract summary: We present an automated pipeline to convert the ImageNet training set into a multi-label dataset, without human annotations.<n>Our labels show strong alignment with human judgment in qualitative evaluations.<n>Compared to traditional single-label scheme, models trained with our multi-label supervision achieve consistently better in-domain accuracy.
• Score: 22.555839995810988
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The original ImageNet benchmark enforces a single-label assumption, despite many images depicting multiple objects. This leads to label noise and limits the richness of the learning signal. Multi-label annotations more accurately reflect real-world visual scenes, where multiple objects co-occur and contribute to semantic understanding, enabling models to learn richer and more robust representations. While prior efforts (e.g., ReaL, ImageNetv2) have improved the validation set, there has not yet been a scalable, high-quality multi-label annotation for the training set. To this end, we present an automated pipeline to convert the ImageNet training set into a multi-label dataset, without human annotations. Using self-supervised Vision Transformers, we perform unsupervised object discovery, select regions aligned with original labels to train a lightweight classifier, and apply it to all regions to generate coherent multi-label annotations across the dataset. Our labels show strong alignment with human judgment in qualitative evaluations and consistently improve performance across quantitative benchmarks. Compared to traditional single-label scheme, models trained with our multi-label supervision achieve consistently better in-domain accuracy across architectures (up to +2.0 top-1 accuracy on ReaL and +1.5 on ImageNet-V2) and exhibit stronger transferability to downstream tasks (up to +4.2 and +2.3 mAP on COCO and VOC, respectively). These results underscore the importance of accurate multi-label annotations for enhancing both classification performance and representation learning. Project code and the generated multi-label annotations are available at https://github.com/jchen175/MultiLabel-ImageNet.

Related papers

• Adapting Vision-Language Models from Iconic to Inclusive for Multi-Label Recognition Without Labels [14.373553090930612]
  Vision-language models (VLMs) enable zero-shot recognition without labeled data.<n>These models often prioritize the most iconic object and omit other contextual positives.<n>We propose an unsupervised framework that adapts VLMs from iconic recognition toward inclusive understanding.
  arXiv  Detail & Related papers  (2026-06-10T03:41:50Z)
• The Impact of the Single-Label Assumption in Image Recognition Benchmarking [1.4828022319975973]
  Deep neural networks (DNNs) are typically evaluated under the assumption that each image has a single correct label.<n>Many images in benchmarks like ImageNet contain multiple valid labels, creating a mismatch between evaluation protocols and the actual complexity of visual data.<n>We rigorously assess the impact of multi-label characteristics on reported accuracy gaps.
  arXiv  Detail & Related papers  (2024-12-24T12:55:31Z)
• Vision-Language Pseudo-Labels for Single-Positive Multi-Label Learning [11.489541220229798]
  In general multi-label learning, a model learns to predict multiple labels or categories for a single input image. This is in contrast with standard multi-class image classification, where the task is predicting a single label from many possible labels for an image.
  arXiv  Detail & Related papers  (2023-10-24T16:36:51Z)
• DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotations [79.433122872973]
  Multi-label image recognition in the low-label regime is a task of great challenge and practical significance. We leverage the powerful alignment between textual and visual features pretrained with millions of auxiliary image-text pairs. We introduce an efficient and effective framework called Evidence-guided Dual Context Optimization (DualCoOp++)
  arXiv  Detail & Related papers  (2023-08-03T17:33:20Z)
• DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations [61.41339201200135]
  We propose Dual Context Optimization (DualCoOp) as a unified framework for partial-label MLR and zero-shot MLR. Since DualCoOp only introduces a very light learnable overhead upon the pretrained vision-language framework, it can quickly adapt to multi-label recognition tasks.
  arXiv  Detail & Related papers  (2022-06-20T02:36:54Z)
• Masked Unsupervised Self-training for Zero-shot Image Classification [98.23094305347709]
  Masked Unsupervised Self-Training (MUST) is a new approach which leverages two different and complimentary sources of supervision: pseudo-labels and raw images. MUST improves upon CLIP by a large margin and narrows the performance gap between unsupervised and supervised classification.
  arXiv  Detail & Related papers  (2022-06-07T02:03:06Z)
• Diverse Instance Discovery: Vision-Transformer for Instance-Aware Multi-Label Image Recognition [24.406654146411682]
  Vision Transformer (ViT) is the research base for this paper. Our goal is to leverage ViT's patch tokens and self-attention mechanism to mine rich instances in multi-label images. We propose a weakly supervised object localization-based approach to extract multi-scale local features.
  arXiv  Detail & Related papers  (2022-04-22T14:38:40Z)
• Multi-label Iterated Learning for Image Classification with Label Ambiguity [3.5736176624479654]
  We propose multi-label iterated learning (MILe) to incorporate the inductive biases of multi-label learning from single labels. MILe is a simple yet effective procedure that builds a multi-label description of the image by propagating binary predictions. We show that MILe is effective reducing label noise, achieving state-of-the-art performance on real-world large-scale noisy data such as WebVision.
  arXiv  Detail & Related papers  (2021-11-23T22:10:00Z)
• Multi-Label Image Classification with Contrastive Learning [57.47567461616912]
  We show that a direct application of contrastive learning can hardly improve in multi-label cases. We propose a novel framework for multi-label classification with contrastive learning in a fully supervised setting.
  arXiv  Detail & Related papers  (2021-07-24T15:00:47Z)
• Towards Good Practices for Efficiently Annotating Large-Scale Image Classification Datasets [90.61266099147053]
  We investigate efficient annotation strategies for collecting multi-class classification labels for a large collection of images. We propose modifications and best practices aimed at minimizing human labeling effort. Simulated experiments on a 125k image subset of the ImageNet100 show that it can be annotated to 80% top-1 accuracy with 0.35 annotations per image on average.
  arXiv  Detail & Related papers  (2021-04-26T16:29:32Z)
• Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization [112.68171734288237]
  We propose a novel framework for discriminative pixel-level tasks using a generative model of both images and labels. We learn a generative adversarial network that captures the joint image-label distribution and is trained efficiently using a large set of unlabeled images. We demonstrate strong in-domain performance compared to several baselines, and are the first to showcase extreme out-of-domain generalization.
  arXiv  Detail & Related papers  (2021-04-12T21:41:25Z)
• Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels [34.13899937264952]
  ImageNet has been arguably the most popular image classification benchmark, but it is also the one with a significant level of label noise. Recent studies have shown that many samples contain multiple classes, despite being assumed to be a single-label benchmark. We argue that the mismatch between single-label annotations and effectively multi-label images is equally, if not more, problematic in the training setup, where random crops are applied.
  arXiv  Detail & Related papers  (2021-01-13T11:55:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.