Label Anything: An Interpretable, High-Fidelity and Prompt-Free Annotator

• URL: http://arxiv.org/abs/2502.02972v1
• Date: Wed, 05 Feb 2025 08:14:52 GMT
• Title: Label Anything: An Interpretable, High-Fidelity and Prompt-Free Annotator
• Authors: Wei-Bin Kou, Guangxu Zhu, Rongguang Ye, Shuai Wang, Ming Tang, Yik-Chung Wu,
• Abstract summary: Traditional manual labeling involves high cost to annotate vast amount of required data for training robust model. We propose a Label Anything Model (LAM) serving as an interpretable, high-fidelity, and prompt-free data annotator. LAM can generate high-fidelity annotations (almost 100% in mIoU) for multiple real-world datasets.
• Score: 29.2532061585323
• License:
• Abstract: Learning-based street scene semantic understanding in autonomous driving (AD) has advanced significantly recently, but the performance of the AD model is heavily dependent on the quantity and quality of the annotated training data. However, traditional manual labeling involves high cost to annotate the vast amount of required data for training robust model. To mitigate this cost of manual labeling, we propose a Label Anything Model (denoted as LAM), serving as an interpretable, high-fidelity, and prompt-free data annotator. Specifically, we firstly incorporate a pretrained Vision Transformer (ViT) to extract the latent features. On top of ViT, we propose a semantic class adapter (SCA) and an optimization-oriented unrolling algorithm (OptOU), both with a quite small number of trainable parameters. SCA is proposed to fuse ViT-extracted features to consolidate the basis of the subsequent automatic annotation. OptOU consists of multiple cascading layers and each layer contains an optimization formulation to align its output with the ground truth as closely as possible, though which OptOU acts as being interpretable rather than learning-based blackbox nature. In addition, training SCA and OptOU requires only a single pre-annotated RGB seed image, owing to their small volume of learnable parameters. Extensive experiments clearly demonstrate that the proposed LAM can generate high-fidelity annotations (almost 100% in mIoU) for multiple real-world datasets (i.e., Camvid, Cityscapes, and Apolloscapes) and CARLA simulation dataset.

Related papers

• Aggregate-and-Adapt Natural Language Prompts for Downstream Generalization of CLIP [24.22470408549266]
  We dub prompt embedding as Aggregate-and-Adapted Prompt Embedding (AAPE) AAPE is shown to be able to generalize to different downstream data distributions and tasks, including vision-language understanding tasks. We also show AAPE is particularly helpful to handle non-canonical and OOD examples.
  arXiv  Detail & Related papers  (2024-10-31T07:41:13Z)
• Parameter-Efficient and Memory-Efficient Tuning for Vision Transformer: A Disentangled Approach [87.8330887605381]
  We show how to adapt a pre-trained Vision Transformer to downstream recognition tasks with only a few learnable parameters. We synthesize a task-specific query with a learnable and lightweight module, which is independent of the pre-trained model. Our method achieves state-of-the-art performance under memory constraints, showcasing its applicability in real-world situations.
  arXiv  Detail & Related papers  (2024-07-09T15:45:04Z)
• A Good Foundation is Worth Many Labels: Label-Efficient Panoptic Segmentation [22.440065488051047]
  Key challenge for the widespread application of learning-based models for robotic perception is to significantly reduce the required amount of annotated training data. We exploit the groundwork paved by visual foundation models to train two lightweight network heads for semantic segmentation and object boundary detection. We demonstrate that PASTEL significantly outperforms previous methods for label-efficient segmentation even when using fewer annotations.
  arXiv  Detail & Related papers  (2024-05-29T12:23:29Z)
• Spatio-Temporal Side Tuning Pre-trained Foundation Models for Video-based Pedestrian Attribute Recognition [58.79807861739438]
  Existing pedestrian recognition (PAR) algorithms are mainly developed based on a static image. We propose to understand human attributes using video frames that can fully use temporal information.
  arXiv  Detail & Related papers  (2024-04-27T14:43:32Z)
• Self-Evolved Diverse Data Sampling for Efficient Instruction Tuning [47.02160072880698]
  We introduce a self-evolving mechanism that allows the model itself to actively sample subsets that are equally or even more effective. The key to our data sampling technique lies in the enhancement of diversity in the chosen subsets. Extensive experiments across three datasets and benchmarks demonstrate the effectiveness of DiverseEvol.
  arXiv  Detail & Related papers  (2023-11-14T14:10:40Z)
• Annotator: A Generic Active Learning Baseline for LiDAR Semantic Segmentation [40.803251337200656]
  Annotator is a general and efficient active learning baseline. voxel-centric online selection strategy is tailored to efficiently probe and annotate the salient and exemplar voxel girds within each LiDAR scan. Annotator excels in diverse settings, with a particular focus on active learning (AL), active source-free domain adaptation (ASFDA), and active domain adaptation (ADA)
  arXiv  Detail & Related papers  (2023-10-31T09:04:39Z)
• From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning [52.257422715393574]
  We introduce a self-guided methodology for Large Language Models (LLMs) to autonomously discern and select cherry samples from open-source datasets. Our key innovation, the Instruction-Following Difficulty (IFD) metric, emerges as a pivotal metric to identify discrepancies between a model's expected responses and its intrinsic generation capability.
  arXiv  Detail & Related papers  (2023-08-23T09:45:29Z)
• A Benchmark Generative Probabilistic Model for Weak Supervised Learning [2.0257616108612373]
  Weak Supervised Learning approaches have been developed to alleviate the annotation burden. We show that latent variable models (PLVMs) achieve state-of-the-art performance across four datasets.
  arXiv  Detail & Related papers  (2023-03-31T07:06:24Z)
• M-Tuning: Prompt Tuning with Mitigated Label Bias in Open-Set Scenarios [58.617025733655005]
  We propose a vision-language prompt tuning method with mitigated label bias (M-Tuning) It introduces open words from the WordNet to extend the range of words forming the prompt texts from only closed-set label words to more, and thus prompts are tuned in a simulated open-set scenario. Our method achieves the best performance on datasets with various scales, and extensive ablation studies also validate its effectiveness.
  arXiv  Detail & Related papers  (2023-03-09T09:05:47Z)
• LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds [62.49198183539889]
  We propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds. Our method co-designs an efficient labeling process with semi/weakly supervised learning. Our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.
  arXiv  Detail & Related papers  (2022-10-14T19:13:36Z)
• Revisiting Self-Training for Few-Shot Learning of Language Model [61.173976954360334]
  Unlabeled data carry rich task-relevant information, they are proven useful for few-shot learning of language model. In this work, we revisit the self-training technique for language model fine-tuning and present a state-of-the-art prompt-based few-shot learner, SFLM.
  arXiv  Detail & Related papers  (2021-10-04T08:51:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.