Cross-Modality Clustering-based Self-Labeling for Multimodal Data Classification

• URL: http://arxiv.org/abs/2408.02568v1
• Date: Mon, 5 Aug 2024 15:43:56 GMT
• Title: Cross-Modality Clustering-based Self-Labeling for Multimodal Data Classification
• Authors: Paweł Zyblewski, Leandro L. Minku,
• Abstract summary: Cross-Modality Clustering-based Self-Labeling ( CMCSL) CMCSL groups instances belonging to each modality in the deep feature space and then propagates known labels within the resulting clusters. Experimental evaluation conducted on 20 datasets derived from the MM-IMDb dataset.
• Score: 2.666791490663749
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Technological advances facilitate the ability to acquire multimodal data, posing a challenge for recognition systems while also providing an opportunity to use the heterogeneous nature of the information to increase the generalization capability of models. An often overlooked issue is the cost of the labeling process, which is typically high due to the need for a significant investment in time and money associated with human experts. Existing semi-supervised learning methods often focus on operating in the feature space created by the fusion of available modalities, neglecting the potential for cross-utilizing complementary information available in each modality. To address this problem, we propose Cross-Modality Clustering-based Self-Labeling (CMCSL). Based on a small set of pre-labeled data, CMCSL groups instances belonging to each modality in the deep feature space and then propagates known labels within the resulting clusters. Next, information about the instances' class membership in each modality is exchanged based on the Euclidean distance to ensure more accurate labeling. Experimental evaluation conducted on 20 datasets derived from the MM-IMDb dataset indicates that cross-propagation of labels between modalities -- especially when the number of pre-labeled instances is small -- can allow for more reliable labeling and thus increase the classification performance in each modality.

Related papers

• Generalized Category Discovery with Clustering Assignment Consistency [56.92546133591019]
  Generalized category discovery (GCD) is a recently proposed open-world task. We propose a co-training-based framework that encourages clustering consistency. Our method achieves state-of-the-art performance on three generic benchmarks and three fine-grained visual recognition datasets.
  arXiv  Detail & Related papers  (2023-10-30T00:32:47Z)
• Tackling Diverse Minorities in Imbalanced Classification [80.78227787608714]
  Imbalanced datasets are commonly observed in various real-world applications, presenting significant challenges in training classifiers. We propose generating synthetic samples iteratively by mixing data samples from both minority and majority classes. We demonstrate the effectiveness of our proposed framework through extensive experiments conducted on seven publicly available benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-28T18:48:34Z)
• Feature Correlation-guided Knowledge Transfer for Federated Self-supervised Learning [19.505644178449046]
  We propose a novel and general method named Federated Self-supervised Learning with Feature-correlation based Aggregation (FedFoA) Our insight is to utilize feature correlation to align the feature mappings and calibrate the local model updates across clients during their local training process. We prove that FedFoA is a model-agnostic training framework and can be easily compatible with state-of-the-art unsupervised FL methods.
  arXiv  Detail & Related papers  (2022-11-14T13:59:50Z)
• Evolving Multi-Label Fuzzy Classifier [5.53329677986653]
  Multi-label classification has attracted much attention in the machine learning community to address the problem of assigning single samples to more than one class at the same time. We propose an evolving multi-label fuzzy classifier (EFC-ML) which is able to self-adapt and self-evolve its structure with new incoming multi-label samples in an incremental, single-pass manner.
  arXiv  Detail & Related papers  (2022-03-29T08:01:03Z)
• Meta Clustering Learning for Large-scale Unsupervised Person Re-identification [124.54749810371986]
  We propose a "small data for big task" paradigm dubbed Meta Clustering Learning (MCL) MCL only pseudo-labels a subset of the entire unlabeled data via clustering to save computing for the first-phase training. Our method significantly saves computational cost while achieving a comparable or even better performance compared to prior works.
  arXiv  Detail & Related papers  (2021-11-19T04:10:18Z)
• Joint Representation Learning and Novel Category Discovery on Single- and Multi-modal Data [16.138075558585516]
  We present a generic, end-to-end framework to jointly learn a reliable representation and assign clusters to unlabelled data. We employ Winner-Take-All (WTA) hashing algorithm on the shared representation space to generate pairwise pseudo labels for unlabelled data. We thoroughly evaluate our framework on large-scale multi-modal video benchmarks Kinetics-400 and VGG-Sound, and image benchmarks CIFAR10, CIFAR100 and ImageNet.
  arXiv  Detail & Related papers  (2021-04-26T15:56:16Z)
• Dual-Refinement: Joint Label and Feature Refinement for Unsupervised Domain Adaptive Person Re-Identification [51.98150752331922]
  Unsupervised domain adaptive (UDA) person re-identification (re-ID) is a challenging task due to the missing of labels for the target domain data. We propose a novel approach, called Dual-Refinement, that jointly refines pseudo labels at the off-line clustering phase and features at the on-line training phase. Our method outperforms the state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2020-12-26T07:35:35Z)
• Bayesian Semi-supervised Crowdsourcing [71.20185379303479]
  Crowdsourcing has emerged as a powerful paradigm for efficiently labeling large datasets and performing various learning tasks. This work deals with semi-supervised crowdsourced classification, under two regimes of semi-supervision.
  arXiv  Detail & Related papers  (2020-12-20T23:18:51Z)
• TCGM: An Information-Theoretic Framework for Semi-Supervised Multi-Modality Learning [35.76792527025377]
  We propose a novel information-theoretic approach, namely textbfTotal textbfCorrelation textbfGain textbfMaximization (TCGM) for semi-supervised multi-modal learning. We apply our method to various tasks and achieve state-of-the-art results, including news classification, emotion recognition and disease prediction.
  arXiv  Detail & Related papers  (2020-07-14T03:32:03Z)
• Classify and Generate Reciprocally: Simultaneous Positive-Unlabelled Learning and Conditional Generation with Extra Data [77.31213472792088]
  The scarcity of class-labeled data is a ubiquitous bottleneck in many machine learning problems. We address this problem by leveraging Positive-Unlabeled(PU) classification and the conditional generation with extra unlabeled data. We present a novel training framework to jointly target both PU classification and conditional generation when exposed to extra data.
  arXiv  Detail & Related papers  (2020-06-14T08:27:40Z)
• Global Multiclass Classification and Dataset Construction via Heterogeneous Local Experts [37.27708297562079]
  We show how to minimize the number of labelers while ensuring the reliability of the resulting dataset. Experiments with the MNIST and CIFAR-10 datasets demonstrate the favorable accuracy of our aggregation scheme.
  arXiv  Detail & Related papers  (2020-05-21T18:07:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.