Intersection Regularization for Extracting Semantic Attributes

• URL: http://arxiv.org/abs/2103.11888v1
• Date: Mon, 22 Mar 2021 14:32:44 GMT
• Title: Intersection Regularization for Extracting Semantic Attributes
• Authors: Ameen Ali, Tomer Galanti, Evgeniy Zheltonozhskiy, Chaim Baskin, Lior Wolf
• Abstract summary: We consider the problem of supervised classification, such that the features that the network extracts match an unseen set of semantic attributes. For example, when learning to classify images of birds into species, we would like to observe the emergence of features that zoologists use to classify birds. We propose training a neural network with discrete top-level activations, which is followed by a multi-layered perceptron (MLP) and a parallel decision tree.
• Score: 72.53481390411173
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We consider the problem of supervised classification, such that the features that the network extracts match an unseen set of semantic attributes, without any additional supervision. For example, when learning to classify images of birds into species, we would like to observe the emergence of features that zoologists use to classify birds. We propose training a neural network with discrete top-level activations, which is followed by a multi-layered perceptron (MLP) and a parallel decision tree. We present a theoretical analysis as well as a practical method for learning in the intersection of two hypothesis classes. Since real-world features are often sparse, a randomized sparsity regularization is also applied. Our results on multiple benchmarks show an improved ability to extract a set of features that are highly correlated with the set of unseen attributes.

Related papers

• Multi-annotator Deep Learning: A Probabilistic Framework for Classification [2.445702550853822]
  Training standard deep neural networks leads to subpar performances in multi-annotator supervised learning settings. We address this issue by presenting a probabilistic training framework named multi-annotator deep learning (MaDL) A modular network architecture enables us to make varying assumptions regarding annotators' performances. Our findings show MaDL's state-of-the-art performance and robustness against many correlated, spamming annotators.
  arXiv  Detail & Related papers  (2023-04-05T16:00:42Z)
• Understanding Imbalanced Semantic Segmentation Through Neural Collapse [81.89121711426951]
  We show that semantic segmentation naturally brings contextual correlation and imbalanced distribution among classes. We introduce a regularizer on feature centers to encourage the network to learn features closer to the appealing structure. Our method ranks 1st and sets a new record on the ScanNet200 test leaderboard.
  arXiv  Detail & Related papers  (2023-01-03T13:51:51Z)
• Equivariance with Learned Canonicalization Functions [77.32483958400282]
  We show that learning a small neural network to perform canonicalization is better than using predefineds. Our experiments show that learning the canonicalization function is competitive with existing techniques for learning equivariant functions across many tasks.
  arXiv  Detail & Related papers  (2022-11-11T21:58:15Z)
• Semi-supervised Predictive Clustering Trees for (Hierarchical) Multi-label Classification [2.706328351174805]
  We propose a hierarchical multi-label classification method based on semi-supervised learning of predictive clustering trees. We also extend the method towards ensemble learning and propose a method based on the random forest approach.
  arXiv  Detail & Related papers  (2022-07-19T12:49:00Z)
• A Top-down Supervised Learning Approach to Hierarchical Multi-label Classification in Networks [0.21485350418225244]
  This paper presents a general prediction model to hierarchical multi-label classification (HMC), where the attributes to be inferred can be specified as a strict poset. It is based on a top-down classification approach that addresses hierarchical multi-label classification with supervised learning by building a local classifier per class. The proposed model is showcased with a case study on the prediction of gene functions for Oryza sativa Japonica, a variety of rice.
  arXiv  Detail & Related papers  (2022-03-23T17:29:17Z)
• Semantic Clustering based Deduction Learning for Image Recognition and Classification [19.757743366620613]
  The paper proposes a semantic clustering based deduction learning by mimicking the learning and thinking process of human brains. The proposed approach is supported theoretically and empirically through extensive experiments.
  arXiv  Detail & Related papers  (2021-12-25T01:31:21Z)
• Learning Debiased and Disentangled Representations for Semantic Segmentation [52.35766945827972]
  We propose a model-agnostic and training scheme for semantic segmentation. By randomly eliminating certain class information in each training iteration, we effectively reduce feature dependencies among classes. Models trained with our approach demonstrate strong results on multiple semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2021-10-31T16:15:09Z)
• A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention [96.77554122595578]
  We introduce a parametrized representation of fixed size, which embeds and then aggregates elements from a given input set according to the optimal transport plan between the set and a trainable reference. Our approach scales to large datasets and allows end-to-end training of the reference, while also providing a simple unsupervised learning mechanism with small computational cost.
  arXiv  Detail & Related papers  (2020-06-22T08:35:58Z)
• Learning from Aggregate Observations [82.44304647051243]
  We study the problem of learning from aggregate observations where supervision signals are given to sets of instances. We present a general probabilistic framework that accommodates a variety of aggregate observations. Simple maximum likelihood solutions can be applied to various differentiable models.
  arXiv  Detail & Related papers  (2020-04-14T06:18:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.