Learn to Predict Sets Using Feed-Forward Neural Networks
- URL: http://arxiv.org/abs/2001.11845v2
- Date: Mon, 25 Oct 2021 06:33:27 GMT
- Title: Learn to Predict Sets Using Feed-Forward Neural Networks
- Authors: Hamid Rezatofighi, Tianyu Zhu, Roman Kaskman, Farbod T. Motlagh,
Qinfeng Shi, Anton Milan, Daniel Cremers, Laura Leal-Taix\'e, Ian Reid
- Abstract summary: This paper addresses the task of set prediction using deep feed-forward neural networks.
We present a novel approach for learning to predict sets with unknown permutation and cardinality.
We demonstrate the validity of our set formulations on relevant vision problems.
- Score: 63.91494644881925
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This paper addresses the task of set prediction using deep feed-forward
neural networks. A set is a collection of elements which is invariant under
permutation and the size of a set is not fixed in advance. Many real-world
problems, such as image tagging and object detection, have outputs that are
naturally expressed as sets of entities. This creates a challenge for
traditional deep neural networks which naturally deal with structured outputs
such as vectors, matrices or tensors. We present a novel approach for learning
to predict sets with unknown permutation and cardinality using deep neural
networks. In our formulation we define a likelihood for a set distribution
represented by a) two discrete distributions defining the set cardinally and
permutation variables, and b) a joint distribution over set elements with a
fixed cardinality. Depending on the problem under consideration, we define
different training models for set prediction using deep neural networks. We
demonstrate the validity of our set formulations on relevant vision problems
such as: 1) multi-label image classification where we outperform the other
competing methods on the PASCAL VOC and MS COCO datasets, 2) object detection,
for which our formulation outperforms popular state-of-the-art detectors, and
3) a complex CAPTCHA test, where we observe that, surprisingly, our set-based
network acquired the ability of mimicking arithmetics without any rules being
coded.
Related papers
- Quantification using Permutation-Invariant Networks based on Histograms [47.47360392729245]
Quantification is the supervised learning task in which a model is trained to predict the prevalence of each class in a given bag of examples.
This paper investigates the application of deep neural networks to tasks of quantification in scenarios where it is possible to apply a symmetric supervised approach.
We propose HistNetQ, a novel neural architecture that relies on a permutation-invariant representation based on histograms.
arXiv Detail & Related papers (2024-03-22T11:25:38Z) - Enhancing Neural Subset Selection: Integrating Background Information into Set Representations [53.15923939406772]
We show that when the target value is conditioned on both the input set and subset, it is essential to incorporate an textitinvariant sufficient statistic of the superset into the subset of interest.
This ensures that the output value remains invariant to permutations of the subset and its corresponding superset, enabling identification of the specific superset from which the subset originated.
arXiv Detail & Related papers (2024-02-05T16:09:35Z) - Discrete Graph Auto-Encoder [52.50288418639075]
We introduce a new framework named Discrete Graph Auto-Encoder (DGAE)
We first use a permutation-equivariant auto-encoder to convert graphs into sets of discrete latent node representations.
In the second step, we sort the sets of discrete latent representations and learn their distribution with a specifically designed auto-regressive model.
arXiv Detail & Related papers (2023-06-13T12:40:39Z) - Graph Neural Networks with Adaptive Readouts [5.575293536755126]
We show the effectiveness of neural readouts on more than 40 datasets spanning different domains and graph characteristics.
We observe a consistent improvement over standard readouts relative to the number of neighborhood aggregation and different convolutional operators.
arXiv Detail & Related papers (2022-11-09T15:21:09Z) - Set Interdependence Transformer: Set-to-Sequence Neural Networks for
Permutation Learning and Structure Prediction [6.396288020763144]
Set-to-sequence problems occur in natural language processing, computer vision and structure prediction.
Previous attention-based methods require $n$ layers of their set transformations to explicitly represent $n$-th order relations.
We propose a novel neural set encoding method called the Set Interdependence Transformer, capable of relating the set's permutation invariant representation to its elements within sets of any cardinality.
arXiv Detail & Related papers (2022-06-08T07:46:49Z) - PICASO: Permutation-Invariant Cascaded Attentional Set Operator [6.845913709297514]
We propose a permutation-invariant cascaded attentional set operator (PICASO) for set-input deep networks.
The proposed operator is a stand-alone module that can be adapted and extended to serve different machine learning tasks.
We demonstrate the utilities of PICASO in four diverse scenarios: (i) clustering, (ii) image classification under novel viewpoints, (iii) image anomaly detection, and (iv) state prediction.
arXiv Detail & Related papers (2021-07-17T19:21:30Z) - Bayesian Attention Belief Networks [59.183311769616466]
Attention-based neural networks have achieved state-of-the-art results on a wide range of tasks.
This paper introduces Bayesian attention belief networks, which construct a decoder network by modeling unnormalized attention weights.
We show that our method outperforms deterministic attention and state-of-the-art attention in accuracy, uncertainty estimation, generalization across domains, and adversarial attacks.
arXiv Detail & Related papers (2021-06-09T17:46:22Z) - Predicting Temporal Sets with Deep Neural Networks [50.53727580527024]
We propose an integrated solution based on the deep neural networks for temporal sets prediction.
A unique perspective is to learn element relationship by constructing set-level co-occurrence graph.
We design an attention-based module to adaptively learn the temporal dependency of elements and sets.
arXiv Detail & Related papers (2020-06-20T03:29:02Z) - Set Distribution Networks: a Generative Model for Sets of Images [22.405670277339023]
We introduce Set Distribution Networks (SDNs), a framework that learns to autoencode and freely generate sets.
We show that SDNs are able to reconstruct image sets that preserve salient attributes of the inputs in our benchmark datasets.
We examine the sets generated by SDN with a pre-trained 3D reconstruction network and a face verification network, respectively, as a novel way to evaluate the quality of generated sets of images.
arXiv Detail & Related papers (2020-06-18T17:38:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.