Exploiting Field Dependencies for Learning on Categorical Data
- URL: http://arxiv.org/abs/2307.09321v1
- Date: Tue, 18 Jul 2023 15:03:56 GMT
- Title: Exploiting Field Dependencies for Learning on Categorical Data
- Authors: Zhibin Li, Piotr Koniusz, Lu Zhang, Daniel Edward Pagendam, Peyman
Moghadam
- Abstract summary: We propose a novel method for learning on categorical data with the goal of exploiting dependencies between fields.
Our method is simple yet it outperforms several state-of-the-art methods on six popular dataset benchmarks.
- Score: 33.2727127163419
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Traditional approaches for learning on categorical data underexploit the
dependencies between columns (\aka fields) in a dataset because they rely on
the embedding of data points driven alone by the classification/regression
loss. In contrast, we propose a novel method for learning on categorical data
with the goal of exploiting dependencies between fields. Instead of modelling
statistics of features globally (i.e., by the covariance matrix of features),
we learn a global field dependency matrix that captures dependencies between
fields and then we refine the global field dependency matrix at the
instance-wise level with different weights (so-called local dependency
modelling) w.r.t. each field to improve the modelling of the field
dependencies. Our algorithm exploits the meta-learning paradigm, i.e., the
dependency matrices are refined in the inner loop of the meta-learning
algorithm without the use of labels, whereas the outer loop intertwines the
updates of the embedding matrix (the matrix performing projection) and global
dependency matrix in a supervised fashion (with the use of labels). Our method
is simple yet it outperforms several state-of-the-art methods on six popular
dataset benchmarks. Detailed ablation studies provide additional insights into
our method.
Related papers
- Potential Field Based Deep Metric Learning [8.670873561640903]
Deep metric learning involves training a network to learn a semantically meaningful representation space.
We present a novel, compositional DML model inspired by electrostatic fields in physics.
We show that such decay helps improve performance on real world datasets with large intra-class variations and label noise.
arXiv Detail & Related papers (2024-05-28T20:10:06Z) - Learning Representations without Compositional Assumptions [79.12273403390311]
We propose a data-driven approach that learns feature set dependencies by representing feature sets as graph nodes and their relationships as learnable edges.
We also introduce LEGATO, a novel hierarchical graph autoencoder that learns a smaller, latent graph to aggregate information from multiple views dynamically.
arXiv Detail & Related papers (2023-05-31T10:36:10Z) - Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning [112.69497636932955]
Federated learning aims to train models across different clients without the sharing of data for privacy considerations.
We study how data heterogeneity affects the representations of the globally aggregated models.
We propose sc FedDecorr, a novel method that can effectively mitigate dimensional collapse in federated learning.
arXiv Detail & Related papers (2022-10-01T09:04:17Z) - Automatic universal taxonomies for multi-domain semantic segmentation [1.4364491422470593]
Training semantic segmentation models on multiple datasets has sparked a lot of recent interest in the computer vision community.
established datasets have mutually incompatible labels which disrupt principled inference in the wild.
We address this issue by automatic construction of universal through iterative dataset integration.
arXiv Detail & Related papers (2022-07-18T08:53:17Z) - Self-Taught Metric Learning without Labels [47.832107446521626]
We present a novel self-taught framework for unsupervised metric learning.
It alternates between predicting class-equivalence relations between data through a moving average of an embedding model and learning the model with the predicted relations as pseudo labels.
arXiv Detail & Related papers (2022-05-04T05:48:40Z) - Disentanglement and Generalization Under Correlation Shifts [22.499106910581958]
Correlations between factors of variation are prevalent in real-world data.
Machine learning algorithms may benefit from exploiting such correlations, as they can increase predictive performance on noisy data.
We aim to learn representations which capture different factors of variation in latent subspaces.
arXiv Detail & Related papers (2021-12-29T18:55:17Z) - Deep Transfer Learning for Multi-source Entity Linkage via Domain
Adaptation [63.24594955429465]
Multi-source entity linkage is critical in high-impact applications such as data cleaning and user stitching.
AdaMEL is a deep transfer learning framework that learns generic high-level knowledge to perform multi-source entity linkage.
Our framework achieves state-of-the-art results with 8.21% improvement on average over methods based on supervised learning.
arXiv Detail & Related papers (2021-10-27T15:20:41Z) - Inferring Latent Domains for Unsupervised Deep Domain Adaptation [54.963823285456925]
Unsupervised Domain Adaptation (UDA) refers to the problem of learning a model in a target domain where labeled data are not available.
This paper introduces a novel deep architecture which addresses the problem of UDA by automatically discovering latent domains in visual datasets.
We evaluate our approach on publicly available benchmarks, showing that it outperforms state-of-the-art domain adaptation methods.
arXiv Detail & Related papers (2021-03-25T14:33:33Z) - Field-wise Learning for Multi-field Categorical Data [27.100048708707593]
We propose a new method for learning with multi-field categorical data.
In doing this, the models can be fitted to each category and thus can better capture the underlying differences in data.
The experiment results on two large-scale datasets show the superior performance of our model.
arXiv Detail & Related papers (2020-12-01T01:10:14Z) - Learning to Combine: Knowledge Aggregation for Multi-Source Domain
Adaptation [56.694330303488435]
We propose a Learning to Combine for Multi-Source Domain Adaptation (LtC-MSDA) framework.
In the nutshell, a knowledge graph is constructed on the prototypes of various domains to realize the information propagation among semantically adjacent representations.
Our approach outperforms existing methods with a remarkable margin.
arXiv Detail & Related papers (2020-07-17T07:52:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.