On the Transferability of VAE Embeddings using Relational Knowledge with Semi-Supervision

• URL: http://arxiv.org/abs/2011.07137v1
• Date: Fri, 13 Nov 2020 21:40:32 GMT
• Title: On the Transferability of VAE Embeddings using Relational Knowledge with Semi-Supervision
• Authors: Harald Str\"omfelt, Luke Dickens, Artur d'Avila Garcez, Alessandra Russo
• Abstract summary: We propose a new model for relational VAE semi-supervision capable of balancing disentanglement and low complexity modelling of relations with different symbolic properties. We compare the relative benefits of relation-decoder and latent space structure on both inductive and transductive transfer learning.
• Score: 67.96748304066827
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a new model for relational VAE semi-supervision capable of balancing disentanglement and low complexity modelling of relations with different symbolic properties. We compare the relative benefits of relation-decoder complexity and latent space structure on both inductive and transductive transfer learning. Our results depict a complex picture where enforcing structure on semi-supervised representations can greatly improve zero-shot transductive transfer, but may be less favourable or even impact negatively the capacity for inductive transfer.

Related papers

• Strengthening Structural Inductive Biases by Pre-training to Perform Syntactic Transformations [75.14793516745374]
  We propose to strengthen the structural inductive bias of a Transformer by intermediate pre-training. Our experiments confirm that this helps with few-shot learning of syntactic tasks such as chunking. Our analysis shows that the intermediate pre-training leads to attention heads that keep track of which syntactic transformation needs to be applied to which token.
  arXiv  Detail & Related papers  (2024-07-05T14:29:44Z)
• Boosting Vision-Language Models with Transduction [12.281505126587048]
  We present TransCLIP, a novel and computationally efficient transductive approach for vision-language models. TransCLIP is applicable as a plug-and-play module on top of popular inductive zero- and few-shot models.
  arXiv  Detail & Related papers  (2024-06-03T23:09:30Z)
• Why Does Little Robustness Help? Understanding and Improving Adversarial Transferability from Surrogate Training [24.376314203167016]
  Adversarial examples (AEs) for DNNs have been shown to be transferable. In this paper, we take a further step towards understanding adversarial transferability.
  arXiv  Detail & Related papers  (2023-07-15T19:20:49Z)
• The Self-Optimal-Transport Feature Transform [2.804721532913997]
  We show how to upgrade the set of features of a data instance to facilitate downstream matching or grouping related tasks. A particular min-cost-max-flow fractional matching problem, whose entropy regularized version can be approximated by an optimal transport (OT) optimization, results in our transductive transform. Empirically, the transform is highly effective and flexible in its use, consistently improving networks it is inserted into.
  arXiv  Detail & Related papers  (2022-04-06T20:00:39Z)
• TRS: Transferability Reduced Ensemble via Encouraging Gradient Diversity and Model Smoothness [14.342349428248887]
  Adversarial Transferability is an intriguing property of adversarial examples. This paper theoretically analyzes sufficient conditions for transferability between models. We propose a practical algorithm to reduce transferability within an ensemble to improve its robustness.
  arXiv  Detail & Related papers  (2021-04-01T17:58:35Z)
• Comparing Probability Distributions with Conditional Transport [63.11403041984197]
  We propose conditional transport (CT) as a new divergence and approximate it with the amortized CT (ACT) cost. ACT amortizes the computation of its conditional transport plans and comes with unbiased sample gradients that are straightforward to compute. On a wide variety of benchmark datasets generative modeling, substituting the default statistical distance of an existing generative adversarial network with ACT is shown to consistently improve the performance.
  arXiv  Detail & Related papers  (2020-12-28T05:14:22Z)
• RatE: Relation-Adaptive Translating Embedding for Knowledge Graph Completion [51.64061146389754]
  We propose a relation-adaptive translation function built upon a novel weighted product in complex space. We then present our Relation-adaptive translating Embedding (RatE) approach to score each graph triple.
  arXiv  Detail & Related papers  (2020-10-10T01:30:30Z)
• Generalization Properties of Optimal Transport GANs with Latent Distribution Learning [52.25145141639159]
  We study how the interplay between the latent distribution and the complexity of the pushforward map affects performance. Motivated by our analysis, we advocate learning the latent distribution as well as the pushforward map within the GAN paradigm.
  arXiv  Detail & Related papers  (2020-07-29T07:31:33Z)
• Adversarial Training Reduces Information and Improves Transferability [81.59364510580738]
  Recent results show that features of adversarially trained networks for classification, in addition to being robust, enable desirable properties such as invertibility. We show that the Adversarial Training can improve linear transferability to new tasks, from which arises a new trade-off between transferability of representations and accuracy on the source task.
  arXiv  Detail & Related papers  (2020-07-22T08:30:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.