Latent Space Translation via Inverse Relative Projection

• URL: http://arxiv.org/abs/2406.15057v1
• Date: Fri, 21 Jun 2024 11:11:46 GMT
• Title: Latent Space Translation via Inverse Relative Projection
• Authors: Valentino Maiorca, Luca Moschella, Marco Fumero, Francesco Locatello, Emanuele Rodolà,
• Abstract summary: "Latent space communication" can be achieved by independently mapping the original spaces to a shared or relative one. We combine the two into a novel method to obtain latent space translation through the relative space. Our method has significant potential for facilitating the reuse of models in a practical manner via compositionality.
• Score: 35.873300268472335
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The emergence of similar representations between independently trained neural models has sparked significant interest in the representation learning community, leading to the development of various methods to obtain communication between latent spaces. "Latent space communication" can be achieved in two ways: i) by independently mapping the original spaces to a shared or relative one; ii) by directly estimating a transformation from a source latent space to a target one. In this work, we combine the two into a novel method to obtain latent space translation through the relative space. By formalizing the invertibility of angle-preserving relative representations and assuming the scale invariance of decoder modules in neural models, we can effectively use the relative space as an intermediary, independently projecting onto and from other semantically similar spaces. Extensive experiments over various architectures and datasets validate our scale invariance assumption and demonstrate the high accuracy of our method in latent space translation. We also apply our method to zero-shot stitching between arbitrary pre-trained text and image encoders and their classifiers, even across modalities. Our method has significant potential for facilitating the reuse of models in a practical manner via compositionality.

Related papers

• Self-Supervised Representation Learning with Spatial-Temporal Consistency for Sign Language Recognition [96.62264528407863]
  We propose a self-supervised contrastive learning framework to excavate rich context via spatial-temporal consistency. Inspired by the complementary property of motion and joint modalities, we first introduce first-order motion information into sign language modeling. Our method is evaluated with extensive experiments on four public benchmarks, and achieves new state-of-the-art performance with a notable margin.
  arXiv  Detail & Related papers  (2024-06-15T04:50:19Z)
• Latent Space Translation via Semantic Alignment [29.2401314068038]
  We show how representations learned from different neural modules can be translated between different pre-trained networks. Our method directly estimates a transformation between two given latent spaces, thereby enabling effective stitching of encoders and decoders without additional training. Notably, we show how it is possible to zero-shot stitch text encoders and vision decoders, or vice-versa, yielding surprisingly good classification performance in this multimodal setting.
  arXiv  Detail & Related papers  (2023-11-01T17:12:00Z)
• Learning Semantic Textual Similarity via Topic-informed Discrete Latent Variables [17.57873577962635]
  We develop a topic-informed discrete latent variable model for semantic textual similarity. Our model learns a shared latent space for sentence-pair representation via vector quantization. We show that our model is able to surpass several strong neural baselines in semantic textual similarity tasks.
  arXiv  Detail & Related papers  (2022-11-07T15:09:58Z)
• Robust Unsupervised Cross-Lingual Word Embedding using Domain Flow Interpolation [48.32604585839687]
  Previous adversarial approaches have shown promising results in inducing cross-lingual word embedding without parallel data. We propose to make use of a sequence of intermediate spaces for smooth bridging.
  arXiv  Detail & Related papers  (2022-10-07T04:37:47Z)
• Relative representations enable zero-shot latent space communication [19.144630518400604]
  Neural networks embed the geometric structure of a data manifold lying in a high-dimensional space into latent representations. We show how neural architectures can leverage these relative representations to guarantee, in practice, latent isometry invariance.
  arXiv  Detail & Related papers  (2022-09-30T12:37:03Z)
• Consistency and Diversity induced Human Motion Segmentation [231.36289425663702]
  We propose a novel Consistency and Diversity induced human Motion (CDMS) algorithm. Our model factorizes the source and target data into distinct multi-layer feature spaces. A multi-mutual learning strategy is carried out to reduce the domain gap between the source and target data.
  arXiv  Detail & Related papers  (2022-02-10T06:23:56Z)
• Learning Conditional Invariance through Cycle Consistency [60.85059977904014]
  We propose a novel approach to identify meaningful and independent factors of variation in a dataset. Our method involves two separate latent subspaces for the target property and the remaining input information. We demonstrate on synthetic and molecular data that our approach identifies more meaningful factors which lead to sparser and more interpretable models.
  arXiv  Detail & Related papers  (2021-11-25T17:33:12Z)
• Collaborative Training of GANs in Continuous and Discrete Spaces for Text Generation [21.435286755934534]
  We propose a novel text GAN architecture that promotes the collaborative training of the continuous-space and discrete-space methods. Our model substantially outperforms state-of-the-art text GANs with respect to quality, diversity, and global consistency.
  arXiv  Detail & Related papers  (2020-10-16T07:51:16Z)
• Improve Variational Autoencoder for Text Generationwith Discrete Latent Bottleneck [52.08901549360262]
  Variational autoencoders (VAEs) are essential tools in end-to-end representation learning. VAEs tend to ignore latent variables with a strong auto-regressive decoder. We propose a principled approach to enforce an implicit latent feature matching in a more compact latent space.
  arXiv  Detail & Related papers  (2020-04-22T14:41:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.