Measuring disentangled generative spatio-temporal representation
- URL: http://arxiv.org/abs/2202.04821v1
- Date: Thu, 10 Feb 2022 03:57:06 GMT
- Title: Measuring disentangled generative spatio-temporal representation
- Authors: Sichen Zhao, Wei Shao, Jeffrey Chan, Flora D. Salim
- Abstract summary: We adopt two state-the-art disentangled representation learning methods and apply them to three large-scale public-temporal datasets.
We find that our methods can be used to discover real-world-world semantics to describe the variables in the learned representation.
- Score: 9.264758623908813
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Disentangled representation learning offers useful properties such as
dimension reduction and interpretability, which are essential to modern deep
learning approaches. Although deep learning techniques have been widely applied
to spatio-temporal data mining, there has been little attention to further
disentangle the latent features and understanding their contribution to the
model performance, particularly their mutual information and correlation across
features. In this study, we adopt two state-of-the-art disentangled
representation learning methods and apply them to three large-scale public
spatio-temporal datasets. To evaluate their performance, we propose an internal
evaluation metric focusing on the degree of correlations among latent variables
of the learned representations and the prediction performance of the downstream
tasks. Empirical results show that our modified method can learn disentangled
representations that achieve the same level of performance as existing
state-of-the-art ST deep learning methods in a spatio-temporal sequence
forecasting problem. Additionally, we find that our methods can be used to
discover real-world spatial-temporal semantics to describe the variables in the
learned representation.
Related papers
- Learning Discriminative Spatio-temporal Representations for Semi-supervised Action Recognition [23.44320273156057]
We propose an Adaptive Contrastive Learning(ACL) strategy and a Multi-scale Temporal Learning(MTL) strategy.
ACL strategy assesses the confidence of all unlabeled samples by the class prototypes of the labeled data, and adaptively selects positive-negative samples from a pseudo-labeled sample bank to construct contrastive learning.
MTL strategy could highlight informative semantics from long-term clips and integrate them into the short-term clip while suppressing noisy information.
arXiv Detail & Related papers (2024-04-25T08:49:08Z) - Revisiting Self-supervised Learning of Speech Representation from a
Mutual Information Perspective [68.20531518525273]
We take a closer look into existing self-supervised methods of speech from an information-theoretic perspective.
We use linear probes to estimate the mutual information between the target information and learned representations.
We explore the potential of evaluating representations in a self-supervised fashion, where we estimate the mutual information between different parts of the data without using any labels.
arXiv Detail & Related papers (2024-01-16T21:13:22Z) - Revisiting the Temporal Modeling in Spatio-Temporal Predictive Learning
under A Unified View [73.73667848619343]
We introduce USTEP (Unified S-TEmporal Predictive learning), an innovative framework that reconciles the recurrent-based and recurrent-free methods by integrating both micro-temporal and macro-temporal scales.
arXiv Detail & Related papers (2023-10-09T16:17:42Z) - Latent Properties of Lifelong Learning Systems [59.50307752165016]
We introduce an algorithm-agnostic explainable surrogate-modeling approach to estimate latent properties of lifelong learning algorithms.
We validate the approach for estimating these properties via experiments on synthetic data.
arXiv Detail & Related papers (2022-07-28T20:58:13Z) - Self-Attention Neural Bag-of-Features [103.70855797025689]
We build on the recently introduced 2D-Attention and reformulate the attention learning methodology.
We propose a joint feature-temporal attention mechanism that learns a joint 2D attention mask highlighting relevant information.
arXiv Detail & Related papers (2022-01-26T17:54:14Z) - Fair Representation Learning using Interpolation Enabled Disentanglement [9.043741281011304]
We propose a novel method to address two key issues: (a) Can we simultaneously learn fair disentangled representations while ensuring the utility of the learned representation for downstream tasks, and (b)Can we provide theoretical insights into when the proposed approach will be both fair and accurate.
To address the former, we propose the method FRIED, Fair Representation learning using Interpolation Enabled Disentanglement.
arXiv Detail & Related papers (2021-07-31T17:32:12Z) - Layer-wise Analysis of a Self-supervised Speech Representation Model [26.727775920272205]
Self-supervised learning approaches have been successful for pre-training speech representation models.
Not much has been studied about the type or extent of information encoded in the pre-trained representations themselves.
arXiv Detail & Related papers (2021-07-10T02:13:25Z) - On Contrastive Representations of Stochastic Processes [53.21653429290478]
Learning representations of processes is an emerging problem in machine learning.
We show that our methods are effective for learning representations of periodic functions, 3D objects and dynamical processes.
arXiv Detail & Related papers (2021-06-18T11:00:24Z) - Progressive Spatio-Temporal Bilinear Network with Monte Carlo Dropout
for Landmark-based Facial Expression Recognition with Uncertainty Estimation [93.73198973454944]
The performance of our method is evaluated on three widely used datasets.
It is comparable to that of video-based state-of-the-art methods while it has much less complexity.
arXiv Detail & Related papers (2021-06-08T13:40:30Z) - Learning Representations that Support Extrapolation [39.84463809100903]
We consider the challenge of learning representations that support extrapolation.
We introduce a novel visual analogy benchmark that allows the graded evaluation of extrapolation.
We also introduce a simple technique, temporal context normalization, that encourages representations that emphasize the relations between objects.
arXiv Detail & Related papers (2020-07-09T20:53:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.