Meta-learning for Matrix Factorization without Shared Rows or Columns
- URL: http://arxiv.org/abs/2106.15133v1
- Date: Tue, 29 Jun 2021 07:40:20 GMT
- Title: Meta-learning for Matrix Factorization without Shared Rows or Columns
- Authors: Tomoharu Iwata
- Abstract summary: The proposed method uses a neural network that takes a matrix as input, and generates prior distributions of factorized matrices of the given matrix.
The neural network is meta-learned such that the expected imputation error is minimized.
In our experiments with three user-item rating datasets, we demonstrate that our proposed method can impute the missing values from a limited number of observations in unseen matrices.
- Score: 39.56814839510978
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We propose a method that meta-learns a knowledge on matrix factorization from
various matrices, and uses the knowledge for factorizing unseen matrices. The
proposed method uses a neural network that takes a matrix as input, and
generates prior distributions of factorized matrices of the given matrix. The
neural network is meta-learned such that the expected imputation error is
minimized when the factorized matrices are adapted to each matrix by a maximum
a posteriori (MAP) estimation. We use a gradient descent method for the MAP
estimation, which enables us to backpropagate the expected imputation error
through the gradient descent steps for updating neural network parameters since
each gradient descent step is written in a closed form and is differentiable.
The proposed method can meta-learn from matrices even when their rows and
columns are not shared, and their sizes are different from each other. In our
experiments with three user-item rating datasets, we demonstrate that our
proposed method can impute the missing values from a limited number of
observations in unseen matrices after being trained with different matrices.
Related papers
- Spectral Entry-wise Matrix Estimation for Low-Rank Reinforcement
Learning [53.445068584013896]
We study matrix estimation problems arising in reinforcement learning (RL) with low-rank structure.
In low-rank bandits, the matrix to be recovered specifies the expected arm rewards, and for low-rank Markov Decision Processes (MDPs), it may for example characterize the transition kernel of the MDP.
We show that simple spectral-based matrix estimation approaches efficiently recover the singular subspaces of the matrix and exhibit nearly-minimal entry-wise error.
arXiv Detail & Related papers (2023-10-10T17:06:41Z) - A Structured Sparse Neural Network and Its Matrix Calculations Algorithm [0.0]
We introduce a nonsymmetric, tridiagonal matrix with offdiagonal sparse entries and offset sub and super-diagonals.
For the cases where the matrix inverse does not exist, a least square type pseudoinverse is provided.
Results show significant improvement in computational costs specially when the size of matrix increases.
arXiv Detail & Related papers (2022-07-02T19:38:48Z) - Memory-Efficient Backpropagation through Large Linear Layers [107.20037639738433]
In modern neural networks like Transformers, linear layers require significant memory to store activations during backward pass.
This study proposes a memory reduction approach to perform backpropagation through linear layers.
arXiv Detail & Related papers (2022-01-31T13:02:41Z) - Sparse Factorization of Large Square Matrices [10.94053598642913]
In this paper, we propose to approximate a large square matrix with a product of sparse full-rank matrices.
In the approximation, our method needs only $N(log N)2$ non-zero numbers for an $Ntimes N$ full matrix.
We show that our method gives a better approximation when the approximated matrix is sparse and high-rank.
arXiv Detail & Related papers (2021-09-16T18:42:21Z) - Variance Reduction for Matrix Computations with Applications to Gaussian
Processes [0.0]
We focus on variance reduction for matrix computations via matrix factorization.
We show how computing the square root factorization of the matrix can achieve in some important cases arbitrarily better performance.
arXiv Detail & Related papers (2021-06-28T10:41:22Z) - Non-PSD Matrix Sketching with Applications to Regression and
Optimization [56.730993511802865]
We present dimensionality reduction methods for non-PSD and square-roots" matrices.
We show how these techniques can be used for multiple downstream tasks.
arXiv Detail & Related papers (2021-06-16T04:07:48Z) - Deep Two-way Matrix Reordering for Relational Data Analysis [41.60125423028092]
Matrix reordering is a task to permute rows and columns of a given observed matrix.
We propose a new matrix reordering method, Deep Two-way Matrix Reordering (DeepTMR), using a neural network model.
We demonstrate the effectiveness of proposed DeepTMR by applying it to both synthetic and practical data sets.
arXiv Detail & Related papers (2021-03-26T01:31:24Z) - Sketching Transformed Matrices with Applications to Natural Language
Processing [76.6222695417524]
We propose a space-efficient sketching algorithm for computing the product of a given small matrix with the transformed matrix.
We show that our approach obtains small error and is efficient in both space and time.
arXiv Detail & Related papers (2020-02-23T03:07:31Z) - Optimal Iterative Sketching with the Subsampled Randomized Hadamard
Transform [64.90148466525754]
We study the performance of iterative sketching for least-squares problems.
We show that the convergence rate for Haar and randomized Hadamard matrices are identical, andally improve upon random projections.
These techniques may be applied to other algorithms that employ randomized dimension reduction.
arXiv Detail & Related papers (2020-02-03T16:17:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.