Learning the Structure of Auto-Encoding Recommenders
- URL: http://arxiv.org/abs/2008.07956v1
- Date: Tue, 18 Aug 2020 14:37:40 GMT
- Title: Learning the Structure of Auto-Encoding Recommenders
- Authors: Farhan Khawar, Leonard Kin Man Poon, Nevin Lianwen Zhang
- Abstract summary: We introduce structure learning for autoencoder recommenders by taking advantage of the inherent item groups present in the collaborative filtering domain.
Based on this, we propose a method that first learns groups of related items and then uses this information to determine the connectivity structure of an auto-encoding neural network.
The resultant sparse network considerably outperforms the state-of-the-art methods like textscMult-vae/Mult-dae on multiple benchmarked datasets.
- Score: 1.9981375888949475
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Autoencoder recommenders have recently shown state-of-the-art performance in
the recommendation task due to their ability to model non-linear item
relationships effectively. However, existing autoencoder recommenders use
fully-connected neural network layers and do not employ structure learning.
This can lead to inefficient training, especially when the data is sparse as
commonly found in collaborative filtering. The aforementioned results in lower
generalization ability and reduced performance. In this paper, we introduce
structure learning for autoencoder recommenders by taking advantage of the
inherent item groups present in the collaborative filtering domain. Due to the
nature of items in general, we know that certain items are more related to each
other than to other items. Based on this, we propose a method that first learns
groups of related items and then uses this information to determine the
connectivity structure of an auto-encoding neural network. This results in a
network that is sparsely connected. This sparse structure can be viewed as a
prior that guides the network training. Empirically we demonstrate that the
proposed structure learning enables the autoencoder to converge to a local
optimum with a much smaller spectral norm and generalization error bound than
the fully-connected network. The resultant sparse network considerably
outperforms the state-of-the-art methods like \textsc{Mult-vae/Mult-dae} on
multiple benchmarked datasets even when the same number of parameters and flops
are used. It also has a better cold-start performance.
Related papers
- Pushing the Efficiency Limit Using Structured Sparse Convolutions [82.31130122200578]
We propose Structured Sparse Convolution (SSC), which leverages the inherent structure in images to reduce the parameters in the convolutional filter.
We show that SSC is a generalization of commonly used layers (depthwise, groupwise and pointwise convolution) in efficient architectures''
Architectures based on SSC achieve state-of-the-art performance compared to baselines on CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet classification benchmarks.
arXiv Detail & Related papers (2022-10-23T18:37:22Z) - Collaborative Reflection-Augmented Autoencoder Network for Recommender
Systems [23.480069921831344]
We develop a Collaborative Reflection-Augmented Autoencoder Network (CRANet)
CRANet is capable of exploring transferable knowledge from observed and unobserved user-item interactions.
We experimentally validate CRANet on four diverse benchmark datasets corresponding to two recommendation tasks.
arXiv Detail & Related papers (2022-01-10T04:36:15Z) - Learning Structures for Deep Neural Networks [99.8331363309895]
We propose to adopt the efficient coding principle, rooted in information theory and developed in computational neuroscience.
We show that sparse coding can effectively maximize the entropy of the output signals.
Our experiments on a public image classification dataset demonstrate that using the structure learned from scratch by our proposed algorithm, one can achieve a classification accuracy comparable to the best expert-designed structure.
arXiv Detail & Related papers (2021-05-27T12:27:24Z) - BCFNet: A Balanced Collaborative Filtering Network with Attention
Mechanism [106.43103176833371]
Collaborative Filtering (CF) based recommendation methods have been widely studied.
We propose a novel recommendation model named Balanced Collaborative Filtering Network (BCFNet)
In addition, an attention mechanism is designed to better capture the hidden information within implicit feedback and strengthen the learning ability of the neural network.
arXiv Detail & Related papers (2021-03-10T14:59:23Z) - Network Support for High-performance Distributed Machine Learning [17.919773898228716]
We propose a system model that captures both learning nodes (that perform computations) and information nodes (that provide data)
We then formulate the problem of selecting (i) which learning and information nodes should cooperate to complete the learning task, and (ii) the number of iterations to perform.
We devise an algorithm, named DoubleClimb, that can find a 1+1/|I|-competitive solution with cubic worst-case complexity.
arXiv Detail & Related papers (2021-02-05T19:38:57Z) - Representation Extraction and Deep Neural Recommendation for
Collaborative Filtering [9.367612782346207]
This paper investigates the usage of novel representation learning algorithms to extract users and items representations from rating matrix.
We propose a modular algorithm consisted of two main phases: REpresentation eXtraction and a deep neural NETwork (RexNet)
RexNet is not dependent on unstructured auxiliary data such as visual and textual information, instead, it uses only the user-item rate matrix as its input.
arXiv Detail & Related papers (2020-12-09T11:15:23Z) - Fast Few-Shot Classification by Few-Iteration Meta-Learning [173.32497326674775]
We introduce a fast optimization-based meta-learning method for few-shot classification.
Our strategy enables important aspects of the base learner objective to be learned during meta-training.
We perform a comprehensive experimental analysis, demonstrating the speed and effectiveness of our approach.
arXiv Detail & Related papers (2020-10-01T15:59:31Z) - On the use of local structural properties for improving the efficiency
of hierarchical community detection methods [77.34726150561087]
We study how local structural network properties can be used as proxies to improve the efficiency of hierarchical community detection.
We also check the performance impact of network prunings as an ancillary tactic to make hierarchical community detection more efficient.
arXiv Detail & Related papers (2020-09-15T00:16:12Z) - Adaptive Hierarchical Decomposition of Large Deep Networks [4.272649614101117]
As datasets get larger, a natural question is if existing deep learning architectures can be extended to handle the 50+K classes thought to be perceptible by a typical human.
This paper introduces a framework that automatically analyzes and configures a family of smaller deep networks as a replacement to a singular, larger network.
The resulting smaller networks are highly scalable, parallel and more practical to train, and achieve higher classification accuracy.
arXiv Detail & Related papers (2020-07-17T21:04:50Z) - Large-Scale Gradient-Free Deep Learning with Recursive Local
Representation Alignment [84.57874289554839]
Training deep neural networks on large-scale datasets requires significant hardware resources.
Backpropagation, the workhorse for training these networks, is an inherently sequential process that is difficult to parallelize.
We propose a neuro-biologically-plausible alternative to backprop that can be used to train deep networks.
arXiv Detail & Related papers (2020-02-10T16:20:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.