EGAD: Evolving Graph Representation Learning with Self-Attention and
Knowledge Distillation for Live Video Streaming Events
- URL: http://arxiv.org/abs/2011.05705v1
- Date: Wed, 11 Nov 2020 11:16:52 GMT
- Title: EGAD: Evolving Graph Representation Learning with Self-Attention and
Knowledge Distillation for Live Video Streaming Events
- Authors: Stefanos Antaris, Dimitrios Rafailidis, Sarunas Girdzijauskas
- Abstract summary: We present a dynamic graph representation learning model on weighted graphs to accurately predict the network capacity of connections between viewers in a live video streaming event.
We propose EGAD, a neural network architecture to capture the graph evolution by introducing a self-attention mechanism on the weights between consecutive graph convolutional networks.
- Score: 4.332367445046418
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In this study, we present a dynamic graph representation learning model on
weighted graphs to accurately predict the network capacity of connections
between viewers in a live video streaming event. We propose EGAD, a neural
network architecture to capture the graph evolution by introducing a
self-attention mechanism on the weights between consecutive graph convolutional
networks. In addition, we account for the fact that neural architectures
require a huge amount of parameters to train, thus increasing the online
inference latency and negatively influencing the user experience in a live
video streaming event. To address the problem of the high online inference of a
vast number of parameters, we propose a knowledge distillation strategy. In
particular, we design a distillation loss function, aiming to first pretrain a
teacher model on offline data, and then transfer the knowledge from the teacher
to a smaller student model with less parameters. We evaluate our proposed model
on the link prediction task on three real-world datasets, generated by live
video streaming events. The events lasted 80 minutes and each viewer exploited
the distribution solution provided by the company Hive Streaming AB. The
experiments demonstrate the effectiveness of the proposed model in terms of
link prediction accuracy and number of required parameters, when evaluated
against state-of-the-art approaches. In addition, we study the distillation
performance of the proposed model in terms of compression ratio for different
distillation strategies, where we show that the proposed model can achieve a
compression ratio up to 15:100, preserving high link prediction accuracy. For
reproduction purposes, our evaluation datasets and implementation are publicly
available at https://stefanosantaris.github.io/EGAD.
Related papers
- Better with Less: A Data-Active Perspective on Pre-Training Graph Neural
Networks [39.71761440499148]
Pre-training on graph neural networks (GNNs) aims to learn transferable knowledge for downstream tasks with unlabeled data.
We propose a better-with-less framework for graph pre-training: fewer, but carefully chosen data are fed into a GNN model.
Experiment results show that the proposed APT is able to obtain an efficient pre-training model with fewer training data and better downstream performance.
arXiv Detail & Related papers (2023-11-02T07:09:59Z) - A Control-Centric Benchmark for Video Prediction [69.22614362800692]
We propose a benchmark for action-conditioned video prediction in the form of a control benchmark.
Our benchmark includes simulated environments with 11 task categories and 310 task instance definitions.
We then leverage our benchmark to study the effects of scaling model size, quantity of training data, and model ensembling.
arXiv Detail & Related papers (2023-04-26T17:59:45Z) - Directed Acyclic Graph Factorization Machines for CTR Prediction via
Knowledge Distillation [65.62538699160085]
We propose a Directed Acyclic Graph Factorization Machine (KD-DAGFM) to learn the high-order feature interactions from existing complex interaction models for CTR prediction via Knowledge Distillation.
KD-DAGFM achieves the best performance with less than 21.5% FLOPs of the state-of-the-art method on both online and offline experiments.
arXiv Detail & Related papers (2022-11-21T03:09:42Z) - Robust Causal Graph Representation Learning against Confounding Effects [21.380907101361643]
We propose Robust Causal Graph Representation Learning (RCGRL) to learn robust graph representations against confounding effects.
RCGRL introduces an active approach to generate instrumental variables under unconditional moment restrictions, which empowers the graph representation learning model to eliminate confounders.
arXiv Detail & Related papers (2022-08-18T01:31:25Z) - A Graph-Enhanced Click Model for Web Search [67.27218481132185]
We propose a novel graph-enhanced click model (GraphCM) for web search.
We exploit both intra-session and inter-session information for the sparsity and cold-start problems.
arXiv Detail & Related papers (2022-06-17T08:32:43Z) - Meta-Reinforcement Learning via Buffering Graph Signatures for Live
Video Streaming Events [4.332367445046418]
We present a meta-learning model to adapt the predictions of the network's capacity between viewers who participate in a live video streaming event.
We evaluate the proposed model on the link weight prediction task on three real-world of live video streaming events.
arXiv Detail & Related papers (2021-10-03T14:03:22Z) - STAR: Sparse Transformer-based Action Recognition [61.490243467748314]
This work proposes a novel skeleton-based human action recognition model with sparse attention on the spatial dimension and segmented linear attention on the temporal dimension of data.
Experiments show that our model can achieve comparable performance while utilizing much less trainable parameters and achieve high speed in training and inference.
arXiv Detail & Related papers (2021-07-15T02:53:11Z) - FitVid: Overfitting in Pixel-Level Video Prediction [117.59339756506142]
We introduce a new architecture, named FitVid, which is capable of severe overfitting on the common benchmarks.
FitVid outperforms the current state-of-the-art models across four different video prediction benchmarks on four different metrics.
arXiv Detail & Related papers (2021-06-24T17:20:21Z) - TCL: Transformer-based Dynamic Graph Modelling via Contrastive Learning [87.38675639186405]
We propose a novel graph neural network approach, called TCL, which deals with the dynamically-evolving graph in a continuous-time fashion.
To the best of our knowledge, this is the first attempt to apply contrastive learning to representation learning on dynamic graphs.
arXiv Detail & Related papers (2021-05-17T15:33:25Z) - Distill2Vec: Dynamic Graph Representation Learning with Knowledge
Distillation [4.568777157687959]
We propose Distill2Vec, a knowledge distillation strategy to train a compact model with a low number of trainable parameters.
Our experiments with publicly available datasets show the superiority of our proposed model over several state-of-the-art approaches.
arXiv Detail & Related papers (2020-11-11T09:49:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.