Large Language Model Meets Graph Neural Network in Knowledge Distillation

• URL: http://arxiv.org/abs/2402.05894v4
• Date: Tue, 11 Jun 2024 13:17:12 GMT
• Title: Large Language Model Meets Graph Neural Network in Knowledge Distillation
• Authors: Shengxiang Hu, Guobing Zou, Song Yang, Yanglan Gan, Bofeng Zhang, Yixin Chen,
• Abstract summary: We propose a temporal-aware framework for predicting Quality of Service (QoS) in service-oriented architectures. Our proposed TOGCL framework significantly outperforms state-of-the-art methods across multiple metrics, achieving improvements of up to 38.80%.
• Score: 7.686812700685084
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In service-oriented architectures, accurately predicting the Quality of Service (QoS) is crucial for maintaining reliability and enhancing user satisfaction. However, significant challenges remain due to existing methods always overlooking high-order latent collaborative relationships between users and services and failing to dynamically adjust feature learning for every specific user-service invocation, which are critical for learning accurate features. Additionally, reliance on RNNs for capturing QoS evolution hampers models' ability to detect long-term trends due to difficulties in managing long-range dependencies. To address these challenges, we propose the \underline{T}arget-Prompt \underline{O}nline \underline{G}raph \underline{C}ollaborative \underline{L}earning (TOGCL) framework for temporal-aware QoS prediction. TOGCL leverages a dynamic user-service invocation graph to model historical interactions, providing a comprehensive representation of user-service relationships. Building on this graph, it develops a target-prompt graph attention network to extract online deep latent features of users and services at each time slice, simultaneously considering implicit collaborative relationships between target users/services and their neighbors, as well as relevant historical QoS values. Additionally, a multi-layer Transformer encoder is employed to uncover temporal feature evolution patterns of users and services, leading to temporal-aware QoS prediction. Extensive experiments conducted on the WS-DREAM dataset demonstrate that our proposed TOGCL framework significantly outperforms state-of-the-art methods across multiple metrics, achieving improvements of up to 38.80\%. These results underscore the effectiveness of the TOGCL framework for precise temporal QoS prediction.

Related papers

• Anomaly Resilient Temporal QoS Prediction using Hypergraph Convoluted Transformer Network [0.47248250311484113]
  Quality-of-Service (QoS) prediction is a critical task in the service lifecycle. Traditional methods often encounter data sparsity and cold-start issues. We introduce a real-time, trust-aware framework for temporal prediction.
  arXiv  Detail & Related papers  (2024-10-23T11:01:39Z)
• Satellite Streaming Video QoE Prediction: A Real-World Subjective Database and Network-Level Prediction Models [59.061552498630874]
  We introduce the LIVE-Viasat Real-World Satellite QoE Database. This database consists of 179 videos recorded from real-world streaming services affected by various authentic distortion patterns. We demonstrate the usefulness of this unique new resource by evaluating the efficacy of QoE-prediction models on it. We also created a new model that maps the network parameters to predicted human perception scores, which can be used by ISPs to optimize the video streaming quality of their networks.
  arXiv  Detail & Related papers  (2024-10-17T18:22:50Z)
• Web Service QoS Prediction via Extended Canonical Polyadic-based Tensor Network [2.2083091880368855]
  A Canonical Polyadic (CP)-based tensor network model has proven to be efficient for predicting dynamic data. Current CP-based tensor network models do not consider the correlation of users and services in the low-dimensional latent feature space. This paper proposes an Extended polyadic-based Network (ECTN) model to improve prediction accuracy.
  arXiv  Detail & Related papers  (2024-08-29T05:56:35Z)
• GACL: Graph Attention Collaborative Learning for Temporal QoS Prediction [5.040979636805073]
  We propose a novel Graph Collaborative Learning (GACL) framework for temporal prediction. It builds on a dynamic user-service graph to comprehensively model historical interactions. Experiments on the WS-DREAM dataset demonstrate that GACL significantly outperforms state-of-the-art methods for temporal prediction.
  arXiv  Detail & Related papers  (2024-08-20T05:38:47Z)
• A Pure Transformer Pretraining Framework on Text-attributed Graphs [50.833130854272774]
  We introduce a feature-centric pretraining perspective by treating graph structure as a prior. Our framework, Graph Sequence Pretraining with Transformer (GSPT), samples node contexts through random walks. GSPT can be easily adapted to both node classification and link prediction, demonstrating promising empirical success on various datasets.
  arXiv  Detail & Related papers  (2024-06-19T22:30:08Z)
• ARRQP: Anomaly Resilient Real-time QoS Prediction Framework with Graph Convolution [0.16317061277456998]
  We introduce a real-time prediction framework (called ARRQP) with a specific emphasis on improving resilience to anomalies in the data. ARRQP integrates both contextual information and collaborative insights, enabling a comprehensive understanding of user-service interactions. Results on the benchmark WS-DREAM dataset demonstrate the framework's effectiveness in achieving accurate and timely predictions.
  arXiv  Detail & Related papers  (2023-09-22T04:37:51Z)
• TPMCF: Temporal QoS Prediction using Multi-Source Collaborative Features [0.5161531917413706]
  Temporal Prediction is essential to identify a suitable service over time. Recent methods hardly achieved desired accuracy due to various limitations. This paper proposes a scalable strategy for Temporal Prediction using Multi-source Collaborative-Features.
  arXiv  Detail & Related papers  (2023-03-30T06:49:53Z)
• Dynamic Graph Message Passing Networks for Visual Recognition [112.49513303433606]
  Modelling long-range dependencies is critical for scene understanding tasks in computer vision. A fully-connected graph is beneficial for such modelling, but its computational overhead is prohibitive. We propose a dynamic graph message passing network, that significantly reduces the computational complexity.
  arXiv  Detail & Related papers  (2022-09-20T14:41:37Z)
• Ordinal Graph Gamma Belief Network for Social Recommender Systems [54.9487910312535]
  We develop a hierarchical Bayesian model termed ordinal graph factor analysis (OGFA), which jointly models user-item and user-user interactions. OGFA not only achieves good recommendation performance, but also extracts interpretable latent factors corresponding to representative user preferences. We extend OGFA to ordinal graph gamma belief network, which is a multi-stochastic-layer deep probabilistic model.
  arXiv  Detail & Related papers  (2022-09-12T09:19:22Z)
• Topology-based Clusterwise Regression for User Segmentation and Demand Forecasting [63.78344280962136]
  Using a public and a novel proprietary data set of commercial data, this research shows that the proposed system enables analysts to both cluster their user base and plan demand at a granular level. This work seeks to introduce TDA-based clustering of time series and clusterwise regression with matrix factorization methods as viable tools for the practitioner.
  arXiv  Detail & Related papers  (2020-09-08T12:10:10Z)
• Tensor Graph Convolutional Networks for Multi-relational and Robust Learning [74.05478502080658]
  This paper introduces a tensor-graph convolutional network (TGCN) for scalable semi-supervised learning (SSL) from data associated with a collection of graphs, that are represented by a tensor. The proposed architecture achieves markedly improved performance relative to standard GCNs, copes with state-of-the-art adversarial attacks, and leads to remarkable SSL performance over protein-to-protein interaction networks.
  arXiv  Detail & Related papers  (2020-03-15T02:33:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.