BESS: Balanced Entity Sampling and Sharing for Large-Scale Knowledge Graph Completion

• URL: http://arxiv.org/abs/2211.12281v1
• Date: Tue, 22 Nov 2022 13:51:33 GMT
• Title: BESS: Balanced Entity Sampling and Sharing for Large-Scale Knowledge Graph Completion
• Authors: Alberto Cattaneo, Daniel Justus, Harry Mellor, Douglas Orr, Jerome Maloberti, Zhenying Liu, Thorin Farnsworth, Andrew Fitzgibbon, Blazej Banaszewski, Carlo Luschi
• Abstract summary: We present the award-winning submission to the WikiKG90Mv2 track of OGB-LSC@NeurIPS 2022. The task is link-prediction on the large-scale knowledge graph WikiKG90Mv2, consisting of 90M+ nodes and 600M+ edges.
• Score: 1.083504248254529
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We present the award-winning submission to the WikiKG90Mv2 track of OGB-LSC@NeurIPS 2022. The task is link-prediction on the large-scale knowledge graph WikiKG90Mv2, consisting of 90M+ nodes and 600M+ edges. Our solution uses a diverse ensemble of $85$ Knowledge Graph Embedding models combining five different scoring functions (TransE, TransH, RotatE, DistMult, ComplEx) and two different loss functions (log-sigmoid, sampled softmax cross-entropy). Each individual model is trained in parallel on a Graphcore Bow Pod$_{16}$ using BESS (Balanced Entity Sampling and Sharing), a new distribution framework for KGE training and inference based on balanced collective communications between workers. Our final model achieves a validation MRR of 0.2922 and a test-challenge MRR of 0.2562, winning the first place in the competition. The code is publicly available at: https://github.com/graphcore/distributed-kge-poplar/tree/2022-ogb-submission.

Related papers

• Graph Sparsification via Mixture of Graphs [67.40204130771967]
  We introduce Mixture-of-Graphs (MoG) to dynamically select tailored pruning solutions for each node. MoG incorporates multiple sparsifier experts, each characterized by unique sparsity levels and pruning criteria, and selects the appropriate experts for each node. Experiments on four large-scale OGB datasets and two superpixel datasets, equipped with five GNNs, demonstrate that MoG identifies subgraphs at higher sparsity levels.
  arXiv  Detail & Related papers  (2024-05-23T07:40:21Z)
• Cooperative Multi-Agent Graph Bandits: UCB Algorithm and Regret Analysis [5.02063914741425]
  We formulate the multi-agent graph bandit problem as a multi-agent extension of the graph bandit problem introduced by Zhang, Johansson, and Li. We propose an Upper Confidence Bound (UCB)-based learning algorithm, Multi-G-UCB, and prove that its expected regret over $T$ steps is bounded by $O(gamma Nlog(T)[sqrtKT + DK])$.
  arXiv  Detail & Related papers  (2024-01-18T21:36:17Z)
• Inferences on Mixing Probabilities and Ranking in Mixed-Membership Models [5.992878098797828]
  Network data is prevalent in numerous big data applications including economics and health networks. In this paper, we model the network using the Degree-Corrected Mixed Membership (DCMM) model. We derive novel finite-sample expansion for the $boldsymbolpi_i(k)$s which allows us to obtain distributions and confidence interval of the membership mixing probabilities and other related population quantities.
  arXiv  Detail & Related papers  (2023-08-29T02:35:45Z)
• Distributionally Robust Classification on a Data Budget [26.69877485937123]
  We show that standard ResNet-50 trained with the cross-entropy loss on 2.4 million image samples can attain comparable robustness to a CLIP ResNet-50 trained on 400 million samples. This is the first result showing (near) state-of-the-art distributional robustness on limited data budgets.
  arXiv  Detail & Related papers  (2023-08-07T15:30:02Z)
• I$^2$SB: Image-to-Image Schr\"odinger Bridge [87.43524087956457]
  Image-to-Image Schr"odinger Bridge (I$2$SB) is a new class of conditional diffusion models. I$2$SB directly learns the nonlinear diffusion processes between two given distributions. We show that I$2$SB surpasses standard conditional diffusion models with more interpretable generative processes.
  arXiv  Detail & Related papers  (2023-02-12T08:35:39Z)
• Hierarchical Inference of the Lensing Convergence from Photometric Catalogs with Bayesian Graph Neural Networks [0.0]
  We introduce fluctuations on galaxy-galaxy lensing scales of $sim$1$''$ and extract random sightlines to train our BGNN. For each test set of 1,000 sightlines, the BGNN infers the individual $kappa$ posteriors, which we combine in a hierarchical Bayesian model. For a test field well sampled by the training set, the BGNN recovers the population mean of $kappa$ precisely and without bias.
  arXiv  Detail & Related papers  (2022-11-15T00:29:20Z)
• Modality-Aware Triplet Hard Mining for Zero-shot Sketch-Based Image Retrieval [51.42470171051007]
  This paper tackles the Zero-Shot Sketch-Based Image Retrieval (ZS-SBIR) problem from the viewpoint of cross-modality metric learning. By combining two fundamental learning approaches in DML, e.g., classification training and pairwise training, we set up a strong baseline for ZS-SBIR. We show that Modality-Aware Triplet Hard Mining (MATHM) enhances the baseline with three types of pairwise learning.
  arXiv  Detail & Related papers  (2021-12-15T08:36:44Z)
• Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoods [69.13371028670153]
  We introduce feature-contribution ratio (FCR) to study the viability of using inductive GNNs to solve the Strict Cold Start (SCS) problem. We experimentally show FCR disentangles the contributions of various components of graph datasets and demonstrate the superior performance of Cold Brew.
  arXiv  Detail & Related papers  (2021-11-08T21:29:25Z)
• Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction [123.20238648121445]
  We propose a new self-supervised learning framework, Graph Information Aided Node feature exTraction (GIANT) GIANT makes use of the eXtreme Multi-label Classification (XMC) formalism, which is crucial for fine-tuning the language model based on graph information. We demonstrate the superior performance of GIANT over the standard GNN pipeline on Open Graph Benchmark datasets.
  arXiv  Detail & Related papers  (2021-10-29T19:55:12Z)
• GraphMixup: Improving Class-Imbalanced Node Classification on Graphs by Self-supervised Context Prediction [25.679620842010422]
  This paper presents GraphMixup, a novel mixup-based framework for improving class-imbalanced node classification on graphs. We develop a emphReinforcement Mixup mechanism to adaptively determine how many samples are to be generated by mixup for those minority classes. Experiments on three real-world datasets show that GraphMixup yields truly encouraging results for class-imbalanced node classification tasks.
  arXiv  Detail & Related papers  (2021-06-21T14:12:16Z)
• Sequential Graph Convolutional Network for Active Learning [53.99104862192055]
  We propose a novel pool-based Active Learning framework constructed on a sequential Graph Convolution Network (GCN) With a small number of randomly sampled images as seed labelled examples, we learn the parameters of the graph to distinguish labelled vs unlabelled nodes. We exploit these characteristics of GCN to select the unlabelled examples which are sufficiently different from labelled ones.
  arXiv  Detail & Related papers  (2020-06-18T00:55:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.