Language Semantic Graph Guided Data-Efficient Learning
- URL: http://arxiv.org/abs/2311.08782v1
- Date: Wed, 15 Nov 2023 08:54:57 GMT
- Title: Language Semantic Graph Guided Data-Efficient Learning
- Authors: Wenxuan Ma and Shuang Li and Lincan Cai and Jingxuan Kang
- Abstract summary: We introduce a Language Semantic Graph (LSG) which is constructed from labels manifest as natural language descriptions.
An auxiliary graph neural network is trained to extract high-level semantic relations and then used to guide the training of the primary model.
Our in-depth analysis shows that the LSG method also expedites the training process.
- Score: 10.039953846594805
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Developing generalizable models that can effectively learn from limited data
and with minimal reliance on human supervision is a significant objective
within the machine learning community, particularly in the era of deep neural
networks. Therefore, to achieve data-efficient learning, researchers typically
explore approaches that can leverage more related or unlabeled data without
necessitating additional manual labeling efforts, such as Semi-Supervised
Learning (SSL), Transfer Learning (TL), and Data Augmentation (DA). SSL
leverages unlabeled data in the training process, while TL enables the transfer
of expertise from related data distributions. DA broadens the dataset by
synthesizing new data from existing examples. However, the significance of
additional knowledge contained within labels has been largely overlooked in
research. In this paper, we propose a novel perspective on data efficiency that
involves exploiting the semantic information contained in the labels of the
available data. Specifically, we introduce a Language Semantic Graph (LSG)
which is constructed from labels manifest as natural language descriptions.
Upon this graph, an auxiliary graph neural network is trained to extract
high-level semantic relations and then used to guide the training of the
primary model, enabling more adequate utilization of label knowledge. Across
image, video, and audio modalities, we utilize the LSG method in both TL and
SSL scenarios and illustrate its versatility in significantly enhancing
performance compared to other data-efficient learning approaches. Additionally,
our in-depth analysis shows that the LSG method also expedites the training
process.
Related papers
- ExGRG: Explicitly-Generated Relation Graph for Self-Supervised Representation Learning [4.105236597768038]
Self-supervised learning has emerged as a powerful technique in pre-training deep learning models.
This paper introduces a novel non-contrastive SSL approach to Explicitly Generate a compositional Relation Graph.
arXiv Detail & Related papers (2024-02-09T19:16:04Z) - Efficient Large Language Models Fine-Tuning On Graphs [23.19795835873144]
Learning from Text-Attributed Graphs (TAGs) has attracted significant attention due to its wide range of real-world applications.
We introduce a novel and efficient approach for the end-to-end fine-tuning of Large Language Models (LLMs) on TAGs, named LEADING.
arXiv Detail & Related papers (2023-12-07T22:35:16Z) - Robotic Skill Acquisition via Instruction Augmentation with
Vision-Language Models [70.82705830137708]
We introduce Data-driven Instruction Augmentation for Language-conditioned control (DIAL)
We utilize semi-language labels leveraging the semantic understanding of CLIP to propagate knowledge onto large datasets of unlabelled demonstration data.
DIAL enables imitation learning policies to acquire new capabilities and generalize to 60 novel instructions unseen in the original dataset.
arXiv Detail & Related papers (2022-11-21T18:56:00Z) - Does Decentralized Learning with Non-IID Unlabeled Data Benefit from
Self Supervision? [51.00034621304361]
We study decentralized learning with unlabeled data through the lens of self-supervised learning (SSL)
We study the effectiveness of contrastive learning algorithms under decentralized learning settings.
arXiv Detail & Related papers (2022-10-20T01:32:41Z) - A Survey of Learning on Small Data: Generalization, Optimization, and
Challenge [101.27154181792567]
Learning on small data that approximates the generalization ability of big data is one of the ultimate purposes of AI.
This survey follows the active sampling theory under a PAC framework to analyze the generalization error and label complexity of learning on small data.
Multiple data applications that may benefit from efficient small data representation are surveyed.
arXiv Detail & Related papers (2022-07-29T02:34:19Z) - Self-supervised on Graphs: Contrastive, Generative,or Predictive [25.679620842010422]
Self-supervised learning (SSL) is emerging as a new paradigm for extracting informative knowledge through well-designed pretext tasks.
We divide existing graph SSL methods into three categories: contrastive, generative, and predictive.
We also summarize the commonly used datasets, evaluation metrics, downstream tasks, and open-source implementations of various algorithms.
arXiv Detail & Related papers (2021-05-16T03:30:03Z) - Graph-based Semi-supervised Learning: A Comprehensive Review [51.26862262550445]
Semi-supervised learning (SSL) has tremendous value in practice due to its ability to utilize both labeled data and unlabelled data.
An important class of SSL methods is to naturally represent data as graphs, which corresponds to graph-based semi-supervised learning (GSSL) methods.
GSSL methods have demonstrated their advantages in various domains due to their uniqueness of structure, the universality of applications, and their scalability to large scale data.
arXiv Detail & Related papers (2021-02-26T05:11:09Z) - Contrastive and Generative Graph Convolutional Networks for Graph-based
Semi-Supervised Learning [64.98816284854067]
Graph-based Semi-Supervised Learning (SSL) aims to transfer the labels of a handful of labeled data to the remaining massive unlabeled data via a graph.
A novel GCN-based SSL algorithm is presented in this paper to enrich the supervision signals by utilizing both data similarities and graph structure.
arXiv Detail & Related papers (2020-09-15T13:59:28Z) - Uncovering the structure of clinical EEG signals with self-supervised
learning [64.4754948595556]
Supervised learning paradigms are often limited by the amount of labeled data that is available.
This phenomenon is particularly problematic in clinically-relevant data, such as electroencephalography (EEG)
By extracting information from unlabeled data, it might be possible to reach competitive performance with deep neural networks.
arXiv Detail & Related papers (2020-07-31T14:34:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.