Automatic Smart Contract Comment Generation via Large Language Models
and In-Context Learning
- URL: http://arxiv.org/abs/2311.10388v2
- Date: Tue, 16 Jan 2024 07:58:25 GMT
- Title: Automatic Smart Contract Comment Generation via Large Language Models
and In-Context Learning
- Authors: Junjie Zhao and Xiang Chen and Guang Yang and Yiheng Shen
- Abstract summary: In this study, we propose an approach SCCLLM based on large language models (LLMs) and in-context learning.
Specifically, in the demonstration selection phase, SCCLLM retrieves the top-k code snippets from the historical corpus.
In the in-context learning phase, SCCLLM utilizes the retrieved code snippets as demonstrations.
- Score: 11.52122354673779
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The previous smart contract code comment (SCC) generation approaches can be
divided into two categories: fine-tuning paradigm-based approaches and
information retrieval-based approaches. However, for the fine-tuning
paradigm-based approaches, the performance may be limited by the quality of the
gathered dataset for the downstream task and they may have knowledge-forgetting
issues. While for the information retrieval-based approaches, it is difficult
for them to generate high-quality comments if similar code does not exist in
the historical repository. Therefore we want to utilize the domain knowledge
related to SCC generation in large language models (LLMs) to alleviate the
disadvantages of these two types of approaches. In this study, we propose an
approach SCCLLM based on LLMs and in-context learning. Specifically, in the
demonstration selection phase, SCCLLM retrieves the top-k code snippets from
the historical corpus by considering syntax, semantics, and lexical
information. In the in-context learning phase, SCCLLM utilizes the retrieved
code snippets as demonstrations, which can help to utilize the related
knowledge for this task. We select a large corpus from a smart contract
community Etherscan.io as our experimental subject. Extensive experimental
results show the effectiveness of SCCLLM when compared with baselines in
automatic evaluation and human evaluation.
Related papers
- Soft Prompting for Unlearning in Large Language Models [11.504012974208466]
This work focuses on investigating machine unlearning for Large Language Models motivated by data protection regulations.
We propose a framework textbfSoft textbfPrompting for textbfUntextbflearning (SPUL) that learns prompt tokens that can be appended to an arbitrary query to induce unlearning.
arXiv Detail & Related papers (2024-06-17T19:11:40Z) - TokenRec: Learning to Tokenize ID for LLM-based Generative Recommendation [16.93374578679005]
TokenRec is a novel framework for tokenizing and retrieving large-scale language models (LLMs) based Recommender Systems (RecSys)
Our strategy, Masked Vector-Quantized (MQ) Tokenizer, quantizes the masked user/item representations learned from collaborative filtering into discrete tokens.
Our generative retrieval paradigm is designed to efficiently recommend top-$K$ items for users to eliminate the need for auto-regressive decoding and beam search processes.
arXiv Detail & Related papers (2024-06-15T00:07:44Z) - Evaluating Generative Language Models in Information Extraction as Subjective Question Correction [49.729908337372436]
We propose a new evaluation method, SQC-Score.
Inspired by the principles in subjective question correction, we propose a new evaluation method, SQC-Score.
Results on three information extraction tasks show that SQC-Score is more preferred by human annotators than the baseline metrics.
arXiv Detail & Related papers (2024-04-04T15:36:53Z) - Generative Context-aware Fine-tuning of Self-supervised Speech Models [54.389711404209415]
We study the use of generative large language models (LLM) generated context information.
We propose an approach to distill the generated information during fine-tuning of self-supervised speech models.
We evaluate the proposed approach using the SLUE and Libri-light benchmarks for several downstream tasks: automatic speech recognition, named entity recognition, and sentiment analysis.
arXiv Detail & Related papers (2023-12-15T15:46:02Z) - Large Language Models Are Zero-Shot Text Classifiers [3.617781755808837]
Large language models (LLMs) have become extensively used across various sub-disciplines of natural language processing (NLP)
In NLP, text classification problems have garnered considerable focus, but still faced with some limitations related to expensive computational cost, time consumption, and robust performance to unseen classes.
With the proposal of chain of thought prompting (CoT), LLMs can be implemented using zero-shot learning (ZSL) with the step by step reasoning prompts.
arXiv Detail & Related papers (2023-12-02T06:33:23Z) - Large Language Models can Contrastively Refine their Generation for Better Sentence Representation Learning [57.74233319453229]
Large language models (LLMs) have emerged as a groundbreaking technology and their unparalleled text generation capabilities have sparked interest in their application to the fundamental sentence representation learning task.
We propose MultiCSR, a multi-level contrastive sentence representation learning framework that decomposes the process of prompting LLMs to generate a corpus.
Our experiments reveal that MultiCSR enables a less advanced LLM to surpass the performance of ChatGPT, while applying it to ChatGPT achieves better state-of-the-art results.
arXiv Detail & Related papers (2023-10-17T03:21:43Z) - Large Language Model-Aware In-Context Learning for Code Generation [75.68709482932903]
Large language models (LLMs) have shown impressive in-context learning (ICL) ability in code generation.
We propose a novel learning-based selection approach named LAIL (LLM-Aware In-context Learning) for code generation.
arXiv Detail & Related papers (2023-10-15T06:12:58Z) - Scalable Learning of Latent Language Structure With Logical Offline
Cycle Consistency [71.42261918225773]
Conceptually, LOCCO can be viewed as a form of self-learning where the semantic being trained is used to generate annotations for unlabeled text.
As an added bonus, the annotations produced by LOCCO can be trivially repurposed to train a neural text generation model.
arXiv Detail & Related papers (2023-05-31T16:47:20Z) - Large Language Models are Few-Shot Summarizers: Multi-Intent Comment
Generation via In-Context Learning [34.006227676170504]
This study investigates the feasibility of utilizing large language models (LLMs) to generate comments that can fulfill developers' diverse intents.
Experiments on two large-scale datasets demonstrate the rationale of our insights.
arXiv Detail & Related papers (2023-04-22T12:26:24Z) - Compositional Exemplars for In-context Learning [21.961094715261133]
Large pretrained language models (LMs) have shown impressive In-Context Learning (ICL) ability.
We propose CEIL (Compositional Exemplars for In-context Learning) to model the interaction between the given input and in-context examples.
We validate CEIL on 12 classification and generation datasets from 7 distinct NLP tasks, including sentiment analysis, paraphrase detection, natural language inference, commonsense reasoning, open-domain question answering, code generation, and semantic parsing.
arXiv Detail & Related papers (2023-02-11T14:02:08Z) - Using Representation Expressiveness and Learnability to Evaluate
Self-Supervised Learning Methods [61.49061000562676]
We introduce Cluster Learnability (CL) to assess learnability.
CL is measured in terms of the performance of a KNN trained to predict labels obtained by clustering the representations with K-means.
We find that CL better correlates with in-distribution model performance than other competing recent evaluation schemes.
arXiv Detail & Related papers (2022-06-02T19:05:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.