A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond

• URL: http://arxiv.org/abs/2403.14734v4
• Date: Fri, 01 Nov 2024 12:45:16 GMT
• Title: A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond
• Authors: Qiushi Sun, Zhirui Chen, Fangzhi Xu, Kanzhi Cheng, Chang Ma, Zhangyue Yin, Jianing Wang, Chengcheng Han, Renyu Zhu, Shuai Yuan, Qipeng Guo, Xipeng Qiu, Pengcheng Yin, Xiaoli Li, Fei Yuan, Lingpeng Kong, Xiang Li, Zhiyong Wu,
• Abstract summary: This survey presents a systematic review of the advancements in code intelligence. It covers over 50 representative models and their variants, more than 20 categories of tasks, and an extensive coverage of over 680 related works. Building on our examination of the developmental trajectories, we further investigate the emerging synergies between code intelligence and broader machine intelligence.
• Score: 84.95530356322621
• License:
• Abstract: Neural Code Intelligence -- leveraging deep learning to understand, generate, and optimize code -- holds immense potential for transformative impacts on the whole society. Bridging the gap between Natural Language and Programming Language, this domain has drawn significant attention from researchers in both research communities over the past few years. This survey presents a systematic and chronological review of the advancements in code intelligence, encompassing over 50 representative models and their variants, more than 20 categories of tasks, and an extensive coverage of over 680 related works. We follow the historical progression to trace the paradigm shifts across different research phases (e.g., from modeling code with recurrent neural networks to the era of Large Language Models). Concurrently, we highlight the major technical transitions in models, tasks, and evaluations spanning through different stages. For applications, we also observe a co-evolving shift. It spans from initial endeavors to tackling specific scenarios, through exploring a diverse array of tasks during its rapid expansion, to currently focusing on tackling increasingly complex and varied real-world challenges. Building on our examination of the developmental trajectories, we further investigate the emerging synergies between code intelligence and broader machine intelligence, uncovering new cross-domain opportunities and illustrating the substantial influence of code intelligence across various domains. Finally, we delve into both the opportunities and challenges associated with this field, alongside elucidating our insights on the most promising research directions. An ongoing, dynamically updated project and resources associated with this survey have been released at https://github.com/QiushiSun/Awesome-Code-Intelligence.

Related papers

• Quality Diversity for Robot Learning: Limitations and Future Directions [6.818634856205417]
  Quality Diversity (QD) has shown great success in discovering high-performing, diverse policies for robot skill learning. We argue that new paradigms must be developed to facilitate open-ended search and generalizability.
  arXiv  Detail & Related papers  (2024-07-09T23:29:54Z)
• Coding for Intelligence from the Perspective of Category [66.14012258680992]
  Coding targets compressing and reconstructing data, and intelligence. Recent trends demonstrate the potential homogeneity of these two fields. We propose a novel problem of Coding for Intelligence from the category theory view.
  arXiv  Detail & Related papers  (2024-07-01T07:05:44Z)
• Large Language Models for Networking: Workflow, Advances and Challenges [19.104593453342304]
  The networking field is characterized by its high complexity and rapid iteration. Traditional machine learning-based methods struggle to generalize and automate complex tasks in networking. The recent emergence of large language models (LLMs) has sparked a new wave of possibilities in addressing these challenges.
  arXiv  Detail & Related papers  (2024-04-19T14:17:02Z)
• On the Challenges and Opportunities in Generative AI [135.2754367149689]
  We argue that current large-scale generative AI models do not sufficiently address several fundamental issues that hinder their widespread adoption across domains. In this work, we aim to identify key unresolved challenges in modern generative AI paradigms that should be tackled to further enhance their capabilities, versatility, and reliability.
  arXiv  Detail & Related papers  (2024-02-28T15:19:33Z)
• Recent Advances in Hate Speech Moderation: Multimodality and the Role of Large Models [52.24001776263608]
  This comprehensive survey delves into the recent strides in HS moderation. We highlight the burgeoning role of large language models (LLMs) and large multimodal models (LMMs) We identify existing gaps in research, particularly in the context of underrepresented languages and cultures.
  arXiv  Detail & Related papers  (2024-01-30T03:51:44Z)
• A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning [76.47138162283714]
  Forgetting refers to the loss or deterioration of previously acquired information or knowledge. Forgetting is a prevalent phenomenon observed in various other research domains within deep learning. Survey argues that forgetting is a double-edged sword and can be beneficial and desirable in certain cases.
  arXiv  Detail & Related papers  (2023-07-16T16:27:58Z)
• A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks [60.38369406877899]
  Transformer is a deep neural network that employs a self-attention mechanism to comprehend the contextual relationships within sequential data. transformer models excel in handling long dependencies between input sequence elements and enable parallel processing. Our survey encompasses the identification of the top five application domains for transformer-based models.
  arXiv  Detail & Related papers  (2023-06-11T23:13:51Z)
• Evolutionary Multitask Optimization: a Methodological Overview, Challenges and Future Research Directions [8.14509634354919]
  We consider multitasking in the context of solving multiple optimization problems simultaneously by conducting a single search process. The emerging paradigm of Evolutionary Multitasking tackles multitask optimization scenarios by using as inspiration concepts drawn from Evolutionary Computation.
  arXiv  Detail & Related papers  (2021-02-04T11:48:11Z)
• Multimodal Research in Vision and Language: A Review of Current and Emerging Trends [41.07256031348454]
  We present a detailed overview of the latest trends in research pertaining to visual and language modalities. We look at its applications in their task formulations and how to solve various problems related to semantic perception and content generation. We shed some light on multi-disciplinary patterns and insights that have emerged in the recent past, directing this field towards more modular and transparent intelligent systems.
  arXiv  Detail & Related papers  (2020-10-19T13:55:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.