Advancing Language Models for Code-related Tasks

• URL: http://arxiv.org/abs/2601.04526v1
• Date: Thu, 08 Jan 2026 02:48:01 GMT
• Title: Advancing Language Models for Code-related Tasks
• Authors: Zhao Tian,
• Abstract summary: Recent advances in language models (LMs) have driven significant progress in various software engineering tasks.<n>However, existing LMs still struggle with complex programming scenarios due to limitations in data quality, model architecture, and reasoning capability.<n>This research systematically addresses these challenges through three complementary directions.
• Score: 3.1580161263510855
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in language models (LMs) have driven significant progress in various software engineering tasks. However, existing LMs still struggle with complex programming scenarios due to limitations in data quality, model architecture, and reasoning capability. This research systematically addresses these challenges through three complementary directions: (1) improving code data quality with a code difference-guided adversarial augmentation technique (CODA) and a code denoising technique (CodeDenoise); (2) enhancing model architecture via syntax-guided code LMs (LEAM and LEAM++); and (3) advancing model reasoning with a prompting technique (muFiX) and an agent-based technique (Specine). These techniques aim to promote the practical adoption of LMs in software development and further advance intelligent software engineering.

Related papers

• From Code Foundation Models to Agents and Applications: A Practical Guide to Code Intelligence [150.3696990310269]
  Large language models (LLMs) have transformed automated software development by enabling direct translation of natural language descriptions into functional code.<n>We provide a comprehensive synthesis and practical guide (a series of analytic and probing experiments) about code LLMs.<n>We analyze the code capability of the general LLMs (GPT-4, Claude, LLaMA) and code-specialized LLMs (StarCoder, Code LLaMA, DeepSeek-Coder, and QwenCoder)
  arXiv  Detail & Related papers  (2025-11-23T17:09:34Z)
• A Survey of Vibe Coding with Large Language Models [93.88284590533242]
  "Vibe Coding" is a development methodology where developers validate AI-generated implementations through outcome observation.<n>Despite its transformative potential, the effectiveness of this emergent paradigm remains under-explored.<n>This survey provides the first comprehensive and systematic review of Vibe Coding with large language models.
  arXiv  Detail & Related papers  (2025-10-14T11:26:56Z)
• A Survey on Code Generation with LLM-based Agents [61.474191493322415]
  Code generation agents powered by large language models (LLMs) are revolutionizing the software development paradigm.<n>LLMs are characterized by three core features.<n>This paper presents a systematic survey of the field of LLM-based code generation agents.
  arXiv  Detail & Related papers  (2025-07-31T18:17:36Z)
• Verbal Process Supervision Elicits Better Coding Agents [0.9558392439655016]
  This work introduces CURA, a code understanding and reasoning agent system enhanced with verbal process supervision (VPS)<n>CURA achieves a 3.65% improvement over baseline models on challenging benchmarks like BigCodeBench.
  arXiv  Detail & Related papers  (2025-03-24T09:48:59Z)
• Do Advanced Language Models Eliminate the Need for Prompt Engineering in Software Engineering? [18.726229967976316]
  This paper reevaluates various prompt engineering techniques within the context of advanced Large Language Models (LLMs) Our findings reveal that prompt engineering techniques developed for earlier LLMs may provide diminished benefits or even hinder performance when applied to advanced models.
  arXiv  Detail & Related papers  (2024-11-04T13:56:37Z)
• Augmenting software engineering with AI and developing it further towards AI-assisted model-driven software engineering [0.0]
  The paper provides an overview of the current state of AI-augmented software engineering and develops a corresponding taxonomy, ai4se.<n>A vision of AI-assisted big models in software development is put forth, with the aim of capitalising on the advantages inherent to both approaches.
  arXiv  Detail & Related papers  (2024-09-26T16:49:57Z)
• SIaM: Self-Improving Code-Assisted Mathematical Reasoning of Large Language Models [54.78329741186446]
  We propose a novel paradigm that uses a code-based critic model to guide steps including question-code data construction, quality control, and complementary evaluation. Experiments across both in-domain and out-of-domain benchmarks in English and Chinese demonstrate the effectiveness of the proposed paradigm.
  arXiv  Detail & Related papers  (2024-08-28T06:33:03Z)
• Data-Driven and SE-assisted AI Model Signal-Awareness Enhancement and Introspection [61.571331422347875]
  We propose a data-driven approach to enhance models' signal-awareness. We combine the SE concept of code complexity with the AI technique of curriculum learning. We achieve up to 4.8x improvement in model signal awareness.
  arXiv  Detail & Related papers  (2021-11-10T17:58:18Z)
• Technology Readiness Levels for AI & ML [79.22051549519989]
  Development of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. Engineering systems follow well-defined processes and testing standards to streamline development for high-quality, reliable results. We propose a proven systems engineering approach for machine learning development and deployment.
  arXiv  Detail & Related papers  (2020-06-21T17:14:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.