Data-Driven and SE-assisted AI Model Signal-Awareness Enhancement and Introspection

• URL: http://arxiv.org/abs/2111.05827v1
• Date: Wed, 10 Nov 2021 17:58:18 GMT
• Title: Data-Driven and SE-assisted AI Model Signal-Awareness Enhancement and Introspection
• Authors: Sahil Suneja, Yufan Zhuang, Yunhui Zheng, Jim Laredo, Alessandro Morari
• Abstract summary: We propose a data-driven approach to enhance models' signal-awareness. We combine the SE concept of code complexity with the AI technique of curriculum learning. We achieve up to 4.8x improvement in model signal awareness.
• Score: 61.571331422347875
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: AI modeling for source code understanding tasks has been making significant progress, and is being adopted in production development pipelines. However, reliability concerns, especially whether the models are actually learning task-related aspects of source code, are being raised. While recent model-probing approaches have observed a lack of signal awareness in many AI-for-code models, i.e. models not capturing task-relevant signals, they do not offer solutions to rectify this problem. In this paper, we explore data-driven approaches to enhance models' signal-awareness: 1) we combine the SE concept of code complexity with the AI technique of curriculum learning; 2) we incorporate SE assistance into AI models by customizing Delta Debugging to generate simplified signal-preserving programs, augmenting them to the training dataset. With our techniques, we achieve up to 4.8x improvement in model signal awareness. Using the notion of code complexity, we further present a novel model learning introspection approach from the perspective of the dataset.

Related papers

• SRA-MCTS: Self-driven Reasoning Augmentation with Monte Carlo Tree Search for Code Generation [14.786100203787194]
  Large language models demonstrate exceptional performance in simple code generation tasks but face challenges in tackling complex problems. We propose a reasoning-augmented data generation process, SRA-MCTS, which guides the model to autonomously generate high-quality intermediate reasoning paths. Our method operates entirely through the model itself without requiring additional supervision.
  arXiv  Detail & Related papers  (2024-11-17T12:31:04Z)
• Two-Timescale Model Caching and Resource Allocation for Edge-Enabled AI-Generated Content Services [55.0337199834612]
  Generative AI (GenAI) has emerged as a transformative technology, enabling customized and personalized AI-generated content (AIGC) services. These services require executing GenAI models with billions of parameters, posing significant obstacles to resource-limited wireless edge. We introduce the formulation of joint model caching and resource allocation for AIGC services to balance a trade-off between AIGC quality and latency metrics.
  arXiv  Detail & Related papers  (2024-11-03T07:01:13Z)
• Next-Gen Software Engineering: AI-Assisted Big Models [0.0]
  This paper aims to facilitate a synthesis between models and AI in software engineering. The paper provides an overview of the current status of AI-assisted software engineering. A vision of AI-assisted Big Models in SE is put forth, with the aim of capitalising on the advantages inherent to both approaches.
  arXiv  Detail & Related papers  (2024-09-26T16:49:57Z)
• A comprehensible analysis of the efficacy of Ensemble Models for Bug Prediction [0.0]
  We present a comparison and analysis of the efficacy of two AI-based approaches, namely single AI models and ensemble AI models, for predicting the probability of a Java class being buggy. Our experimental findings indicate that the ensemble of AI models can outperform the results of applying individual AI models.
  arXiv  Detail & Related papers  (2023-10-18T17:43:54Z)
• AlerTiger: Deep Learning for AI Model Health Monitoring at LinkedIn [4.020770981811131]
  AlerTiger helps AI teams across the company monitor their AI models' health. System consists of four major steps: model statistics generation, deep-learning-based anomaly detection, anomaly post-processing, and user alerting.
  arXiv  Detail & Related papers  (2023-06-03T01:21:58Z)
• Enhancing Multiple Reliability Measures via Nuisance-extended Information Bottleneck [77.37409441129995]
  In practical scenarios where training data is limited, many predictive signals in the data can be rather from some biases in data acquisition. We consider an adversarial threat model under a mutual information constraint to cover a wider class of perturbations in training. We propose an autoencoder-based training to implement the objective, as well as practical encoder designs to facilitate the proposed hybrid discriminative-generative training.
  arXiv  Detail & Related papers  (2023-03-24T16:03:21Z)
• AI Model Utilization Measurements For Finding Class Encoding Patterns [2.702380921892937]
  This work addresses the problems of designing utilization measurements of trained artificial intelligence (AI) models. The problems are motivated by the lack of explainability of AI models in security and safety critical applications.
  arXiv  Detail & Related papers  (2022-12-12T02:18:10Z)
• CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning [92.36705236706678]
  "CodeRL" is a new framework for program synthesis tasks through pretrained LMs and deep reinforcement learning. During inference, we introduce a new generation procedure with a critical sampling strategy. For the model backbones, we extended the encoder-decoder architecture of CodeT5 with enhanced learning objectives.
  arXiv  Detail & Related papers  (2022-07-05T02:42:15Z)
• Enabling Automated Machine Learning for Model-Driven AI Engineering [60.09869520679979]
  We propose a novel approach to enable Model-Driven Software Engineering and Model-Driven AI Engineering. In particular, we support Automated ML, thus assisting software engineers without deep AI knowledge in developing AI-intensive systems.
  arXiv  Detail & Related papers  (2022-03-06T10:12:56Z)
• Model-Based Deep Learning [155.063817656602]
  Signal processing, communications, and control have traditionally relied on classical statistical modeling techniques. Deep neural networks (DNNs) use generic architectures which learn to operate from data, and demonstrate excellent performance. We are interested in hybrid techniques that combine principled mathematical models with data-driven systems to benefit from the advantages of both approaches.
  arXiv  Detail & Related papers  (2020-12-15T16:29:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.