Computer Science Achievement and Writing Skills Predict Vibe Coding Proficiency

• URL: http://arxiv.org/abs/2603.14133v1
• Date: Sat, 14 Mar 2026 21:42:22 GMT
• Title: Computer Science Achievement and Writing Skills Predict Vibe Coding Proficiency
• Abstract summary: "vibe coding" allows one to specify programs in natural language and iterate from observed behavior without editing source code.<n>While its adoption is accelerating, little is known about which skills best predict success in this workflow.<n>We report a preregistered cross-sectional study with tertiary-level students who completed measures of computer-science achievement, domain-general cognitive skills, written-communication proficiency, and a vibe-coding assessment.
• Score: 5.449649175237595
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many software development platforms now support LLM-driven programming, or "vibe coding", a technique that allows one to specify programs in natural language and iterate from observed behavior, all without directly editing source code. While its adoption is accelerating, little is known about which skills best predict success in this workflow. We report a preregistered cross-sectional study with tertiary-level students (N = 100) who completed measures of computer-science achievement, domain-general cognitive skills, written-communication proficiency, and a vibe-coding assessment. Tasks were curated via an eight-expert consensus process and executed in a purpose-built, vibe-coding environment that mirrors commercial tools while enabling controlled evaluation. We find that both writing skill and CS achievement are significant predictors of vibe-coding performance, and that CS achievement remains a significant predictor after controlling for domain-general cognitive skills. The results may inform tool and curriculum design, including when to emphasize prompt-writing versus CS fundamentals to support future software creators.

Related papers

• Beyond the Traceback: Using LLMs for Adaptive Explanations of Programming Errors [52.619793817715326]
  We present a multi-stage crowdsourced study evaluating skill-targeted, LLM-generated Python error messages.<n>We show that while LLM-rewritten messages significantly improved subjective evaluations, these perceived gains did not translate into statistically significant improvements in objective debug performance.<n>This highlights a critical human-AI complementarity gap: explanations that feel better to users do not necessarily make them more effective debuggers.
  arXiv  Detail & Related papers  (2026-08-21T09:14:28Z)
• Vibe Coding Ate My Homework: An evaluation of AI approaches to greenfield software engineering and programming [0.0]
  This paper aims to evaluate the viability of vibe coding for greenfield software engineering tasks.<n>We have developed an evaluation suite for analysing an LLM's proficiency in carrying out simple, isolated greenfield programming tasks in Python.
  arXiv  Detail & Related papers  (2026-06-15T18:49:02Z)
• Code for All: Educational Applications of the "Vibe Coding" Hackathon in Programming Education across All Skill Levels [10.291969598378179]
  vibe coding is a natural language approach to programming in which users describe intent and AI generates or revises code.<n>We investigate educational value through a month-long online hackathon that welcomed participants from multiple countries.<n>Our findings describe how participants with different backgrounds engage with vibe coding as task complexity increases.
  arXiv  Detail & Related papers  (2026-04-24T17:48:40Z)
• Feasibility of AI-Assisted Programming for End-User Development [0.0]
  This paper investigates whether AI-assisted end-user coding is a feasible paradigm for end-user development.<n>We conducted a case study in which non-programmers were asked to develop a basic web app through interaction with AI assistants.<n>The majority of study participants successfully completed the task in reasonable time and also expressed support for AI-assisted end-user coding.
  arXiv  Detail & Related papers  (2025-12-05T12:13:21Z)
• "Can you feel the vibes?": An exploration of novice programmer engagement with vibe coding [42.82674998306379]
  "vibe coding" refers to creating software via natural language prompts rather than direct code authorship.<n>This paper reports on a one-day educational hackathon investigating how novice programmers and mixed-experience teams engage with vibe coding.
  arXiv  Detail & Related papers  (2025-12-02T13:32:23Z)
• From Code Foundation Models to Agents and Applications: A Practical Guide to Code Intelligence [150.3696990310269]
  Large language models (LLMs) have transformed automated software development by enabling direct translation of natural language descriptions into functional code.<n>We provide a comprehensive synthesis and practical guide (a series of analytic and probing experiments) about code LLMs.<n>We analyze the code capability of the general LLMs (GPT-4, Claude, LLaMA) and code-specialized LLMs (StarCoder, Code LLaMA, DeepSeek-Coder, and QwenCoder)
  arXiv  Detail & Related papers  (2025-11-23T17:09:34Z)
• From Code to Career: Assessing Competitive Programmers for Industry Placement [0.0]
  This study focuses on predicting the potential of Codeforces users to secure various levels of software engineering jobs.<n>We collect user data using the Codeforces API, process key performance metrics, and build a prediction model using a Random Forest.<n>The model categorizes users into four levels of employability, ranging from those needing further development to those ready for top-tier tech jobs.
  arXiv  Detail & Related papers  (2025-08-01T16:52:44Z)
• MERA Code: A Unified Framework for Evaluating Code Generation Across Tasks [56.34018316319873]
  We propose MERA Code, a benchmark for evaluating code for the latest code generation LLMs in Russian.<n>This benchmark includes 11 evaluation tasks that span 8 programming languages.<n>We evaluate open LLMs and frontier API models, analyzing their limitations in terms of practical coding tasks in non-English languages.
  arXiv  Detail & Related papers  (2025-07-16T14:31:33Z)
• CoderAgent: Simulating Student Behavior for Personalized Programming Learning with Large Language Models [34.62411261398559]
  We propose a LLM-based agent, CoderAgent, to simulate students' programming processes in a fine-grained manner without relying on real data.<n>Specifically, we equip each human learner with an intelligent agent, the core of which lies in capturing the cognitive states of the human programming practice process.
  arXiv  Detail & Related papers  (2025-05-27T02:43:38Z)
• Is Compression Really Linear with Code Intelligence? [60.123628177110206]
  textitFormat Annealing is a lightweight, transparent training methodology designed to assess the intrinsic capabilities of pre-trained models equitably.<n>Our empirical results reveal a fundamental logarithmic relationship between measured code intelligence and bits-per-character (BPC)<n>Our work provides a more nuanced understanding of compression's role in developing code intelligence and contributes a robust evaluation framework in the code domain.
  arXiv  Detail & Related papers  (2025-05-16T16:59:14Z)
• Toward Finding and Supporting Struggling Students in a Programming Course with an Early Warning System [1.192436948211501]
  We assess different cognitive skills of students of an introductory programming course. Most of the cognitive skills can predict whether students acquire programming skills to a certain degree. We found a significant positive effect of the syntax drill-and-practice exercises on the success of a course.
  arXiv  Detail & Related papers  (2024-01-25T12:55:27Z)
• Using Machine Learning To Identify Software Weaknesses From Software Requirement Specifications [49.1574468325115]
  This research focuses on finding an efficient machine learning algorithm to identify software weaknesses from requirement specifications. Keywords extracted using latent semantic analysis help map the CWE categories to PROMISE_exp. Naive Bayes, support vector machine (SVM), decision trees, neural network, and convolutional neural network (CNN) algorithms were tested.
  arXiv  Detail & Related papers  (2023-08-10T13:19:10Z)
• ChatDev: Communicative Agents for Software Development [84.90400377131962]
  ChatDev is a chat-powered software development framework in which specialized agents are guided in what to communicate. These agents actively contribute to the design, coding, and testing phases through unified language-based communication.
  arXiv  Detail & Related papers  (2023-07-16T02:11:34Z)
• Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completions [54.55334589363247]
  We study whether conveying information about uncertainty enables programmers to more quickly and accurately produce code. We find that highlighting tokens with the highest predicted likelihood of being edited leads to faster task completion and more targeted edits.
  arXiv  Detail & Related papers  (2023-02-14T18:43:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.