ChatGPT in the Classroom: An Analysis of Its Strengths and Weaknesses for Solving Undergraduate Computer Science Questions

• URL: http://arxiv.org/abs/2304.14993v4
• Date: Fri, 6 Oct 2023 11:45:52 GMT
• Title: ChatGPT in the Classroom: An Analysis of Its Strengths and Weaknesses for Solving Undergraduate Computer Science Questions
• Authors: Ishika Joshi, Ritvik Budhiraja, Harshal Dev, Jahnvi Kadia, M. Osama Ataullah, Sayan Mitra, Dhruv Kumar, Harshal D. Akolekar
• Abstract summary: ChatGPT is an AI language model developed by OpenAI that can understand and generate human-like text. There is concern that students may leverage ChatGPT to complete take-home assignments and exams and obtain favorable grades without genuinely acquiring knowledge.
• Score: 5.962828109329824
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: ChatGPT is an AI language model developed by OpenAI that can understand and generate human-like text. It can be used for a variety of use cases such as language generation, question answering, text summarization, chatbot development, language translation, sentiment analysis, content creation, personalization, text completion, and storytelling. While ChatGPT has garnered significant positive attention, it has also generated a sense of apprehension and uncertainty in academic circles. There is concern that students may leverage ChatGPT to complete take-home assignments and exams and obtain favorable grades without genuinely acquiring knowledge. This paper adopts a quantitative approach to demonstrate ChatGPT's high degree of unreliability in answering a diverse range of questions pertaining to topics in undergraduate computer science. Our analysis shows that students may risk self-sabotage by blindly depending on ChatGPT to complete assignments and exams. We build upon this analysis to provide constructive recommendations to both students and instructors.

Related papers

• ChatGPT in Research and Education: Exploring Benefits and Threats [1.9466452723529557]
  ChatGPT is a powerful language model developed by OpenAI. It offers personalized feedback, enhances accessibility, enables interactive conversations, assists with lesson preparation and evaluation, and introduces new methods for teaching complex subjects. ChatGPT also poses challenges to traditional education and research systems. These challenges include the risk of cheating on online exams, the generation of human-like text that may compromise academic integrity, and difficulties in assessing the reliability of information generated by AI.
  arXiv  Detail & Related papers  (2024-11-05T05:29:00Z)
• Integrating ChatGPT in a Computer Science Course: Students Perceptions and Suggestions [0.0]
  This experience report explores students' perceptions and suggestions for integrating ChatGPT in a computer science course. Findings show the importance of carefully balancing using ChatGPT in computer science courses.
  arXiv  Detail & Related papers  (2023-12-22T10:48:34Z)
• Exploring ChatGPT's Capabilities on Vulnerability Management [56.4403395100589]
  We explore ChatGPT's capabilities on 6 tasks involving the complete vulnerability management process with a large-scale dataset containing 70,346 samples. One notable example is ChatGPT's proficiency in tasks like generating titles for software bug reports. Our findings reveal the difficulties encountered by ChatGPT and shed light on promising future directions.
  arXiv  Detail & Related papers  (2023-11-11T11:01:13Z)
• Chatbot-supported Thesis Writing: An Autoethnographic Report [0.0]
  ChatGPT might be applied to formats that require learners to generate text, such as bachelor theses or student research papers. ChatGPT is to be valued as a beneficial tool in thesis writing. However, writing a conclusive thesis still requires the learner's meaningful engagement.
  arXiv  Detail & Related papers  (2023-10-14T09:09:26Z)
• Can ChatGPT pass the Vietnamese National High School Graduation Examination? [0.0]
  The study dataset included 30 essays in the literature test case and 1,700 multiple-choice questions designed for other subjects. ChatGPT was able to pass the examination with an average score of 6-7, demonstrating the technology's potential to revolutionize the educational landscape.
  arXiv  Detail & Related papers  (2023-06-15T14:47:03Z)
• Uncovering the Potential of ChatGPT for Discourse Analysis in Dialogue: An Empirical Study [51.079100495163736]
  This paper systematically inspects ChatGPT's performance in two discourse analysis tasks: topic segmentation and discourse parsing. ChatGPT demonstrates proficiency in identifying topic structures in general-domain conversations yet struggles considerably in specific-domain conversations. Our deeper investigation indicates that ChatGPT can give more reasonable topic structures than human annotations but only linearly parses the hierarchical rhetorical structures.
  arXiv  Detail & Related papers  (2023-05-15T07:14:41Z)
• Can ChatGPT Pass An Introductory Level Functional Language Programming Course? [2.3456295046913405]
  This paper aims to explore how well ChatGPT can perform in an introductory-level functional language programming course. Our comprehensive evaluation provides valuable insights into ChatGPT's impact from both student and instructor perspectives.
  arXiv  Detail & Related papers  (2023-04-29T20:30:32Z)
• ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning [70.57126720079971]
  Large language models (LLMs) have emerged as the most important breakthroughs in natural language processing (NLP) This paper evaluates ChatGPT on 7 different tasks, covering 37 diverse languages with high, medium, low, and extremely low resources. Compared to the performance of previous models, our extensive experimental results demonstrate a worse performance of ChatGPT for different NLP tasks and languages.
  arXiv  Detail & Related papers  (2023-04-12T05:08:52Z)
• To ChatGPT, or not to ChatGPT: That is the question! [78.407861566006]
  This study provides a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection. We have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains. Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.
  arXiv  Detail & Related papers  (2023-04-04T03:04:28Z)
• Is ChatGPT a General-Purpose Natural Language Processing Task Solver? [113.22611481694825]
  Large language models (LLMs) have demonstrated the ability to perform a variety of natural language processing (NLP) tasks zero-shot. Recently, the debut of ChatGPT has drawn a great deal of attention from the natural language processing (NLP) community. It is not yet known whether ChatGPT can serve as a generalist model that can perform many NLP tasks zero-shot.
  arXiv  Detail & Related papers  (2023-02-08T09:44:51Z)
• A Categorical Archive of ChatGPT Failures [47.64219291655723]
  ChatGPT, developed by OpenAI, has been trained using massive amounts of data and simulates human conversation. It has garnered significant attention due to its ability to effectively answer a broad range of human inquiries. However, a comprehensive analysis of ChatGPT's failures is lacking, which is the focus of this study.
  arXiv  Detail & Related papers  (2023-02-06T04:21:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.