Performance of ChatGPT on the US Fundamentals of Engineering Exam: Comprehensive Assessment of Proficiency and Potential Implications for Professional Environmental Engineering Practice

• URL: http://arxiv.org/abs/2304.12198v1
• Date: Thu, 20 Apr 2023 16:54:34 GMT
• Title: Performance of ChatGPT on the US Fundamentals of Engineering Exam: Comprehensive Assessment of Proficiency and Potential Implications for Professional Environmental Engineering Practice
• Authors: Vinay Pursnani, Yusuf Sermet, Ibrahim Demir
• Abstract summary: This study investigates the feasibility and effectiveness of using ChatGPT, a GPT-4 based model, in achieving satisfactory performance on the Fundamentals of Engineering (FE) Environmental Exam. The findings reflect remarkable improvements in mathematical capabilities across successive iterations of ChatGPT models, showcasing their potential in solving complex engineering problems.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In recent years, advancements in artificial intelligence (AI) have led to the development of large language models like GPT-4, demonstrating potential applications in various fields, including education. This study investigates the feasibility and effectiveness of using ChatGPT, a GPT-4 based model, in achieving satisfactory performance on the Fundamentals of Engineering (FE) Environmental Exam. This study further shows a significant improvement in the model's accuracy when answering FE exam questions through noninvasive prompt modifications, substantiating the utility of prompt modification as a viable approach to enhance AI performance in educational contexts. Furthermore, the findings reflect remarkable improvements in mathematical capabilities across successive iterations of ChatGPT models, showcasing their potential in solving complex engineering problems. Our paper also explores future research directions, emphasizing the importance of addressing AI challenges in education, enhancing accessibility and inclusion for diverse student populations, and developing AI-resistant exam questions to maintain examination integrity. By evaluating the performance of ChatGPT in the context of the FE Environmental Exam, this study contributes valuable insights into the potential applications and limitations of large language models in educational settings. As AI continues to evolve, these findings offer a foundation for further research into the responsible and effective integration of AI models across various disciplines, ultimately optimizing the learning experience and improving student outcomes.

Related papers

• Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvement [62.94719119451089]
  Lingma SWE-GPT series learns from and simulating real-world code submission activities. Lingma SWE-GPT 72B resolves 30.20% of GitHub issues, marking a significant improvement in automatic issue resolution.
  arXiv  Detail & Related papers  (2024-11-01T14:27:16Z)
• Evaluation of OpenAI o1: Opportunities and Challenges of AGI [112.0812059747033]
  o1-preview demonstrated remarkable capabilities, often achieving human-level or superior performance. The model excelled in tasks requiring intricate reasoning and knowledge integration across various fields. Overall results indicate significant progress towards artificial general intelligence.
  arXiv  Detail & Related papers  (2024-09-27T06:57:00Z)
• Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants [175.9723801486487]
  We evaluate whether two AI assistants, GPT-3.5 and GPT-4, can adequately answer assessment questions. GPT-4 answers an average of 65.8% of questions correctly, and can even produce the correct answer across at least one prompting strategy for 85.1% of questions. Our results call for revising program-level assessment design in higher education in light of advances in generative AI.
  arXiv  Detail & Related papers  (2024-08-07T12:11:49Z)
• Autonomous Prompt Engineering in Large Language Models [0.0]
  This research introduces the Automatic Prompt Engineering Toolbox (APET), which enables GPT-4 to autonomously apply prompt engineering techniques. APET empowers GPT-4 to dynamically optimize prompts, resulting in substantial improvements in tasks like Word Sorting. This research represents a significant leap in AI development, presenting a robust framework for future innovations in autonomous AI systems.
  arXiv  Detail & Related papers  (2024-06-25T10:14:44Z)
• The AI Companion in Education: Analyzing the Pedagogical Potential of ChatGPT in Computer Science and Engineering [1.120999712480549]
  This study aims to comprehensively analyze the pedagogical potential of ChatGPT in CSE education. We employ a systematic approach, creating a diverse range of educational practice problems within CSE field. According to our examinations, certain question types, like conceptual knowledge queries, typically do not pose significant challenges to ChatGPT.
  arXiv  Detail & Related papers  (2024-04-23T21:42:30Z)
• The Evolution of Learning: Assessing the Transformative Impact of Generative AI on Higher Education [0.0]
  Generative Artificial Intelligence models such as ChatGPT have experienced a surge in popularity. This research paper investigates the impact of GAI on university students and Higher Education Institutions.
  arXiv  Detail & Related papers  (2024-04-16T13:19:57Z)
• Integration of cognitive tasks into artificial general intelligence test for large models [54.72053150920186]
  We advocate for a comprehensive framework of cognitive science-inspired artificial general intelligence (AGI) tests. The cognitive science-inspired AGI tests encompass the full spectrum of intelligence facets, including crystallized intelligence, fluid intelligence, social intelligence, and embodied intelligence.
  arXiv  Detail & Related papers  (2024-02-04T15:50:42Z)
• Bringing Generative AI to Adaptive Learning in Education [58.690250000579496]
  We shed light on the intersectional studies of generative AI and adaptive learning. We argue that this union will contribute significantly to the development of the next-stage learning format in education.
  arXiv  Detail & Related papers  (2024-02-02T23:54:51Z)
• Evaluating Large Language Models on the GMAT: Implications for the Future of Business Education [0.13654846342364302]
  This study introduces the first benchmark to assess the performance of seven major Large Language Models (LLMs) Our analysis shows that most LLMs outperform human candidates, with GPT-4 Turbo not only outperforming the other models but also surpassing the average scores of graduate students at top business schools. While AI's promise in education, assessment, and tutoring is clear, challenges remain.
  arXiv  Detail & Related papers  (2024-01-02T03:54:50Z)
• AI-assisted Learning for Electronic Engineering Courses in High Education [2.67766280323297]
  The study includes the evaluation and reflection of different stakeholders: students, lecturers, and engineers. The findings of this study shed light on the benefits and limitations of ChatGPT as an AI tool, paving the way for innovative learning approaches in technical disciplines.
  arXiv  Detail & Related papers  (2023-11-02T07:48:10Z)
• Pre-Trained Models: Past, Present and Future [126.21572378910746]
  Large-scale pre-trained models (PTMs) have recently achieved great success and become a milestone in the field of artificial intelligence (AI) By storing knowledge into huge parameters and fine-tuning on specific tasks, the rich knowledge implicitly encoded in huge parameters can benefit a variety of downstream tasks. It is now the consensus of the AI community to adopt PTMs as backbone for downstream tasks rather than learning models from scratch.
  arXiv  Detail & Related papers  (2021-06-14T02:40:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.