AGI Is Coming... Right After AI Learns to Play Wordle

• URL: http://arxiv.org/abs/2504.15434v1
• Date: Mon, 21 Apr 2025 20:58:58 GMT
• Title: AGI Is Coming... Right After AI Learns to Play Wordle
• Authors: Sarath Shekkizhar, Romain Cosentino,
• Abstract summary: multimodal agents, in particular, OpenAI's Computer-User Agent (CUA), trained to control and complete tasks through a standard computer interface, similar to humans.<n>We evaluated the agent's performance on the New York Times Wordle game to elicit model behaviors and identify shortcomings.
• Score: 4.2909314120969855
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper investigates multimodal agents, in particular, OpenAI's Computer-User Agent (CUA), trained to control and complete tasks through a standard computer interface, similar to humans. We evaluated the agent's performance on the New York Times Wordle game to elicit model behaviors and identify shortcomings. Our findings revealed a significant discrepancy in the model's ability to recognize colors correctly depending on the context. The model had a $5.36\%$ success rate over several hundred runs across a week of Wordle. Despite the immense enthusiasm surrounding AI agents and their potential to usher in Artificial General Intelligence (AGI), our findings reinforce the fact that even simple tasks present substantial challenges for today's frontier AI models. We conclude with a discussion of the potential underlying causes, implications for future development, and research directions to improve these AI systems.

Related papers

• Evaluating Intelligence via Trial and Error [59.80426744891971]
  We introduce Survival Game as a framework to evaluate intelligence based on the number of failed attempts in a trial-and-error process.<n>When the expectation and variance of failure counts are both finite, it signals the ability to consistently find solutions to new challenges.<n>Our results show that while AI systems achieve the Autonomous Level in simple tasks, they are still far from it in more complex tasks.
  arXiv  Detail & Related papers  (2025-02-26T05:59:45Z)
• Are Large Language Models Ready for Business Integration? A Study on Generative AI Adoption [0.6144680854063939]
  This research examines the readiness of other Large Language Models (LLMs) such as Google Gemini for potential business applications.<n>A dataset of 42,654 reviews from distinct Disneyland branches was employed.<n>Results presented a spectrum of responses, including 75% successful simplifications, 25% errors, and instances of model self-reference.
  arXiv  Detail & Related papers  (2025-01-28T21:01:22Z)
• AI-Driven Agents with Prompts Designed for High Agreeableness Increase the Likelihood of Being Mistaken for a Human in the Turing Test [0.0]
  GPT agents with varying levels of agreeableness were tested in a Turing Test. All exceeded a 50% confusion rate, with the highly agreeable AI agent surpassing 60%. This agent was also recognized as exhibiting the most human-like traits.
  arXiv  Detail & Related papers  (2024-11-20T23:12:49Z)
• ML Research Benchmark [0.0]
  We present the ML Research Benchmark (MLRB), comprising 7 competition-level tasks derived from recent machine learning conference tracks. This paper introduces a novel benchmark and evaluates it using agent scaffolds powered by frontier models, including Claude-3 and GPT-4o. The results indicate that the Claude-3.5 Sonnet agent performs best across our benchmark, excelling in planning and developing machine learning models.
  arXiv  Detail & Related papers  (2024-10-29T21:38:42Z)
• Position Paper: Agent AI Towards a Holistic Intelligence [53.35971598180146]
  We emphasize developing Agent AI -- an embodied system that integrates large foundation models into agent actions. In this paper, we propose a novel large action model to achieve embodied intelligent behavior, the Agent Foundation Model.
  arXiv  Detail & Related papers  (2024-02-28T16:09:56Z)
• Generative AI in Writing Research Papers: A New Type of Algorithmic Bias and Uncertainty in Scholarly Work [0.38850145898707145]
  Large language models (LLMs) and generative AI tools present challenges in identifying and addressing biases. generative AI tools are susceptible to goal misgeneralization, hallucinations, and adversarial attacks such as red teaming prompts. We find that incorporating generative AI in the process of writing research manuscripts introduces a new type of context-induced algorithmic bias.
  arXiv  Detail & Related papers  (2023-12-04T04:05:04Z)
• Exploration with Principles for Diverse AI Supervision [88.61687950039662]
  Training large transformers using next-token prediction has given rise to groundbreaking advancements in AI. While this generative AI approach has produced impressive results, it heavily leans on human supervision. This strong reliance on human oversight poses a significant hurdle to the advancement of AI innovation. We propose a novel paradigm termed Exploratory AI (EAI) aimed at autonomously generating high-quality training data.
  arXiv  Detail & Related papers  (2023-10-13T07:03:39Z)
• The Rise and Potential of Large Language Model Based Agents: A Survey [91.71061158000953]
  Large language models (LLMs) are regarded as potential sparks for Artificial General Intelligence (AGI) We start by tracing the concept of agents from its philosophical origins to its development in AI, and explain why LLMs are suitable foundations for agents. We explore the extensive applications of LLM-based agents in three aspects: single-agent scenarios, multi-agent scenarios, and human-agent cooperation.
  arXiv  Detail & Related papers  (2023-09-14T17:12:03Z)
• AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges [60.56413461109281]
  Artificial Intelligence for IT operations (AIOps) aims to combine the power of AI with the big data generated by IT Operations processes. We discuss in depth the key types of data emitted by IT Operations activities, the scale and challenges in analyzing them, and where they can be helpful. We categorize the key AIOps tasks as - incident detection, failure prediction, root cause analysis and automated actions.
  arXiv  Detail & Related papers  (2023-04-10T15:38:12Z)
• ProcTHOR: Large-Scale Embodied AI Using Procedural Generation [55.485985317538194]
  ProcTHOR is a framework for procedural generation of Embodied AI environments. We demonstrate state-of-the-art results across 6 embodied AI benchmarks for navigation, rearrangement, and arm manipulation.
  arXiv  Detail & Related papers  (2022-06-14T17:09:35Z)
• Brittle AI, Causal Confusion, and Bad Mental Models: Challenges and Successes in the XAI Program [17.52385105997044]
  Deep neural network driven models have surpassed human level performance in benchmark autonomy tasks. The underlying policies for these agents, however, are not easily interpretable. This paper discusses the origins of these takeaways, provides amplifying information, and suggestions for future work.
  arXiv  Detail & Related papers  (2021-06-10T05:21:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.