End-to-End Evaluation of a Spoken Dialogue System for Learning Basic Mathematics

• URL: http://arxiv.org/abs/2211.03511v1
• Date: Mon, 7 Nov 2022 12:58:24 GMT
• Title: End-to-End Evaluation of a Spoken Dialogue System for Learning Basic Mathematics
• Authors: Eda Okur, Saurav Sahay, Roddy Fuentes Alba, Lama Nachman
• Abstract summary: This work presents a task-oriented Spoken Dialogue System (SDS) built to support play-based learning of basic math concepts for early childhood education. The system has been evaluated via real-world deployments at school while the students are practicing early math concepts with multimodal interactions.
• Score: 8.819665252533104
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The advances in language-based Artificial Intelligence (AI) technologies applied to build educational applications can present AI for social-good opportunities with a broader positive impact. Across many disciplines, enhancing the quality of mathematics education is crucial in building critical thinking and problem-solving skills at younger ages. Conversational AI systems have started maturing to a point where they could play a significant role in helping students learn fundamental math concepts. This work presents a task-oriented Spoken Dialogue System (SDS) built to support play-based learning of basic math concepts for early childhood education. The system has been evaluated via real-world deployments at school while the students are practicing early math concepts with multimodal interactions. We discuss our efforts to improve the SDS pipeline built for math learning, for which we explore utilizing MathBERT representations for potential enhancement to the Natural Language Understanding (NLU) module. We perform an end-to-end evaluation using real-world deployment outputs from the Automatic Speech Recognition (ASR), Intent Recognition, and Dialogue Manager (DM) components to understand how error propagation affects the overall performance in real-world scenarios.

Related papers

• Knowledge Tagging System on Math Questions via LLMs with Flexible Demonstration Retriever [48.5585921817745]
  Large Language Models (LLMs) are used to automate the knowledge tagging task. We show the strong performance of zero- and few-shot results over math questions knowledge tagging tasks. By proposing a reinforcement learning-based demonstration retriever, we successfully exploit the great potential of different-sized LLMs.
  arXiv  Detail & Related papers  (2024-06-19T23:30:01Z)
• Enabling High-Level Machine Reasoning with Cognitive Neuro-Symbolic Systems [67.01132165581667]
  We propose to enable high-level reasoning in AI systems by integrating cognitive architectures with external neuro-symbolic components. We illustrate a hybrid framework centered on ACT-R and we discuss the role of generative models in recent and future applications.
  arXiv  Detail & Related papers  (2023-11-13T21:20:17Z)
• Beyond Traditional Teaching: The Potential of Large Language Models and Chatbots in Graduate Engineering Education [0.0]
  This paper explores the potential integration of large language models (LLMs) and chatbots into graduate engineering education. We develop a question bank from the course material and assess the bot's ability to provide accurate, insightful responses. We demonstrate how powerful plugins like Wolfram Alpha for mathematical problem-solving and code interpretation can significantly extend the bot's capabilities.
  arXiv  Detail & Related papers  (2023-09-09T13:37:22Z)
• Brain in a Vat: On Missing Pieces Towards Artificial General Intelligence in Large Language Models [83.63242931107638]
  We propose four characteristics of generally intelligent agents. We argue that active engagement with objects in the real world delivers more robust signals for forming conceptual representations. We conclude by outlining promising future research directions in the field of artificial general intelligence.
  arXiv  Detail & Related papers  (2023-07-07T13:58:16Z)
• Inspecting Spoken Language Understanding from Kids for Basic Math Learning at Home [8.819665252533104]
  This work explores Spoken Language Understanding (SLU) pipeline within a task-oriented dialogue system developed for Kid Space. Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) components evaluated on our home deployment data.
  arXiv  Detail & Related papers  (2023-06-01T09:31:57Z)
• New Era of Artificial Intelligence in Education: Towards a Sustainable Multifaceted Revolution [2.94944680995069]
  ChatGPT's high performance on standardized academic tests has thrust the topic of artificial intelligence (AI) into the mainstream conversation about the future of education. This research aims to investigate the potential impact of AI on education through review and analysis of the existing literature across three major axes: applications, advantages, and challenges.
  arXiv  Detail & Related papers  (2023-05-12T08:22:54Z)
• Enhancing STEM Learning with ChatGPT and Bing Chat as Objects to Think With: A Case Study [0.0]
  This study investigates the potential of ChatGPT and Bing Chat, advanced conversational AIs, as "objects-to-think-with" The study concludes that ChatGPT and Bing Chat as objects-to-think-with offer promising avenues to revolutionise STEM education.
  arXiv  Detail & Related papers  (2023-05-01T12:20:18Z)
• Learning by Applying: A General Framework for Mathematical Reasoning via Enhancing Explicit Knowledge Learning [47.96987739801807]
  We propose a framework to enhance existing models (backbones) in a principled way by explicit knowledge learning. In LeAp, we perform knowledge learning in a novel problem-knowledge-expression paradigm. We show that LeAp improves all backbones' performances, learns accurate knowledge, and achieves a more interpretable reasoning process.
  arXiv  Detail & Related papers  (2023-02-11T15:15:41Z)
• A Survey of Deep Learning for Mathematical Reasoning [71.88150173381153]
  We review the key tasks, datasets, and methods at the intersection of mathematical reasoning and deep learning over the past decade. Recent advances in large-scale neural language models have opened up new benchmarks and opportunities to use deep learning for mathematical reasoning.
  arXiv  Detail & Related papers  (2022-12-20T18:46:16Z)
• Data Augmentation with Paraphrase Generation and Entity Extraction for Multimodal Dialogue System [9.912419882236918]
  We are working towards a multimodal dialogue system for younger kids learning basic math concepts. This work explores the potential benefits of data augmentation with paraphrase generation for the Natural Language Understanding module of the Spoken Dialogue Systems pipeline. We have shown that paraphrasing with model-in-the-loop (MITL) strategies using small seed data is a promising approach yielding improved performance results for the Intent Recognition task.
  arXiv  Detail & Related papers  (2022-05-09T02:21:20Z)
• Rethinking Supervised Learning and Reinforcement Learning in Task-Oriented Dialogue Systems [58.724629408229205]
  We demonstrate how traditional supervised learning and a simulator-free adversarial learning method can be used to achieve performance comparable to state-of-the-art RL-based methods. Our main goal is not to beat reinforcement learning with supervised learning, but to demonstrate the value of rethinking the role of reinforcement learning and supervised learning in optimizing task-oriented dialogue systems.
  arXiv  Detail & Related papers  (2020-09-21T12:04:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.