Prosody as a Teaching Signal for Agent Learning: Exploratory Studies and Algorithmic Implications

• URL: http://arxiv.org/abs/2410.23554v1
• Date: Thu, 31 Oct 2024 01:51:23 GMT
• Title: Prosody as a Teaching Signal for Agent Learning: Exploratory Studies and Algorithmic Implications
• Authors: Matilda Knierim, Sahil Jain, Murat Han Aydoğan, Kenneth Mitra, Kush Desai, Akanksha Saran, Kim Baraka,
• Abstract summary: This paper advocates for the integration of prosody as a teaching signal to enhance agent learning from human teachers. Our findings suggest that prosodic features, when coupled with explicit feedback, can enhance reinforcement learning outcomes.
• Score: 2.8243597585456017
• License:
• Abstract: Agent learning from human interaction often relies on explicit signals, but implicit social cues, such as prosody in speech, could provide valuable information for more effective learning. This paper advocates for the integration of prosody as a teaching signal to enhance agent learning from human teachers. Through two exploratory studies--one examining voice feedback in an interactive reinforcement learning setup and the other analyzing restricted audio from human demonstrations in three Atari games--we demonstrate that prosody carries significant information about task dynamics. Our findings suggest that prosodic features, when coupled with explicit feedback, can enhance reinforcement learning outcomes. Moreover, we propose guidelines for prosody-sensitive algorithm design and discuss insights into teaching behavior. Our work underscores the potential of leveraging prosody as an implicit signal for more efficient agent learning, thus advancing human-agent interaction paradigms.

Related papers

• RLIF: Interactive Imitation Learning as Reinforcement Learning [56.997263135104504]
  We show how off-policy reinforcement learning can enable improved performance under assumptions that are similar but potentially even more practical than those of interactive imitation learning. Our proposed method uses reinforcement learning with user intervention signals themselves as rewards. This relaxes the assumption that intervening experts in interactive imitation learning should be near-optimal and enables the algorithm to learn behaviors that improve over the potential suboptimal human expert.
  arXiv  Detail & Related papers  (2023-11-21T21:05:21Z)
• "You might think about slightly revising the title": identifying hedges in peer-tutoring interactions [1.0466434989449724]
  Hedges play an important role in the management of conversational interaction. We use a multimodal peer-tutoring dataset to construct a computational framework for identifying hedges. We employ a model explainability tool to explore the features that characterize hedges in peer-tutoring conversations.
  arXiv  Detail & Related papers  (2023-06-18T12:47:54Z)
• Yes, this Way! Learning to Ground Referring Expressions into Actions with Intra-episodic Feedback from Supportive Teachers [15.211628096103475]
  We present an initial study that evaluates intra-episodic feedback given in a collaborative setting. Our results show that intra-episodic feedback allows the follower to generalize on aspects of scene complexity.
  arXiv  Detail & Related papers  (2023-05-22T10:01:15Z)
• Intra-agent speech permits zero-shot task acquisition [13.19051572784014]
  We take inspiration from processes of "inner speech" in humans to better understand the role of intra-agent speech in embodied behavior. We develop algorithms that enable visually grounded captioning with little labeled language data. We incorporate intra-agent speech into an embodied, mobile manipulator agent operating in a 3D virtual world.
  arXiv  Detail & Related papers  (2022-06-07T09:28:10Z)
• Teachable Reinforcement Learning via Advice Distillation [161.43457947665073]
  We propose a new supervision paradigm for interactive learning based on "teachable" decision-making systems that learn from structured advice provided by an external teacher. We show that agents that learn from advice can acquire new skills with significantly less human supervision than standard reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2022-03-19T03:22:57Z)
• Natural Language Communication with a Teachable Agent [0.0]
  This work investigates the effect of teaching modality when interacting with a virtual agent via the Curiosity Notebook. A method of teaching the agent by selecting sentences from source material is compared to a method paraphrasing the source material and typing text input to teach. The results indicate that teaching via paraphrasing and text input has a positive effect on learning outcomes for the material covered, and also on aspects of affective engagement.
  arXiv  Detail & Related papers  (2022-03-17T01:31:23Z)
• Unsupervised Domain Adaptive Person Re-Identification via Human Learning Imitation [67.52229938775294]
  In past years, researchers propose to utilize the teacher-student framework in their methods to decrease the domain gap between different person re-identification datasets. Inspired by recent teacher-student framework based methods, we propose to conduct further exploration to imitate the human learning process from different aspects.
  arXiv  Detail & Related papers  (2021-11-28T01:14:29Z)
• Backprop-Free Reinforcement Learning with Active Neural Generative Coding [84.11376568625353]
  We propose a computational framework for learning action-driven generative models without backpropagation of errors (backprop) in dynamic environments. We develop an intelligent agent that operates even with sparse rewards, drawing inspiration from the cognitive theory of planning as inference. The robust performance of our agent offers promising evidence that a backprop-free approach for neural inference and learning can drive goal-directed behavior.
  arXiv  Detail & Related papers  (2021-07-10T19:02:27Z)
• PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training [94.87393610927812]
  We present an off-policy, interactive reinforcement learning algorithm that capitalizes on the strengths of both feedback and off-policy learning. We demonstrate that our approach is capable of learning tasks of higher complexity than previously considered by human-in-the-loop methods.
  arXiv  Detail & Related papers  (2021-06-09T14:10:50Z)
• Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation [20.837228359591663]
  We present the first study of using human visual explanations in human-in-the-loop reinforcement learning. We propose EXPAND to encourage the model to encode task-relevant features through a context-aware data augmentation.
  arXiv  Detail & Related papers  (2020-06-26T05:40:05Z)
• On the interaction between supervision and self-play in emergent communication [82.290338507106]
  We investigate the relationship between two categories of learning signals with the ultimate goal of improving sample efficiency. We find that first training agents via supervised learning on human data followed by self-play outperforms the converse.
  arXiv  Detail & Related papers  (2020-02-04T02:35:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.