A Graph-to-Text Approach to Knowledge-Grounded Response Generation in
Human-Robot Interaction
- URL: http://arxiv.org/abs/2311.16137v1
- Date: Fri, 3 Nov 2023 15:44:28 GMT
- Title: A Graph-to-Text Approach to Knowledge-Grounded Response Generation in
Human-Robot Interaction
- Authors: Nicholas Thomas Walker, Stefan Ultes, Pierre Lison
- Abstract summary: This paper presents a novel conversational model for human--robot interaction that rests upon a graph-based representation of the dialogue state.
The neural conversational model employed to respond to user utterances relies on a simple but effective graph-to-text mechanism.
The proposed approach is empirically evaluated through a user study with a humanoid robot.
- Score: 2.3590037806133024
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Knowledge graphs are often used to represent structured information in a
flexible and efficient manner, but their use in situated dialogue remains
under-explored. This paper presents a novel conversational model for
human--robot interaction that rests upon a graph-based representation of the
dialogue state. The knowledge graph representing the dialogue state is
continuously updated with new observations from the robot sensors, including
linguistic, situated and multimodal inputs, and is further enriched by other
modules, in particular for spatial understanding. The neural conversational
model employed to respond to user utterances relies on a simple but effective
graph-to-text mechanism that traverses the dialogue state graph and converts
the traversals into a natural language form. This conversion of the state graph
into text is performed using a set of parameterized functions, and the values
for those parameters are optimized based on a small set of Wizard-of-Oz
interactions. After this conversion, the text representation of the dialogue
state graph is included as part of the prompt of a large language model used to
decode the agent response. The proposed approach is empirically evaluated
through a user study with a humanoid robot that acts as conversation partner to
evaluate the impact of the graph-to-text mechanism on the response generation.
After moving a robot along a tour of an indoor environment, participants
interacted with the robot using spoken dialogue and evaluated how well the
robot was able to answer questions about what the robot observed during the
tour. User scores show a statistically significant improvement in the perceived
factuality of the robot responses when the graph-to-text approach is employed,
compared to a baseline using inputs structured as semantic triples.
Related papers
- Learning Manipulation by Predicting Interaction [85.57297574510507]
We propose a general pre-training pipeline that learns Manipulation by Predicting the Interaction.
The experimental results demonstrate that MPI exhibits remarkable improvement by 10% to 64% compared with previous state-of-the-art in real-world robot platforms.
arXiv Detail & Related papers (2024-06-01T13:28:31Z) - A Multi-Modal Explainability Approach for Human-Aware Robots in Multi-Party Conversation [39.87346821309096]
We present an addressee estimation model with improved performance in comparison with the previous SOTA.
We also propose several ways to incorporate explainability and transparency in the aforementioned architecture.
arXiv Detail & Related papers (2024-05-20T13:09:32Z) - I Was Blind but Now I See: Implementing Vision-Enabled Dialogue in
Social Robots [0.040792653193642496]
This paper presents an initial implementation of a dialogue manager that enhances the traditional text-based prompts with real-time visual input.
The system's prompt engineering, incorporating dialogue with summarisation of the images, ensures a balance between context preservation and computational efficiency.
arXiv Detail & Related papers (2023-11-15T13:47:00Z) - Retrieval-Augmented Neural Response Generation Using Logical Reasoning
and Relevance Scoring [2.3590037806133024]
This paper presents a novel approach to knowledge-grounded response generation.
It combines retrieval-augmented language models with logical reasoning.
Experimental results show that the combination of (probabilistic) logical reasoning with conversational relevance scoring does increase both the factuality and fluency of the responses.
arXiv Detail & Related papers (2023-10-20T15:05:18Z) - Interactive Natural Language Processing [67.87925315773924]
Interactive Natural Language Processing (iNLP) has emerged as a novel paradigm within the field of NLP.
This paper offers a comprehensive survey of iNLP, starting by proposing a unified definition and framework of the concept.
arXiv Detail & Related papers (2023-05-22T17:18:29Z) - GraphWOZ: Dialogue Management with Conversational Knowledge Graphs [2.938377447673471]
We present a new approach to dialogue management using conversational knowledge graphs as core representation of the dialogue state.
We introduce a new dataset, GraphWOZ, which comprises Wizard-of-Oz dialogues in which human participants interact with a robot acting as a receptionist.
arXiv Detail & Related papers (2022-11-23T10:53:21Z) - Multimodal Emotion Recognition using Transfer Learning from Speaker
Recognition and BERT-based models [53.31917090073727]
We propose a neural network-based emotion recognition framework that uses a late fusion of transfer-learned and fine-tuned models from speech and text modalities.
We evaluate the effectiveness of our proposed multimodal approach on the interactive emotional dyadic motion capture dataset.
arXiv Detail & Related papers (2022-02-16T00:23:42Z) - GRADE: Automatic Graph-Enhanced Coherence Metric for Evaluating
Open-Domain Dialogue Systems [133.13117064357425]
We propose a new evaluation metric GRADE, which stands for Graph-enhanced Representations for Automatic Dialogue Evaluation.
Specifically, GRADE incorporates both coarse-grained utterance-level contextualized representations and fine-grained topic-level graph representations to evaluate dialogue coherence.
Experimental results show that our GRADE significantly outperforms other state-of-the-art metrics on measuring diverse dialogue models.
arXiv Detail & Related papers (2020-10-08T14:07:32Z) - How To Evaluate Your Dialogue System: Probe Tasks as an Alternative for
Token-level Evaluation Metrics [47.20761880464552]
generative dialogue modeling is widely seen as a language modeling task.
The task demands an agent to have a complex natural language understanding of its input text to carry a meaningful interaction with an user.
The automatic metrics used evaluate the quality of the generated text as a proxy to the holistic interaction of the agent.
arXiv Detail & Related papers (2020-08-24T13:28:35Z) - You Impress Me: Dialogue Generation via Mutual Persona Perception [62.89449096369027]
The research in cognitive science suggests that understanding is an essential signal for a high-quality chit-chat conversation.
Motivated by this, we propose P2 Bot, a transmitter-receiver based framework with the aim of explicitly modeling understanding.
arXiv Detail & Related papers (2020-04-11T12:51:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.