A Graph Reasoning Network for Multi-turn Response Selection via Customized Pre-training

• URL: http://arxiv.org/abs/2012.11099v2
• Date: Fri, 15 Jan 2021 02:12:11 GMT
• Title: A Graph Reasoning Network for Multi-turn Response Selection via Customized Pre-training
• Authors: Yongkang Liu, Shi Feng, Daling Wang, Kaisong Song, Feiliang Ren, Yifei Zhang
• Abstract summary: We propose a graph-reasoning network (GRN) to address the problem. GRN first conducts pre-training based on ALBERT. We then fine-tune the model on an integrated network with sequence reasoning and graph reasoning structures.
• Score: 11.532734330690584
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We investigate response selection for multi-turn conversation in retrieval-based chatbots. Existing studies pay more attention to the matching between utterances and responses by calculating the matching score based on learned features, leading to insufficient model reasoning ability. In this paper, we propose a graph-reasoning network (GRN) to address the problem. GRN first conducts pre-training based on ALBERT using next utterance prediction and utterance order prediction tasks specifically devised for response selection. These two customized pre-training tasks can endow our model with the ability of capturing semantical and chronological dependency between utterances. We then fine-tune the model on an integrated network with sequence reasoning and graph reasoning structures. The sequence reasoning module conducts inference based on the highly summarized context vector of utterance-response pairs from the global perspective. The graph reasoning module conducts the reasoning on the utterance-level graph neural network from the local perspective. Experiments on two conversational reasoning datasets show that our model can dramatically outperform the strong baseline methods and can achieve performance which is close to human-level.

Related papers

• Integrating Large Language Models with Graph-based Reasoning for Conversational Question Answering [58.17090503446995]
  We focus on a conversational question answering task which combines the challenges of understanding questions in context and reasoning over evidence gathered from heterogeneous sources like text, knowledge graphs, tables, and infoboxes. Our method utilizes a graph structured representation to aggregate information about a question and its context.
  arXiv  Detail & Related papers  (2024-06-14T13:28:03Z)
• Can Graph Learning Improve Planning in LLM-based Agents? [61.47027387839096]
  Task planning in language agents is emerging as an important research topic alongside the development of large language models (LLMs) In this paper, we explore graph learning-based methods for task planning, a direction that is to the prevalent focus on prompt design. Our interest in graph learning stems from a theoretical discovery: the biases of attention and auto-regressive loss impede LLMs' ability to effectively navigate decision-making on graphs.
  arXiv  Detail & Related papers  (2024-05-29T14:26:24Z)
• On Discprecncies between Perturbation Evaluations of Graph Neural Network Attributions [49.8110352174327]
  We assess attribution methods from a perspective not previously explored in the graph domain: retraining. The core idea is to retrain the network on important (or not important) relationships as identified by the attributions. We run our analysis on four state-of-the-art GNN attribution methods and five synthetic and real-world graph classification datasets.
  arXiv  Detail & Related papers  (2024-01-01T02:03:35Z)
• Retrieval-Augmented Neural Response Generation Using Logical Reasoning and Relevance Scoring [2.3590037806133024]
  This paper presents a novel approach to knowledge-grounded response generation. It combines retrieval-augmented language models with logical reasoning. Experimental results show that the combination of (probabilistic) logical reasoning with conversational relevance scoring does increase both the factuality and fluency of the responses.
  arXiv  Detail & Related papers  (2023-10-20T15:05:18Z)
• IRRGN: An Implicit Relational Reasoning Graph Network for Multi-turn Response Selection [4.471148909362883]
  Implicit Reasoning to Graph Network aims to implicitly extract between utterances, as well as utterances and options. Model surpasses human performance for the first time on the MuTual dataset.
  arXiv  Detail & Related papers  (2022-12-01T13:17:25Z)
• Neural Graph Matching for Pre-training Graph Neural Networks [72.32801428070749]
  Graph neural networks (GNNs) have been shown powerful capacity at modeling structural data. We present a novel Graph Matching based GNN Pre-Training framework, called GMPT. The proposed method can be applied to fully self-supervised pre-training and coarse-grained supervised pre-training.
  arXiv  Detail & Related papers  (2022-03-03T09:53:53Z)
• Question-Answer Sentence Graph for Joint Modeling Answer Selection [122.29142965960138]
  We train and integrate state-of-the-art (SOTA) models for computing scores between question-question, question-answer, and answer-answer pairs. Online inference is then performed to solve the AS2 task on unseen queries.
  arXiv  Detail & Related papers  (2022-02-16T05:59:53Z)
• A Hierarchical Reasoning Graph Neural Network for The Automatic Scoring of Answer Transcriptions in Video Job Interviews [14.091472037847499]
  We propose a Hierarchical Reasoning Graph Neural Network (HRGNN) for the automatic assessment of question-answer pairs. We employ a semantic-level reasoning graph attention network to model the interaction states of the current QA session. Finally, we propose a gated recurrent unit encoder to represent the temporal question-answer pairs for the final prediction.
  arXiv  Detail & Related papers  (2020-12-22T12:27:45Z)
• Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues [88.73739515457116]
  We introduce four self-supervised tasks including next session prediction, utterance restoration, incoherence detection and consistency discrimination. We jointly train the PLM-based response selection model with these auxiliary tasks in a multi-task manner. Experiment results indicate that the proposed auxiliary self-supervised tasks bring significant improvement for multi-turn response selection.
  arXiv  Detail & Related papers  (2020-09-14T08:44:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.