MPC-BERT: A Pre-Trained Language Model for Multi-Party Conversation Understanding

• URL: http://arxiv.org/abs/2106.01541v1
• Date: Thu, 3 Jun 2021 01:49:12 GMT
• Title: MPC-BERT: A Pre-Trained Language Model for Multi-Party Conversation Understanding
• Authors: Jia-Chen Gu, Chongyang Tao, Zhen-Hua Ling, Can Xu, Xiubo Geng, Daxin Jiang
• Abstract summary: We present MPC-BERT, a pre-trained model for MPC understanding. We evaluate MPC-BERT on three downstream tasks including addressee recognition, speaker identification and response selection.
• Score: 58.95156916558384
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, various neural models for multi-party conversation (MPC) have achieved impressive improvements on a variety of tasks such as addressee recognition, speaker identification and response prediction. However, these existing methods on MPC usually represent interlocutors and utterances individually and ignore the inherent complicated structure in MPC which may provide crucial interlocutor and utterance semantics and would enhance the conversation understanding process. To this end, we present MPC-BERT, a pre-trained model for MPC understanding that considers learning who says what to whom in a unified model with several elaborated self-supervised tasks. Particularly, these tasks can be generally categorized into (1) interlocutor structure modeling including reply-to utterance recognition, identical speaker searching and pointer consistency distinction, and (2) utterance semantics modeling including masked shared utterance restoration and shared node detection. We evaluate MPC-BERT on three downstream tasks including addressee recognition, speaker identification and response selection. Experimental results show that MPC-BERT outperforms previous methods by large margins and achieves new state-of-the-art performance on all three downstream tasks at two benchmarks.

Related papers

• M3TCM: Multi-modal Multi-task Context Model for Utterance Classification in Motivational Interviews [1.8100046713740954]
  We present M3TCM, a Multi-modal, Multi-task Context Model for utterance classification. Our approach for the first time employs multi-task learning to effectively model both joint and individual components of therapist and client behaviour. With our novel approach, we outperform the state of the art for utterance classification on the recently introduced AnnoMI dataset with a relative improvement of 20% for the client- and by 15% for therapist utterance classification.
  arXiv  Detail & Related papers  (2024-04-04T09:17:22Z)
• 'What are you referring to?' Evaluating the Ability of Multi-Modal Dialogue Models to Process Clarificational Exchanges [65.03196674816772]
  Referential ambiguities arise in dialogue when a referring expression does not uniquely identify the intended referent for the addressee. Addressees usually detect such ambiguities immediately and work with the speaker to repair it using meta-communicative, Clarification Exchanges (CE): a Clarification Request (CR) and a response. Here, we argue that the ability to generate and respond to CRs imposes specific constraints on the architecture and objective functions of multi-modal, visually grounded dialogue models.
  arXiv  Detail & Related papers  (2023-07-28T13:44:33Z)
• MPCHAT: Towards Multimodal Persona-Grounded Conversation [54.800425322314105]
  We extend persona-based dialogue to the multimodal domain and make two main contributions. First, we present the first multimodal persona-based dialogue dataset named MPCHAT. Second, we empirically show that incorporating multimodal persona, as measured by three proposed multimodal persona-grounded dialogue tasks, leads to statistically significant performance improvements.
  arXiv  Detail & Related papers  (2023-05-27T06:46:42Z)
• GIFT: Graph-Induced Fine-Tuning for Multi-Party Conversation Understanding [51.37738394062851]
  GIFT can adapt various Transformer-based pre-trained language models for universal MPC understanding. Four types of edges are designed to integrate graph-induced signals into attention mechanisms.
  arXiv  Detail & Related papers  (2023-05-16T11:35:24Z)
• Multi-tasking Dialogue Comprehension with Discourse Parsing [43.352833140317486]
  We propose the first multi-task model for jointly performing QA and discourse parsing (DP) on the multi-party dialogue MRC task. Our results indicate that training with complementary tasks indeed benefits not only QA task, but also DP task itself.
  arXiv  Detail & Related papers  (2021-10-07T08:51:49Z)
• Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answering [29.545937716796082]
  Spoken question answering (SQA) requires fine-grained understanding of both spoken documents and questions. We propose novel training schemes for spoken question answering with a self-supervised training stage and a contrastive representation learning stage. Our model achieves state-of-the-art results on three SQA benchmarks.
  arXiv  Detail & Related papers  (2021-09-08T01:13:14Z)
• Filling the Gap of Utterance-aware and Speaker-aware Representation for Multi-turn Dialogue [76.88174667929665]
  A multi-turn dialogue is composed of multiple utterances from two or more different speaker roles. In the existing retrieval-based multi-turn dialogue modeling, the pre-trained language models (PrLMs) as encoder represent the dialogues coarsely. We propose a novel model to fill such a gap by modeling the effective utterance-aware and speaker-aware representations entailed in a dialogue history.
  arXiv  Detail & Related papers  (2020-09-14T15:07:19Z)
• Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues [88.73739515457116]
  We introduce four self-supervised tasks including next session prediction, utterance restoration, incoherence detection and consistency discrimination. We jointly train the PLM-based response selection model with these auxiliary tasks in a multi-task manner. Experiment results indicate that the proposed auxiliary self-supervised tasks bring significant improvement for multi-turn response selection.
  arXiv  Detail & Related papers  (2020-09-14T08:44:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.