SS-MPC: A Sequence-Structured Multi-Party Conversation System

• URL: http://arxiv.org/abs/2502.16920v1
• Date: Mon, 24 Feb 2025 07:25:19 GMT
• Title: SS-MPC: A Sequence-Structured Multi-Party Conversation System
• Abstract summary: We propose textbfSS-MPC, a response generation model for Multi-Party Conversation (MPC) models.<n>Unlike existing models that depend on graphs to analyze conversation structures, SS-MPC internally encodes the dialogue structure as a sequential input.<n>Human evaluation confirms that SS-MPC generates more fluent and accurate responses compared to existing MPC models.
• Score: 12.943428847927123
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent Multi-Party Conversation (MPC) models typically rely on graph-based approaches to capture dialogue structures. However, these methods have limitations, such as information loss during the projection of utterances into structural embeddings and constraints in leveraging pre-trained language models directly. In this paper, we propose \textbf{SS-MPC}, a response generation model for MPC that eliminates the need for explicit graph structures. Unlike existing models that depend on graphs to analyze conversation structures, SS-MPC internally encodes the dialogue structure as a sequential input, enabling direct utilization of pre-trained language models. Experimental results show that \textbf{SS-MPC} achieves \textbf{15.60\% BLEU-1} and \textbf{12.44\% ROUGE-L} score, outperforming the current state-of-the-art MPC response generation model by \textbf{3.91\%p} in \textbf{BLEU-1} and \textbf{0.62\%p} in \textbf{ROUGE-L}. Additionally, human evaluation confirms that SS-MPC generates more fluent and accurate responses compared to existing MPC models.

Related papers

• LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence [54.21298601916397]
  LimiX-2 is developed through model and data scaling guided by our previously established scaling laws.<n>LimiX-2 adopts the Contextual Mechanism Networks (CMNs) paradigm and is pretrained with Context-Conditional Masked Modeling ( CCMM)<n> CCMM shifts the organizing principle of in-context learning from target-centric prediction to mechanism-oriented joint modeling.
  arXiv  Detail & Related papers  (2026-09-15T17:30:02Z)
• Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics [49.443264461057645]
  Diffusion and continuous flow-based language models have emerged as the leading non-autoregressive alternatives to language modeling.<n>Progress in both paradigms is overwhelmingly tracked by generative perplexity (gen-PPL): the per-token negative log-likelihood of samples under a frozen autoregressive (AR) scorer.<n>We argue that this metric is unsound. By construction, gen-PPL measures only predictability under the scoring AR, not grammaticality or semantic coherence.
  arXiv  Detail & Related papers  (2026-06-07T02:35:56Z)
• Relational Probing: LM-to-Graph Adaptation for Financial Prediction [3.7582774908258294]
  Language models can be used to identify relationships between financial entities in text.<n>We propose emphRelational Probing, which replaces the standard languagemodel head with a relational graph.<n>It enables language-model outputs to go beyond text, allowing them to be reshaped into task-specific formats for downstream models.
  arXiv  Detail & Related papers  (2026-04-11T13:43:56Z)
• AULLM++: Structural Reasoning with Large Language Models for Micro-Expression Recognition [30.62262381459829]
  AULLM++ is a reasoning-oriented framework leveraging Large Language Models (LLMs)<n>It formulates AU prediction into three stages: evidence construction, structure modeling, and deduction-based prediction.<n>Experiments demonstrate AULLM++ achieves state-of-the-art performance on standard benchmarks.
  arXiv  Detail & Related papers  (2026-03-09T13:45:21Z)
• Instruction-Tuning Open-Weight Language Models for BPMN Model Generation [0.0]
  We investigate whether open-weight large language models, adapted via instruction tuning, can generate high-quality BPMN process models.<n>We introduce InstruBPM, a reproducible approach that prepares paired text-diagram data and instruction tunes an open source large language model.<n>We compare the tuned model with untuned open-weight baselines and strong proprietary models under consistent prompting regimes.
  arXiv  Detail & Related papers  (2025-12-12T22:07:51Z)
• KBQA-R1: Reinforcing Large Language Models for Knowledge Base Question Answering [64.62317305868264]
  We present textbfKBQA-R1, a framework that shifts the paradigm from text imitation to interaction optimization via Reinforcement Learning.<n>Treating KBQA as a multi-turn decision process, our model learns to navigate the knowledge base using a list of actions.<n>Experiments on WebQSP, GrailQA, and GraphQuestions demonstrate that KBQA-R1 achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-12-10T17:45:42Z)
• Context-level Language Modeling by Learning Predictive Context Embeddings [79.00607069677393]
  We introduce textbfContextLM, a framework that augments standard pretraining with an inherent textbfnext-context prediction objective.<n>This mechanism trains the model to learn predictive representations of multi-token contexts, leveraging error signals derived from future token chunks.<n>Experiments on the GPT2 and Pythia model families, scaled up to $1.5$B parameters, show that ContextLM delivers consistent improvements in both perplexity and downstream task performance.
  arXiv  Detail & Related papers  (2025-10-23T07:09:45Z)
• A Novel LLM-based Two-stage Summarization Approach for Long Dialogues [9.835499880812646]
  This study proposes a hierarchical framework that segments and condenses information from long documents. The condensation stage utilizes an unsupervised generation model to generate condensed data. The summarization stage fine-tunes the abstractive summarization model on the condensed data to generate the final results.
  arXiv  Detail & Related papers  (2024-10-09T03:42:40Z)
• Adaptive Reconstruction for Graph Neural Networks [7.307436175842646]
  Graph Neural Networks (GNNs) have become fundamental in semi-supervised learning for graph representation.<n>Recent trend in GNN research focuses on textbfadaptive k-hop structure learning<n>We propose an textbfadaptive reconstruction framework that dynamically refines k-hop structure learning.
  arXiv  Detail & Related papers  (2024-06-25T05:12:51Z)
• Unsupervised Pre-training with Language-Vision Prompts for Low-Data Instance Segmentation [105.23631749213729]
  We propose a novel method for unsupervised pre-training in low-data regimes. Inspired by the recently successful prompting technique, we introduce a new method, Unsupervised Pre-training with Language-Vision Prompts. We show that our method can converge faster and perform better than CNN-based models in low-data regimes.
  arXiv  Detail & Related papers  (2024-05-22T06:48:43Z)
• CELA: Cost-Efficient Language Model Alignment for CTR Prediction [70.65910069412944]
  Click-Through Rate (CTR) prediction holds a paramount position in recommender systems.<n>Recent efforts have sought to mitigate these challenges by integrating Pre-trained Language Models (PLMs)<n>We propose textbfCost-textbfEfficient textbfLanguage Model textbfAlignment (textbfCELA) for CTR prediction.
  arXiv  Detail & Related papers  (2024-05-17T07:43:25Z)
• Enhancing Pre-Trained Generative Language Models with Question Attended Span Extraction on Machine Reading Comprehension [6.602323571343169]
  Integrated during the fine-tuning phase of pre-trained generative language models (PLMs), QASE significantly enhances their performance. The efficacy of the QASE module has been rigorously tested across various datasets.
  arXiv  Detail & Related papers  (2024-04-27T19:42:51Z)
• Sample Complexity Characterization for Linear Contextual MDPs [67.79455646673762]
  Contextual decision processes (CMDPs) describe a class of reinforcement learning problems in which the transition kernels and reward functions can change over time with different MDPs indexed by a context variable. CMDPs serve as an important framework to model many real-world applications with time-varying environments. We study CMDPs under two linear function approximation models: Model I with context-varying representations and common linear weights for all contexts; and Model II with common representations for all contexts and context-varying linear weights.
  arXiv  Detail & Related papers  (2024-02-05T03:25:04Z)
• Expedited Training of Visual Conditioned Language Generation via Redundancy Reduction [61.16125290912494]
  $textEVL_textGen$ is a framework designed for the pre-training of visually conditioned language generation models. We show that our approach accelerates the training of vision-language models by a factor of 5 without a noticeable impact on overall performance.
  arXiv  Detail & Related papers  (2023-10-05T03:40:06Z)
• Grounded Image Text Matching with Mismatched Relation Reasoning [39.524420144738684]
  Grounded Image Text Matching with Mismatched Relation (GITM-MR) is a novel visual-linguistic joint task. GITM-MR requires a model to first determine if an expression describes an image, then localize referred objects or ground the mismatched parts of the text. We propose the Relation-sensitive Correspondence Reasoning Network (RCRN), which incorporates relation-aware reasoning via bi-directional message propagation.
  arXiv  Detail & Related papers  (2023-08-02T15:44:36Z)
• Autoregressive Structured Prediction with Language Models [73.11519625765301]
  We describe an approach to model structures as sequences of actions in an autoregressive manner with PLMs. Our approach achieves the new state-of-the-art on all the structured prediction tasks we looked at.
  arXiv  Detail & Related papers  (2022-10-26T13:27:26Z)
• SAICL: Student Modelling with Interaction-level Auxiliary Contrastive Tasks for Knowledge Tracing and Dropout Prediction [15.116940192251029]
  This study introduces a novel student modeling framework, SAICL. By combining cross-entropy with contrastive objectives, the proposed SAICL achieved comparable knowledge tracing and dropout prediction performance.
  arXiv  Detail & Related papers  (2022-10-07T14:17:22Z)
• Enhancing Pre-trained Models with Text Structure Knowledge for Question Generation [2.526624977753083]
  We model text structure as answer position and syntactic dependency, and propose answer localness modeling and syntactic mask attention to address these limitations. Experiments on SQuAD dataset show that our proposed two modules improve performance over the strong pre-trained model ProphetNet.
  arXiv  Detail & Related papers  (2022-09-09T08:33:47Z)
• Low-Rank Constraints for Fast Inference in Structured Models [110.38427965904266]
  This work demonstrates a simple approach to reduce the computational and memory complexity of a large class of structured models. Experiments with neural parameterized structured models for language modeling, polyphonic music modeling, unsupervised grammar induction, and video modeling show that our approach matches the accuracy of standard models at large state spaces.
  arXiv  Detail & Related papers  (2022-01-08T00:47:50Z)
• Representation Learning for Conversational Data using Discourse Mutual Information Maximization [9.017156603976915]
  We argue that the structure-unaware word-by-word generation is not suitable for effective conversation modeling. We propose a structure-aware Mutual Information based loss-function DMI for training dialog-representation models. Our models show the most promising performance on the dialog evaluation task DailyDialog++, in both random and adversarial negative scenarios.
  arXiv  Detail & Related papers  (2021-12-04T13:17:07Z)
• Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representations [27.244943870086175]
  Sequence-to-sequence (seq2seq) models have been found to struggle at out-of-distribution compositional generalization. We study the impact of intermediate representations on compositional generalization in pre-trained seq2seq models.
  arXiv  Detail & Related papers  (2021-04-15T14:15:14Z)
• Structural Adapters in Pretrained Language Models for AMR-to-text Generation [59.50420985074769]
  Previous work on text generation from graph-structured data relies on pretrained language models (PLMs) We propose StructAdapt, an adapter method to encode graph structure into PLMs.
  arXiv  Detail & Related papers  (2021-03-16T15:06:50Z)
• Unsupervised Paraphrasing with Pretrained Language Models [85.03373221588707]
  We propose a training pipeline that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting. Our recipe consists of task-adaptation, self-supervision, and a novel decoding algorithm named Dynamic Blocking. We show with automatic and human evaluations that our approach achieves state-of-the-art performance on both the Quora Question Pair and the ParaNMT datasets.
  arXiv  Detail & Related papers  (2020-10-24T11:55:28Z)
• Struct-MMSB: Mixed Membership Stochastic Blockmodels with Interpretable Structured Priors [13.712395104755783]
  Mixed membership blockmodel (MMSB) is a popular framework for community detection and network generation. We present a flexible MMSB model, textitStruct-MMSB, that uses a recently developed statistical relational learning model, hinge-loss Markov random fields (HL-MRFs) Our model is capable of learning latent characteristics in real-world networks via meaningful latent variables encoded as a complex combination of observed features and membership distributions.
  arXiv  Detail & Related papers  (2020-02-21T19:32:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.