AMELIA: A Family of Multi-task End-to-end Language Models for Argumentation

• URL: http://arxiv.org/abs/2508.17926v1
• Date: Mon, 25 Aug 2025 11:51:39 GMT
• Title: AMELIA: A Family of Multi-task End-to-end Language Models for Argumentation
• Authors: Henri Savigny, Bruno Yun,
• Abstract summary: Argument mining aims to automatically extract argumentative structures and their relations from natural language texts.<n>This paper investigates how a single large language model can be leveraged to perform one or several argument mining tasks.
• Score: 0.8379286663107844
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Argument mining is a subfield of argumentation that aims to automatically extract argumentative structures and their relations from natural language texts. This paper investigates how a single large language model can be leveraged to perform one or several argument mining tasks. Our contributions are two-fold. First, we construct a multi-task dataset by surveying and converting 19 well-known argument mining datasets from the literature into a unified format. Second, we explore various training strategies using Meta AI's Llama-3.1-8B-Instruct model: (1) fine-tuning on individual tasks, (2) fine-tuning jointly on multiple tasks, and (3) merging models fine-tuned separately on individual tasks. Our experiments show that task-specific fine-tuning significantly improves individual performance across all tasks. Moreover, multi-task fine-tuning maintains strong performance without degradation, suggesting effective transfer learning across related tasks. Finally, we demonstrate that model merging offers a viable compromise: it yields competitive performance while mitigating the computational costs associated with full multi-task fine-tuning.

Related papers

• Single-Input Multi-Output Model Merging: Leveraging Foundation Models for Dense Multi-Task Learning [46.51245338355645]
  Model merging is a flexible and computationally tractable approach to merge single-task checkpoints into a multi-task model.<n>We show that it qualitatively differs from the single-input-multiple-output model merging settings studied in the literature due to the existence of task-specific decoders.<n>We propose two simple and efficient fixes for the SIMO setting to re-align the feature representation after merging.
  arXiv  Detail & Related papers  (2025-04-15T15:10:46Z)
• Multi-Task Learning for Front-End Text Processing in TTS [15.62497569424995]
  We propose a multi-task learning (MTL) model for jointly performing three tasks that are commonly solved in a text-to-speech front-end. Our framework utilizes a tree-like structure with a trunk that learns shared representations, followed by separate task-specific heads.
  arXiv  Detail & Related papers  (2024-01-12T02:13:21Z)
• Multi-Task Learning Improves Performance In Deep Argument Mining Models [2.2312474084968024]
  We show that different argument mining tasks share common semantic and logical structure by implementing a multi-task approach to argument mining. Our results are important for argument mining as they show that different tasks share substantial similarities and suggest a holistic approach to the extraction of argumentative techniques from text.
  arXiv  Detail & Related papers  (2023-07-03T23:42:29Z)
• Musketeer: Joint Training for Multi-task Vision Language Model with Task Explanation Prompts [75.75548749888029]
  We present a vision-language model whose parameters are jointly trained on all tasks and fully shared among multiple heterogeneous tasks. With a single model, Musketeer achieves results comparable to or better than strong baselines trained on single tasks, almost uniformly across multiple tasks.
  arXiv  Detail & Related papers  (2023-05-11T17:57:49Z)
• Effective Cross-Task Transfer Learning for Explainable Natural Language Inference with T5 [50.574918785575655]
  We compare sequential fine-tuning with a model for multi-task learning in the context of boosting performance on two tasks. Our results show that while sequential multi-task learning can be tuned to be good at the first of two target tasks, it performs less well on the second and additionally struggles with overfitting.
  arXiv  Detail & Related papers  (2022-10-31T13:26:08Z)
• Coarse-to-Fine: Hierarchical Multi-task Learning for Natural Language Understanding [51.31622274823167]
  We propose a hierarchical framework with a coarse-to-fine paradigm, with the bottom level shared to all the tasks, the mid-level divided to different groups, and the top-level assigned to each of the tasks. This allows our model to learn basic language properties from all tasks, boost performance on relevant tasks, and reduce the negative impact from irrelevant tasks.
  arXiv  Detail & Related papers  (2022-08-19T02:46:20Z)
• Diversity Over Size: On the Effect of Sample and Topic Sizes for Topic-Dependent Argument Mining Datasets [49.65208986436848]
  We investigate the effect of Argument Mining dataset composition in few- and zero-shot settings. Our findings show that, while fine-tuning is mandatory to achieve acceptable model performance, using carefully composed training samples and reducing the training sample size by up to almost 90% can still yield 95% of the maximum performance.
  arXiv  Detail & Related papers  (2022-05-23T17:14:32Z)
• Exploring Relational Context for Multi-Task Dense Prediction [76.86090370115]
  We consider a multi-task environment for dense prediction tasks, represented by a common backbone and independent task-specific heads. We explore various attention-based contexts, such as global and local, in the multi-task setting. We propose an Adaptive Task-Relational Context module, which samples the pool of all available contexts for each task pair.
  arXiv  Detail & Related papers  (2021-04-28T16:45:56Z)
• Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference [75.95287293847697]
  Two common challenges in developing multi-task models are often overlooked in literature. First, enabling the model to be inherently incremental, continuously incorporating information from new tasks without forgetting the previously learned ones (incremental learning) Second, eliminating adverse interactions amongst tasks, which has been shown to significantly degrade the single-task performance in a multi-task setup (task interference)
  arXiv  Detail & Related papers  (2020-07-24T14:44:46Z)
• Modelling Latent Skills for Multitask Language Generation [15.126163032403811]
  We present a generative model for multitask conditional language generation. Our guiding hypothesis is that a shared set of latent skills underlies many disparate language generation tasks. We instantiate this task embedding space as a latent variable in a latent variable sequence-to-sequence model.
  arXiv  Detail & Related papers  (2020-02-21T20:39:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.