SilverSight: A Multi-Task Chinese Financial Large Language Model Based on Adaptive Semantic Space Learning

• URL: http://arxiv.org/abs/2404.04949v1
• Date: Sun, 7 Apr 2024 13:02:21 GMT
• Title: SilverSight: A Multi-Task Chinese Financial Large Language Model Based on Adaptive Semantic Space Learning
• Authors: Yuhang Zhou, Zeping Li, Siyu Tian, Yuchen Ni, Sen Liu, Guangnan Ye, Hongfeng Chai,
• Abstract summary: This study introduces an Adaptive Semantic Space Learning (ASSL) framework to enhance the performance and selection efficacy of multi-expert models. Our research findings demonstrate that our framework can achieve results close to those obtained with full data training using only 10% of the data, while also exhibiting strong generalization capabilities.
• Score: 4.540505713937026
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are increasingly being applied across various specialized fields, leveraging their extensive knowledge to empower a multitude of scenarios within these domains. However, each field encompasses a variety of specific tasks that require learning, and the diverse, heterogeneous data across these domains can lead to conflicts during model task transfer. In response to this challenge, our study introduces an Adaptive Semantic Space Learning (ASSL) framework, which utilizes the adaptive reorganization of data distributions within the semantic space to enhance the performance and selection efficacy of multi-expert models. Utilizing this framework, we trained a financial multi-task LLM named "SilverSight". Our research findings demonstrate that our framework can achieve results close to those obtained with full data training using only 10% of the data, while also exhibiting strong generalization capabilities.

Related papers

• Dynamic Adaptive Optimization for Effective Sentiment Analysis Fine-Tuning on Large Language Models [0.0]
  Large language models (LLMs) have become a popular paradigm for sentiment analysis, leveraging multi-task learning to address specific tasks concurrently. We propose a novel multi-task learning framework with a dynamic adaptive optimization (DAO) module. This work improves the Mean Squared Error (MSE) and Accuracy (ACC) by 15.58% and 1.24% respectively, compared with previous work.
  arXiv  Detail & Related papers  (2024-08-15T19:13:38Z)
• SoupLM: Model Integration in Large Language and Multi-Modal Models [51.12227693121004]
  Training large language models (LLMs) requires significant computing resources. Existing publicly available LLMs are typically pre-trained on diverse, privately curated datasets spanning various tasks.
  arXiv  Detail & Related papers  (2024-07-11T05:38:15Z)
• Mixture-of-Skills: Learning to Optimize Data Usage for Fine-Tuning Large Language Models [45.51085356985464]
  Large language models (LLMs) are typically fine-tuned on diverse and extensive datasets sourced from various origins. MoS learns to optimize data usage automatically during the fine-tuning process. MoSpec harnesses the utilities of various datasets for a specific purpose.
  arXiv  Detail & Related papers  (2024-06-13T05:01:28Z)
• Grounding Multimodal Large Language Models in Actions [65.88208317380793]
  We study how to best ground a MLLM into different embodiments and their associated action spaces. For continuous actions, we show that a learned tokenization allows for sufficient modeling precision. For discrete actions, we demonstrate that semantically aligning these actions with the native output token space of the MLLM leads to the strongest performance.
  arXiv  Detail & Related papers  (2024-06-12T06:12:04Z)
• Scalable Language Model with Generalized Continual Learning [58.700439919096155]
  The Joint Adaptive Re-ization (JARe) is integrated with Dynamic Task-related Knowledge Retrieval (DTKR) to enable adaptive adjustment of language models based on specific downstream tasks. Our method demonstrates state-of-the-art performance on diverse backbones and benchmarks, achieving effective continual learning in both full-set and few-shot scenarios with minimal forgetting.
  arXiv  Detail & Related papers  (2024-04-11T04:22:15Z)
• Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challenges [47.45993726498343]
  Data augmentation (DA) has emerged as a pivotal technique for enhancing model performance by diversifying training examples without the need for additional data collection. This survey explores the transformative impact of large language models (LLMs) on DA, particularly addressing the unique challenges and opportunities they present in the context of natural language processing (NLP) and beyond.
  arXiv  Detail & Related papers  (2024-03-05T14:11:54Z)
• Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey [100.24095818099522]
  Large language models (LLMs) have significantly advanced the field of natural language processing (NLP) They provide a highly useful, task-agnostic foundation for a wide range of applications. However, directly applying LLMs to solve sophisticated problems in specific domains meets many hurdles.
  arXiv  Detail & Related papers  (2023-05-30T03:00:30Z)
• Semi-supervised Multi-task Learning for Semantics and Depth [88.77716991603252]
  Multi-Task Learning (MTL) aims to enhance the model generalization by sharing representations between related tasks for better performance. We propose the Semi-supervised Multi-Task Learning (MTL) method to leverage the available supervisory signals from different datasets. We present a domain-aware discriminator structure with various alignment formulations to mitigate the domain discrepancy issue among datasets.
  arXiv  Detail & Related papers  (2021-10-14T07:43:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.