SPEAR-MM: Selective Parameter Evaluation and Restoration via Model Merging for Efficient Financial LLM Adaptation

• URL: http://arxiv.org/abs/2511.08500v1
• Date: Wed, 12 Nov 2025 02:01:45 GMT
• Title: SPEAR-MM: Selective Parameter Evaluation and Restoration via Model Merging for Efficient Financial LLM Adaptation
• Abstract summary: Large language models (LLMs) adapted to financial domains often suffer from catastrophic forgetting of general reasoning capabilities.<n>We introduce Selective Evaluation and Restoration via Model Merging (SPEAR-MM), a practical framework that preserves critical capabilities while enabling domain adaptation.
• Score: 7.761546505292692
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) adapted to financial domains often suffer from catastrophic forgetting of general reasoning capabilities essential for customer interactions and complex financial analysis. We introduce Selective Parameter Evaluation and Restoration via Model Merging (SPEAR-MM), a practical framework that preserves critical capabilities while enabling domain adaptation. Our method approximates layer-wise impact on external benchmarks through post-hoc analysis, then selectively freezes or restores transformer layers via spherical interpolation merging. Applied to LLaMA-3.1-8B for financial tasks, SPEAR-MM achieves 91.2% retention of general capabilities versus 69.7% for standard continual pretraining, while maintaining 94% of domain adaptation gains. The approach provides interpretable trade-off control and reduces computational costs by 90% crucial for resource-constrained financial institutions.

Related papers

• IPM-FM: A Foundation Model with Consensus Feature Selection for Industrial Process Monitoring [59.59850511287819]
  We propose the industrial process monitoring foundation model (IPM-FM)<n>It first learns general-purpose representations from unlabeled industrial process data through self-supervised pretraining.<n>It adapts to specific monitoring tasks using a small amount of task-labeled data, and finally produces calibrated predictions.
  arXiv  Detail & Related papers  (2026-09-08T07:43:04Z)
• PandaAI: A Practical Agent CQ2 for Neuro-symbolic Data Analysis And Integrated Decision-Making in Quantitative Finance [11.790070440436294]
  textbfPandaAI is a closed-loop neuro-symbolic LLM agent with market regime modeling and constrained alpha generation.<n>Our experiments show textbfPandaAI achieves a $18.2%$ higher Rank IC and $25.7%$ lower maximum drawdown than state-of-the-art time-series models.
  arXiv  Detail & Related papers  (2026-06-05T01:56:28Z)
• YouZhi: Towards High-Concurrency Financial LLMs via Adaptive GQA-to-MLA Transition [78.52944812151664]
  Large language models (LLMs) drive significant financial innovations, yet their deployment is bottlenecked by KV cache memory overhead.<n>We propose YouZhi-LLM, a highly efficient financial LLM empowered by a comprehensive structural transition and training pipeline built on the Huawei Ascend ecosystem.
  arXiv  Detail & Related papers  (2026-06-04T08:44:37Z)
• Exploring the impact of fairness-aware criteria in AutoML [0.5156484100374058]
  We study the impact of integrating fairness directly into the optimisation component of an AutoML framework.<n>Our work incorporates complementary fairness metrics to capture different dimensions of fairness during the optimisation.<n>Despite a 9.4% decrease in predictive power, the average fairness improved by 14.5%, accompanied by a 35.7% reduction in data usage.
  arXiv  Detail & Related papers  (2026-04-11T14:15:15Z)
• ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inference [60.958331943869126]
  ODAR-Expert is an adaptive routing framework that optimize the accuracy-efficiency trade-off via principled resource allocation.<n>We show strong and consistent gains, including 98.2% accuracy on MATH and 54.8% on Humanity's Last Exam.
  arXiv  Detail & Related papers  (2026-02-27T05:22:01Z)
• $φ$-DPO: Fairness Direct Preference Optimization Approach to Continual Learning in Large Multimodal Models [58.217707070069885]
  This paper presents a novel Fairness Direct Preference Optimization (FaiDPO or $$-DPO) framework for continual learning in LMMs.<n>We first propose a new continual learning paradigm based on Direct Preference Optimization (DPO) to mitigate catastrophic forgetting by aligning learning with pairwise preference signals.<n> Extensive experiments and ablation studies show the proposed $$-DPO achieves State-of-the-Art performance across multiple benchmarks.
  arXiv  Detail & Related papers  (2026-02-26T04:14:33Z)
• Uni-FinLLM: A Unified Multimodal Large Language Model with Modular Task Heads for Micro-Level Stock Prediction and Macro-Level Systemic Risk Assessment [6.015507338546882]
  Financial institutions and regulators require systems that integrate heterogeneous data to assess risks from stock fluctuations to systemic vulnerabilities.<n>We propose Uni-FinLLM, a unified multimodal large language model that uses a shared Transformer backbone and modular task heads to jointly process financial text, numerical time series, fundamentals, and visual data.
  arXiv  Detail & Related papers  (2026-01-06T03:22:51Z)
• Reliable LLM-Based Edge-Cloud-Expert Cascades for Telecom Knowledge Systems [54.916243942641444]
  Large language models (LLMs) are emerging as key enablers of automation in domains such as telecommunications.<n>We study an edge-cloud-expert cascaded LLM-based knowledge system that supports decision-making through a question-and-answer pipeline.
  arXiv  Detail & Related papers  (2025-12-23T03:10:09Z)
• CALM Before the STORM: Unlocking Native Reasoning for Optimization Modeling [60.55856973678002]
  Large Reasoning Models (LRMs) have demonstrated strong capabilities in complex multi-step reasoning.<n>Existing domain adaptation methods, originally designed for earlier instruction-tuned models, often fail to exploit the advanced reasoning patterns of modern LRMs.<n>We propose textbfCALM, a framework that progressively refines LRMs within their native reasoning modes for optimization modeling tasks.
  arXiv  Detail & Related papers  (2025-10-05T13:38:31Z)
• Hierarchical Evaluation Function: A Multi-Metric Approach for Optimizing Demand Forecasting Models [0.479839492673697]
  The Hierarchical Evaluation Function (HEF) is proposed as a multi-metric framework for hyperparameter optimization.<n>HEF integrates explanatory power (R2), sensitivity to extreme errors (RMSE), and average accuracy (MAE)<n>The performance of HEF was assessed using four widely recognized benchmark datasets in the forecasting domain.
  arXiv  Detail & Related papers  (2025-08-18T16:25:49Z)
• FinHEAR: Human Expertise and Adaptive Risk-Aware Temporal Reasoning for Financial Decision-Making [58.04602111184477]
  FinHEAR is a framework for Human Expertise and Adaptive Risk-aware reasoning.<n>It orchestrates specialized agents to analyze historical trends, interpret current events, and retrieve expert-informed precedents.<n> Empirical results on financial datasets show that FinHEAR consistently outperforms strong baselines across trend prediction and trading tasks.
  arXiv  Detail & Related papers  (2025-06-10T04:06:51Z)
• Speculative Reward Model Boosts Decision Making Ability of LLMs Cost-Effectively [13.40488551654639]
  We introduce the 3E Criteria to assess the cost-effectiveness of search strategies.<n>We propose the Speculative Reward Model (SRM), a plug-and-play framework that integrates seamlessly with existing search strategies.<n> Experimental results show that RM reduces costs to 1/10 of the original search framework on average while maintaining effectiveness.
  arXiv  Detail & Related papers  (2025-05-31T05:32:12Z)
• Modality-Balancing Preference Optimization of Large Multimodal Models by Adversarial Negative Mining [75.14823970163685]
  We propose a novel preference learning framework, Modality-Balancing Preference Optimization (MBPO), to address the modality imbalance in LMMs.<n>MBPO constructs a more effective offline preference dataset by generating hard negatives, i.e., rejected responses misled by LLM biases.<n>It can enhance LMM performance on challenging vision-language tasks and effectively reduce hallucinations.
  arXiv  Detail & Related papers  (2025-05-20T03:59:05Z)
• Cache-Efficient Posterior Sampling for Reinforcement Learning with LLM-Derived Priors Across Discrete and Continuous Domains [2.1797343876622097]
  Large language models (LLMs) as priors in reinforcement learning (RL) offers significant advantages but comes with substantial computational costs.<n>We present a principled cache-efficient framework for posterior sampling with LLM-derived priors that dramatically reduces these costs while maintaining high performance.
  arXiv  Detail & Related papers  (2025-05-12T06:53:24Z)
• EMAFusion: A Self-Optimizing System for Seamless LLM Selection and Integration [9.019951211182969]
  We introduce EMAFusion, a new framework that self-optimizes for LLM selection and reliable execution for a given query.<n>We find EMAFusion outperforms the best individual models by over 2.6 percentage points (94.3% vs. 91.7%) while being 4X cheaper than the average cost.<n>Our combined routing approach delivers 94.3% accuracy compared to taxonomy-based (88.1%) and learned model predictor-based (91.7%) methods alone.
  arXiv  Detail & Related papers  (2025-04-14T20:04:54Z)
• Democratizing LLMs: An Exploration of Cost-Performance Trade-offs in Self-Refined Open-Source Models [53.859446823312126]
  SoTA open source models of varying sizes from 7B - 65B, on average, improve 8.2% from their baseline performance. Strikingly, even models with extremely small memory footprints, such as Vicuna-7B, show a 11.74% improvement overall and up to a 25.39% improvement in high-creativity, open ended tasks.
  arXiv  Detail & Related papers  (2023-10-11T15:56:00Z)
• Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models [11.154814189699735]
  Large Language Models (LLMs) pre-trained on extensive corpora have demonstrated superior performance across various NLP tasks. We introduce a retrieval-augmented LLMs framework for financial sentiment analysis. Our approach achieves 15% to 48% performance gain in accuracy and F1 score.
  arXiv  Detail & Related papers  (2023-10-06T05:40:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.