Compact Model Parameter Extraction via Derivative-Free Optimization

• URL: http://arxiv.org/abs/2406.16355v2
• Date: Mon, 11 Nov 2024 20:00:02 GMT
• Title: Compact Model Parameter Extraction via Derivative-Free Optimization
• Authors: Rafael Perez Martinez, Masaya Iwamoto, Kelly Woo, Zhengliang Bian, Roberto Tinti, Stephen Boyd, Srabanti Chowdhury,
• Abstract summary: Traditionally, parameter extraction is performed manually by dividing the complete set of parameters into smaller subsets. We employ derivative-free optimization to identify a good parameter set that best fits the compact model without performing an exhaustive number of simulations. We demonstrate the effectiveness of our approach by successfully modeling a diamond Schottky diode with the SPICE diode model and a GaN-on-SiC HEMT with the ASM-HEMT model.
• Score: 0.0
• License:
• Abstract: In this paper, we address the problem of compact model parameter extraction to simultaneously extract tens of parameters via derivative-free optimization. Traditionally, parameter extraction is performed manually by dividing the complete set of parameters into smaller subsets, each targeting different operational regions of the device, a process that can take several days or weeks. Our approach streamlines this process by employing derivative-free optimization to identify a good parameter set that best fits the compact model without performing an exhaustive number of simulations. We further enhance the optimization process to address three critical issues in device modeling by carefully choosing a loss function that focuses on relative errors rather than absolute errors to ensure consistent performance across different orders of magnitude, prioritizes accuracy in key operational regions above a specific threshold, and reduces sensitivity to outliers. Furthermore, we utilize the concept of train-test split to assess the model fit and avoid overfitting. We demonstrate the effectiveness of our approach by successfully modeling a diamond Schottky diode with the SPICE diode model and a GaN-on-SiC HEMT with the ASM-HEMT model. For the latter, which involves extracting 35 parameters for the ASM-HEMT DC model, we identified the best set of parameters in under 6,000 trials. Additional examples using both devices are provided to demonstrate robustness to outliers, showing that an excellent fit is achieved even with over 25% of the data purposely corrupted. These examples demonstrate the practicality of our approach, highlighting the benefits of derivative-free optimization in device modeling.

Related papers

• Model Fusion through Bayesian Optimization in Language Model Fine-Tuning [16.86812534268461]
  Fine-tuning pre-trained models for downstream tasks is a widely adopted technique known for its adaptability and reliability across various domains. We introduce a novel model fusion technique that optimize both the desired metric and loss through multi-objective Bayesian optimization. Experiments across various downstream tasks show considerable performance improvements using our Bayesian optimization-guided method.
  arXiv  Detail & Related papers  (2024-11-11T04:36:58Z)
• SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models [85.67096251281191]
  We present an innovative approach to model fusion called zero-shot Sparse MIxture of Low-rank Experts (SMILE) construction. SMILE allows for the upscaling of source models into an MoE model without extra data or further training. We conduct extensive experiments across diverse scenarios, such as image classification and text generation tasks, using full fine-tuning and LoRA fine-tuning.
  arXiv  Detail & Related papers  (2024-08-19T17:32:15Z)
• Scaling Exponents Across Parameterizations and Optimizers [94.54718325264218]
  We propose a new perspective on parameterization by investigating a key assumption in prior work. Our empirical investigation includes tens of thousands of models trained with all combinations of threes. We find that the best learning rate scaling prescription would often have been excluded by the assumptions in prior work.
  arXiv  Detail & Related papers  (2024-07-08T12:32:51Z)
• Spectrum-Aware Parameter Efficient Fine-Tuning for Diffusion Models [73.88009808326387]
  We propose a novel spectrum-aware adaptation framework for generative models. Our method adjusts both singular values and their basis vectors of pretrained weights. We introduce Spectral Ortho Decomposition Adaptation (SODA), which balances computational efficiency and representation capacity.
  arXiv  Detail & Related papers  (2024-05-31T17:43:35Z)
• Sine Activated Low-Rank Matrices for Parameter Efficient Learning [25.12262017296922]
  We propose a novel theoretical framework that integrates a sinusoidal function within the low-rank decomposition process. Our method proves to be an enhancement for existing low-rank models, as evidenced by its successful application in Vision Transformers (ViT), Large Language Models (LLMs), Neural Radiance Fields (NeRF)
  arXiv  Detail & Related papers  (2024-03-28T08:58:20Z)
• E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuning [55.50908600818483]
  Fine-tuning large-scale pretrained vision models for new tasks has become increasingly parameter-intensive. We propose an Effective and Efficient Visual Prompt Tuning (E2VPT) approach for large-scale transformer-based model adaptation. Our approach outperforms several state-of-the-art baselines on two benchmarks.
  arXiv  Detail & Related papers  (2023-07-25T19:03:21Z)
• Active-Learning-Driven Surrogate Modeling for Efficient Simulation of Parametric Nonlinear Systems [0.0]
  In absence of governing equations, we need to construct the parametric reduced-order surrogate model in a non-intrusive fashion. Our work provides a non-intrusive optimality criterion to efficiently populate the parameter snapshots. We propose an active-learning-driven surrogate model using kernel-based shallow neural networks.
  arXiv  Detail & Related papers  (2023-06-09T18:01:14Z)
• On the Effectiveness of Parameter-Efficient Fine-Tuning [79.6302606855302]
  Currently, many research works propose to only fine-tune a small portion of the parameters while keeping most of the parameters shared across different tasks. We show that all of the methods are actually sparse fine-tuned models and conduct a novel theoretical analysis of them. Despite the effectiveness of sparsity grounded by our theory, it still remains an open problem of how to choose the tunable parameters.
  arXiv  Detail & Related papers  (2022-11-28T17:41:48Z)
• Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models [90.24999406296867]
  In contrast with the standard fine-tuning, delta tuning only fine-tunes a small portion of the model parameters while keeping the rest untouched. Recent studies have demonstrated that a series of delta tuning methods with distinct tuned parameter selection could achieve performance on a par with full- parameter fine-tuning.
  arXiv  Detail & Related papers  (2022-03-14T07:56:32Z)
• On the Parameter Combinations That Matter and on Those That do Not [0.0]
  We present a data-driven approach to characterizing nonidentifiability of a model's parameters. By employing Diffusion Maps and their extensions, we discover the minimal combinations of parameters required to characterize the dynamic output behavior.
  arXiv  Detail & Related papers  (2021-10-13T13:46:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.