Hybrid Modeling Design Patterns

• URL: http://arxiv.org/abs/2401.00033v1
• Date: Fri, 29 Dec 2023 15:40:38 GMT
• Title: Hybrid Modeling Design Patterns
• Authors: Maja Rudolph, Stefan Kurz, Barbara Rakitsch
• Abstract summary: We provide four base patterns that can serve as blueprints for combining data-driven components with domain knowledge into a hybrid approach. We also present two composition patterns that govern the combination of the base patterns into more complex hybrid models. Each design pattern is illustrated by typical use cases from application areas such as climate modeling, engineering, and physics.
• Score: 10.266928164137635
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Design patterns provide a systematic way to convey solutions to recurring modeling challenges. This paper introduces design patterns for hybrid modeling, an approach that combines modeling based on first principles with data-driven modeling techniques. While both approaches have complementary advantages there are often multiple ways to combine them into a hybrid model, and the appropriate solution will depend on the problem at hand. In this paper, we provide four base patterns that can serve as blueprints for combining data-driven components with domain knowledge into a hybrid approach. In addition, we also present two composition patterns that govern the combination of the base patterns into more complex hybrid models. Each design pattern is illustrated by typical use cases from application areas such as climate modeling, engineering, and physics.

Related papers

• Model Assembly Learning with Heterogeneous Layer Weight Merging [57.8462476398611]
  We introduce Model Assembly Learning (MAL), a novel paradigm for model merging. MAL integrates parameters from diverse models in an open-ended model zoo to enhance the base model's capabilities.
  arXiv  Detail & Related papers  (2025-03-27T16:21:53Z)
• Training-free Heterogeneous Model Merging [40.681362819808136]
  We propose an innovative model merging framework designed for heterogeneous models. We show that the merging of structurally heterogeneous models can achieve performance levels comparable to those of homogeneous merging. Our code is publicly available at https://github.com/zju-vipa/training_free_heterogeneous_model_merging.
  arXiv  Detail & Related papers  (2024-12-29T04:49:11Z)
• Jet: A Modern Transformer-Based Normalizing Flow [62.2573739835562]
  We revisit the design of the coupling-based normalizing flow models by carefully ablating prior design choices. We achieve state-of-the-art quantitative and qualitative performance with a much simpler architecture.
  arXiv  Detail & Related papers  (2024-12-19T18:09:42Z)
• Learnable & Interpretable Model Combination in Dynamic Systems Modeling [0.0]
  We discuss which types of models are usually combined and propose a model interface that is capable of expressing a variety of mixed equation based models. We propose a new wildcard topology, that is capable of describing the generic connection between two combined models in an easy to interpret fashion. The contributions of this paper are highlighted at a proof of concept: Different connection topologies between two models are learned, interpreted and compared.
  arXiv  Detail & Related papers  (2024-06-12T11:17:11Z)
• FusionBench: A Comprehensive Benchmark of Deep Model Fusion [78.80920533793595]
  Deep model fusion is a technique that unifies the predictions or parameters of several deep neural networks into a single model. FusionBench is the first comprehensive benchmark dedicated to deep model fusion.
  arXiv  Detail & Related papers  (2024-06-05T13:54:28Z)
• Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models [54.132297393662654]
  We introduce a hybrid method that fine-tunes cutting-edge diffusion models by optimizing reward models through RL. We demonstrate the capability of our approach to outperform the best designs in offline data, leveraging the extrapolation capabilities of reward models.
  arXiv  Detail & Related papers  (2024-05-30T03:57:29Z)
• Model Synthesis for Zero-Shot Model Attribution [26.835046772924258]
  generative models are shaping various fields such as art, design, and human-computer interaction. We propose a model synthesis technique, which generates numerous synthetic models mimicking the fingerprint patterns of real-world generative models. Our experiments demonstrate that this fingerprint extractor, trained solely on synthetic models, achieves impressive zero-shot generalization on a wide range of real-world generative models.
  arXiv  Detail & Related papers  (2023-07-29T13:00:42Z)
• Planning with Diffusion for Flexible Behavior Synthesis [125.24438991142573]
  We consider what it would look like to fold as much of the trajectory optimization pipeline as possible into the modeling problem. The core of our technical approach lies in a diffusion probabilistic model that plans by iteratively denoising trajectories.
  arXiv  Detail & Related papers  (2022-05-20T07:02:03Z)
• An Ample Approach to Data and Modeling [1.0152838128195467]
  We describe a framework for modeling how models can be built that integrates concepts and methods from a wide range of fields. The reference M* meta model framework is presented, which relies critically in associating whole datasets and respective models in terms of a strict equivalence relation. Several considerations about how the developed framework can provide insights about data clustering, complexity, collaborative research, deep learning, and creativity are then presented.
  arXiv  Detail & Related papers  (2021-10-05T01:26:09Z)
• Hybrid modeling of the human cardiovascular system using NeuralFMUs [0.0]
  We show that the hybrid modeling process is more comfortable, needs less system knowledge and is less error-prone compared to modeling solely based on first principle. The resulting hybrid model has improved in computation performance, compared to a pure first principle white-box model. The considered use-case can serve as example for other modeling and simulation applications in and beyond the medical domain.
  arXiv  Detail & Related papers  (2021-09-10T13:48:43Z)
• Model Complexity of Deep Learning: A Survey [79.20117679251766]
  We conduct a systematic overview of the latest studies on model complexity in deep learning. We review the existing studies on those two categories along four important factors, including model framework, model size, optimization process and data complexity.
  arXiv  Detail & Related papers  (2021-03-08T22:39:32Z)
• Reconstruction of Pairwise Interactions using Energy-Based Models [3.553493344868414]
  We show that hybrid models, which combine a pairwise model and a neural network, can lead to significant improvements in the reconstruction of pairwise interactions. This is in line with the general idea that simple interpretable models and complex black-box models are not necessarily a dichotomy.
  arXiv  Detail & Related papers  (2020-12-11T20:15:10Z)
• CoSE: Compositional Stroke Embeddings [52.529172734044664]
  We present a generative model for complex free-form structures such as stroke-based drawing tasks. Our approach is suitable for interactive use cases such as auto-completing diagrams.
  arXiv  Detail & Related papers  (2020-06-17T15:22:54Z)
• Hybrid modeling: Applications in real-time diagnosis [64.5040763067757]
  We outline a novel hybrid modeling approach that combines machine learning inspired models and physics-based models. We are using such models for real-time diagnosis applications.
  arXiv  Detail & Related papers  (2020-03-04T00:44:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.