Convex Hybrid Modeling: An Operator-Based Approach

• URL: http://arxiv.org/abs/2605.23151v1
• Date: Fri, 22 May 2026 01:56:51 GMT
• Title: Convex Hybrid Modeling: An Operator-Based Approach
• Abstract summary: This paper considers hybrid modeling by formulating convex learning problems that account for interpretability systematically.<n>In the more general setting, the system is regarded as a kernel-based mixture of interpretable models.
• Score: 0.989901717499058
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: While machine learning can accurately model process systems, models for decision making should also be structurally simple and physically interpretable. In process control, for example, (nearly) linear models are favored than nonlinear ones, promoting the use of operator theory, which ``universally'' represents a nonlinear system by a nonparametric operator. On the other hand, interpretability requires by a ``non-universal'', parametric nonlinear model family satisfying first principles; these constraints tend to complicate the learning procedure. This paper considers hybrid modeling by formulating convex learning problems that account for interpretability systematically and give surrogate models efficiently. Three settings are discussed -- (i) regularization around a particular ``reference model'', (ii) restriction on an ``interpretable subspace'', and more generally, (iii) restriction on a ``interpretable manifold'' that is nonlinearly parameterized. In the more general setting, by introducing an operator-theoretic technique to re-parameterize models in the ``lifted'' parameters (``canonical features'', potentially infinite-dimensional), the system is regarded as a kernel-based mixture of interpretable models. Application to both static and dynamic models are exemplified in numerical studies.

Related papers

• LiD-GLM: Lipschitz-constrained Deep Generalized Linear Models [1.389948527681755]
  Combination of traditional statistical models and neural network (NN) components is an intriguing approach to construct hybrid models.<n>We propose invertible residual neural networks (i-ResNets) to equip models with a flexible correction of distributional assumptions.<n>We develop specific inherent interpretation techniques for our model and enforce model identifiability through an adapted post-hocization.
  arXiv  Detail & Related papers  (2026-08-17T09:45:35Z)
• Non-Parametric Rehearsal Learning via Conditional Mean Embeddings [88.89267783967263]
  We propose a non-parametric rehearsal learning approach for textitavoiding undesired future (AUF) problems.<n>Specifically, we use kernel machinery to reformulate the AUF objective into a unified representation that disentangles desirability modeling from action-induced distributional changes.
  arXiv  Detail & Related papers  (2026-05-09T15:30:52Z)
• Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Models [51.85815025140659]
  Modern Machine Learning (ML) and Deep Neural Networks (DNNs) often operate on high-dimensional data.<n>In particular, the proportional regime where the data dimension, sample size, and number of model parameters are all large gives rise to novel and sometimes counterintuitive behaviors.<n>This paper extends traditional Random Matrix Theory (RMT) beyond eigenvalue-based analysis of linear models to address the challenges posed by nonlinear ML models.
  arXiv  Detail & Related papers  (2025-06-16T06:54:08Z)
• Nonlinear Model Order Reduction of Dynamical Systems in Process Engineering: Review and Comparison [50.0791489606211]
  We review state-of-the-art nonlinear model order reduction methods.<n>We discuss both general-purpose methods and tailored approaches for (chemical) process systems.
  arXiv  Detail & Related papers  (2025-06-15T11:39:12Z)
• Efficient identification of linear, parameter-varying, and nonlinear systems with noise models [1.6385815610837167]
  We present a general system identification procedure capable of estimating a broad spectrum of state-space dynamical models.<n>We show that for this general class of model structures, the model dynamics can be separated into a deterministic process and a noise part.<n>We parameterize the involved nonlinear functional relations by means of artificial neural-networks (ANNs)
  arXiv  Detail & Related papers  (2025-04-16T11:23:30Z)
• No Equations Needed: Learning System Dynamics Without Relying on Closed-Form ODEs [56.78271181959529]
  This paper proposes a conceptual shift to modeling low-dimensional dynamical systems by departing from the traditional two-step modeling process.<n>Instead of first discovering a closed-form equation and then analyzing it, our approach, direct semantic modeling, predicts the semantic representation of the dynamical system.<n>Our approach not only simplifies the modeling pipeline but also enhances the transparency and flexibility of the resulting models.
  arXiv  Detail & Related papers  (2025-01-30T18:36:48Z)
• BINDy -- Bayesian identification of nonlinear dynamics with reversible-jump Markov-chain Monte-Carlo [0.0]
  Model parsimony is an important emphcognitive bias in data-driven modelling that aids interpretability and helps to prevent over-fitting.<n>Sparse identification of nonlinear dynamics (SINDy) methods are able to learn sparse representations of complex dynamics directly from data.<n>A novel Bayesian treatment of dictionary learning system identification, as an alternative to SINDy, is envisaged.
  arXiv  Detail & Related papers  (2024-08-15T10:03:30Z)
• Scaling and renormalization in high-dimensional regression [72.59731158970894]
  We present a unifying perspective on recent results on ridge regression.<n>We use the basic tools of random matrix theory and free probability, aimed at readers with backgrounds in physics and deep learning.<n>Our results extend and provide a unifying perspective on earlier models of scaling laws.
  arXiv  Detail & Related papers  (2024-05-01T15:59:00Z)
• Meaning Representations from Trajectories in Autoregressive Models [106.63181745054571]
  We propose to extract meaning representations from autoregressive language models by considering the distribution of all possible trajectories extending an input text. This strategy is prompt-free, does not require fine-tuning, and is applicable to any pre-trained autoregressive model. We empirically show that the representations obtained from large models align well with human annotations, outperform other zero-shot and prompt-free methods on semantic similarity tasks, and can be used to solve more complex entailment and containment tasks that standard embeddings cannot handle.
  arXiv  Detail & Related papers  (2023-10-23T04:35:58Z)
• Physics-informed regularization and structure preservation for learning stable reduced models from data with operator inference [0.0]
  Operator inference learns low-dimensional dynamical-system models with nonlinear terms from trajectories of high-dimensional physical systems. A regularizer for operator inference that induces a stability bias onto quadratic models is proposed. A formulation of operator inference is proposed that enforces model constraints for preserving structure.
  arXiv  Detail & Related papers  (2021-07-06T13:15:54Z)
• Hessian Eigenspectra of More Realistic Nonlinear Models [73.31363313577941]
  We make a emphprecise characterization of the Hessian eigenspectra for a broad family of nonlinear models. Our analysis takes a step forward to identify the origin of many striking features observed in more complex machine learning models.
  arXiv  Detail & Related papers  (2021-03-02T06:59:52Z)
• Operator inference for non-intrusive model reduction of systems with non-polynomial nonlinear terms [6.806310449963198]
  This work presents a non-intrusive model reduction method to learn low-dimensional models of dynamical systems with non-polynomial nonlinear terms that are spatially local. The proposed approach requires only the non-polynomial terms in analytic form and learns the rest of the dynamics from snapshots computed with a potentially black-box full-model solver. The proposed method is demonstrated on three problems governed by partial differential equations, namely the diffusion-reaction Chafee-Infante model, a tubular reactor model for reactive flows, and a batch-chromatography model that describes a chemical separation process.
  arXiv  Detail & Related papers  (2020-02-22T16:27:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.