Exponentially Weighted l_2 Regularization Strategy in Constructing Reinforced Second-order Fuzzy Rule-based Model

• URL: http://arxiv.org/abs/2007.01208v1
• Date: Thu, 2 Jul 2020 15:42:15 GMT
• Title: Exponentially Weighted l_2 Regularization Strategy in Constructing Reinforced Second-order Fuzzy Rule-based Model
• Authors: Congcong Zhang, Sung-Kwun Oh, Witold Pedrycz, Zunwei Fu and Shanzhen Lu
• Abstract summary: In the conventional Takagi-Sugeno-Kang (TSK)-type fuzzy models, constant or linear functions are usually utilized as the consequent parts of the fuzzy rules. We introduce an exponential weight approach inspired by the weight function theory encountered in harmonic analysis.
• Score: 72.57056258027336
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the conventional Takagi-Sugeno-Kang (TSK)-type fuzzy models, constant or linear functions are usually utilized as the consequent parts of the fuzzy rules, but they cannot effectively describe the behavior within local regions defined by the antecedent parts. In this article, a theoretical and practical design methodology is developed to address this problem. First, the information granulation (Fuzzy C-Means) method is applied to capture the structure in the data and split the input space into subspaces, as well as form the antecedent parts. Second, the quadratic polynomials (QPs) are employed as the consequent parts. Compared with constant and linear functions, QPs can describe the input-output behavior within the local regions (subspaces) by refining the relationship between input and output variables. However, although QP can improve the approximation ability of the model, it could lead to the deterioration of the prediction ability of the model (e.g., overfitting). To handle this issue, we introduce an exponential weight approach inspired by the weight function theory encountered in harmonic analysis. More specifically, we adopt the exponential functions as the targeted penalty terms, which are equipped with l2 regularization (l2) (i.e., exponential weighted l2, ewl_2) to match the proposed reinforced second-order fuzzy rule-based model (RSFRM) properly. The advantage of el 2 compared to ordinary l2 lies in separately identifying and penalizing different types of polynomial terms in the coefficient estimation, and its results not only alleviate the overfitting and prevent the deterioration of generalization ability but also effectively release the prediction potential of the model.

Related papers

• Variational formulation based on duality to solve partial differential equations: Use of B-splines and machine learning approximants [0.0]
  Partial differential equations (PDEs) in fluid mechanics, inelastic deformation in solids, and transient parabolic and hyperbolic equations do not have an exact, primal variational structure. A variational principle based on the dual (Lagrange multiplier) field was proposed to treat the given PDE as constraints. The vanishing of the first variation of the dual functional is, up to Dirichlet boundary conditions on dual fields, the weak form of the primal PDE problem. We derive the dual weak form for the linear, one-dimensional, transient convection-diffusion equation.
  arXiv  Detail & Related papers  (2024-12-02T07:53:47Z)
• Scaling and renormalization in high-dimensional regression [72.59731158970894]
  This paper presents a succinct derivation of the training and generalization performance of a variety of high-dimensional ridge regression models. We provide an introduction and review of recent results on these topics, aimed at readers with backgrounds in physics and deep learning.
  arXiv  Detail & Related papers  (2024-05-01T15:59:00Z)
• Double Duality: Variational Primal-Dual Policy Optimization for Constrained Reinforcement Learning [132.7040981721302]
  We study the Constrained Convex Decision Process (MDP), where the goal is to minimize a convex functional of the visitation measure. Design algorithms for a constrained convex MDP faces several challenges, including handling the large state space.
  arXiv  Detail & Related papers  (2024-02-16T16:35:18Z)
• Mean-Field Analysis for Learning Subspace-Sparse Polynomials with Gaussian Input [10.594689377150356]
  We study the mean-field flow for learning subspace-sparses using gradient descent and two-layer neural networks. We establish a necessary condition for SGD-learnability, involving both the characteristics of the target function and the expressiveness of the activation function.
  arXiv  Detail & Related papers  (2024-02-14T05:34:24Z)
• Wasserstein proximal operators describe score-based generative models and resolve memorization [12.321631823103894]
  We first formulate SGMs in terms of the Wasserstein proximal operator (WPO) We show that WPO describes the inductive bias of diffusion and score-based models. We present an interpretable kernel-based model for the score function which dramatically improves the performance of SGMs.
  arXiv  Detail & Related papers  (2024-02-09T03:33:13Z)
• PDE-constrained Gaussian process surrogate modeling with uncertain data locations [1.943678022072958]
  We propose a Bayesian approach that integrates the variability of input data into the Gaussian process regression for function and partial differential equation approximation. A consistently good performance of generalization is observed, and a substantial reduction in the predictive uncertainties is achieved.
  arXiv  Detail & Related papers  (2023-05-19T10:53:08Z)
• Data-Driven Influence Functions for Optimization-Based Causal Inference [105.5385525290466]
  We study a constructive algorithm that approximates Gateaux derivatives for statistical functionals by finite differencing. We study the case where probability distributions are not known a priori but need to be estimated from data.
  arXiv  Detail & Related papers  (2022-08-29T16:16:22Z)
• Estimation of Bivariate Structural Causal Models by Variational Gaussian Process Regression Under Likelihoods Parametrised by Normalising Flows [74.85071867225533]
  Causal mechanisms can be described by structural causal models. One major drawback of state-of-the-art artificial intelligence is its lack of explainability.
  arXiv  Detail & Related papers  (2021-09-06T14:52:58Z)
• Learning Likelihoods with Conditional Normalizing Flows [54.60456010771409]
  Conditional normalizing flows (CNFs) are efficient in sampling and inference. We present a study of CNFs where the base density to output space mapping is conditioned on an input x, to model conditional densities p(y|x)
  arXiv  Detail & Related papers  (2019-11-29T19:17:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.