Fuzzy Rule-based Differentiable Representation Learning

• URL: http://arxiv.org/abs/2503.13548v1
• Date: Sun, 16 Mar 2025 14:00:34 GMT
• Title: Fuzzy Rule-based Differentiable Representation Learning
• Authors: Wei Zhang, Zhaohong Deng, Guanjin Wang, Kup-Sze Choi,
• Abstract summary: This paper introduces a novel representation learning method grounded in an interpretable fuzzy rule-based model.<n>It is built upon the Takagi-Sugeno-Kang fuzzy system (TSK-FS) to initially map input data to a high-dimensional fuzzy feature space.<n>A novel differentiable optimization method is proposed for the consequence part learning which can preserve the model's interpretability and transparency.
• Score: 16.706014479049493
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Representation learning has emerged as a crucial focus in machine and deep learning, involving the extraction of meaningful and useful features and patterns from the input data, thereby enhancing the performance of various downstream tasks such as classification, clustering, and prediction. Current mainstream representation learning methods primarily rely on non-linear data mining techniques such as kernel methods and deep neural networks to extract abstract knowledge from complex datasets. However, most of these methods are black-box, lacking transparency and interpretability in the learning process, which constrains their practical utility. To this end, this paper introduces a novel representation learning method grounded in an interpretable fuzzy rule-based model. Specifically, it is built upon the Takagi-Sugeno-Kang fuzzy system (TSK-FS) to initially map input data to a high-dimensional fuzzy feature space through the antecedent part of the TSK-FS. Subsequently, a novel differentiable optimization method is proposed for the consequence part learning which can preserve the model's interpretability and transparency while further exploring the nonlinear relationships within the data. This optimization method retains the essence of traditional optimization, with certain parts of the process parameterized corresponding differentiable modules constructed, and a deep optimization process implemented. Consequently, this method not only enhances the model's performance but also ensures its interpretability. Moreover, a second-order geometry preservation method is introduced to further improve the robustness of the proposed method. Extensive experiments conducted on various benchmark datasets validate the superiority of the proposed method, highlighting its potential for advancing representation learning methodologies.

Related papers

• Novel Saliency Analysis for the Forward Forward Algorithm [0.0]
  We introduce the Forward Forward algorithm into neural network training. This method involves executing two forward passes the first with actual data to promote positive reinforcement, and the second with synthetically generated negative data to enable discriminative learning. To overcome the limitations inherent in traditional saliency techniques, we developed a bespoke saliency algorithm specifically tailored for the Forward Forward framework.
  arXiv  Detail & Related papers  (2024-09-18T17:21:59Z)
• High-Performance Few-Shot Segmentation with Foundation Models: An Empirical Study [64.06777376676513]
  We develop a few-shot segmentation (FSS) framework based on foundation models. To be specific, we propose a simple approach to extract implicit knowledge from foundation models to construct coarse correspondence. Experiments on two widely used datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-09-10T08:04:11Z)
• MARS: Meta-Learning as Score Matching in the Function Space [79.73213540203389]
  We present a novel approach to extracting inductive biases from a set of related datasets. We use functional Bayesian neural network inference, which views the prior as a process and performs inference in the function space. Our approach can seamlessly acquire and represent complex prior knowledge by metalearning the score function of the data-generating process.
  arXiv  Detail & Related papers  (2022-10-24T15:14:26Z)
• Object Representations as Fixed Points: Training Iterative Refinement Algorithms with Implicit Differentiation [88.14365009076907]
  Iterative refinement is a useful paradigm for representation learning. We develop an implicit differentiation approach that improves the stability and tractability of training.
  arXiv  Detail & Related papers  (2022-07-02T10:00:35Z)
• MACE: An Efficient Model-Agnostic Framework for Counterfactual Explanation [132.77005365032468]
  We propose a novel framework of Model-Agnostic Counterfactual Explanation (MACE) In our MACE approach, we propose a novel RL-based method for finding good counterfactual examples and a gradient-less descent method for improving proximity. Experiments on public datasets validate the effectiveness with better validity, sparsity and proximity.
  arXiv  Detail & Related papers  (2022-05-31T04:57:06Z)
• Model-Based Deep Learning: On the Intersection of Deep Learning and Optimization [101.32332941117271]
  Decision making algorithms are used in a multitude of different applications. Deep learning approaches that use highly parametric architectures tuned from data without relying on mathematical models are becoming increasingly popular. Model-based optimization and data-centric deep learning are often considered to be distinct disciplines.
  arXiv  Detail & Related papers  (2022-05-05T13:40:08Z)
• Unsupervised feature selection via self-paced learning and low-redundant regularization [6.083524716031565]
  An unsupervised feature selection is proposed by integrating the framework of self-paced learning and subspace learning. The convergence of the method is proved theoretically and experimentally. The experimental results show that the proposed method can improve the performance of clustering methods and outperform other compared algorithms.
  arXiv  Detail & Related papers  (2021-12-14T08:28:19Z)
• Transfer Learning with Gaussian Processes for Bayesian Optimization [9.933956770453438]
  We provide a unified view on hierarchical GP models for transfer learning, which allows us to analyze the relationship between methods. We develop a novel closed-form boosted GP transfer model that fits between existing approaches in terms of complexity. We evaluate the performance of the different approaches in large-scale experiments and highlight strengths and weaknesses of the different transfer-learning methods.
  arXiv  Detail & Related papers  (2021-11-22T14:09:45Z)
• Inducing Semantic Grouping of Latent Concepts for Explanations: An Ante-Hoc Approach [18.170504027784183]
  We show that by exploiting latent and properly modifying different parts of the model can result better explanation as well as provide superior predictive performance. We also proposed a technique of using two different self-supervision techniques to extract meaningful concepts related to the type of self-supervision considered.
  arXiv  Detail & Related papers  (2021-08-25T07:09:57Z)
• Deep Unfolding Network for Image Super-Resolution [159.50726840791697]
  This paper proposes an end-to-end trainable unfolding network which leverages both learning-based methods and model-based methods. The proposed network inherits the flexibility of model-based methods to super-resolve blurry, noisy images for different scale factors via a single model.
  arXiv  Detail & Related papers  (2020-03-23T17:55:42Z)
• Multi-Objective Genetic Programming for Manifold Learning: Balancing Quality and Dimensionality [4.4181317696554325]
  State-of-the-art manifold learning algorithms are opaque in how they perform this transformation. We introduce a multi-objective approach that automatically balances the competing objectives of manifold quality and dimensionality. Our proposed approach is competitive with a range of baseline and state-of-the-art manifold learning methods.
  arXiv  Detail & Related papers  (2020-01-05T23:24:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.