TMoE-P: Towards the Pareto Optimum for Multivariate Soft Sensors
- URL: http://arxiv.org/abs/2302.10477v1
- Date: Tue, 21 Feb 2023 06:49:09 GMT
- Title: TMoE-P: Towards the Pareto Optimum for Multivariate Soft Sensors
- Authors: Licheng Pan, Hao Wang, Zhichao Chen, Yuxing Huang, Xinggao Liu
- Abstract summary: We reformulate the multi-variate soft sensor to a multi-objective problem, to address both issues and advance state-of-the-art performance.
To handle the negative transfer issue, we first propose an Objective-aware Mixture-of-Experts (OMoE) module, utilizing objective-specific and objective-shared experts for parameter sharing.
To address the seesaw phenomenon, we then propose a Task-aware Mixture-of-Experts framework for achieving the optimum routing.
- Score: 7.236362889442992
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Multi-variate soft sensor seeks accurate estimation of multiple quality
variables using measurable process variables, which have emerged as a key
factor in improving the quality of industrial manufacturing. The current
progress stays in some direct applications of multitask network architectures;
however, there are two fundamental issues remain yet to be investigated with
these approaches: (1) negative transfer, where sharing representations despite
the difference of discriminate representations for different objectives
degrades performance; (2) seesaw phenomenon, where the optimizer focuses on one
dominant yet simple objective at the expense of others. In this study, we
reformulate the multi-variate soft sensor to a multi-objective problem, to
address both issues and advance state-of-the-art performance. To handle the
negative transfer issue, we first propose an Objective-aware Mixture-of-Experts
(OMoE) module, utilizing objective-specific and objective-shared experts for
parameter sharing while maintaining the distinction between objectives. To
address the seesaw phenomenon, we then propose a Pareto Objective Routing (POR)
module, adjusting the weights of learning objectives dynamically to achieve the
Pareto optimum, with solid theoretical supports. We further present a
Task-aware Mixture-of-Experts framework for achieving the Pareto optimum
(TMoE-P) in multi-variate soft sensor, which consists of a stacked OMoE module
and a POR module. We illustrate the efficacy of TMoE-P with an open soft sensor
benchmark, where TMoE-P effectively alleviates the negative transfer and seesaw
issues and outperforms the baseline models.
Related papers
- UCB-driven Utility Function Search for Multi-objective Reinforcement Learning [75.11267478778295]
In Multi-objective Reinforcement Learning (MORL) agents are tasked with optimising decision-making behaviours.
We focus on the case of linear utility functions parameterised by weight vectors w.
We introduce a method based on Upper Confidence Bound to efficiently search for the most promising weight vectors during different stages of the learning process.
arXiv Detail & Related papers (2024-05-01T09:34:42Z) - Intuition-aware Mixture-of-Rank-1-Experts for Parameter Efficient Finetuning [50.73666458313015]
Large Language Models (LLMs) have demonstrated significant potential in performing multiple tasks in multimedia applications.
MoE has been emerged as a promising solution with its sparse architecture for effective task decoupling.
Intuition-MoR1E achieves superior efficiency and 2.15% overall accuracy improvement across 14 public datasets.
arXiv Detail & Related papers (2024-04-13T12:14:58Z) - Controllable Preference Optimization: Toward Controllable
Multi-Objective Alignment [107.63756895544842]
Alignment in artificial intelligence pursues consistency between model responses and human preferences as well as values.
Existing alignment techniques are mostly unidirectional, leading to suboptimal trade-offs and poor flexibility over various objectives.
We introduce controllable preference optimization (CPO), which explicitly specifies preference scores for different objectives.
arXiv Detail & Related papers (2024-02-29T12:12:30Z) - Interactive Hyperparameter Optimization in Multi-Objective Problems via
Preference Learning [65.51668094117802]
We propose a human-centered interactive HPO approach tailored towards multi-objective machine learning (ML)
Instead of relying on the user guessing the most suitable indicator for their needs, our approach automatically learns an appropriate indicator.
arXiv Detail & Related papers (2023-09-07T09:22:05Z) - BOtied: Multi-objective Bayesian optimization with tied multivariate ranks [33.414682601242006]
In this paper, we show a natural connection between non-dominated solutions and the extreme quantile of the joint cumulative distribution function.
Motivated by this link, we propose the Pareto-compliant CDF indicator and the associated acquisition function, BOtied.
Our experiments on a variety of synthetic and real-world problems demonstrate that BOtied outperforms state-of-the-art MOBO acquisition functions.
arXiv Detail & Related papers (2023-06-01T04:50:06Z) - Multi-Objective GFlowNets [59.16787189214784]
We study the problem of generating diverse candidates in the context of Multi-Objective Optimization.
In many applications of machine learning such as drug discovery and material design, the goal is to generate candidates which simultaneously optimize a set of potentially conflicting objectives.
We propose Multi-Objective GFlowNets (MOGFNs), a novel method for generating diverse optimal solutions, based on GFlowNets.
arXiv Detail & Related papers (2022-10-23T16:15:36Z) - Multi-Fidelity Multi-Objective Bayesian Optimization: An Output Space
Entropy Search Approach [44.25245545568633]
We study the novel problem of blackbox optimization of multiple objectives via multi-fidelity function evaluations.
Our experiments on several synthetic and real-world benchmark problems show that MF-OSEMO, with both approximations, significantly improves over the state-of-the-art single-fidelity algorithms.
arXiv Detail & Related papers (2020-11-02T06:59:04Z) - Empirical Study on the Benefits of Multiobjectivization for Solving
Single-Objective Problems [0.0]
Local optima are often preventing algorithms from making progress and thus pose a severe threat.
With the use of a sophisticated visualization technique based on the multi-objective gradients, the properties of the arising multi-objective landscapes are illustrated and examined.
We will empirically show that the multi-objective COCO MOGSA is able to exploit these properties to overcome local traps.
arXiv Detail & Related papers (2020-06-25T14:04:37Z) - MOPT: Multi-Object Panoptic Tracking [33.77171216778909]
We introduce a novel perception task denoted as multi-object panoptic tracking (MOPT)
MOPT allows for exploiting pixel-level semantic information of 'thing' and'stuff' classes, temporal coherence, and pixel-level associations over time.
We present extensive quantitative and qualitative evaluations of both vision-based and LiDAR-based MOPT that demonstrate encouraging results.
arXiv Detail & Related papers (2020-04-17T11:45:28Z) - A Unified Object Motion and Affinity Model for Online Multi-Object
Tracking [127.5229859255719]
We propose a novel MOT framework that unifies object motion and affinity model into a single network, named UMA.
UMA integrates single object tracking and metric learning into a unified triplet network by means of multi-task learning.
We equip our model with a task-specific attention module, which is used to boost task-aware feature learning.
arXiv Detail & Related papers (2020-03-25T09:36:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.