Auto-COP: Adaptation Generation in Context-Oriented Programming using
Reinforcement Learning Options
- URL: http://arxiv.org/abs/2103.06757v2
- Date: Thu, 3 Aug 2023 13:47:39 GMT
- Title: Auto-COP: Adaptation Generation in Context-Oriented Programming using
Reinforcement Learning Options
- Authors: Nicol\'as Cardozo and Ivana Dusparic
- Abstract summary: We propose Auto-COP, a new technique to enable generation of adaptations at run time.
We present two case studies exhibiting different system characteristics and application domains.
We confirm that the generated adaptations exhibit correct system behavior measured by domain-specific performance metrics.
- Score: 2.984934409689467
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Self-adaptive software systems continuously adapt in response to internal and
external changes in their execution environment, captured as contexts. The COP
paradigm posits a technique for the development of self-adaptive systems,
capturing their main characteristics with specialized programming language
constructs. COP adaptations are specified as independent modules composed in
and out of the base system as contexts are activated and deactivated in
response to sensed circumstances from the surrounding environment. However, the
definition of adaptations, their contexts and associated specialized behavior,
need to be specified at design time. In complex CPS this is intractable due to
new unpredicted operating conditions. We propose Auto-COP, a new technique to
enable generation of adaptations at run time. Auto-COP uses RL options to build
action sequences, based on the previous instances of the system execution.
Options are explored in interaction with the environment, and the most suitable
options for each context are used to generate adaptations exploiting COP. To
validate Auto-COP, we present two case studies exhibiting different system
characteristics and application domains: a driving assistant and a robot
delivery system. We present examples of Auto-COP code generated at run time, to
illustrate the types of circumstances (contexts) requiring adaptation, and the
corresponding generated adaptations for each context. We confirm that the
generated adaptations exhibit correct system behavior measured by
domain-specific performance metrics, while reducing the number of required
execution/actuation steps by a factor of two showing that the adaptations are
regularly selected by the running system as adaptive behavior is more
appropriate than the execution of primitive actions.
Related papers
- CAPE: Context-Adaptive Positional Encoding for Length Extrapolation [60.18239094672938]
Positional encoding plays a crucial role in transformers, significantly impacting model performance and length generalization.
We propose a Context-Adaptive Positional.
CAPE method, which adjusts semantically based on input context and learned priors.
We successfully train the model on sequence length 128 and achieve better performance at evaluation sequence length 8192, compared with other static positional encoding methods.
arXiv Detail & Related papers (2024-05-23T15:51:24Z) - Generalized Preference Optimization: A Unified Approach to Offline Alignment [54.97015778517253]
We propose generalized preference optimization (GPO), a family of offline losses parameterized by a general class of convex functions.
GPO enables a unified view over preference optimization, encompassing existing algorithms such as DPO, IPO and SLiC as special cases.
Our results present new algorithmic toolkits and empirical insights to alignment practitioners.
arXiv Detail & Related papers (2024-02-08T15:33:09Z) - Reducing Large Adaptation Spaces in Self-Adaptive Systems Using Machine
Learning [10.444983001376874]
We present ML2ASR+, short for Machine Learning to Adaptation Space Reduction Plus.
We evaluate ML2ASR+ for two applications with different sizes of adaptation spaces: an Internet-of-Things application and a service-based system.
The results demonstrate that ML2ASR+ can be applied to deal with different types of goals and is able to reduce the adaptation space and hence the time to make adaptation decisions with over 90%, with negligible effect on the realization of the adaptation goals.
arXiv Detail & Related papers (2023-06-02T09:49:33Z) - Condition-Invariant Semantic Segmentation [77.10045325743644]
We implement Condition-Invariant Semantic (CISS) on the current state-of-the-art domain adaptation architecture.
Our method achieves the second-best performance on the normal-to-adverse Cityscapes$to$ACDC benchmark.
CISS is shown to generalize well to domains unseen during training, such as BDD100K-night and ACDC-night.
arXiv Detail & Related papers (2023-05-27T03:05:07Z) - Deep Learning for Effective and Efficient Reduction of Large Adaptation
Spaces in Self-Adaptive Systems [12.341380735802568]
We present 'Deep Learning for Adaptation Space Reduction Plus' -- DLASeR+ in short.
DLASeR+ offers an extendable learning framework for online adaptation space reduction.
It supports three common types of adaptation goals: threshold, optimization, and set-point goals.
Results show that DLASeR+ is effective with a negligible effect on the realization of the adaptation goals.
arXiv Detail & Related papers (2022-04-13T08:51:06Z) - REPTILE: A Proactive Real-Time Deep Reinforcement Learning Self-adaptive
Framework [0.6335848702857039]
A general framework is proposed to support the development of software systems that are able to adapt their behaviour according to the operating environment changes.
The proposed approach, named REPTILE, works in a complete proactive manner and relies on Deep Reinforcement Learning-based agents to react to events.
In our framework, two types of novelties are taken into account: those related to the context/environment and those related to the physical architecture itself.
The framework, predicting those novelties before their occurrence, extracts time-changing models of the environment and uses a suitable Markov Decision Process to deal with the real-time setting.
arXiv Detail & Related papers (2022-03-28T12:38:08Z) - Lifelong Unsupervised Domain Adaptive Person Re-identification with
Coordinated Anti-forgetting and Adaptation [127.6168183074427]
We propose a new task, Lifelong Unsupervised Domain Adaptive (LUDA) person ReID.
This is challenging because it requires the model to continuously adapt to unlabeled data of the target environments.
We design an effective scheme for this task, dubbed CLUDA-ReID, where the anti-forgetting is harmoniously coordinated with the adaptation.
arXiv Detail & Related papers (2021-12-13T13:19:45Z) - Realistic simulation of users for IT systems in cyber ranges [63.20765930558542]
We instrument each machine by means of an external agent to generate user activity.
This agent combines both deterministic and deep learning based methods to adapt to different environment.
We also propose conditional text generation models to facilitate the creation of conversations and documents.
arXiv Detail & Related papers (2021-11-23T10:53:29Z) - Generalize then Adapt: Source-Free Domain Adaptive Semantic Segmentation [78.38321096371106]
Prior arts assume concurrent access to both labeled source and unlabeled target, making them unsuitable for scenarios demanding source-free adaptation.
In this work, we enable source-free DA by partitioning the task into two: a) source-only domain generalization and b) source-free target adaptation.
We introduce a novel conditional prior-enforcing auto-encoder that discourages spatial irregularities, thereby enhancing the pseudo-label quality.
arXiv Detail & Related papers (2021-08-25T14:18:59Z) - Towards Better Adaptive Systems by Combining MAPE, Control Theory, and
Machine Learning [16.998805882711864]
Two established approaches to engineer adaptive systems are architecture-based adaptation that uses a Monitor-Analysis-Planning-Executing loop, and control-based adaptation that relies on principles of control theory (CT) to realize adaptation.
We are concerned with the question of how these approaches are related with one another and whether combining them and supporting them with machine learning can produce better adaptive systems.
We motivate the combined use of different adaptation approaches using a scenario of a cloud-based enterprise system and illustrate the analysis when combining the different approaches.
arXiv Detail & Related papers (2021-03-19T15:00:08Z) - Time Adaptive Reinforcement Learning [2.0305676256390934]
Reinforcement learning (RL) allows to solve complex tasks such as Go often with a stronger performance than humans.
Here we consider the case of adapting RL agents to different time restrictions, such as finishing a task with a given time limit that might change from one task execution to the next.
We introduce two model-free, value-based algorithms: the Independent Gamma-Ensemble and the n-Step Ensemble.
arXiv Detail & Related papers (2020-04-18T11:52:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.