Detecting Performance-Relevant Changes in Configurable Software Systems

• URL: http://arxiv.org/abs/2511.17271v1
• Date: Fri, 21 Nov 2025 14:20:08 GMT
• Title: Detecting Performance-Relevant Changes in Configurable Software Systems
• Abstract summary: ConfFLARE estimates whether a change potentially impacts performance by identifying data-flow interactions with performance-relevant code.<n>It correctly detects performance regressions in almost all cases and identifies relevant features in all but two cases.<n>It reduces the number of configurations to be tested on average by $79%$ for synthetic and by $70%$ for real-world regression scenarios saving hours of performance testing time.
• Score: 8.036869046810223
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Performance is a volatile property of a software system and frequent performance profiling is required to keep the knowledge about a software system's performance behavior up to date. Repeating all performance measurements after every revision is a cost-intensive task, especially in the presence of configurability, where one has to measure multiple configurations to obtain a comprehensive picture. Configuration sampling is a common approach to control the measurement cost. However, it cannot guarantee completeness and might miss performance regressions, especially if they only affect few configurations. As an alternative to solve the cost reduction problem, we present ConfFLARE: ConfFLARE estimates whether a change potentially impacts performance by identifying data-flow interactions with performance-relevant code and extracts which software features participate in such interactions. Based on these features, we can select a subset of relevant configurations to focus performance profiling efforts on. In a study conducted on both, synthetic and real-world software systems, ConfFLARE correctly detects performance regressions in almost all cases and identifies relevant features in all but two cases, reducing the number of configurations to be tested on average by $79\%$ for synthetic and by $70\%$ for real-world regression scenarios saving hours of performance testing time.

Related papers

• Beyond Isolation: Unlocking Reinforcement Learning Component Synergy for Sample-Efficient Continuous Control [59.15149762610843]
  Reinforcement learning systems are significantly more complex than other machine learning paradigms.<n>Despite advances in individual algorithmic components, their functional interdependencies remain underexplored.<n>We propose ROSER, an RL framework that coordinates three critical dimensions: Model-based Representation, Optimization Stability, and Experience Replay.
  arXiv  Detail & Related papers  (2026-08-07T10:38:31Z)
• ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox [61.862814740220806]
  $textbfComplexMCP$ is a benchmark designed to evaluate agents in rigorous conditions.<n>Built on the Model Context Protocol (MCP), $textbfComplexMCP$ provides over 300 meticulously tested tools.
  arXiv  Detail & Related papers  (2026-05-11T16:20:51Z)
• On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows [71.92083784393418]
  Agentic AI (systems that autonomously plan and act) are becoming widespread, yet their task success rate on complex tasks remains low.<n>Inference-time alignment relies on three components: sampling, evaluation, and feedback.<n>We introduce Iterative Agent Decoding (IAD), a procedure that repeatedly inserts feedback extracted from different forms of critiques.
  arXiv  Detail & Related papers  (2025-04-02T17:40:47Z)
• Tracing Optimization for Performance Modeling and Regression Detection [15.99435412859094]
  A performance model analytically describes the relationship between the performance of a system and its runtime activities. We propose statistical approaches to reduce tracing overhead by identifying and excluding performance-insensitive code regions. Our approach is fully automated, making it ready to be used in production environments with minimal human effort.
  arXiv  Detail & Related papers  (2024-11-26T16:11:55Z)
• Early Detection of Performance Regressions by Bridging Local Performance Data and Architectural Models [12.581051275141537]
  During software development, developers often make numerous modifications to the software to address existing issues or implement new features. To ensure that the performance of new software releases does not degrade, existing practices rely on system-level performance testing. We propose a novel approach to early detection of performance regressions by bridging the local performance data generated by component-level testing and the system-level architectural models.
  arXiv  Detail & Related papers  (2024-08-15T13:33:20Z)
• Restore Anything Model via Efficient Degradation Adaptation [129.38475243424563]
  RAM takes a unified path that leverages inherent similarities across various degradations to enable efficient and comprehensive restoration.<n> RAM's SOTA performance confirms RAM's SOTA performance, reducing model complexity by approximately 82% in trainable parameters and 85% in FLOPs.
  arXiv  Detail & Related papers  (2024-07-18T10:26:53Z)
• FamiCom: Further Demystifying Prompts for Language Models with Task-Agnostic Performance Estimation [73.454943870226]
  Language models have shown impressive in-context-learning capabilities. We propose a measure called FamiCom, providing a more comprehensive measure for task-agnostic performance estimation.
  arXiv  Detail & Related papers  (2024-06-17T06:14:55Z)
• Controllable Prompt Tuning For Balancing Group Distributional Robustness [53.336515056479705]
  We introduce an optimization scheme to achieve good performance across groups and find a good solution for all without severely sacrificing performance on any of them. We propose Controllable Prompt Tuning (CPT), which couples our approach with prompt-tuning techniques. On spurious correlation benchmarks, our procedures achieve state-of-the-art results across both transformer and non-transformer architectures, as well as unimodal and multimodal data.
  arXiv  Detail & Related papers  (2024-03-05T06:23:55Z)
• Large Language Models are Miscalibrated In-Context Learners [22.30783674111999]
  In this work, we deliver an in-depth analysis of the behavior across different choices of learning methods.<n>We observe that the miscalibration problem exists across all learning methods in low-resource setups.<n>We find that self-ensembling with max probability produces robust and calibrated predictions.
  arXiv  Detail & Related papers  (2023-12-21T11:55:10Z)
• Analyzing the Influence of Processor Speed and Clock Speed on Remaining Useful Life Estimation of Software Systems [0.9831489366502301]
  This research extends the analysis to assess how changes in environmental attributes, such as operating system and clock speed, affect RUL estimation in software. Findings are rigorously validated using real performance data from controlled test beds and compared with predictive model-generated data. This exploration yields actionable knowledge for software maintenance and optimization strategies.
  arXiv  Detail & Related papers  (2023-09-22T04:46:34Z)
• Identifying Performance Issues in Cloud Service Systems Based on Relational-Temporal Features [11.83269525626691]
  Cloud systems are susceptible to performance issues, which may cause service-level agreement violations and financial losses. We propose a learning-based approach that leverages both the relational and temporal features of metrics to identify performance issues.
  arXiv  Detail & Related papers  (2023-07-20T13:41:26Z)
• FuzzyFlow: Leveraging Dataflow To Find and Squash Program Optimization Bugs [92.47146416628965]
  FuzzyFlow is a fault localization and test case extraction framework designed to test program optimizations. We leverage dataflow program representations to capture a fully reproducible system state and area-of-effect for optimizations. To reduce testing time, we design an algorithm for minimizing test inputs, trading off memory for recomputation.
  arXiv  Detail & Related papers  (2023-06-28T13:00:17Z)
• Transformer-based Context Condensation for Boosting Feature Pyramids in Object Detection [77.50110439560152]
  Current object detectors typically have a feature pyramid (FP) module for multi-level feature fusion (MFF) We propose a novel and efficient context modeling mechanism that can help existing FPs deliver better MFF results. In particular, we introduce a novel insight that comprehensive contexts can be decomposed and condensed into two types of representations for higher efficiency.
  arXiv  Detail & Related papers  (2022-07-14T01:45:03Z)
• Evolving Pareto-Optimal Actor-Critic Algorithms for Generalizability and Stability [67.8426046908398]
  Generalizability and stability are two key objectives for operating reinforcement learning (RL) agents in the real world. This paper presents MetaPG, an evolutionary method for automated design of actor-critic loss functions.
  arXiv  Detail & Related papers  (2022-04-08T20:46:16Z)
• Cost-Effective Federated Learning Design [37.16466118235272]
  Federated learning (FL) is a distributed learning paradigm that enables a large number of devices to collaboratively learn a model without sharing their raw data. Despite its efficiency and effectiveness, the iterative on-device learning process incurs a considerable cost in terms of learning time and energy consumption. We analyze how to design adaptive FL that optimally chooses essential control variables to minimize the total cost while ensuring convergence.
  arXiv  Detail & Related papers  (2020-12-15T14:45:11Z)
• Dynamic Federated Learning [57.14673504239551]
  Federated learning has emerged as an umbrella term for centralized coordination strategies in multi-agent environments. We consider a federated learning model where at every iteration, a random subset of available agents perform local updates based on their data. Under a non-stationary random walk model on the true minimizer for the aggregate optimization problem, we establish that the performance of the architecture is determined by three factors, namely, the data variability at each agent, the model variability across all agents, and a tracking term that is inversely proportional to the learning rate of the algorithm.
  arXiv  Detail & Related papers  (2020-02-20T15:00:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.