LONViZ: Unboxing the black-box of Configurable Software Systems from a Complex Networks Perspective

• URL: http://arxiv.org/abs/2201.01429v1
• Date: Wed, 5 Jan 2022 03:14:39 GMT
• Title: LONViZ: Unboxing the black-box of Configurable Software Systems from a Complex Networks Perspective
• Authors: Ke Li, Peili Mao, Tao Chen
• Abstract summary: This paper proposes a tool, dubbed LONViZ, to facilitate the exploratory analysis of black-boxconfigured software systems. In experiments, we choose four widely used real-world software systems to develop benchmark platforms under 42 different running environments. We find that LONViZ enables both qualitative and quantitative analysis and disclose various interesting hidden patterns and properties of different software systems.
• Score: 9.770775293243934
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most, if not all, modern software systems are highly configurable to tailor both their functional and non-functional properties to a variety of stakeholders. Due to the black-box nature, it is difficult, if not impossible, to analyze and understand its behavior, such as the interaction between combinations of configuration options with regard to the performance, in particular, which is of great importance to advance the controllability of the underlying software system. This paper proposes a tool, dubbed LONViZ, which is the first of its kind, to facilitate the exploratory analysis of black-box configurable software systems. It starts from a systematic sampling over the configuration space of the underlying system. Then LONViZ seeks to construct a structurally stable LON by synthesizing multiple repeats of sampling results. Finally, exploratory analysis can be conducted on the stable LON from both qualitative and quantitative perspectives. In experiments, we choose four widely used real-world configurable software systems to develop benchmark platforms under 42 different running environments. From our empirical study, we find that LONViZ enables both qualitative and quantitative analysis and disclose various interesting hidden patterns and properties of different software systems.

Related papers

• Prompting for Performance: Exploring LLMs for Configuring Software [10.764205218438759]
  Large language models (LLMs) can assist in performance-oriented software configuration through prompts.<n>We evaluate several LLMs on tasks including identifying relevant options, ranking configurations, and recommending performant configurations across various systems.
  arXiv  Detail & Related papers  (2025-07-13T21:05:01Z)
• BinMetric: A Comprehensive Binary Analysis Benchmark for Large Language Models [50.17907898478795]
  We introduce BinMetric, a benchmark designed to evaluate the performance of large language models on binary analysis tasks.<n>BinMetric comprises 1,000 questions derived from 20 real-world open-source projects across 6 practical binary analysis tasks.<n>Our empirical study on this benchmark investigates the binary analysis capabilities of various state-of-the-art LLMs, revealing their strengths and limitations in this field.
  arXiv  Detail & Related papers  (2025-05-12T08:54:07Z)
• PuzzleBench: A Fully Dynamic Evaluation Framework for Large Multimodal Models on Puzzle Solving [50.50405233978406]
  We propose a fully dynamic multimodal evaluation framework, named Open-ended Visual Puzzle Generation (OVPG) OVPG aims to generate fresh, diverse, and verifiable evaluation data automatically in puzzle-solving tasks. Built upon OVPG, we construct PuzzleBench, a dynamic and scalable benchmark comprising 11,840 VQA samples.
  arXiv  Detail & Related papers  (2025-04-15T05:29:31Z)
• OmniParser V2: Structured-Points-of-Thought for Unified Visual Text Parsing and Its Generality to Multimodal Large Language Models [58.45517851437422]
  Visually-situated text parsing (VsTP) has recently seen notable advancements, driven by the growing demand for automated document understanding. Existing solutions often rely on task-specific architectures and objectives for individual tasks. In this paper, we introduce Omni V2, a universal model that unifies VsTP typical tasks, including text spotting, key information extraction, table recognition, and layout analysis.
  arXiv  Detail & Related papers  (2025-02-22T09:32:01Z)
• Rethinking Performance Analysis for Configurable Software Systems: A Case Study from a Fitness Landscape Perspective [3.845572815195074]
  We advocate a novel perspective to rethink performance analysis -- modeling the configuration space as a structured landscape'' We apply our framework to $86$M benchmarked configurations from $32$ running workloads of $3$ real-world systems.
  arXiv  Detail & Related papers  (2024-12-22T06:51:33Z)
• Code-Survey: An LLM-Driven Methodology for Analyzing Large-Scale Codebases [3.8153349016958074]
  We introduce Code-Survey, the first LLM-driven methodology designed to explore and analyze large-scales. By carefully designing surveys, Code-Survey transforms unstructured data, such as commits, emails, into organized, structured, and analyzable datasets. This enables quantitative analysis of complex software evolution and uncovers valuable insights related to design, implementation, maintenance, reliability, and security.
  arXiv  Detail & Related papers  (2024-09-24T17:08:29Z)
• Diversity-Driven View Subset Selection for Indoor Novel View Synthesis [54.468355408388675]
  We propose a novel subset selection framework that integrates a comprehensive diversity-based measurement with well-designed utility functions.<n>We show that our framework consistently outperforms baseline strategies while using only 5-20% of the data.
  arXiv  Detail & Related papers  (2024-09-11T08:36:49Z)
• SysBench: Can Large Language Models Follow System Messages? [30.701602680394686]
  Large Language Models (LLMs) have become instrumental across various applications, with the customization of these models to specific scenarios becoming increasingly critical. Despite the recognized potential of system messages to optimize AI-driven solutions, there is a notable absence of a benchmark for evaluating how well LLMs follow system messages. We introduce SysBench, a benchmark that systematically analyzes system message following ability in terms of three limitations of existing LLMs.
  arXiv  Detail & Related papers  (2024-08-20T15:33:16Z)
• MMSci: A Dataset for Graduate-Level Multi-Discipline Multimodal Scientific Understanding [59.41495657570397]
  This dataset includes figures such as schematic diagrams, simulated images, macroscopic/microscopic photos, and experimental visualizations. We developed benchmarks for scientific figure captioning and multiple-choice questions, evaluating six proprietary and over ten open-source models. The dataset and benchmarks will be released to support further research.
  arXiv  Detail & Related papers  (2024-07-06T00:40:53Z)
• Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs [56.391404083287235]
  We introduce Cambrian-1, a family of multimodal LLMs (MLLMs) designed with a vision-centric approach. Our study uses LLMs and visual instruction tuning as an interface to evaluate various visual representations. We provide model weights, code, supporting tools, datasets, and detailed instruction-tuning and evaluation recipes.
  arXiv  Detail & Related papers  (2024-06-24T17:59:42Z)
• BEACON: A Bayesian Optimization Strategy for Novelty Search in Expensive Black-Box Systems [1.204357447396532]
  Novelty search (NS) refers to a class of exploration algorithms that automatically uncover diverse system behaviors through simulations or experiments. We propose a sample-efficient NS method inspired by Bayesian optimization principles. We show that BEACON comprehensively outperforms existing baselines by finding substantially larger sets of diverse behaviors under limited sampling budgets.
  arXiv  Detail & Related papers  (2024-06-05T20:23:52Z)
• BEHAVIOR Vision Suite: Customizable Dataset Generation via Simulation [57.40024206484446]
  We introduce the BEHAVIOR Vision Suite (BVS), a set of tools and assets to generate fully customized synthetic data for systematic evaluation of computer vision models. BVS supports a large number of adjustable parameters at the scene level. We showcase three example application scenarios.
  arXiv  Detail & Related papers  (2024-05-15T17:57:56Z)
• Can LLMs Configure Software Tools [0.76146285961466]
  In software engineering, the meticulous configuration of software tools is crucial in ensuring optimal performance within intricate systems. In this study, we embark on an exploration of leveraging Large-Language Models (LLMs) to streamline the software configuration process. Our work presents a novel approach that employs LLMs, such as Chat-GPT, to identify starting conditions and narrow down the search space, improving configuration efficiency.
  arXiv  Detail & Related papers  (2023-12-11T05:03:02Z)
• Multilayer Environment and Toolchain for Holistic NetwOrk Design and Analysis [2.7763199324745966]
  This work analyses in detail the requirements for distributed systems assessment. Our approach emphasizes setting up and assessing a broader spectrum of distributed systems. We demonstrate the framework's capabilities to provide valuable insights across various use cases.
  arXiv  Detail & Related papers  (2023-10-24T21:18:25Z)
• DISPATCH: Design Space Exploration of Cyber-Physical Systems [5.273291582861981]
  Design of cyber-physical systems (CPSs) is a challenging task that involves searching over a large search space of various CPS configurations. We propose DIS, a two-step methodology for sample-efficient search over the design space.
  arXiv  Detail & Related papers  (2020-09-21T23:14:51Z)
• Integrated Benchmarking and Design for Reproducible and Accessible Evaluation of Robotic Agents [61.36681529571202]
  We describe a new concept for reproducible robotics research that integrates development and benchmarking. One of the central components of this setup is the Duckietown Autolab, a standardized setup that is itself relatively low-cost and reproducible. We validate the system by analyzing the repeatability of experiments conducted using the infrastructure and show that there is low variance across different robot hardware and across different remote labs.
  arXiv  Detail & Related papers  (2020-09-09T15:31:29Z)
• BUSTLE: Bottom-Up Program Synthesis Through Learning-Guided Exploration [72.88493072196094]
  We present a new synthesis approach that leverages learning to guide a bottom-up search over programs. In particular, we train a model to prioritize compositions of intermediate values during search conditioned on a set of input-output examples. We show that the combination of learning and bottom-up search is remarkably effective, even with simple supervised learning approaches.
  arXiv  Detail & Related papers  (2020-07-28T17:46:18Z)
• NAS-Count: Counting-by-Density with Neural Architecture Search [74.92941571724525]
  We automate the design of counting models with Neural Architecture Search (NAS) We introduce an end-to-end searched encoder-decoder architecture, Automatic Multi-Scale Network (AMSNet)
  arXiv  Detail & Related papers  (2020-02-29T09:18:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.