The Sustainability Assessment Framework Toolkit: A Decade of Modeling Experience

• URL: http://arxiv.org/abs/2405.01391v2
• Date: Sat, 19 Oct 2024 14:45:11 GMT
• Title: The Sustainability Assessment Framework Toolkit: A Decade of Modeling Experience
• Abstract summary: We present the Sustainability Assessment Framework Toolkit -- a set of instruments we developed to support software architects and design decision makers in modeling sustainability as a software quality property. We illustrate the toolkit with examples stemming from various cases. We extract our lessons learned, and our current research and future plans to extend the SAF Toolkit for further architecture modeling and measurement.
• Score: 9.879300829023467
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Software intensive systems play a crucial role in most, if not all, aspects of modern society. As such, both their sustainability and their role in supporting sustainable processes, must be realized by design. To this aim, the architecture of software intensive systems should be designed to support sustainability goals; and measured to understand how effectively they do so. In this paper, we present the Sustainability Assessment Framework (SAF) Toolkit -- a set of instruments we developed to support software architects and design decision makers in modeling sustainability as a software quality property. The SAF Toolkit is the result of our experience gained in over a decade of cases in collaboration with industrial partners. We illustrate the toolkit with examples stemming from various cases. We extract our lessons learned, and our current research and future plans to extend the SAF Toolkit for further architecture modeling and measurement.

Related papers

• No One Left Behind: Cross-Level Analysis for Sustainable Software Engineering [3.0332210076508326]
  We introduce the concept of sustainability antipatterns to capture conditions that systematically produce unsustainable outcomes.<n>To guide future research, we outline concrete directions for identifying, formalizing, detecting, and mitigating sustainability anti-patterns.
  arXiv  Detail & Related papers  (2026-09-03T13:50:39Z)
• A Survey on the Green Development of Large Models: From Resource-Efficient Architectures to Hardware-Software Co-Design [71.32556809242666]
  The rapid expansion of large-scale AI models has led to significant performance breakthroughs across diverse domains.<n>It has also raised critical concerns regarding computational costs, energy consumption, and environmental sustainability.<n>This survey aims to offer a holistic roadmap toward sustainable, scalable, and socially responsible development of large models.
  arXiv  Detail & Related papers  (2026-07-10T04:02:22Z)
• VESTA: Visual Exploration with Statistical Tool Agents [56.76643750765825]
  We introduce VESTA: Visual Exploration with Statistical Tool Agents.<n>We show that VESTA's dynamic tool creation outperforms prior agentic pipelines.<n>We also introduce DAWN, a benchmark targeting distribution fitting and time series.
  arXiv  Detail & Related papers  (2026-05-29T21:57:37Z)
• The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestration [58.61149924275458]
  Tool use enables large language models to access external information, invoke software systems, and act in digital environments beyond what can be solved from model parameters alone.<n>As agent systems evolve, the central problem has shifted from isolated invocation to multi-tool orchestration over long trajectories with intermediate state, execution feedback, changing environments, and practical constraints such as safety, cost, and verifiability.<n>We organize the literature around six core dimensions: inference-time planning and execution, training and trajectory construction, safety and control, efficiency under resource constraints, capability completeness in open environments, and benchmark design and evaluation.
  arXiv  Detail & Related papers  (2026-03-24T07:05:05Z)
• A Benchmark for Language Models in Real-World System Building [56.549267258789904]
  Cross-ISA software package repair is a critical task for ensuring the reliability of software deployment and the stability of modern operating systems.<n>We introduce a new benchmark designed for software package build repair across diverse architectures and languages.<n>We evaluate six state-of-the-art LLMs on the benchmark, and the results show that cross-ISA software package repair remains difficult and requires further advances.
  arXiv  Detail & Related papers  (2026-01-19T10:30:46Z)
• Injecting Sustainability in Software Architecture: A Rapid Review [4.049713100624969]
  Sustainability has evolved from an emerging concern into a fundamental responsibility in software design, development, and operation.<n>Research increasingly explores how sustainability can be systematically integrated into existing software engineering practices.<n>We combine a rapid review of secondary studies with a focus group of practitioners. The review identifies challenges and opportunities in embedding sustainability in software architecture.<n>We derive five tangible takeaways to inform architects working in the field, and to guide our industry partners in the integration of sustainability concerns in architecture practices.
  arXiv  Detail & Related papers  (2025-11-27T10:47:16Z)
• LoCoBench-Agent: An Interactive Benchmark for LLM Agents in Long-Context Software Engineering [90.84806758077536]
  We introduce textbfLoCoBench-Agent, a comprehensive evaluation framework specifically designed to assess large language models (LLMs) agents in realistic, long-context software engineering.<n>Our framework extends LoCoBench's 8,000 scenarios into interactive agent environments, enabling systematic evaluation of multi-turn conversations.<n>Our framework provides agents with 8 specialized tools (file operations, search, code analysis) and evaluates them across context lengths ranging from 10K to 1M tokens.
  arXiv  Detail & Related papers  (2025-11-17T23:57:24Z)
• Uncovering Scientific Software Sustainability through Community Engagement and Software Quality Metrics [0.0]
  This paper explores the sustainability of scientific open-source software (Sci-OSS) projects hosted on GitHub.<n>We map sustainability to repository metrics from the literature and mined data from ten prominent Sci-OSS projects.<n>Our visualization and analysis methods offer researchers, funders, and developers key insights into long-term software sustainability.
  arXiv  Detail & Related papers  (2025-11-11T05:34:27Z)
• A Survey of Vibe Coding with Large Language Models [93.88284590533242]
  "Vibe Coding" is a development methodology where developers validate AI-generated implementations through outcome observation.<n>Despite its transformative potential, the effectiveness of this emergent paradigm remains under-explored.<n>This survey provides the first comprehensive and systematic review of Vibe Coding with large language models.
  arXiv  Detail & Related papers  (2025-10-14T11:26:56Z)
• SEER: Sustainability Enhanced Engineering of Software Requirements [3.8168092489216385]
  We introduce SEER, a framework which addresses sustainability concerns in the early software development phase.<n>The framework operates in three stages: (i) it identifies sustainability requirements relevant to a specific software product from a general taxonomy; (ii) it evaluates how sustainable system requirements are based on the identified SRs; and (iii) it optimize system requirements that fail to satisfy any SR.<n>Results generated using Gemini 2.5 reasoning model demonstrate the effectiveness of the proposed approach in accurately identifying a broad range of sustainability concerns across diverse domains.
  arXiv  Detail & Related papers  (2025-10-10T03:48:30Z)
• Towards a Knowledge Base of Common Sustainability Weaknesses in Green Software Development [9.521952718902973]
  In this paper, we motivate the need for the development of a standard knowledge base of commonly occurring sustainability weaknesses in code.<n>We demonstrate why existing knowledge regarding software weaknesses cannot be re-tagged "as is" to sustainability without significant due diligence.
  arXiv  Detail & Related papers  (2025-06-10T14:03:58Z)
• ToolACE-DEV: Self-Improving Tool Learning via Decomposition and EVolution [77.86222359025011]
  We propose ToolACE-DEV, a self-improving framework for tool learning.<n>First, we decompose the tool-learning objective into sub-tasks that enhance basic tool-making and tool-using abilities.<n>We then introduce a self-evolving paradigm that allows lightweight models to self-improve, reducing reliance on advanced LLMs.
  arXiv  Detail & Related papers  (2025-05-12T12:48:30Z)
• Identifying Critical Dependencies in Large-Scale Continuous Software Engineering [3.4225872357136136]
  Continuous Software Engineering (CSE) is widely adopted in the industry, integrating practices such as Continuous Integration and Continuous Deployment (CI/CD) Coordinating these activities in large-scale product development involves multiple stakeholders, increasing complexity. This study aims to address this complexity by identifying and analyzing critical dependencies in large-scale CSE.
  arXiv  Detail & Related papers  (2025-04-30T08:52:32Z)
• Incubation and Beyond: A Comparative Analysis of ASF Projects Sustainability Impacts on Software Quality [2.7059033823627923]
  Free and Open Source Software (FOSS) communities' sustainability, meaning to remain operational without signs of weakening or interruptions to its development, is fundamental for the resilience and continuity of society's digital infrastructure. This study seeks to understand how the different aspects of FOSS sustainability impact software quality from a life-cycle perspective.
  arXiv  Detail & Related papers  (2025-04-13T07:51:40Z)
• AISafetyLab: A Comprehensive Framework for AI Safety Evaluation and Improvement [73.0700818105842]
  We introduce AISafetyLab, a unified framework and toolkit that integrates representative attack, defense, and evaluation methodologies for AI safety. AISafetyLab features an intuitive interface that enables developers to seamlessly apply various techniques. We conduct empirical studies on Vicuna, analyzing different attack and defense strategies to provide valuable insights into their comparative effectiveness.
  arXiv  Detail & Related papers  (2025-02-24T02:11:52Z)
• Using Sustainability Impact Scores for Software Architecture Evaluation [5.33605239628904]
  We present an improved version of the Sustainability Impact Score (SIS) The SIS facilitates the identification and quantification of trade-offs in terms of their sustainability impact. Our study reveals that technical quality concerns have significant, often unrecognized impacts across sustainability dimensions.
  arXiv  Detail & Related papers  (2025-01-28T15:00:45Z)
• Integrating Sustainability Concerns into Agile Software Development Process [1.959302443472258]
  This research investigates the current state of sustainability consideration within the software engineering industry through a survey. The results highlight a lack of progress in practically integrating sustainability considerations into software development activities. To address this gap, a case study with an industry partner is conducted to demonstrate how sustainability concerns and effects can be integrated into agile software development.
  arXiv  Detail & Related papers  (2024-07-09T11:58:00Z)
• The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resources [100.23208165760114]
  Foundation model development attracts a rapidly expanding body of contributors, scientists, and applications. To help shape responsible development practices, we introduce the Foundation Model Development Cheatsheet.
  arXiv  Detail & Related papers  (2024-06-24T15:55:49Z)
• Bridging Gaps, Building Futures: Advancing Software Developer Diversity and Inclusion Through Future-Oriented Research [50.545824691484796]
  We present insights from SE researchers and practitioners on challenges and solutions regarding diversity and inclusion in SE. We share potential utopian and dystopian visions of the future and provide future research directions and implications for academia and industry.
  arXiv  Detail & Related papers  (2024-04-10T16:18:11Z)
• Charting a Path to Efficient Onboarding: The Role of Software Visualization [49.1574468325115]
  The present study aims to explore the familiarity of managers, leaders, and developers with software visualization tools. This approach incorporated quantitative and qualitative analyses of data collected from practitioners using questionnaires and semi-structured interviews.
  arXiv  Detail & Related papers  (2024-01-17T21:30:45Z)
• SusDevOps: Promoting Sustainability to a First Principle in Software Delivery [0.5439020425819]
  SusDevOps is a framework that situates sustainability-related activities within the software delivery lifecycle. We demonstrate the lifecycle phases and techniques of SusDevOps through the case of a software development startup company.
  arXiv  Detail & Related papers  (2023-12-22T17:15:58Z)
• Architecture Knowledge Representation and Communication Industry Survey [0.0]
  We aim to understand the current practice in architecture knowledge, and to explore where sustainability can be applied to address sustainability in software architecture in the future. We used a survey, which utilized a questionnaire containing 34 questions and collected responses from 45 architects working at a prominent bank in the Netherlands.
  arXiv  Detail & Related papers  (2023-09-20T18:17:16Z)
• An Open Community-Driven Model For Sustainable Research Software: Sustainable Research Software Institute [0.586336038845426]
  The Sustainable Research Software Institute (SRSI) Model promotes sustainable practices in the research software community. This white paper provides an in-depth overview of the SRSI Model, outlining its objectives, services, funding mechanisms, collaborations, and the potential impact it could have on the research software community.
  arXiv  Detail & Related papers  (2023-08-29T01:00:32Z)
• Machine Learning-Enabled Software and System Architecture Frameworks [48.87872564630711]
  The stakeholders with data science and Machine Learning related concerns, such as data scientists and data engineers, are yet to be included in existing architecture frameworks. We surveyed 61 subject matter experts from over 25 organizations in 10 countries.
  arXiv  Detail & Related papers  (2023-08-09T21:54:34Z)
• Tool Learning with Foundation Models [158.8640687353623]
  With the advent of foundation models, AI systems have the potential to be equally adept in tool use as humans. Despite its immense potential, there is still a lack of a comprehensive understanding of key challenges, opportunities, and future endeavors in this field.
  arXiv  Detail & Related papers  (2023-04-17T15:16:10Z)
• Software Architecture for Next-Generation AI Planning Systems [0.0]
  We propose a service-oriented planning architecture to be at the core of the ability to design, develop and use next-generation AI planning systems. We incorporate software design principles and patterns into the architecture to allow for usability, interoperability and reusability of the planning capabilities.
  arXiv  Detail & Related papers  (2021-02-22T13:43:45Z)
• Technology Readiness Levels for Machine Learning Systems [107.56979560568232]
  Development and deployment of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. We have developed a proven systems engineering approach for machine learning development and deployment. Our "Machine Learning Technology Readiness Levels" framework defines a principled process to ensure robust, reliable, and responsible systems.
  arXiv  Detail & Related papers  (2021-01-11T15:54:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.