Assessing Trustworthiness of Autonomous Systems
- URL: http://arxiv.org/abs/2305.03411v2
- Date: Thu, 11 May 2023 10:14:05 GMT
- Title: Assessing Trustworthiness of Autonomous Systems
- Authors: Gregory Chance and Dhaminda B. Abeywickrama and Beckett LeClair and
Owen Kerr and Kerstin Eder
- Abstract summary: As Autonomous Systems (AS) become more ubiquitous in society, more responsible for our safety and our interaction with them more frequent, it is essential that they are trustworthy.
Assessing the trustworthiness of AS is a mandatory challenge for the verification and development community.
This will require appropriate standards and suitable metrics that may serve to objectively and comparatively judge trustworthiness of AS across the broad range of current and future applications.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: As Autonomous Systems (AS) become more ubiquitous in society, more
responsible for our safety and our interaction with them more frequent, it is
essential that they are trustworthy. Assessing the trustworthiness of AS is a
mandatory challenge for the verification and development community. This will
require appropriate standards and suitable metrics that may serve to
objectively and comparatively judge trustworthiness of AS across the broad
range of current and future applications. The meta-expression `trustworthiness'
is examined in the context of AS capturing the relevant qualities that comprise
this term in the literature. Recent developments in standards and frameworks
that support assurance of autonomous systems are reviewed. A list of key
challenges are identified for the community and we present an outline of a
process that can be used as a trustworthiness assessment framework for AS.
Related papers
- Cross-Modality Safety Alignment [73.8765529028288]
We introduce a novel safety alignment challenge called Safe Inputs but Unsafe Output (SIUO) to evaluate cross-modality safety alignment.
To empirically investigate this problem, we developed the SIUO, a cross-modality benchmark encompassing 9 critical safety domains, such as self-harm, illegal activities, and privacy violations.
Our findings reveal substantial safety vulnerabilities in both closed- and open-source LVLMs, underscoring the inadequacy of current models to reliably interpret and respond to complex, real-world scenarios.
arXiv Detail & Related papers (2024-06-21T16:14:15Z) - When to Trust LLMs: Aligning Confidence with Response Quality [49.371218210305656]
We propose CONfidence-Quality-ORDer-preserving alignment approach (CONQORD)
It integrates quality reward and order-preserving alignment reward functions.
Experiments demonstrate that CONQORD significantly improves the alignment performance between confidence and response accuracy.
arXiv Detail & Related papers (2024-04-26T09:42:46Z) - On Specifying for Trustworthiness [39.845582350253515]
We look across a range of AS domains with consideration of the resilience, trust, functionality, verifiability, security, and governance and regulation of AS.
We highlight the intellectual challenges that are involved with specifying for trustworthiness in AS that cut across domains and are exacerbated by the inherent uncertainty involved with the environments in which AS need to operate.
arXiv Detail & Related papers (2022-06-22T23:37:18Z) - Towards a multi-stakeholder value-based assessment framework for
algorithmic systems [76.79703106646967]
We develop a value-based assessment framework that visualizes closeness and tensions between values.
We give guidelines on how to operationalize them, while opening up the evaluation and deliberation process to a wide range of stakeholders.
arXiv Detail & Related papers (2022-05-09T19:28:32Z) - Designing for Responsible Trust in AI Systems: A Communication
Perspective [56.80107647520364]
We draw from communication theories and literature on trust in technologies to develop a conceptual model called MATCH.
We highlight transparency and interaction as AI systems' affordances that present a wide range of trustworthiness cues to users.
We propose a checklist of requirements to help technology creators identify appropriate cues to use.
arXiv Detail & Related papers (2022-04-29T00:14:33Z) - Defining Security Requirements with the Common Criteria: Applications,
Adoptions, and Challenges [17.700647389830774]
The adoption of ICT products with security properties depends on consumers' confidence and markets' trust in the security functionalities.
Common Criteria for Information Technology Security Evaluation (often referred to as Common Criteria or CC) is an international standard for cyber security certification.
Best practices on developing Protection Profiles, recommendations, and future directions for trusted cybersecurity advancement are presented.
arXiv Detail & Related papers (2022-01-19T05:05:33Z) - Reliability Testing for Natural Language Processing Systems [14.393308846231083]
We argue for the need for reliability testing and contextualize it among existing work on improving accountability.
We show how adversarial attacks can be reframed for this goal, via a framework for developing reliability tests.
arXiv Detail & Related papers (2021-05-06T11:24:58Z) - How Trustworthy are Performance Evaluations for Basic Vision Tasks? [46.0590176230731]
This paper examines performance evaluation criteria for basic vision tasks involving sets of objects namely, object detection, instance-level segmentation and multi-object tracking.
The rankings of algorithms by an existing criterion can fluctuate with different choices of parameters, making their evaluations unreliable.
This work suggests a notion of trustworthiness for performance criteria, which requires (i) robustness to parameters for reliability, (ii) contextual meaningfulness in sanity tests, and (iii) consistency with mathematical requirements such as the metric properties.
arXiv Detail & Related papers (2020-08-08T14:21:15Z) - Quantifying Assurance in Learning-enabled Systems [3.0938904602244355]
Dependability assurance of systems embedding machine learning components is a key step for their use in safety-critical applications.
This paper develops a quantitative notion of assurance that an LES is dependable, as a core component of its assurance case.
We illustrate the utility of assurance measures by application to a real world autonomous aviation system.
arXiv Detail & Related papers (2020-06-18T08:11:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.