Towards Standards-Compliant Assistive Technology Product Specifications via LLMs
- URL: http://arxiv.org/abs/2404.03122v1
- Date: Thu, 4 Apr 2024 00:10:39 GMT
- Title: Towards Standards-Compliant Assistive Technology Product Specifications via LLMs
- Authors: Chetan Arora, John Grundy, Louise Puli, Natasha Layton,
- Abstract summary: We introduce CompliAT, a pioneering framework designed to streamline the compliance process of AT product specifications.
CompliAT addresses three critical tasks: checking consistency terminology, classifying products according to standards, and tracing key product specifications to standard requirements.
We propose a novel approach for product classification, leveraging a retrieval-augmented generation model to accurately categorize AT products aligning to international standards.
- Score: 7.30389619012625
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In the rapidly evolving field of assistive technology (AT), ensuring that products meet national and international standards is essential for user safety, efficacy, and accessibility. In this vision paper, we introduce CompliAT, a pioneering framework designed to streamline the compliance process of AT product specifications with these standards through the innovative use of Large Language Models (LLMs). CompliAT addresses three critical tasks: checking terminology consistency, classifying products according to standards, and tracing key product specifications to standard requirements. We tackle the challenge of terminology consistency to ensure that the language used in product specifications aligns with relevant standards, reducing misunderstandings and non-compliance risks. We propose a novel approach for product classification, leveraging a retrieval-augmented generation model to accurately categorize AT products aligning to international standards, despite the sparse availability of training data. Finally, CompliAT implements a traceability and compliance mechanism from key product specifications to standard requirements, ensuring all aspects of an AT product are thoroughly vetted against the corresponding standards. By semi-automating these processes, CompliAT aims to significantly reduce the time and effort required for AT product standards compliance and uphold quality and safety standards. We outline our planned implementation and evaluation plan for CompliAT.
Related papers
- AuditNet: A Conversational AI-based Security Assistant [DEMO] [10.941722434218262]
We propose a versatile conversational AI assistant framework designed to facilitate compliance checking on the go.
Our framework automates the review, indexing, and retrieval of relevant, context-aware information.
This AI assistant not only reduces the manual effort involved in compliance checks but also enhances accuracy and efficiency.
arXiv Detail & Related papers (2024-07-19T08:33:07Z) - ConU: Conformal Uncertainty in Large Language Models with Correctness Coverage Guarantees [68.33498595506941]
Uncertainty quantification in natural language generation (NLG) tasks remains an open challenge.
This study investigates adapting conformal prediction (CP), which can convert any measure of uncertainty into rigorous theoretical guarantees.
We propose a sampling-based uncertainty measure leveraging self-consistency and develop a conformal uncertainty criterion.
arXiv Detail & Related papers (2024-06-29T17:33:07Z) - Navigating the EU AI Act: A Methodological Approach to Compliance for Safety-critical Products [0.0]
This paper presents a methodology for interpreting the EU AI Act requirements for high-risk AI systems.
We first propose an extended product quality model for AI systems, incorporating attributes relevant to the Act not covered by current quality models.
We then propose a contract-based approach to derive technical requirements at the stakeholder level.
arXiv Detail & Related papers (2024-03-25T14:32:18Z) - Towards an Enforceable GDPR Specification [49.1574468325115]
Privacy by Design (PbD) is prescribed by modern privacy regulations such as the EU's.
One emerging technique to realize PbD is enforcement (RE)
We present a set of requirements and an iterative methodology for creating formal specifications of legal provisions.
arXiv Detail & Related papers (2024-02-27T09:38:51Z) - On the relationship between Benchmarking, Standards and Certification in
Robotics and AI [1.1421942894219899]
Benchmarking, standards and certification are closely related processes.
Benchmarking, standards and certification are not only useful but vital to the broader practice of Responsible Innovation.
arXiv Detail & Related papers (2023-09-21T14:59:36Z) - A General Verification Framework for Dynamical and Control Models via Certificate Synthesis [54.959571890098786]
We provide a framework to encode system specifications and define corresponding certificates.
We present an automated approach to formally synthesise controllers and certificates.
Our approach contributes to the broad field of safe learning for control, exploiting the flexibility of neural networks.
arXiv Detail & Related papers (2023-09-12T09:37:26Z) - An Ensemble-based approach for assigning text to correct Harmonized
system code [2.365702128814616]
Harmonized System (HS) is the most standardized numerical method of classifying traded products among industry classification systems.
A hierarchical ensemble model comprising of Bert- transformer, NER, distance-based approaches, and knowledge-graphs have been developed to address scalability, coverage, ability to capture nuances, automation and auditing requirements when classifying unknown text-descriptions as per HS method.
arXiv Detail & Related papers (2022-11-08T15:32:36Z) - NormSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations
On-the-Fly [61.77957329364812]
We introduce a framework for addressing the novel task of conversation-grounded multi-lingual, multi-cultural norm discovery.
NormSAGE elicits knowledge about norms through directed questions representing the norm discovery task and conversation context.
It further addresses the risk of language model hallucination with a self-verification mechanism ensuring that the norms discovered are correct.
arXiv Detail & Related papers (2022-10-16T18:30:05Z) - The Complexity of Data-Driven Norm Synthesis and Revision [19.235739945526877]
We consider the problem of synthesising a norm from traces of agent behaviour.
Each trace is labelled with whether the behaviour satisfies the system objective.
We show that the norm synthesis problem is NP-complete.
arXiv Detail & Related papers (2021-12-05T17:15:00Z) - The GEM Benchmark: Natural Language Generation, its Evaluation and
Metrics [66.96150429230035]
We introduce GEM, a living benchmark for natural language Generation (NLG), its Evaluation, and Metrics.
Regular updates to the benchmark will help NLG research become more multilingual and evolve the challenge alongside models.
arXiv Detail & Related papers (2021-02-02T18:42:05Z) - Technology Readiness Levels for AI & ML [79.22051549519989]
Development of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end.
Engineering systems follow well-defined processes and testing standards to streamline development for high-quality, reliable results.
We propose a proven systems engineering approach for machine learning development and deployment.
arXiv Detail & Related papers (2020-06-21T17:14:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.