RAISE: A Unified Framework for Responsible AI Scoring and Evaluation

• URL: http://arxiv.org/abs/2510.18559v1
• Date: Tue, 21 Oct 2025 12:15:13 GMT
• Title: RAISE: A Unified Framework for Responsible AI Scoring and Evaluation
• Abstract summary: We introduce RAISE (Responsible AI Scoring and Evaluation), a unified framework that quantifies model performance across four dimensions and aggregates them into a single Responsibility Score.<n>Our findings reveal critical trade-offs: the Transformer excelled in explainability and fairness at a very high environmental cost, and the Tabular ResNet offered a balanced profile.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: As AI systems enter high-stakes domains, evaluation must extend beyond predictive accuracy to include explainability, fairness, robustness, and sustainability. We introduce RAISE (Responsible AI Scoring and Evaluation), a unified framework that quantifies model performance across these four dimensions and aggregates them into a single, holistic Responsibility Score. We evaluated three deep learning models: a Multilayer Perceptron (MLP), a Tabular ResNet, and a Feature Tokenizer Transformer, on structured datasets from finance, healthcare, and socioeconomics. Our findings reveal critical trade-offs: the MLP demonstrated strong sustainability and robustness, the Transformer excelled in explainability and fairness at a very high environmental cost, and the Tabular ResNet offered a balanced profile. These results underscore that no single model dominates across all responsibility criteria, highlighting the necessity of multi-dimensional evaluation for responsible model selection. Our implementation is available at: https://github.com/raise-framework/raise.

Related papers

• Multi-Dimensional Model Integrity and Responsibility Assessment Index and Scoring Framework [1.7038535698746309]
  We propose the Model Integrity and Responsibility Assessment Index (MIRAI) to measure explainability, fairness, robustness, privacy, and sustainability.<n>MIRAI combines established metrics through normalized and direction-aligned dimension scores.<n>We show that higher predictive performance does not necessarily imply better overall integrity and responsibility.
  arXiv  Detail & Related papers  (2026-05-14T08:29:36Z)
• SAVOIR: Learning Social Savoir-Faire via Shapley-based Reward Attribution [82.31558282651811]
  Social intelligence, the ability to navigate complex interpersonal interactions, presents a fundamental challenge for language agents.<n>Existing approaches directly employ language models to distribute episode-level rewards, yielding attributions that are retrospective and lack theoretical grounding.<n>We propose SAVOIR, a novel principled framework grounded in cooperative game theory.
  arXiv  Detail & Related papers  (2026-04-21T02:08:25Z)
• CDRRM: Contrast-Driven Rubric Generation for Reliable and Interpretable Reward Modeling [61.75914342638658]
  CDRRM (Contrast-Driven Reward Model) is a framework for high-quality rubric generation and guided preference judgment.<n>This work offers a scalable, interpretable, and data-efficient path for reward modeling.
  arXiv  Detail & Related papers  (2026-03-09T07:15:23Z)
• A Comprehensive Evaluation of LLM Reasoning: From Single-Model to Multi-Agent Paradigms [20.241519889633285]
  Large Language Models (LLMs) are increasingly deployed as reasoning systems, where reasoning paradigms play a critical role.<n>We conduct a comprehensive and unified evaluation of reasoning paradigms, spanning direct single-model generation, CoT-augmented single-model reasoning, and representative MAS.<n>We introduce MIMeBench, a new open-ended benchmark that targets two foundational yet underexplored semantic capabilities.
  arXiv  Detail & Related papers  (2026-01-19T17:23:45Z)
• Q-REAL: Towards Realism and Plausibility Evaluation for AI-Generated Content [71.46991494014382]
  We introduce Q-Real, a novel dataset for fine-grained evaluation of realism and plausibility in AI-generated images.<n>Q-Real consists of 3,088 images generated by popular text-to-image models.<n>We construct Q-Real Bench to evaluate them on two tasks: judgment and grounding with reasoning.
  arXiv  Detail & Related papers  (2025-11-21T02:43:17Z)
• Unlocking the Black Box: A Five-Dimensional Framework for Evaluating Explainable AI in Credit Risk [17.910002088282624]
  This paper intends to fill the gap in the application between "black box" models and explainability frameworks, such as LIME and SHAP.<n>Authors elaborate on the application of these frameworks on different models and demonstrates the more complex models with better prediction powers could be applied.<n>This research demonstrates the feasibility of employing sophisticated, high performing ML models in regulated financial environments.
  arXiv  Detail & Related papers  (2025-11-07T04:49:29Z)
• Continual Action Quality Assessment via Adaptive Manifold-Aligned Graph Regularization [53.82400605816587]
  Action Quality Assessment (AQA) quantifies human actions in videos, supporting applications in sports scoring, rehabilitation, and skill evaluation.<n>A major challenge lies in the non-stationary nature of quality distributions in real-world scenarios.<n>We introduce Continual AQA (CAQA), which equips with Continual Learning capabilities to handle evolving distributions.
  arXiv  Detail & Related papers  (2025-10-08T10:09:47Z)
• Enhancing Credit Risk Prediction: A Meta-Learning Framework Integrating Baseline Models, LASSO, and ECOC for Superior Accuracy [7.254744067646655]
  This research proposes a comprehensive meta-learning framework that synthesizes multiple complementary models.<n>We implement Permutation Feature Importance analysis for each prediction class across all constituent models.<n>Results demonstrate that our framework significantly enhances the accuracy of financial entity classification.
  arXiv  Detail & Related papers  (2025-09-26T14:09:04Z)
• A Comparative Benchmark of Large Language Models for Labelling Wind Turbine Maintenance Logs [0.0]
  This paper presents a framework for benchmarking Large Language Models (LLMs) on the task of classifying complex industrial records.<n>To promote transparency and encourage further research, this framework has been made publicly available as an open-source tool.<n>We quantify a clear performance hierarchy, identifying top models that exhibit high alignment with a benchmark standard and trustworthy, well-calibrated confidence scores.
  arXiv  Detail & Related papers  (2025-09-08T15:48:17Z)
• Sustainability via LLM Right-sizing [21.17523328451591]
  Large language models (LLMs) have become increasingly embedded in organizational.<n>This study offers an empirical answer by evaluating eleven proprietary and open-weight LLMs across ten everyday occupational tasks.<n>Results show that GPT-4o delivers consistently superior performance but at a significantly higher cost and environmental footprint.
  arXiv  Detail & Related papers  (2025-04-17T04:00:40Z)
• M3-AGIQA: Multimodal, Multi-Round, Multi-Aspect AI-Generated Image Quality Assessment [65.3860007085689]
  M3-AGIQA is a comprehensive framework that enables more human-aligned, holistic evaluation of AI-generated images.<n>By aligning model outputs more closely with human judgment, M3-AGIQA delivers robust and interpretable quality scores.
  arXiv  Detail & Related papers  (2025-02-21T03:05:45Z)
• Benchmarking Zero-Shot Robustness of Multimodal Foundation Models: A Pilot Study [61.65123150513683]
  multimodal foundation models, such as CLIP, produce state-of-the-art zero-shot results. It is reported that these models close the robustness gap by matching the performance of supervised models trained on ImageNet. We show that CLIP leads to a significant robustness drop compared to supervised ImageNet models on our benchmark.
  arXiv  Detail & Related papers  (2024-03-15T17:33:49Z)
• QualEval: Qualitative Evaluation for Model Improvement [82.73561470966658]
  We propose QualEval, which augments quantitative scalar metrics with automated qualitative evaluation as a vehicle for model improvement. QualEval uses a powerful LLM reasoner and our novel flexible linear programming solver to generate human-readable insights. We demonstrate that leveraging its insights, for example, improves the absolute performance of the Llama 2 model by up to 15% points relative.
  arXiv  Detail & Related papers  (2023-11-06T00:21:44Z)
• GREAT Score: Global Robustness Evaluation of Adversarial Perturbation using Generative Models [60.48306899271866]
  We present a new framework, called GREAT Score, for global robustness evaluation of adversarial perturbation using generative models. We show high correlation and significantly reduced cost of GREAT Score when compared to the attack-based model ranking on RobustBench. GREAT Score can be used for remote auditing of privacy-sensitive black-box models.
  arXiv  Detail & Related papers  (2023-04-19T14:58:27Z)
• Exploring the Trade-off between Plausibility, Change Intensity and Adversarial Power in Counterfactual Explanations using Multi-objective Optimization [73.89239820192894]
  We argue that automated counterfactual generation should regard several aspects of the produced adversarial instances. We present a novel framework for the generation of counterfactual examples.
  arXiv  Detail & Related papers  (2022-05-20T15:02:53Z)
• A Comprehensive Evaluation Framework for Deep Model Robustness [44.20580847861682]
  Deep neural networks (DNNs) have achieved remarkable performance across a wide area of applications. They are vulnerable to adversarial examples, which motivates the adversarial defense. This paper presents a model evaluation framework containing a comprehensive, rigorous, and coherent set of evaluation metrics.
  arXiv  Detail & Related papers  (2021-01-24T01:04:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.