TRIAGE: Ethical Benchmarking of AI Models Through Mass Casualty Simulations

• URL: http://arxiv.org/abs/2410.18991v2
• Date: Mon, 04 Nov 2024 12:38:38 GMT
• Title: TRIAGE: Ethical Benchmarking of AI Models Through Mass Casualty Simulations
• Authors: Nathalie Maria Kirch, Konstantin Hebenstreit, Matthias Samwald,
• Abstract summary: We present the TRIAGE Benchmark, a novel machine ethics (ME) benchmark that tests LLMs' ability to make ethical decisions during mass casualty incidents. It uses real-world ethical dilemmas with clear solutions designed by medical professionals, offering a more realistic alternative to annotation-based benchmarks.
• Score: 4.294623208722234
• License:
• Abstract: We present the TRIAGE Benchmark, a novel machine ethics (ME) benchmark that tests LLMs' ability to make ethical decisions during mass casualty incidents. It uses real-world ethical dilemmas with clear solutions designed by medical professionals, offering a more realistic alternative to annotation-based benchmarks. TRIAGE incorporates various prompting styles to evaluate model performance across different contexts. Most models consistently outperformed random guessing, suggesting LLMs may support decision-making in triage scenarios. Neutral or factual scenario formulations led to the best performance, unlike other ME benchmarks where ethical reminders improved outcomes. Adversarial prompts reduced performance but not to random guessing levels. Open-source models made more morally serious errors, and general capability overall predicted better performance.

Related papers

• A Comparative Analysis on Ethical Benchmarking in Large Language Models [0.0]
  This work contributes to the field of Machine Ethics (ME) benchmarking, which develops tests to assess whether intelligent systems accurately represent human values and act accordingly. We identify three major issues with current ME benchmarks: limited ecological validity due to unrealistic ethical dilemmas, unstructured question generation without clear inclusion/exclusion criteria, and a lack of scalability due to reliance on human annotations. We introduce two new ME benchmarks: the Triage Benchmark and the Medical Law (MedLaw) Benchmark, both featuring real-world ethical dilemmas from the medical domain.
  arXiv  Detail & Related papers  (2024-10-11T05:05:21Z)
• Fine-Tuning Language Models for Ethical Ambiguity: A Comparative Study of Alignment with Human Responses [1.566834021297545]
  Language models often misinterpret human intentions due to their handling of ambiguity. We show that human and LLM judgments are poorly aligned in morally ambiguous contexts. Our fine-tuning approach, which improves the model's understanding of text distributions in a text-to-text format, effectively enhances performance and alignment.
  arXiv  Detail & Related papers  (2024-10-10T11:24:04Z)
• MR-Ben: A Meta-Reasoning Benchmark for Evaluating System-2 Thinking in LLMs [55.20845457594977]
  Large language models (LLMs) have shown increasing capability in problem-solving and decision-making. We present a process-based benchmark MR-Ben that demands a meta-reasoning skill. Our meta-reasoning paradigm is especially suited for system-2 slow thinking.
  arXiv  Detail & Related papers  (2024-06-20T03:50:23Z)
• Grade Score: Quantifying LLM Performance in Option Selection [0.0]
  "Grade Score" is a novel metric designed to evaluate the consistency and fairness of Large Language Models (LLMs) The Grade Score combines Entropy, which measures order bias, and Mode Frequency, which assesses choice stability. The study explores techniques such as prompt engineering and option sampling strategies to optimize the Grade Score.
  arXiv  Detail & Related papers  (2024-06-17T19:29:39Z)
• OLMES: A Standard for Language Model Evaluations [64.85905119836818]
  We propose OLMES, a practical, open standard for reproducible language model evaluations. We identify and review the varying factors in evaluation practices adopted by the community. OLMES supports meaningful comparisons between smaller base models that require the unnatural "cloze" formulation of multiple-choice questions.
  arXiv  Detail & Related papers  (2024-06-12T17:37:09Z)
• Uncertainty Aware Learning for Language Model Alignment [97.36361196793929]
  We propose uncertainty-aware learning (UAL) to improve the model alignment of different task scenarios. We implement UAL in a simple fashion -- adaptively setting the label smoothing value of training according to the uncertainty of individual samples. Experiments on widely used benchmarks demonstrate that our UAL significantly and consistently outperforms standard supervised fine-tuning.
  arXiv  Detail & Related papers  (2024-06-07T11:37:45Z)
• MoralBench: Moral Evaluation of LLMs [34.43699121838648]
  This paper introduces a novel benchmark designed to measure and compare the moral reasoning capabilities of large language models (LLMs) We present the first comprehensive dataset specifically curated to probe the moral dimensions of LLM outputs. Our methodology involves a multi-faceted approach, combining quantitative analysis with qualitative insights from ethics scholars to ensure a thorough evaluation of model performance.
  arXiv  Detail & Related papers  (2024-06-06T18:15:01Z)
• Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluation [51.99752147380505]
  This paper presents a benchmark self-evolving framework to dynamically evaluate Large Language Models (LLMs) We utilize a multi-agent system to manipulate the context or question of original instances, reframing new evolving instances with high confidence. Our framework widens performance discrepancies both between different models and within the same model across various tasks.
  arXiv  Detail & Related papers  (2024-02-18T03:40:06Z)
• Don't Make Your LLM an Evaluation Benchmark Cheater [142.24553056600627]
  Large language models(LLMs) have greatly advanced the frontiers of artificial intelligence, attaining remarkable improvement in model capacity. To assess the model performance, a typical approach is to construct evaluation benchmarks for measuring the ability level of LLMs. We discuss the potential risk and impact of inappropriately using evaluation benchmarks and misleadingly interpreting the evaluation results.
  arXiv  Detail & Related papers  (2023-11-03T14:59:54Z)
• When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learning [62.00672284480755]
  This paper aims to recover the structure of rewards and environment dynamics that underlie observed actions in a fixed, finite set of demonstrations from an expert agent. Accurate models of expertise in executing a task has applications in safety-sensitive applications such as clinical decision making and autonomous driving.
  arXiv  Detail & Related papers  (2023-02-15T04:14:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.