Utilizing LLMs to Investigate the Disputed Role of Evidence in Electronic Cigarette Health Policy Formation in Australia and the UK

• URL: http://arxiv.org/abs/2505.06782v1
• Date: Sat, 10 May 2025 23:40:28 GMT
• Title: Utilizing LLMs to Investigate the Disputed Role of Evidence in Electronic Cigarette Health Policy Formation in Australia and the UK
• Authors: Damian Curran, Brian Chapman, Mike Conway,
• Abstract summary: Australia and the UK have developed contrasting approaches to the regulation of electronic cigarettes.<n>We developed and evaluated a Large Language Model-based sentence classifier to perform automated analyses of electronic cigarette-related policy documents.
• Score: 0.7646713951724013
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Australia and the UK have developed contrasting approaches to the regulation of electronic cigarettes, with - broadly speaking - Australia adopting a relatively restrictive approach and the UK adopting a more permissive approach. Notably, these divergent policies were developed from the same broad evidence base. In this paper, to investigate differences in how the two jurisdictions manage and present evidence, we developed and evaluated a Large Language Model-based sentence classifier to perform automated analyses of electronic cigarette-related policy documents drawn from official Australian and UK legislative processes (109 documents in total). Specifically, we utilized GPT-4 to automatically classify sentences based on whether they contained claims that e-cigarettes were broadly helpful or harmful for public health. Our LLM-based classifier achieved an F-score of 0.9. Further, when applying the classifier to our entire sentence-level corpus, we found that Australian legislative documents show a much higher proportion of harmful statements, and a lower proportion of helpful statements compared to the expected values, with the opposite holding for the UK. In conclusion, this work utilized an LLM-based approach to provide evidence to support the contention that - drawing on the same evidence base - Australian ENDS-related policy documents emphasize the harms associated with ENDS products and UK policy documents emphasize the benefits. Further, our approach provides a starting point for using LLM-based methods to investigate the complex relationship between evidence and health policy formation.

Related papers

• Computational Identification of Regulatory Statements in EU Legislation [0.0]
  A computational method is valuable for scaling the identification of such statements from a growing body of EU legislation.<n>We provide a specific definition for what constitutes a regulatory statement based on the institutional grammar tool.
  arXiv  Detail & Related papers  (2025-05-01T12:11:32Z)
• Machine Learning-Driven Convergence Analysis in Multijurisdictional Compliance Using BERT and K-Means Clustering [0.0]
  This study compares various regulations to identify areas where they overlap or diverge.<n>International companies can learn valuable lessons from this report.<n>The study's objective is to "bridge the gap between legal knowledge and technical expertise"
  arXiv  Detail & Related papers  (2025-01-23T22:11:18Z)
• Knowledge-Augmented Reasoning for EUAIA Compliance and Adversarial Robustness of LLMs [1.368472250332885]
  The EU AI Act (EUAIA) introduces requirements for AI systems which intersect with the processes required to establish adversarial robustness. This paper presents a functional architecture that focuses on bridging the two properties. We aim to support developers and auditors with a reasoning layer based on knowledge augmentation.
  arXiv  Detail & Related papers  (2024-10-04T18:23:14Z)
• InternLM-Law: An Open Source Chinese Legal Large Language Model [72.2589401309848]
  InternLM-Law is a specialized LLM tailored for addressing diverse legal queries related to Chinese laws. We meticulously construct a dataset in the Chinese legal domain, encompassing over 1 million queries. InternLM-Law achieves the highest average performance on LawBench, outperforming state-of-the-art models, including GPT-4, on 13 out of 20 subtasks.
  arXiv  Detail & Related papers  (2024-06-21T06:19:03Z)
• Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspective [66.34066553400108]
  We conduct a rigorous evaluation of large language models' implicit bias towards certain demographics.<n>Inspired by psychometric principles, we propose three attack approaches, i.e., Disguise, Deception, and Teaching.<n>Our methods can elicit LLMs' inner bias more effectively than competitive baselines.
  arXiv  Detail & Related papers  (2024-06-20T06:42:08Z)
• Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis [86.49858739347412]
  Large Language Models (LLMs) have sparked intense debate regarding the prevalence of bias in these models and its mitigation. We propose a prompt-based method for the extraction of confounding and mediating attributes which contribute to the decision process. We find that the observed disparate treatment can at least in part be attributed to confounding and mitigating attributes and model misalignment.
  arXiv  Detail & Related papers  (2023-11-15T00:02:25Z)
• LM-Polygraph: Uncertainty Estimation for Language Models [71.21409522341482]
  Uncertainty estimation (UE) methods are one path to safer, more responsible, and more effective use of large language models (LLMs) We introduce LM-Polygraph, a framework with implementations of a battery of state-of-the-art UE methods for LLMs in text generation tasks, with unified program interfaces in Python. It introduces an extendable benchmark for consistent evaluation of UE techniques by researchers, and a demo web application that enriches the standard chat dialog with confidence scores.
  arXiv  Detail & Related papers  (2023-11-13T15:08:59Z)
• A Hierarchical Neural Framework for Classification and its Explanation in Large Unstructured Legal Documents [0.5812284760539713]
  We define this problem as "scarce annotated legal documents" We propose a deep-learning-based classification framework which we call MESc. We also propose an explanation extraction algorithm named ORSE.
  arXiv  Detail & Related papers  (2023-09-19T12:18:28Z)
• Leveraging Large Language Models for Topic Classification in the Domain of Public Affairs [65.9077733300329]
  Large Language Models (LLMs) have the potential to greatly enhance the analysis of public affairs documents. LLMs can be of great use to process domain-specific documents, such as those in the domain of public affairs.
  arXiv  Detail & Related papers  (2023-06-05T13:35:01Z)
• Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset [46.156169284961045]
  We offer an approach to filtering grounded in law, which has directly addressed the tradeoffs in filtering material. First, we gather and make available the Pile of Law, a 256GB dataset of open-source English-language legal and administrative data. Second, we distill the legal norms that governments have developed to constrain the inclusion of toxic or private content into actionable lessons. Third, we show how the Pile of Law offers researchers the opportunity to learn such filtering rules directly from the data.
  arXiv  Detail & Related papers  (2022-07-01T06:25:15Z)
• Eider: Evidence-enhanced Document-level Relation Extraction [56.71004595444816]
  Document-level relation extraction (DocRE) aims at extracting semantic relations among entity pairs in a document. We propose a three-stage evidence-enhanced DocRE framework consisting of joint relation and evidence extraction, evidence-centered relation extraction (RE), and fusion of extraction results.
  arXiv  Detail & Related papers  (2021-06-16T09:43:16Z)
• Towards Automatic Comparison of Data Privacy Documents: A Preliminary Experiment on GDPR-like Laws [1.3537117504260623]
  General Data Protection Regulation (NLP) becomes standard law for protection in many countries. 12 countries adopt their similarities-like regulations, but evaluating differences is time-consuming and needs manual effort from legal experts. In this paper, we investigate a simple natural language processing (NLP) approach to tackle the problem.
  arXiv  Detail & Related papers  (2021-05-21T03:59:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.