Are Models Trained on Indian Legal Data Fair?

• URL: http://arxiv.org/abs/2303.07247v3
• Date: Tue, 14 May 2024 08:44:37 GMT
• Title: Are Models Trained on Indian Legal Data Fair?
• Authors: Sahil Girhepuje, Anmol Goel, Gokul S Krishnan, Shreya Goyal, Satyendra Pandey, Ponnurangam Kumaraguru, Balaraman Ravindran,
• Abstract summary: We present an initial investigation of fairness from the Indian perspective in the legal domain. We show that a decision tree model trained for the bail prediction task has an overall fairness disparity of 0.237 between input features associated with Hindus and Muslims.
• Score: 20.162205920441895
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances and applications of language technology and artificial intelligence have enabled much success across multiple domains like law, medical and mental health. AI-based Language Models, like Judgement Prediction, have recently been proposed for the legal sector. However, these models are strife with encoded social biases picked up from the training data. While bias and fairness have been studied across NLP, most studies primarily locate themselves within a Western context. In this work, we present an initial investigation of fairness from the Indian perspective in the legal domain. We highlight the propagation of learnt algorithmic biases in the bail prediction task for models trained on Hindi legal documents. We evaluate the fairness gap using demographic parity and show that a decision tree model trained for the bail prediction task has an overall fairness disparity of 0.237 between input features associated with Hindus and Muslims. Additionally, we highlight the need for further research and studies in the avenues of fairness/bias in applying AI in the legal sector with a specific focus on the Indian context.

Related papers

• Auditing for Racial Discrimination in the Delivery of Education Ads [50.37313459134418]
  We propose a new third-party auditing method that can evaluate racial bias in the delivery of ads for education opportunities. We find evidence of racial discrimination in Meta's algorithmic delivery of ads for education opportunities, posing legal and ethical concerns.
  arXiv  Detail & Related papers  (2024-06-02T02:00:55Z)
• Towards Explainability in Legal Outcome Prediction Models [64.00172507827499]
  We argue that precedent is a natural way of facilitating explainability for legal NLP models. By developing a taxonomy of legal precedent, we are able to compare human judges and neural models. We find that while the models learn to predict outcomes reasonably well, their use of precedent is unlike that of human judges.
  arXiv  Detail & Related papers  (2024-03-25T15:15:41Z)
• Convolutional Neural Networks can achieve binary bail judgement classification [0.5013868868152144]
  We deploy a Convolutional Neural Network (CNN) architecture on a corpus of Hindi legal documents. We perform a bail Prediction task with the help of a CNN model and achieve an overall accuracy of 93%.
  arXiv  Detail & Related papers  (2024-01-25T12:31:41Z)
• Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaboration [52.57055162778548]
  Legal Judgment Prediction (LJP) has become an increasingly crucial task in Legal AI. Precedents are the previous legal cases with similar facts, which are the basis for the judgment of the subsequent case in national legal systems. Recent advances in deep learning have enabled a variety of techniques to be used to solve the LJP task.
  arXiv  Detail & Related papers  (2023-10-13T16:47:20Z)
• Legal Question-Answering in the Indian Context: Efficacy, Challenges, and Potential of Modern AI Models [3.552993426200889]
  Legal QA platforms bear the promise to metamorphose the manner in which legal experts engage with jurisprudential documents. Our discourse zeroes in on an array of retrieval and QA mechanisms, positioning the OpenAI GPT model as a reference point. The ambit of this study is tethered to the Indian criminal legal landscape, distinguished by its intricate nature and associated logistical constraints.
  arXiv  Detail & Related papers  (2023-09-26T07:56:55Z)
• Fair Enough: Standardizing Evaluation and Model Selection for Fairness Research in NLP [64.45845091719002]
  Modern NLP systems exhibit a range of biases, which a growing literature on model debiasing attempts to correct. This paper seeks to clarify the current situation and plot a course for meaningful progress in fair learning.
  arXiv  Detail & Related papers  (2023-02-11T14:54:00Z)
• Pre-trained Language Models for the Legal Domain: A Case Study on Indian Law [7.366081387295463]
  We re-train two popular legal PLMs, LegalBERT and CaseLawBERT, on Indian legal data, as well as train a model from scratch with a vocabulary based on Indian legal text. We observe our approach not only enhances performance on the new domain (Indian texts) but also over the original domain (European and UK texts)
  arXiv  Detail & Related papers  (2022-09-13T15:01:11Z)
• Decoding Demographic un-fairness from Indian Names [4.402336973466853]
  Demographic classification is essential in fairness assessment in recommender systems or in measuring unintended bias in online networks and voting systems. We collect three publicly available datasets to train state-of-the-art classifiers in the domain of gender and caste classification. We perform cross-testing (training and testing on different datasets) to understand the efficacy of the above models.
  arXiv  Detail & Related papers  (2022-09-07T11:54:49Z)
• Lawformer: A Pre-trained Language Model for Chinese Legal Long Documents [56.40163943394202]
  We release the Longformer-based pre-trained language model, named as Lawformer, for Chinese legal long documents understanding. We evaluate Lawformer on a variety of LegalAI tasks, including judgment prediction, similar case retrieval, legal reading comprehension, and legal question answering.
  arXiv  Detail & Related papers  (2021-05-09T09:39:25Z)
• Re-imagining Algorithmic Fairness in India and Beyond [9.667710168953239]
  We de-center algorithmic fairness and analyse AI power in India. We find that data is not always reliable due to socio-economic factors. We provide a roadmap to re-contextualise data and models, empower oppressed communities, and enable Fair-ML ecosystems.
  arXiv  Detail & Related papers  (2021-01-25T10:20:57Z)
• How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence [81.04070052740596]
  Legal Artificial Intelligence (LegalAI) focuses on applying the technology of artificial intelligence, especially natural language processing, to benefit tasks in the legal domain. This paper introduces the history, the current state, and the future directions of research in LegalAI.
  arXiv  Detail & Related papers  (2020-04-25T14:45:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.