Is a Seat at the Table Enough? Engaging Teachers and Students in Dataset Specification for ML in Education

• URL: http://arxiv.org/abs/2311.05792v1
• Date: Thu, 9 Nov 2023 23:51:08 GMT
• Title: Is a Seat at the Table Enough? Engaging Teachers and Students in Dataset Specification for ML in Education
• Authors: Mei Tan, Hansol Lee, Dakuo Wang, Hariharan Subramonyam
• Abstract summary: Despite the promises of ML in education, its adoption has surfaced numerous issues regarding fairness, accountability, and transparency. A root cause of these issues is the lack of understanding of the complex dynamics of education.
• Score: 28.899007394121416
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the promises of ML in education, its adoption in the classroom has surfaced numerous issues regarding fairness, accountability, and transparency, as well as concerns about data privacy and student consent. A root cause of these issues is the lack of understanding of the complex dynamics of education, including teacher-student interactions, collaborative learning, and classroom environment. To overcome these challenges and fully utilize the potential of ML in education, software practitioners need to work closely with educators and students to fully understand the context of the data (the backbone of ML applications) and collaboratively define the ML data specifications. To gain a deeper understanding of such a collaborative process, we conduct ten co-design sessions with ML software practitioners, educators, and students. In the sessions, teachers and students work with ML engineers, UX designers, and legal practitioners to define dataset characteristics for a given ML application. We find that stakeholders contextualize data based on their domain and procedural knowledge, proactively design data requirements to mitigate downstream harms and data reliability concerns, and exhibit role-based collaborative strategies and contribution patterns. Further, we find that beyond a seat at the table, meaningful stakeholder participation in ML requires structured supports: defined processes for continuous iteration and co-evaluation, shared contextual data quality standards, and information scaffolds for both technical and non-technical stakeholders to traverse expertise boundaries.

Related papers

• Benchmarking Vision Language Model Unlearning via Fictitious Facial Identity Dataset [94.13848736705575]
  We introduce Facial Identity Unlearning Benchmark (FIUBench), a novel VLM unlearning benchmark designed to robustly evaluate the effectiveness of unlearning algorithms. We apply a two-stage evaluation pipeline that is designed to precisely control the sources of information and their exposure levels. Through the evaluation of four baseline VLM unlearning algorithms within FIUBench, we find that all methods remain limited in their unlearning performance.
  arXiv  Detail & Related papers  (2024-11-05T23:26:10Z)
• Exploring Knowledge Tracing in Tutor-Student Dialogues [53.52699766206808]
  We present a first attempt at performing knowledge tracing (KT) in tutor-student dialogues. We propose methods to identify the knowledge components/skills involved in each dialogue turn. We then apply a range of KT methods on the resulting labeled data to track student knowledge levels over an entire dialogue.
  arXiv  Detail & Related papers  (2024-09-24T22:31:39Z)
• Faculty Perspectives on the Potential of RAG in Computer Science Higher Education [0.0]
  We developed Retrieval Augmented Generation (RAG) applications for the two tasks of virtual teaching assistants and teaching aids. This study is the first to gather faculty feedback on the application of LLM-based RAG in education.
  arXiv  Detail & Related papers  (2024-07-28T14:55:22Z)
• SINKT: A Structure-Aware Inductive Knowledge Tracing Model with Large Language Model [64.92472567841105]
  Knowledge Tracing (KT) aims to determine whether students will respond correctly to the next question. Structure-aware Inductive Knowledge Tracing model with large language model (dubbed SINKT) SINKT predicts the student's response to the target question by interacting with the student's knowledge state and the question representation.
  arXiv  Detail & Related papers  (2024-07-01T12:44:52Z)
• FATE in MMLA: A Student-Centred Exploration of Fairness, Accountability, Transparency, and Ethics in Multimodal Learning Analytics [0.0]
  This study assessed students' perceived fairness, accountability, transparency, and ethics (FATE) with MMLA visualisations. Findings highlighted the significance of accurate and comprehensive data representation to ensure visualisation fairness. Students also emphasise the importance of ethical considerations, highlighting a pressing need for the LA and MMLA community to investigate and address FATE issues actively.
  arXiv  Detail & Related papers  (2024-02-29T11:52:06Z)
• Rethinking Machine Unlearning for Large Language Models [85.92660644100582]
  We explore machine unlearning in the domain of large language models (LLMs) This initiative aims to eliminate undesirable data influence (e.g., sensitive or illegal information) and the associated model capabilities.
  arXiv  Detail & Related papers  (2024-02-13T20:51:58Z)
• MachineLearnAthon: An Action-Oriented Machine Learning Didactic Concept [34.6229719907685]
  This paper introduces the MachineLearnAthon format, an innovative didactic concept designed to be inclusive for students of different disciplines. At the heart of the concept lie ML challenges, which make use of industrial data sets to solve real-world problems. These cover the entire ML pipeline, promoting data literacy and practical skills, from data preparation, through deployment, to evaluation.
  arXiv  Detail & Related papers  (2024-01-29T16:50:32Z)
• Identifying Concerns When Specifying Machine Learning-Enabled Systems: A Perspective-Based Approach [1.2184324428571227]
  PerSpecML is a perspective-based approach for specifying ML-enabled systems. It helps practitioners identify which attributes, including ML and non-ML components, are important to contribute to the overall system's quality.
  arXiv  Detail & Related papers  (2023-09-14T18:31:16Z)
• ML-Based Teaching Systems: A Conceptual Framework [0.0]
  We investigate the potential of machine learning (ML) models to facilitate knowledge transfer in an organizational context. We examine key concepts, themes, and dimensions to better understand and design ML-based teaching systems.
  arXiv  Detail & Related papers  (2023-05-12T09:55:34Z)
• Distributed Machine Learning and the Semblance of Trust [66.1227776348216]
  Federated Learning (FL) allows the data owner to maintain data governance and perform model training locally without having to share their data. FL and related techniques are often described as privacy-preserving. We explain why this term is not appropriate and outline the risks associated with over-reliance on protocols that were not designed with formal definitions of privacy in mind.
  arXiv  Detail & Related papers  (2021-12-21T08:44:05Z)
• Understanding the Usability Challenges of Machine Learning In High-Stakes Decision Making [67.72855777115772]
  Machine learning (ML) is being applied to a diverse and ever-growing set of domains. In many cases, domain experts -- who often have no expertise in ML or data science -- are asked to use ML predictions to make high-stakes decisions. We investigate the ML usability challenges present in the domain of child welfare screening through a series of collaborations with child welfare screeners.
  arXiv  Detail & Related papers  (2021-03-02T22:50:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.