Machine Learning in the Wild: Early Evidence of Non-Compliant ML-Automation in Open-Source Software

• URL: http://arxiv.org/abs/2603.29698v1
• Date: Tue, 31 Mar 2026 12:53:01 GMT
• Title: Machine Learning in the Wild: Early Evidence of Non-Compliant ML-Automation in Open-Source Software
• Abstract summary: This paper presents a preliminary investigation into the use of Machine Learning models by 173 open-source projects on GitHub.<n>We evaluate whether models are used to make decisions, the scope of these decisions, and whether any post-processing measures are taken to reduce the risks inherent in fully autonomous systems.<n>This study lays the groundwork for defining guidelines for developers and creating analysis tools that automatically identify potential regulatory violations in the use of ML models in software systems.
• Score: 6.407929369404772
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The increasing availability of Machine Learning (ML) models, particularly foundation models, enables their use across a range of downstream applications, from scenarios with missing data to safety-critical contexts. This, in principle, may contravene not only the models' terms of use, but also governmental principles and regulations. This paper presents a preliminary investigation into the use of ML models by 173 open-source projects on GitHub, spanning 16 application domains. We evaluate whether models are used to make decisions, the scope of these decisions, and whether any post-processing measures are taken to reduce the risks inherent in fully autonomous systems. Lastly, we investigate the models' compliance with established terms of use. This study lays the groundwork for defining guidelines for developers and creating analysis tools that automatically identify potential regulatory violations in the use of ML models in software systems.

Related papers

• On the Robustness of Fairness Practices: A Causal Framework for Systematic Evaluation [17.020173966728507]
  Machine learning algorithms are increasingly deployed to make critical decisions in socioeconomic applications.<n>Due to their data-driven and pattern-seeking nature, ML algorithms may develop decision logic that disproportionately distributes opportunities, benefits, resources, or information among different population groups.<n>The software engineering and ML communities have made significant efforts to establish the best practices for creating fair ML software.
  arXiv  Detail & Related papers  (2026-01-07T06:02:53Z)
• Querying Large Automotive Software Models: Agentic vs. Direct LLM Approaches [3.549427092296418]
  Large language models (LLMs) offer new opportunities for interacting with complex software artifacts, such as software models, through natural language.<n>This paper investigates two approaches for leveraging LLMs to answer questions over software models.<n>We evaluate these approaches using an Ecore metamodel designed for timing analysis and software optimization in automotive domains.
  arXiv  Detail & Related papers  (2025-06-16T07:34:28Z)
• Gradient-Based Model Fingerprinting for LLM Similarity Detection and Family Classification [6.008384763761687]
  Large Language Models (LLMs) are integral software components in modern applications.<n>We presentGuard, a gradient-based fingerprinting framework for similarity detection and family classification.<n>Our approach extracts model-intrinsic behavioral signatures by analyzing responses to random input perturbations.<n>It supports the widely-adopted safetensors format and constructs high-dimensional fingerprints through statistical analysis of gradient features.
  arXiv  Detail & Related papers  (2025-06-02T13:08:01Z)
• Network Resource Optimization for ML-Based UAV Condition Monitoring with Vibration Analysis [54.550658461477106]
  Condition Monitoring (CM) uses Machine Learning (ML) models to identify abnormal and adverse conditions.<n>This work explores the optimization of network resources for ML-based UAV CM frameworks.<n>By leveraging dimensionality reduction techniques, there is a 99.9% reduction in network resource consumption.
  arXiv  Detail & Related papers  (2025-02-21T14:36:12Z)
• Training Compute Thresholds: Features and Functions in AI Regulation [0.7234862895932991]
  Regulators in the US and EU are using thresholds based on training compute to identify GPAI models that may pose risks of large-scale societal harm. We argue that training compute currently is the most suitable metric to identify GPAI models that deserve regulatory oversight and further scrutiny. As GPAI technology and market structures evolve, regulators should update compute thresholds and complement them with other metrics into regulatory review processes.
  arXiv  Detail & Related papers  (2024-05-17T14:10:24Z)
• Unveiling the Misuse Potential of Base Large Language Models via In-Context Learning [61.2224355547598]
  Open-sourcing of large language models (LLMs) accelerates application development, innovation, and scientific progress. Our investigation exposes a critical oversight in this belief. By deploying carefully designed demonstrations, our research demonstrates that base LLMs could effectively interpret and execute malicious instructions.
  arXiv  Detail & Related papers  (2024-04-16T13:22:54Z)
• Benchmarking Automated Machine Learning Methods for Price Forecasting Applications [58.720142291102135]
  We show the possibility of substituting manually created ML pipelines with automated machine learning (AutoML) solutions. Based on the CRISP-DM process, we split the manual ML pipeline into a machine learning and non-machine learning part. We show in a case study for the industrial use case of price forecasting, that domain knowledge combined with AutoML can weaken the dependence on ML experts.
  arXiv  Detail & Related papers  (2023-04-28T10:27:38Z)
• Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models [103.71308117592963]
  We present an algorithm for training self-destructing models leveraging techniques from meta-learning and adversarial learning. In a small-scale experiment, we show MLAC can largely prevent a BERT-style model from being re-purposed to perform gender identification.
  arXiv  Detail & Related papers  (2022-11-27T21:43:45Z)
• A Survey of Machine Unlearning [56.017968863854186]
  Recent regulations now require that, on request, private information about a user must be removed from computer systems. ML models often remember' the old data. Recent works on machine unlearning have not been able to completely solve the problem.
  arXiv  Detail & Related papers  (2022-09-06T08:51:53Z)
• A Model-Driven Engineering Approach to Machine Learning and Software Modeling [0.5156484100374059]
  Models are used in both the Software Engineering (SE) and the Artificial Intelligence (AI) communities. The main focus is on the Internet of Things (IoT) and smart Cyber-Physical Systems (CPS) use cases, where both ML and model-driven SE play a key role.
  arXiv  Detail & Related papers  (2021-07-06T15:50:50Z)
• Practical Machine Learning Safety: A Survey and Primer [81.73857913779534]
  Open-world deployment of Machine Learning algorithms in safety-critical applications such as autonomous vehicles needs to address a variety of ML vulnerabilities. New models and training techniques to reduce generalization error, achieve domain adaptation, and detect outlier examples and adversarial attacks. Our organization maps state-of-the-art ML techniques to safety strategies in order to enhance the dependability of the ML algorithm from different aspects.
  arXiv  Detail & Related papers  (2021-06-09T05:56:42Z)
• AHMoSe: A Knowledge-Based Visual Support System for Selecting Regression Machine Learning Models [2.9998889086656577]
  AHMoSe is a visual support system that allows domain experts to better understand, diagnose and compare different regression models. We describe a use case scenario in the viticulture domain, grape quality prediction, where the system enables users to diagnose and select prediction models that perform better.
  arXiv  Detail & Related papers  (2021-01-28T12:55:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.