Analyzing the Evolution and Maintenance of ML Models on Hugging Face

• URL: http://arxiv.org/abs/2311.13380v2
• Date: Mon, 5 Feb 2024 13:37:21 GMT
• Title: Analyzing the Evolution and Maintenance of ML Models on Hugging Face
• Authors: Joel Casta\~no, Silverio Mart\'inez-Fern\'andez, Xavier Franch, Justus Bogner
• Abstract summary: Hugging Face (HF) has established itself as a crucial platform for the development and sharing of machine learning (ML) models. This repository mining study, which delves into more than 380,000 models using data gathered via the HF Hub API, aims to explore the community engagement, evolution, and maintenance around models hosted on HF.
• Score: 8.409033836300761
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hugging Face (HF) has established itself as a crucial platform for the development and sharing of machine learning (ML) models. This repository mining study, which delves into more than 380,000 models using data gathered via the HF Hub API, aims to explore the community engagement, evolution, and maintenance around models hosted on HF, aspects that have yet to be comprehensively explored in the literature. We first examine the overall growth and popularity of HF, uncovering trends in ML domains, framework usage, authors grouping and the evolution of tags and datasets used. Through text analysis of model card descriptions, we also seek to identify prevalent themes and insights within the developer community. Our investigation further extends to the maintenance aspects of models, where we evaluate the maintenance status of ML models, classify commit messages into various categories (corrective, perfective, and adaptive), analyze the evolution across development stages of commits metrics and introduce a new classification system that estimates the maintenance status of models based on multiple attributes. This study aims to provide valuable insights about ML model maintenance and evolution that could inform future model development strategies on platforms like HF.

Related papers

• How do Machine Learning Models Change? [7.543685248926161]
  This study utilizes both repository mining and longitudinal analysis methods to examine over 200,000 commits and 1,200 releases from over 50,000 models on Hugging Face (HF) We replicate and extend an ML change taxonomy for classifying commits and utilize Bayesian networks to uncover patterns in commit and release activities over time. Our findings indicate that commit activities align with established data science methodologies, such as CRISP-DM, emphasizing iterative refinement and continuous improvement. Additionally, release patterns tend to consolidate significant updates, particularly in documentation, distinguishing between granular changes and milestone-based releases.
  arXiv  Detail & Related papers  (2024-11-14T18:14:32Z)
• Recommendation with Generative Models [35.029116616023586]
  Generative models are AI models capable of creating new instances of data by learning and sampling from their statistical distributions. These models have applications across various domains, such as image generation, text synthesis, and music composition. In recommender systems, generative models, referred to as Gen-RecSys, improve the accuracy and diversity of recommendations.
  arXiv  Detail & Related papers  (2024-09-18T18:29:15Z)
• Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities [89.40778301238642]
  Model merging is an efficient empowerment technique in the machine learning community. There is a significant gap in the literature regarding a systematic and thorough review of these techniques.
  arXiv  Detail & Related papers  (2024-08-14T16:58:48Z)
• DogeRM: Equipping Reward Models with Domain Knowledge through Model Merging [65.41765072566287]
  We propose textbfDomain knowledtextbfge merged textbfReward textbfModel (DogeRM), a novel framework that integrates domain-specific knowledge into a general reward model by model merging.
  arXiv  Detail & Related papers  (2024-07-01T17:01:54Z)
• Lessons Learned from Mining the Hugging Face Repository [5.394314536012109]
  Report synthesizes insights from two comprehensive studies conducted on Hugging Face (HF) Our objective is to provide a practical guide for future researchers embarking on mining software repository studies within the HF ecosystem.
  arXiv  Detail & Related papers  (2024-02-11T22:59:19Z)
• Continual Learning with Pre-Trained Models: A Survey [61.97613090666247]
  Continual Learning aims to overcome the catastrophic forgetting of former knowledge when learning new ones. This paper presents a comprehensive survey of the latest advancements in PTM-based CL.
  arXiv  Detail & Related papers  (2024-01-29T18:27:52Z)
• Comprehensive Exploration of Synthetic Data Generation: A Survey [4.485401662312072]
  This work surveys 417 Synthetic Data Generation models over the last decade. The findings reveal increased model performance and complexity, with neural network-based approaches prevailing. Computer vision dominates, with GANs as primary generative models, while diffusion models, transformers, and RNNs compete.
  arXiv  Detail & Related papers  (2024-01-04T20:23:51Z)
• Model-as-a-Service (MaaS): A Survey [42.70857461774014]
  Foundation models are a form of generative artificial intelligence (GenAI) Model-as-a-Service (M) has emerged as a groundbreaking paradigm that revolutionizes the deployment and utilization of GenAI models.
  arXiv  Detail & Related papers  (2023-11-10T00:35:00Z)
• Towards Graph Foundation Models: A Survey and Beyond [66.37994863159861]
  Foundation models have emerged as critical components in a variety of artificial intelligence applications. The capabilities of foundation models to generalize and adapt motivate graph machine learning researchers to discuss the potential of developing a new graph learning paradigm. This article introduces the concept of Graph Foundation Models (GFMs), and offers an exhaustive explanation of their key characteristics and underlying technologies.
  arXiv  Detail & Related papers  (2023-10-18T09:31:21Z)
• Learn From Model Beyond Fine-Tuning: A Survey [78.80920533793595]
  Learn From Model (LFM) focuses on the research, modification, and design of foundation models (FM) based on the model interface. The study of LFM techniques can be broadly categorized into five major areas: model tuning, model distillation, model reuse, meta learning and model editing. This paper gives a comprehensive review of the current methods based on FM from the perspective of LFM.
  arXiv  Detail & Related papers  (2023-10-12T10:20:36Z)
• Scaling Vision-Language Models with Sparse Mixture of Experts [128.0882767889029]
  We show that mixture-of-experts (MoE) techniques can achieve state-of-the-art performance on a range of benchmarks over dense models of equivalent computational cost. Our research offers valuable insights into stabilizing the training of MoE models, understanding the impact of MoE on model interpretability, and balancing the trade-offs between compute performance when scaling vision-language models.
  arXiv  Detail & Related papers  (2023-03-13T16:00:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.