Dynamic Learning and Productivity for Data Analysts: A Bayesian Hidden Markov Model Perspective

• URL: http://arxiv.org/abs/2503.20233v1
• Date: Wed, 26 Mar 2025 04:57:03 GMT
• Title: Dynamic Learning and Productivity for Data Analysts: A Bayesian Hidden Markov Model Perspective
• Authors: Yue Yin,
• Abstract summary: This study explores how analysts' productivity evolves on a collaborative platform.<n>It focuses on two key learning activities: writing queries and viewing peer queries.<n>Using an industry dataset with 2,001 analysts and 79,797 queries, this study identifies three learning states: novice, intermediate, and advanced.
• Score: 1.9368956071944328
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data analysts are essential in organizations, transforming raw data into insights that drive decision-making and strategy. This study explores how analysts' productivity evolves on a collaborative platform, focusing on two key learning activities: writing queries and viewing peer queries. While traditional research often assumes static models, where performance improves steadily with cumulative learning, such models fail to capture the dynamic nature of real-world learning. To address this, we propose a Hidden Markov Model (HMM) that tracks how analysts transition between distinct learning states based on their participation in these activities. Using an industry dataset with 2,001 analysts and 79,797 queries, this study identifies three learning states: novice, intermediate, and advanced. Productivity increases as analysts advance to higher states, reflecting the cumulative benefits of learning. Writing queries benefits analysts across all states, with the largest gains observed for novices. Viewing peer queries supports novices but may hinder analysts in higher states due to cognitive overload or inefficiencies. Transitions between states are also uneven, with progression from intermediate to advanced being particularly challenging. This study advances understanding of into dynamic learning behavior of knowledge worker and offers practical implications for designing systems, optimizing training, enabling personalized learning, and fostering effective knowledge sharing.

Related papers

• Active Learning Methods for Efficient Data Utilization and Model Performance Enhancement [5.4044723481768235]
  This paper gives a detailed overview of Active Learning (AL), which is a strategy in machine learning that helps models achieve better performance using fewer labeled examples. It introduces the basic concepts of AL and discusses how it is used in various fields such as computer vision, natural language processing, transfer learning, and real-world applications.
  arXiv  Detail & Related papers  (2025-04-21T20:42:13Z)
• LLMs as Educational Analysts: Transforming Multimodal Data Traces into Actionable Reading Assessment Reports [6.523137821124204]
  This study investigates the use of multimodal data sources to derive meaningful reading insights.<n>We employ unsupervised learning techniques to identify distinct reading behavior patterns.<n>A large language model (LLM) synthesizes the derived information into actionable reports for educators.
  arXiv  Detail & Related papers  (2025-03-03T22:34:08Z)
• Spurious Forgetting in Continual Learning of Language Models [20.0936011355535]
  Recent advancements in large language models (LLMs) reveal a perplexing phenomenon in continual learning.<n>Despite extensive training, models experience significant performance declines.<n>This study proposes that such performance drops often reflect a decline in task alignment rather than true knowledge loss.
  arXiv  Detail & Related papers  (2025-01-23T08:09:54Z)
• Data Analysis in the Era of Generative AI [56.44807642944589]
  This paper explores the potential of AI-powered tools to reshape data analysis, focusing on design considerations and challenges. We explore how the emergence of large language and multimodal models offers new opportunities to enhance various stages of data analysis workflow. We then examine human-centered design principles that facilitate intuitive interactions, build user trust, and streamline the AI-assisted analysis workflow across multiple apps.
  arXiv  Detail & Related papers  (2024-09-27T06:31:03Z)
• Translating Expert Intuition into Quantifiable Features: Encode Investigator Domain Knowledge via LLM for Enhanced Predictive Analytics [2.330270848695646]
  This paper explores the potential of Large Language Models to bridge the gap by systematically converting investigator-derived insights into quantifiable, actionable features. We present a framework that leverages LLMs' natural language understanding capabilities to encode these red flags into a structured feature set that can be readily integrated into existing predictive models. The results indicate significant improvements in risk assessment and decision-making accuracy, highlighting the value of blending human experiential knowledge with advanced machine learning techniques.
  arXiv  Detail & Related papers  (2024-05-11T13:23:43Z)
• Towards Automated Knowledge Integration From Human-Interpretable Representations [55.2480439325792]
  We introduce and motivate theoretically the principles of informed meta-learning enabling automated and controllable inductive bias selection.<n>We empirically demonstrate the potential benefits and limitations of informed meta-learning in improving data efficiency and generalisation.
  arXiv  Detail & Related papers  (2024-02-25T15:08:37Z)
• Exploring Federated Unlearning: Analysis, Comparison, and Insights [101.64910079905566]
  federated unlearning enables the selective removal of data from models trained in federated systems. This paper examines existing federated unlearning approaches, examining their algorithmic efficiency, impact on model accuracy, and effectiveness in preserving privacy. We propose the OpenFederatedUnlearning framework, a unified benchmark for evaluating federated unlearning methods.
  arXiv  Detail & Related papers  (2023-10-30T01:34:33Z)
• Accelerating exploration and representation learning with offline pre-training [52.6912479800592]
  We show that exploration and representation learning can be improved by separately learning two different models from a single offline dataset. We show that learning a state representation using noise-contrastive estimation and a model of auxiliary reward can significantly improve the sample efficiency on the challenging NetHack benchmark.
  arXiv  Detail & Related papers  (2023-03-31T18:03:30Z)
• An Empirical Investigation of Commonsense Self-Supervision with Knowledge Graphs [67.23285413610243]
  Self-supervision based on the information extracted from large knowledge graphs has been shown to improve the generalization of language models. We study the effect of knowledge sampling strategies and sizes that can be used to generate synthetic data for adapting language models.
  arXiv  Detail & Related papers  (2022-05-21T19:49:04Z)
• What Makes Good Contrastive Learning on Small-Scale Wearable-based Tasks? [59.51457877578138]
  We study contrastive learning on the wearable-based activity recognition task. This paper presents an open-source PyTorch library textttCL-HAR, which can serve as a practical tool for researchers.
  arXiv  Detail & Related papers  (2022-02-12T06:10:15Z)
• Do we need to go Deep? Knowledge Tracing with Big Data [5.218882272051637]
  We use EdNet, the largest student interaction dataset publicly available in the education domain, to understand how accurately both deep and traditional models predict future student performances. Our work observes that logistic regression models with carefully engineered features outperformed deep models from extensive experimentation.
  arXiv  Detail & Related papers  (2021-01-20T22:40:38Z)
• Principles and Practice of Explainable Machine Learning [12.47276164048813]
  This report focuses on data-driven methods -- machine learning (ML) and pattern recognition models in particular. With the increasing prevalence and complexity of methods, business stakeholders in the very least have a growing number of concerns about the drawbacks of models. We have undertaken a survey to help industry practitioners understand the field of explainable machine learning better.
  arXiv  Detail & Related papers  (2020-09-18T14:50:27Z)
• Human Trajectory Forecasting in Crowds: A Deep Learning Perspective [89.4600982169]
  We present an in-depth analysis of existing deep learning-based methods for modelling social interactions. We propose two knowledge-based data-driven methods to effectively capture these social interactions. We develop a large scale interaction-centric benchmark TrajNet++, a significant yet missing component in the field of human trajectory forecasting.
  arXiv  Detail & Related papers  (2020-07-07T17:19:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.