Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks

• URL: http://arxiv.org/abs/2502.10526v2
• Date: Thu, 20 Feb 2025 17:56:42 GMT
• Title: Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks
• Authors: Venkatesh Sivaraman, Anika Vaishampayan, Xiaotong Li, Brian R Buck, Ziyong Ma, Richard D Boyce, Adam Perer,
• Abstract summary: We develop Tempo, an interactive system that helps data scientists and domain experts collaborate on model specifications.<n>Data scientists can quickly prototype specifications with greater transparency about pre-processing choices.<n> domain experts can assess performance within data subgroups to validate that models behave as expected.
• Score: 14.099791384467274
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Temporal predictive models have the potential to improve decisions in health care, public services, and other domains, yet they often fail to effectively support decision-makers. Prior literature shows that many misalignments between model behavior and decision-makers' expectations stem from issues of model specification, namely how, when, and for whom predictions are made. However, model specifications for predictive tasks are highly technical and difficult for non-data-scientist stakeholders to interpret and critique. To address this challenge we developed Tempo, an interactive system that helps data scientists and domain experts collaboratively iterate on model specifications. Using Tempo's simple yet precise temporal query language, data scientists can quickly prototype specifications with greater transparency about pre-processing choices. Moreover, domain experts can assess performance within data subgroups to validate that models behave as expected. Through three case studies, we demonstrate how Tempo helps multidisciplinary teams quickly prune infeasible specifications and identify more promising directions to explore.

Related papers

• Does Multimodality Lead to Better Time Series Forecasting? [84.74978289870155]
  It remains unclear whether and under what conditions such multimodal integration consistently yields gains.<n>We evaluate two popular multimodal forecasting paradigms: aligning-based methods, which align time series and text representations; and prompting-based methods, which directly prompt large language models for forecasting.<n>Our findings highlight that on the modeling side, incorporating text information is most helpful given (1) high-capacity text models, (2) comparatively weaker time series models, and (3) appropriate aligning strategies.
  arXiv  Detail & Related papers  (2025-06-20T23:55:56Z)
• XForecast: Evaluating Natural Language Explanations for Time Series Forecasting [72.57427992446698]
  Time series forecasting aids decision-making, especially for stakeholders who rely on accurate predictions. Traditional explainable AI (XAI) methods, which underline feature or temporal importance, often require expert knowledge. evaluating forecast NLEs is difficult due to the complex causal relationships in time series data.
  arXiv  Detail & Related papers  (2024-10-18T05:16:39Z)
• Temporal receptive field in dynamic graph learning: A comprehensive analysis [15.161255747900968]
  We present a comprehensive analysis of the temporal receptive field in dynamic graph learning. Our results demonstrate that appropriately chosen temporal receptive field can significantly enhance model performance. For some models, overly large windows may introduce noise and reduce accuracy.
  arXiv  Detail & Related papers  (2024-07-17T07:46:53Z)
• A Reliable Framework for Human-in-the-Loop Anomaly Detection in Time Series [17.08674819906415]
  We introduce HILAD, a novel framework designed to foster a dynamic and bidirectional collaboration between humans and AI. Through our visual interface, HILAD empowers domain experts to detect, interpret, and correct unexpected model behaviors at scale.
  arXiv  Detail & Related papers  (2024-05-06T07:44:07Z)
• A Survey on Diffusion Models for Time Series and Spatio-Temporal Data [92.1255811066468]
  We review the use of diffusion models in time series and S-temporal data, categorizing them by model, task type, data modality, and practical application domain. We categorize diffusion models into unconditioned and conditioned types discuss time series and S-temporal data separately. Our survey covers their application extensively in various fields including healthcare, recommendation, climate, energy, audio, and transportation.
  arXiv  Detail & Related papers  (2024-04-29T17:19:40Z)
• Third-Party Language Model Performance Prediction from Instruction [59.574169249307054]
  Language model-based instruction-following systems have lately shown increasing performance on many benchmark tasks. A user may easily prompt a model with an instruction without any idea of whether the responses should be expected to be accurate. We propose a third party performance prediction framework, where a separate model is trained to predict the metric resulting from evaluating an instruction-following system on a task.
  arXiv  Detail & Related papers  (2024-03-19T03:53:47Z)
• Towards Unifying Diffusion Models for Probabilistic Spatio-Temporal Graph Learning [26.57090339778029]
  We propose to model tasks in a unified probabilistic perspective, viewing them as predictions based on conditional information with shared dependencies. Based on this proposal, we introduce Unified Spatio-temporal Diffusion Models (USTD) to address the tasks uniformly. USTD is holistically designed, comprising a shared-temporal encoder and attention-based denoising decoders are task-specific.
  arXiv  Detail & Related papers  (2023-10-26T12:48:43Z)
• Are Metrics Enough? Guidelines for Communicating and Visualizing Predictive Models to Subject Matter Experts [5.6735884560669]
  We describe an iterative study conducted with both subject matter experts and data scientists to understand the gaps in communication. We derive a set of communication guidelines that use visualization as a common medium for communicating the strengths and weaknesses of a model.
  arXiv  Detail & Related papers  (2022-05-11T19:40:24Z)
• Leveraging the structure of dynamical systems for data-driven modeling [111.45324708884813]
  We consider the impact of the training set and its structure on the quality of the long-term prediction. We show how an informed design of the training set, based on invariants of the system and the structure of the underlying attractor, significantly improves the resulting models.
  arXiv  Detail & Related papers  (2021-12-15T20:09:20Z)
• Model-Attentive Ensemble Learning for Sequence Modeling [86.4785354333566]
  We present Model-Attentive Ensemble learning for Sequence modeling (MAES) MAES is a mixture of time-series experts which leverages an attention-based gating mechanism to specialize the experts on different sequence dynamics and adaptively weight their predictions. We demonstrate that MAES significantly out-performs popular sequence models on datasets subject to temporal shift.
  arXiv  Detail & Related papers  (2021-02-23T05:23:35Z)
• Ambiguity in Sequential Data: Predicting Uncertain Futures with Recurrent Models [110.82452096672182]
  We propose an extension of the Multiple Hypothesis Prediction (MHP) model to handle ambiguous predictions with sequential data. We also introduce a novel metric for ambiguous problems, which is better suited to account for uncertainties.
  arXiv  Detail & Related papers  (2020-03-10T09:15:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.