Eliciting Model Steering Interactions from Users via Data and Visual Design Probes

• URL: http://arxiv.org/abs/2310.09314v1
• Date: Thu, 12 Oct 2023 20:34:02 GMT
• Title: Eliciting Model Steering Interactions from Users via Data and Visual Design Probes
• Authors: Anamaria Crisan, Maddie Shang, Eric Brochu
• Abstract summary: Domain experts increasingly use automated data science tools to incorporate machine learning (ML) models in their work but struggle to " codify" these models when they are incorrect. For these experts, semantic interactions can provide an accessible avenue to guide and refine ML models without having to dive into its technical details. This study examines how experts with a spectrum of ML expertise use semantic interactions to update a simple classification model.
• Score: 8.45602005745865
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Domain experts increasingly use automated data science tools to incorporate machine learning (ML) models in their work but struggle to "debug" these models when they are incorrect. For these experts, semantic interactions can provide an accessible avenue to guide and refine ML models without having to programmatically dive into its technical details. In this research, we conduct an elicitation study using data and visual design probes to examine if and how experts with a spectrum of ML expertise use semantic interactions to update a simple classification model. We use our design probes to facilitate an interactive dialogue with 20 participants and codify their interactions as a set of target-interaction pairs. Interestingly, our findings revealed that many targets of semantic interactions do not directly map to ML model parameters, but instead aim to augment the data a model uses for training. We also identify reasons that participants would hesitate to interact with ML models, including burdens of cognitive load and concerns of injecting bias. Unexpectedly participants also saw the value of using semantic interactions to work collaboratively with members of their team. Participants with less ML expertise found this to be a useful mechanism for communicating their concerns to ML experts. This was an especially important observation, as our study also shows the different needs that correspond to diverse ML expertise. Collectively, we demonstrate that design probes are effective tools for proactively gathering the affordances that should be offered in an interactive machine learning system.

Related papers

• LLM-assisted Explicit and Implicit Multi-interest Learning Framework for Sequential Recommendation [50.98046887582194]
  We propose an explicit and implicit multi-interest learning framework to model user interests on two levels: behavior and semantics. The proposed EIMF framework effectively and efficiently combines small models with LLM to improve the accuracy of multi-interest modeling.
  arXiv  Detail & Related papers  (2024-11-14T13:00:23Z)
• RA-BLIP: Multimodal Adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-training [55.54020926284334]
  Multimodal Large Language Models (MLLMs) have recently received substantial interest, which shows their emerging potential as general-purpose models for various vision-language tasks. Retrieval augmentation techniques have proven to be effective plugins for both LLMs and MLLMs. In this study, we propose multimodal adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-training (RA-BLIP), a novel retrieval-augmented framework for various MLLMs.
  arXiv  Detail & Related papers  (2024-10-18T03:45:19Z)
• MergeNet: Knowledge Migration across Heterogeneous Models, Tasks, and Modalities [72.68829963458408]
  We present MergeNet, which learns to bridge the gap of parameter spaces of heterogeneous models. The core mechanism of MergeNet lies in the parameter adapter, which operates by querying the source model's low-rank parameters. MergeNet is learned alongside both models, allowing our framework to dynamically transfer and adapt knowledge relevant to the current stage.
  arXiv  Detail & Related papers  (2024-04-20T08:34:39Z)
• Revisiting Self-supervised Learning of Speech Representation from a Mutual Information Perspective [68.20531518525273]
  We take a closer look into existing self-supervised methods of speech from an information-theoretic perspective. We use linear probes to estimate the mutual information between the target information and learned representations. We explore the potential of evaluating representations in a self-supervised fashion, where we estimate the mutual information between different parts of the data without using any labels.
  arXiv  Detail & Related papers  (2024-01-16T21:13:22Z)
• Enhancing HOI Detection with Contextual Cues from Large Vision-Language Models [56.257840490146]
  ConCue is a novel approach for improving visual feature extraction in HOI detection. We develop a transformer-based feature extraction module with a multi-tower architecture that integrates contextual cues into both instance and interaction detectors.
  arXiv  Detail & Related papers  (2023-11-26T09:11:32Z)
• Scaling Vision-Language Models with Sparse Mixture of Experts [128.0882767889029]
  We show that mixture-of-experts (MoE) techniques can achieve state-of-the-art performance on a range of benchmarks over dense models of equivalent computational cost. Our research offers valuable insights into stabilizing the training of MoE models, understanding the impact of MoE on model interpretability, and balancing the trade-offs between compute performance when scaling vision-language models.
  arXiv  Detail & Related papers  (2023-03-13T16:00:31Z)
• Leveraging Explanations in Interactive Machine Learning: An Overview [10.284830265068793]
  Explanations have gained an increasing level of interest in the AI and Machine Learning (ML) communities. This paper presents an overview of research where explanations are combined with interactive capabilities.
  arXiv  Detail & Related papers  (2022-07-29T07:46:11Z)
• Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values [27.333641578187887]
  We develop GAM Changer, the first interactive system to help data scientists and domain experts edit Generalized Additive Models (GAMs) With novel interaction techniques, our tool puts interpretability into action--empowering users to analyze, validate, and align model behaviors with their knowledge and values.
  arXiv  Detail & Related papers  (2022-06-30T17:57:12Z)
• Interactive Model Cards: A Human-Centered Approach to Model Documentation [20.880991026743498]
  Deep learning models for natural language processing are increasingly adopted and deployed by analysts without formal training in NLP or machine learning. The documentation intended to convey the model's details and appropriate use is tailored primarily to individuals with ML or NLP expertise. We conduct a design inquiry into interactive model cards, which augment traditionally static model cards with affordances for exploring model documentation and interacting with the models themselves.
  arXiv  Detail & Related papers  (2022-05-05T19:19:28Z)
• Towards Model-informed Precision Dosing with Expert-in-the-loop Machine Learning [0.0]
  We consider a ML framework that may accelerate model learning and improve its interpretability by incorporating human experts into the model learning loop. We propose a novel human-in-the-loop ML framework aimed at dealing with learning problems that the cost of data annotation is high. With an application to precision dosing, our experimental results show that the approach can learn interpretable rules from data and may potentially lower experts' workload.
  arXiv  Detail & Related papers  (2021-06-28T03:45:09Z)
• Intuitively Assessing ML Model Reliability through Example-Based Explanations and Editing Model Inputs [19.09848738521126]
  Interpretability methods aim to help users build trust in and understand the capabilities of machine learning models. We present two interface modules to facilitate a more intuitive assessment of model reliability.
  arXiv  Detail & Related papers  (2021-02-17T02:41:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.