InFiConD: Interactive No-code Fine-tuning with Concept-based Knowledge Distillation

• URL: http://arxiv.org/abs/2406.17838v1
• Date: Tue, 25 Jun 2024 16:56:45 GMT
• Title: InFiConD: Interactive No-code Fine-tuning with Concept-based Knowledge Distillation
• Authors: Jinbin Huang, Wenbin He, Liang Gou, Liu Ren, Chris Bryan,
• Abstract summary: This paper presents InFiConD, a novel framework that leverages visual concepts to implement the knowledge distillation process. We develop a novel knowledge distillation pipeline based on extracting text-aligned visual concepts from a concept corpus. InFiConD's interface allows users to interactively fine-tune the student model by manipulating concept influences directly in the user interface.
• Score: 18.793275018467163
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The emergence of large-scale pre-trained models has heightened their application in various downstream tasks, yet deployment is a challenge in environments with limited computational resources. Knowledge distillation has emerged as a solution in such scenarios, whereby knowledge from large teacher models is transferred into smaller student' models, but this is a non-trivial process that traditionally requires technical expertise in AI/ML. To address these challenges, this paper presents InFiConD, a novel framework that leverages visual concepts to implement the knowledge distillation process and enable subsequent no-code fine-tuning of student models. We develop a novel knowledge distillation pipeline based on extracting text-aligned visual concepts from a concept corpus using multimodal models, and construct highly interpretable linear student models based on visual concepts that mimic a teacher model in a response-based manner. InFiConD's interface allows users to interactively fine-tune the student model by manipulating concept influences directly in the user interface. We validate InFiConD via a robust usage scenario and user study. Our findings indicate that InFiConD's human-in-the-loop and visualization-driven approach enables users to effectively create and analyze student models, understand how knowledge is transferred, and efficiently perform fine-tuning operations. We discuss how this work highlights the potential of interactive and visual methods in making knowledge distillation and subsequent no-code fine-tuning more accessible and adaptable to a wider range of users with domain-specific demands.

Related papers

• Deep Learning and Machine Learning, Advancing Big Data Analytics and Management: Tensorflow Pretrained Models [17.372501468675303]
  The book covers practical implementations of modern architectures like ResNet, MobileNet, and EfficientNet. It compares linear probing and model fine-tuning, offering visualizations using techniques such as PCA, t-SNE, and UMAP. By blending theoretical insights with hands-on practice, this book equips readers with the knowledge to confidently tackle various deep learning challenges.
  arXiv  Detail & Related papers  (2024-09-20T15:07:14Z)
• Hierarchical and Decoupled BEV Perception Learning Framework for Autonomous Driving [52.808273563372126]
  This paper proposes a novel hierarchical BEV perception paradigm, aiming to provide a library of fundamental perception modules and user-friendly graphical interface. We conduct the Pretrain-Finetune strategy to effectively utilize large scale public datasets and streamline development processes. We also present a Multi-Module Learning (MML) approach, enhancing performance through synergistic and iterative training of multiple models.
  arXiv  Detail & Related papers  (2024-07-17T11:17:20Z)
• Restyling Unsupervised Concept Based Interpretable Networks with Generative Models [14.604305230535026]
  We propose a novel method that relies on mapping the concept features to the latent space of a pretrained generative model. We quantitatively ascertain the efficacy of our method in terms of accuracy of the interpretable prediction network, fidelity of reconstruction, as well as faithfulness and consistency of learnt concepts.
  arXiv  Detail & Related papers  (2024-07-01T14:39:41Z)
• Interaction as Explanation: A User Interaction-based Method for Explaining Image Classification Models [1.3597551064547502]
  In computer vision, explainable AI (xAI) methods seek to mitigate the 'black-box' problem. Traditional xAI methods concentrate on visualizing input features that influence model predictions. We present an interaction-based xAI method that enhances user comprehension of image classification models through their interaction.
  arXiv  Detail & Related papers  (2024-04-15T14:26:00Z)
• Visual Concept-driven Image Generation with Text-to-Image Diffusion Model [65.96212844602866]
  Text-to-image (TTI) models have demonstrated impressive results in generating high-resolution images of complex scenes. Recent approaches have extended these methods with personalization techniques that allow them to integrate user-illustrated concepts. However, the ability to generate images with multiple interacting concepts, such as human subjects, as well as concepts that may be entangled in one, or across multiple, image illustrations remains illusive. We propose a concept-driven TTI personalization framework that addresses these core challenges.
  arXiv  Detail & Related papers  (2024-02-18T07:28:37Z)
• Advancing Ante-Hoc Explainable Models through Generative Adversarial Networks [24.45212348373868]
  This paper presents a novel concept learning framework for enhancing model interpretability and performance in visual classification tasks. Our approach appends an unsupervised explanation generator to the primary classifier network and makes use of adversarial training. This work presents a significant step towards building inherently interpretable deep vision models with task-aligned concept representations.
  arXiv  Detail & Related papers  (2024-01-09T16:16:16Z)
• InterVLS: Interactive Model Understanding and Improvement with Vision-Language Surrogates [18.793275018467163]
  Deep learning models are widely used in critical applications, highlighting the need for pre-deployment model understanding and improvement. Visual concept-based methods, while increasingly used for this purpose, face challenges: (1) most concepts lack interpretability, (2) existing methods require model knowledge, often unavailable at run time, and (3) there lacks a no-code method for post-understanding model improvement. We present InterVLS, which facilitates model understanding by discovering text-aligned concepts, measuring their influence with model-agnostic linear surrogates.
  arXiv  Detail & Related papers  (2023-11-06T21:30:59Z)
• Distilling Knowledge from Self-Supervised Teacher by Embedding Graph Alignment [52.704331909850026]
  We formulate a new knowledge distillation framework to transfer the knowledge from self-supervised pre-trained models to any other student network. Inspired by the spirit of instance discrimination in self-supervised learning, we model the instance-instance relations by a graph formulation in the feature embedding space. Our distillation scheme can be flexibly applied to transfer the self-supervised knowledge to enhance representation learning on various student networks.
  arXiv  Detail & Related papers  (2022-11-23T19:27:48Z)
• Explainable Recommender Systems via Resolving Learning Representations [57.24565012731325]
  Explanations could help improve user experience and discover system defects. We propose a novel explainable recommendation model through improving the transparency of the representation learning process.
  arXiv  Detail & Related papers  (2020-08-21T05:30:48Z)
• A Competence-aware Curriculum for Visual Concepts Learning via Question Answering [95.35905804211698]
  We propose a competence-aware curriculum for visual concept learning in a question-answering manner. We design a neural-symbolic concept learner for learning the visual concepts and a multi-dimensional Item Response Theory (mIRT) model for guiding the learning process. Experimental results on CLEVR show that with a competence-aware curriculum, the proposed method achieves state-of-the-art performances.
  arXiv  Detail & Related papers  (2020-07-03T05:08:09Z)
• Plausible Counterfactuals: Auditing Deep Learning Classifiers with Realistic Adversarial Examples [84.8370546614042]
  Black-box nature of Deep Learning models has posed unanswered questions about what they learn from data. Generative Adversarial Network (GAN) and multi-objectives are used to furnish a plausible attack to the audited model. Its utility is showcased within a human face classification task, unveiling the enormous potential of the proposed framework.
  arXiv  Detail & Related papers  (2020-03-25T11:08:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.