LLM4Vis: Explainable Visualization Recommendation using ChatGPT
- URL: http://arxiv.org/abs/2310.07652v2
- Date: Mon, 16 Oct 2023 03:34:47 GMT
- Title: LLM4Vis: Explainable Visualization Recommendation using ChatGPT
- Authors: Lei Wang, Songheng Zhang, Yun Wang, Ee-Peng Lim, Yong Wang
- Abstract summary: We propose a novel ChatGPT-based approach to perform visualization recommendation and return human-like explanations.
Our approach involves feature description, demonstration example selection, explanation generation, demonstration example construction, and inference steps.
- Score: 21.875548217393927
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Data visualization is a powerful tool for exploring and communicating
insights in various domains. To automate visualization choice for datasets, a
task known as visualization recommendation has been proposed. Various
machine-learning-based approaches have been developed for this purpose, but
they often require a large corpus of dataset-visualization pairs for training
and lack natural explanations for their results. To address this research gap,
we propose LLM4Vis, a novel ChatGPT-based prompting approach to perform
visualization recommendation and return human-like explanations using very few
demonstration examples. Our approach involves feature description,
demonstration example selection, explanation generation, demonstration example
construction, and inference steps. To obtain demonstration examples with
high-quality explanations, we propose a new explanation generation
bootstrapping to iteratively refine generated explanations by considering the
previous generation and template-based hint. Evaluations on the VizML dataset
show that LLM4Vis outperforms or performs similarly to supervised learning
models like Random Forest, Decision Tree, and MLP in both few-shot and
zero-shot settings. The qualitative evaluation also shows the effectiveness of
explanations generated by LLM4Vis. We make our code publicly available at
\href{https://github.com/demoleiwang/LLM4Vis}{https://github.com/demoleiwang/LLM4Vis}.
Related papers
- V-RECS, a Low-Cost LLM4VIS Recommender with Explanations, Captioning and Suggestions [3.3235895997314726]
We present V-RECS, the first Visual Recommender augmented with explanations(E), captioning(C), and suggestions(S) for further data exploration.
V-RECS' visualization narratives facilitate both response verification and data exploration by non-expert users.
arXiv Detail & Related papers (2024-06-21T15:50:10Z) - Exploring the Distinctiveness and Fidelity of the Descriptions Generated by Large Vision-Language Models [16.524244395901356]
We study how models like Open-Flamingo, IDEFICS, and MiniGPT-4 can distinguish between similar objects and accurately describe visual features.
We propose the Textual Retrieval-Augmented Classification (TRAC) framework, which allows us to delve deeper into analyzing fine-grained visual description generation.
arXiv Detail & Related papers (2024-04-26T16:59:26Z) - Vi(E)va LLM! A Conceptual Stack for Evaluating and Interpreting
Generative AI-based Visualizations [1.709620026135923]
Large language models (LLM) have become an interesting option for supporting generative tasks related to visualization.
This paper copes with the problem of modeling the evaluation of a generated visualization through an LLM.
We propose a theoretical evaluation stack, EvaLLM, that decomposes the evaluation effort in its atomic components.
arXiv Detail & Related papers (2024-02-03T14:28:55Z) - Silkie: Preference Distillation for Large Visual Language Models [56.10697821410489]
This paper explores preference distillation for large vision language models (LVLMs)
We first build a vision-language feedback dataset utilizing AI annotation.
We adopt GPT-4V to assess the generated outputs regarding helpfulness, visual faithfulness, and ethical considerations.
The resulting model Silkie, achieves 6.9% and 9.5% relative improvement on the MME benchmark regarding the perception and cognition capabilities.
arXiv Detail & Related papers (2023-12-17T09:44:27Z) - From Wrong To Right: A Recursive Approach Towards Vision-Language
Explanation [60.746079839840895]
We present ReVisE: a $textbfRe$cursive $textbfVis$ual $textbfE$xplanation algorithm.
Our method iteratively computes visual features (conditioned on the text input), an answer, and an explanation.
We find that this multi-step approach guides the model to correct its own answers and outperforms single-step explanation generation.
arXiv Detail & Related papers (2023-11-21T07:02:32Z) - What Makes for Good Visual Instructions? Synthesizing Complex Visual
Reasoning Instructions for Visual Instruction Tuning [115.19451843294154]
Visual instruction tuning is an essential approach to improving the zero-shot generalization capability of Multi-modal Large Language Models (MLLMs)
We propose a systematic approach to automatically creating high-quality complex visual reasoning instructions.
Our dataset consistently enhances the performance of all the compared MLLMs, e.g., improving the performance of MiniGPT-4 and BLIP-2 on MME-Cognition by 32.6% and 28.8%, respectively.
arXiv Detail & Related papers (2023-11-02T15:36:12Z) - Evaluating and Explaining Large Language Models for Code Using Syntactic
Structures [74.93762031957883]
This paper introduces ASTxplainer, an explainability method specific to Large Language Models for code.
At its core, ASTxplainer provides an automated method for aligning token predictions with AST nodes.
We perform an empirical evaluation on 12 popular LLMs for code using a curated dataset of the most popular GitHub projects.
arXiv Detail & Related papers (2023-08-07T18:50:57Z) - Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning [92.85265959892115]
This paper introduces the first large and diverse visual instruction tuning dataset, named Large-scale Robust Visual (LRV)-Instruction.
Our dataset comprises 400k visual instructions generated by GPT4, covering 16 vision-and-language tasks with open-ended instructions and answers.
To efficiently measure the hallucination generated by LMMs, we propose GPT4-Assisted Visual Instruction Evaluation (GAVIE), a stable approach to evaluate visual instruction tuning like human experts.
arXiv Detail & Related papers (2023-06-26T10:26:33Z) - e-ViL: A Dataset and Benchmark for Natural Language Explanations in
Vision-Language Tasks [52.918087305406296]
We introduce e-ViL, a benchmark for evaluate explainable vision-language tasks.
We also introduce e-SNLI-VE, the largest existing dataset with NLEs.
We propose a new model that combines UNITER, which learns joint embeddings of images and text, and GPT-2, a pre-trained language model.
arXiv Detail & Related papers (2021-05-08T18:46:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.