Visualization in the Era of Artificial Intelligence: Experiments for Creating Structural Visualizations by Prompting Large Language Models

• URL: http://arxiv.org/abs/2305.03380v2
• Date: Thu, 11 May 2023 09:21:46 GMT
• Title: Visualization in the Era of Artificial Intelligence: Experiments for Creating Structural Visualizations by Prompting Large Language Models
• Authors: Hans-Georg Fill, Fabian Muff
• Abstract summary: Large Language Models (LLMs) have revolutionized natural language processing by generating human-like text and images from textual input. We report initial experiments showing that LLMs can generate 2D/3D visualizations that may be used for legal visualization.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) have revolutionized natural language processing by generating human-like text and images from textual input. However, their potential to generate complex 2D/3D visualizations has been largely unexplored. We report initial experiments showing that LLMs can generate 2D/3D visualizations that may be used for legal visualization. Further research is needed for complex 2D visualizations and 3D scenes. LLMs can become a powerful tool for many industries and applications, generating complex visualizations with minimal training.

Related papers

• Story3D-Agent: Exploring 3D Storytelling Visualization with Large Language Models [57.30913211264333]
  We present Story3D-Agent, a pioneering approach that transforms provided narratives into 3D-rendered visualizations. By integrating procedural modeling, our approach enables precise control over multi-character actions and motions, as well as diverse decorative elements. We have thoroughly evaluated our Story3D-Agent to validate its effectiveness, offering a basic framework to advance 3D story representation.
  arXiv  Detail & Related papers  (2024-08-21T17:43:15Z)
• Unlocking Textual and Visual Wisdom: Open-Vocabulary 3D Object Detection Enhanced by Comprehensive Guidance from Text and Image [70.02187124865627]
  Open-vocabulary 3D object detection (OV-3DDet) aims to localize and recognize both seen and previously unseen object categories within any new 3D scene. We leverage a vision foundation model to provide image-wise guidance for discovering novel classes in 3D scenes. We demonstrate significant improvements in accuracy and generalization, highlighting the potential of foundation models in advancing open-vocabulary 3D object detection.
  arXiv  Detail & Related papers  (2024-07-07T04:50:04Z)
• When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models [113.18524940863841]
  This survey provides a comprehensive overview of the methodologies enabling large language models to process, understand, and generate 3D data. Our investigation spans various 3D data representations, from point clouds to Neural Radiance Fields (NeRFs) It examines their integration with LLMs for tasks such as 3D scene understanding, captioning, question-answering, and dialogue.
  arXiv  Detail & Related papers  (2024-05-16T16:59:58Z)
• Re-Thinking Inverse Graphics With Large Language Models [51.333105116400205]
  Inverse graphics -- inverting an image into physical variables that, when rendered, enable reproduction of the observed scene -- is a fundamental challenge in computer vision and graphics. We propose the Inverse-Graphics Large Language Model (IG-LLM), an inversegraphics framework centered around an LLM. We incorporate a frozen pre-trained visual encoder and a continuous numeric head to enable end-to-end training.
  arXiv  Detail & Related papers  (2024-04-23T16:59:02Z)
• A Vision Check-up for Language Models [61.852026871772914]
  We show how a preliminary visual representation learning system can be trained using models of text. Experiments on self-supervised visual representation learning highlight the potential to train vision models capable of making semantic assessments of natural images.
  arXiv  Detail & Related papers  (2024-01-03T18:09:33Z)
• An Embodied Generalist Agent in 3D World [67.16935110789528]
  We introduce LEO, an embodied multi-modal generalist agent that excels in perceiving, grounding, reasoning, planning, and acting in the 3D world. We collect large-scale datasets comprising diverse object-level and scene-level tasks, which require considerable understanding of and interaction with the 3D world. Through extensive experiments, we demonstrate LEO's remarkable proficiency across a wide spectrum of tasks, including 3D captioning, question answering, embodied reasoning, navigation and manipulation.
  arXiv  Detail & Related papers  (2023-11-18T01:21:38Z)
• Towards Language-guided Interactive 3D Generation: LLMs as Layout Interpreter with Generative Feedback [20.151147653552155]
  Large Language Models (LLMs) have demonstrated impressive reasoning, conversational, and zero-shot generation abilities. We propose a novel language-guided interactive 3D generation system, dubbed LI3D, that integrates LLMs as a 3D layout interpreter. Our system also incorporates LLaVA, a large language and vision assistant, to provide generative feedback from the visual aspect for improving the visual quality of generated content.
  arXiv  Detail & Related papers  (2023-05-25T07:43:39Z)
• WildRefer: 3D Object Localization in Large-scale Dynamic Scenes with Multi-modal Visual Data and Natural Language [31.691159120136064]
  We introduce the task of 3D visual grounding in large-scale dynamic scenes based on natural linguistic descriptions and online captured multi-modal visual data. We present a novel method, dubbed WildRefer, for this task by fully utilizing the rich appearance information in images, the position and geometric clues in point cloud. Our datasets are significant for the research of 3D visual grounding in the wild and has huge potential to boost the development of autonomous driving and service robots.
  arXiv  Detail & Related papers  (2023-04-12T06:48:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.