GenPlot: Increasing the Scale and Diversity of Chart Derendering Data

• URL: http://arxiv.org/abs/2306.11699v1
• Date: Tue, 20 Jun 2023 17:25:53 GMT
• Title: GenPlot: Increasing the Scale and Diversity of Chart Derendering Data
• Authors: Brendan Artley
• Abstract summary: We propose GenPlot, a plot generator that can generate billions of additional plots for chart-derendering using synthetic data. OCR-free chart-to-text translation has achieved state-of-the-art results on visual language tasks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vertical bars, horizontal bars, dot, scatter, and line plots provide a diverse set of visualizations to represent data. To understand these plots, one must be able to recognize textual components, locate data points in a plot, and process diverse visual contexts to extract information. In recent works such as Pix2Struct, Matcha, and Deplot, OCR-free chart-to-text translation has achieved state-of-the-art results on visual language tasks. These results outline the importance of chart-derendering as a pre-training objective, yet existing datasets provide a fixed set of training examples. In this paper, we propose GenPlot; a plot generator that can generate billions of additional plots for chart-derendering using synthetic data.

Related papers

• Text2Chart31: Instruction Tuning for Chart Generation with Automatic Feedback [37.275533538711436]
  We propose a hierarchical pipeline and a new dataset for chart generation. Our dataset, Text2Chart31, includes 31 unique plot types referring to the Matplotlib library. We introduce a reinforcement learning-based instruction tuning technique for chart generation tasks without requiring human feedback.
  arXiv  Detail & Related papers  (2024-10-05T07:25:56Z)
• ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning [54.89249749894061]
  ChartAssistant is a vision-language model for universal chart comprehension and reasoning. It undergoes a two-stage training process, starting with pre-training on chart-to-table parsing to align chart and text. Experimental results demonstrate significant performance gains over the state-of-the-art UniChart and Chartllama method.
  arXiv  Detail & Related papers  (2024-01-04T17:51:48Z)
• ChartLlama: A Multimodal LLM for Chart Understanding and Generation [70.1393163657813]
  We create a high-quality instruction-tuning dataset leveraging GPT-4. Next, we introduce ChartLlama, a multi-modal large language model that we've trained using our created dataset.
  arXiv  Detail & Related papers  (2023-11-27T15:20:23Z)
• StructChart: Perception, Structuring, Reasoning for Visual Chart Understanding [58.38480335579541]
  Current chart-related tasks focus on either chart perception which refers to extracting information from the visual charts, or performing reasoning given the extracted data. In this paper, we aim to establish a unified and label-efficient learning paradigm for joint perception and reasoning tasks. Experiments are conducted on various chart-related tasks, demonstrating the effectiveness and promising potential for a unified chart perception-reasoning paradigm.
  arXiv  Detail & Related papers  (2023-09-20T12:51:13Z)
• UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoning [29.947053208614246]
  We present UniChart, a pretrained model for chart comprehension and reasoning. UniChart encodes the relevant text, data, and visual elements of charts and then uses a chart-grounded text decoder to generate the expected output in natural language. We propose several chart-specific pretraining tasks that include: (i) low-level tasks to extract the visual elements (e.g., bars, lines) and data from charts, and (ii) high-level tasks to acquire chart understanding and reasoning skills.
  arXiv  Detail & Related papers  (2023-05-24T06:11:17Z)
• ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules [89.75395046894809]
  We present ChartReader, a unified framework that seamlessly integrates chart derendering and comprehension tasks. Our approach includes a transformer-based chart component detection module and an extended pre-trained vision-language model for chart-to-X tasks. Our proposed framework can significantly reduce the manual effort involved in chart analysis, providing a step towards a universal chart understanding model.
  arXiv  Detail & Related papers  (2023-04-05T00:25:27Z)
• DePlot: One-shot visual language reasoning by plot-to-table translation [50.28850068391312]
  This paper presents the first one-shot solution to visual language reasoning. A modality conversion module, named as DePlot, translates the image of a plot or chart to a linearized table. The output of DePlot can then be directly used to prompt a pretrained large language model.
  arXiv  Detail & Related papers  (2022-12-20T18:20:50Z)
• VizExtract: Automatic Relation Extraction from Data Visualizations [7.2241069295727955]
  This paper presents a framework for automatically extracting compared variables from statistical charts. We leverage a computer vision based framework to automatically identify and localize visualization facets in line graphs, scatter plots, or bar graphs. In controlled experiments, our framework is able to classify, with 87.5% accuracy, the correlation between variables for graphs with 1-3 series per graph, varying colors, and solid line styles.
  arXiv  Detail & Related papers  (2021-12-07T04:27:08Z)
• Learning to Generate Scene Graph from Natural Language Supervision [52.18175340725455]
  We propose one of the first methods that learn from image-sentence pairs to extract a graphical representation of localized objects and their relationships within an image, known as scene graph. We leverage an off-the-shelf object detector to identify and localize object instances, match labels of detected regions to concepts parsed from captions, and thus create "pseudo" labels for learning scene graph.
  arXiv  Detail & Related papers  (2021-09-06T03:38:52Z)
• Tensor Fields for Data Extraction from Chart Images: Bar Charts and Scatter Plots [0.0]
  Automated chart reading involves data extraction and contextual understanding of the data from chart images. We identify an appropriate tensor field as the model and propose a methodology for the use of its degenerate point extraction for data extraction from chart images. Our results show that tensor voting is effective for data extraction from bar charts and scatter plots, and histograms, as a special case of bar charts.
  arXiv  Detail & Related papers  (2020-10-05T20:19:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.