LineFormer: Rethinking Line Chart Data Extraction as Instance
Segmentation
- URL: http://arxiv.org/abs/2305.01837v1
- Date: Wed, 3 May 2023 00:38:24 GMT
- Title: LineFormer: Rethinking Line Chart Data Extraction as Instance
Segmentation
- Authors: Jay Lal, Aditya Mitkari, Mahesh Bhosale, David Doermann
- Abstract summary: LineFormer is a robust approach to line data extraction using instance segmentation.
We achieve state-of-the-art performance on several benchmark synthetic and real chart datasets.
- Score: 0.07521357985302306
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Data extraction from line-chart images is an essential component of the
automated document understanding process, as line charts are a ubiquitous data
visualization format. However, the amount of visual and structural variations
in multi-line graphs makes them particularly challenging for automated parsing.
Existing works, however, are not robust to all these variations, either taking
an all-chart unified approach or relying on auxiliary information such as
legends for line data extraction. In this work, we propose LineFormer, a robust
approach to line data extraction using instance segmentation. We achieve
state-of-the-art performance on several benchmark synthetic and real chart
datasets. Our implementation is available at
https://github.com/TheJaeLal/LineFormer .
Related papers
- ChartAssisstant: A Universal Chart Multimodal Language Model via
Chart-to-Table Pre-training and Multitask Instruction Tuning [54.89249749894061]
ChartAssistant is a vision-language model for universal chart comprehension and reasoning.
It undergoes a two-stage training process, starting with pre-training on chart-to-table parsing to align chart and text.
Experimental results demonstrate significant performance gains over the state-of-the-art UniChart and Chartllama method.
arXiv Detail & Related papers (2024-01-04T17:51:48Z) - ChartLlama: A Multimodal LLM for Chart Understanding and Generation [70.1393163657813]
We create a high-quality instruction-tuning dataset leveraging GPT-4.
Next, we introduce ChartLlama, a multi-modal large language model that we've trained using our created dataset.
arXiv Detail & Related papers (2023-11-27T15:20:23Z) - StructChart: Perception, Structuring, Reasoning for Visual Chart
Understanding [58.38480335579541]
Current chart-related tasks focus on either chart perception which refers to extracting information from the visual charts, or performing reasoning given the extracted data.
In this paper, we aim to establish a unified and label-efficient learning paradigm for joint perception and reasoning tasks.
Experiments are conducted on various chart-related tasks, demonstrating the effectiveness and promising potential for a unified chart perception-reasoning paradigm.
arXiv Detail & Related papers (2023-09-20T12:51:13Z) - GenPlot: Increasing the Scale and Diversity of Chart Derendering Data [0.0]
We propose GenPlot, a plot generator that can generate billions of additional plots for chart-derendering using synthetic data.
OCR-free chart-to-text translation has achieved state-of-the-art results on visual language tasks.
arXiv Detail & Related papers (2023-06-20T17:25:53Z) - ChartReader: A Unified Framework for Chart Derendering and Comprehension
without Heuristic Rules [89.75395046894809]
We present ChartReader, a unified framework that seamlessly integrates chart derendering and comprehension tasks.
Our approach includes a transformer-based chart component detection module and an extended pre-trained vision-language model for chart-to-X tasks.
Our proposed framework can significantly reduce the manual effort involved in chart analysis, providing a step towards a universal chart understanding model.
arXiv Detail & Related papers (2023-04-05T00:25:27Z) - Instance Segmentation Based Graph Extraction for Handwritten Circuit
Diagram Images [4.365209337828563]
This paper describes an approach for extracting both the electrical components (including their terminals and describing texts) by the means of instance segmentation and keypoint extraction.
The resulting graph extraction process consists of a simple two-step process of model inference and trivial geometric keypoint matching.
arXiv Detail & Related papers (2023-01-09T03:00:20Z) - Multi-task Self-distillation for Graph-based Semi-Supervised Learning [6.277952154365413]
We propose a multi-task self-distillation framework that injects self-supervised learning and self-distillation into graph convolutional networks.
First, we formulate a self-supervision pipeline based on pre-text tasks to capture different levels of similarities in graphs.
Second, self-distillation uses soft labels of the model itself as additional supervision.
arXiv Detail & Related papers (2021-12-02T12:43:41Z) - SOLD2: Self-supervised Occlusion-aware Line Description and Detection [95.8719432775724]
We introduce the first joint detection and description of line segments in a single deep network.
Our method does not require any annotated line labels and can therefore generalize to any dataset.
We evaluate our approach against previous line detection and description methods on several multi-view datasets.
arXiv Detail & Related papers (2021-04-07T19:27:17Z) - Tensor Fields for Data Extraction from Chart Images: Bar Charts and
Scatter Plots [0.0]
Automated chart reading involves data extraction and contextual understanding of the data from chart images.
We identify an appropriate tensor field as the model and propose a methodology for the use of its degenerate point extraction for data extraction from chart images.
Our results show that tensor voting is effective for data extraction from bar charts and scatter plots, and histograms, as a special case of bar charts.
arXiv Detail & Related papers (2020-10-05T20:19:40Z) - Partially-Aligned Data-to-Text Generation with Distant Supervision [69.15410325679635]
We propose a new generation task called Partially-Aligned Data-to-Text Generation (PADTG)
It is more practical since it utilizes automatically annotated data for training and thus considerably expands the application domains.
Our framework outperforms all baseline models as well as verify the feasibility of utilizing partially-aligned data.
arXiv Detail & Related papers (2020-10-03T03:18:52Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.