Table Transformers for Imputing Textual Attributes

• URL: http://arxiv.org/abs/2408.02128v2
• Date: Fri, 1 Nov 2024 00:34:19 GMT
• Title: Table Transformers for Imputing Textual Attributes
• Authors: Ting-Ruen Wei, Yuan Wang, Yoshitaka Inoue, Hsin-Tai Wu, Yi Fang,
• Abstract summary: We propose a novel end-to-end approach called Table Transformers for Imputing Textual Attributes (TTITA) Our approach shows competitive performance outperforming baseline models such as recurrent neural networks and Llama2. We incorporate multi-task learning to simultaneously impute for heterogeneous columns, boosting the performance for text imputation.
• Score: 15.823533688884105
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Missing data in tabular dataset is a common issue as the performance of downstream tasks usually depends on the completeness of the training dataset. Previous missing data imputation methods focus on numeric and categorical columns, but we propose a novel end-to-end approach called Table Transformers for Imputing Textual Attributes (TTITA) based on the transformer to impute unstructured textual columns using other columns in the table. We conduct extensive experiments on three datasets, and our approach shows competitive performance outperforming baseline models such as recurrent neural networks and Llama2. The performance improvement is more significant when the target sequence has a longer length. Additionally, we incorporate multi-task learning to simultaneously impute for heterogeneous columns, boosting the performance for text imputation. We also qualitatively compare with ChatGPT for realistic applications.

Related papers

• A Framework for Fine-Tuning LLMs using Heterogeneous Feedback [69.51729152929413]
  We present a framework for fine-tuning large language models (LLMs) using heterogeneous feedback. First, we combine the heterogeneous feedback data into a single supervision format, compatible with methods like SFT and RLHF. Next, given this unified feedback dataset, we extract a high-quality and diverse subset to obtain performance increases.
  arXiv  Detail & Related papers  (2024-08-05T23:20:32Z)
• Text Role Classification in Scientific Charts Using Multimodal Transformers [4.605099852396135]
  Text role classification involves classifying the semantic role of textual elements within scientific charts. We propose to finetune two pretrained multimodal document layout analysis models, LayoutLMv3 and UDOP, on chart datasets. We investigate whether data augmentation and balancing methods help the performance of the models.
  arXiv  Detail & Related papers  (2024-02-08T13:21:44Z)
• Retrieval-Based Transformer for Table Augmentation [14.460363647772745]
  We introduce a novel approach toward automatic data wrangling. We aim to address table augmentation tasks, including row/column population and data imputation. Our model consistently and substantially outperforms both supervised statistical methods and the current state-of-the-art transformer-based models.
  arXiv  Detail & Related papers  (2023-06-20T18:51:21Z)
• Importance of Synthesizing High-quality Data for Text-to-SQL Parsing [71.02856634369174]
  State-of-the-art text-to-weighted algorithms did not further improve on popular benchmarks when trained with augmented synthetic data. We propose a novel framework that incorporates key relationships from schema, imposes strong typing, and schema-weighted column sampling.
  arXiv  Detail & Related papers  (2022-12-17T02:53:21Z)
• Learning Enhanced Representations for Tabular Data via Neighborhood Propagation [24.485479610138498]
  We construct a hypergraph to model the cross-row and cross-column patterns of data instances. We then perform message propagation to enhance the target data instance representation. Experiments on two important data prediction tasks validate the superiority of the proposed PET model.
  arXiv  Detail & Related papers  (2022-06-14T04:24:52Z)
• HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization [57.798070356553936]
  HETFORMER is a Transformer-based pre-trained model with multi-granularity sparse attentions for extractive summarization. Experiments on both single- and multi-document summarization tasks show that HETFORMER achieves state-of-the-art performance in Rouge F1.
  arXiv  Detail & Related papers  (2021-10-12T22:42:31Z)
• Interpretable Feature Construction for Time Series Extrinsic Regression [0.028675177318965035]
  In some application domains, it occurs that the target variable is numerical and the problem is known as time series extrinsic regression (TSER) We suggest an extension of a Bayesian method for robust and interpretable feature construction and selection in the context of TSER. Our approach exploits a relational way to tackle with TSER: (i), we build various and simple representations of the time series which are stored in a relational data scheme, then, (ii), a propositionalisation technique is applied to build interpretable features from secondary tables to "flatten" the data.
  arXiv  Detail & Related papers  (2021-03-15T08:12:19Z)
• Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation [101.26235068460551]
  Models pretrained with self-supervised objectives on large text corpora achieve state-of-the-art performance on English text summarization tasks. Models are typically fine-tuned on hundreds of thousands of data points, an infeasible requirement when applying summarization to new, niche domains. We introduce a novel and generalizable method, called WikiTransfer, for fine-tuning pretrained models for summarization in an unsupervised, dataset-specific manner.
  arXiv  Detail & Related papers  (2020-10-24T08:36:49Z)
• Partially-Aligned Data-to-Text Generation with Distant Supervision [69.15410325679635]
  We propose a new generation task called Partially-Aligned Data-to-Text Generation (PADTG) It is more practical since it utilizes automatically annotated data for training and thus considerably expands the application domains. Our framework outperforms all baseline models as well as verify the feasibility of utilizing partially-aligned data.
  arXiv  Detail & Related papers  (2020-10-03T03:18:52Z)
• GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing [117.98107557103877]
  We present GraPPa, an effective pre-training approach for table semantic parsing. We construct synthetic question-pairs over high-free tables via a synchronous context-free grammar. To maintain the model's ability to represent real-world data, we also include masked language modeling.
  arXiv  Detail & Related papers  (2020-09-29T08:17:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.