CorDEL: A Contrastive Deep Learning Approach for Entity Linkage
- URL: http://arxiv.org/abs/2009.07203v3
- Date: Thu, 3 Dec 2020 00:30:33 GMT
- Title: CorDEL: A Contrastive Deep Learning Approach for Entity Linkage
- Authors: Zhengyang Wang, Bunyamin Sisman, Hao Wei, Xin Luna Dong, Shuiwang Ji
- Abstract summary: Entity linkage (EL) is a critical problem in data cleaning and integration.
With the ever-increasing growth of new data, deep learning (DL) based approaches have been proposed to alleviate the high cost of EL associated with the traditional models.
We argue that the twin-network architecture is sub-optimal to EL, leading to inherent drawbacks of existing models.
- Score: 70.82533554253335
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Entity linkage (EL) is a critical problem in data cleaning and integration.
In the past several decades, EL has typically been done by rule-based systems
or traditional machine learning models with hand-curated features, both of
which heavily depend on manual human inputs. With the ever-increasing growth of
new data, deep learning (DL) based approaches have been proposed to alleviate
the high cost of EL associated with the traditional models. Existing
exploration of DL models for EL strictly follows the well-known twin-network
architecture. However, we argue that the twin-network architecture is
sub-optimal to EL, leading to inherent drawbacks of existing models. In order
to address the drawbacks, we propose a novel and generic contrastive DL
framework for EL. The proposed framework is able to capture both syntactic and
semantic matching signals and pays attention to subtle but critical
differences. Based on the framework, we develop a contrastive DL approach for
EL, called CorDEL, with three powerful variants. We evaluate CorDEL with
extensive experiments conducted on both public benchmark datasets and a
real-world dataset. CorDEL outperforms previous state-of-the-art models by 5.2%
on public benchmark datasets. Moreover, CorDEL yields a 2.4% improvement over
the current best DL model on the real-world dataset, while reducing the number
of training parameters by 97.6%.
Related papers
- Rethinking Data Synthesis: A Teacher Model Training Recipe with Interpretation [8.013158752919722]
Recent advances in large language model (LLM) training have highlighted the need for diverse, high-quality instruction data.
We propose a paradigm shift named textbfNOMAD by investigating how to specifically train models for data generation.
arXiv Detail & Related papers (2024-10-27T07:38:39Z) - Real World Conversational Entity Linking Requires More Than Zeroshots [50.5691094768954]
We design targeted evaluation scenarios to measure the efficacy of EL models under resource constraints.
We assess EL models' ability to generalize to a new unfamiliar KB using Fandom and a novel zero-shot conversational entity linking dataset.
Results indicate that current zero-shot EL models falter when introduced to new, domain-specific KBs without prior training.
arXiv Detail & Related papers (2024-09-02T10:37:53Z) - Numerical Literals in Link Prediction: A Critical Examination of Models and Datasets [2.5999037208435705]
Link Prediction models that incorporate numerical literals have shown minor improvements on existing benchmark datasets.
It is unclear whether a model is actually better in using numerical literals, or better capable of utilizing the graph structure.
We propose a methodology to evaluate LP models that incorporate numerical literals.
arXiv Detail & Related papers (2024-07-25T17:55:33Z) - A Two-Scale Complexity Measure for Deep Learning Models [2.7446241148152257]
We introduce a novel capacity measure 2sED for statistical models based on the effective dimension.
The new quantity provably bounds the generalization error under mild assumptions on the model.
simulations on standard data sets and popular model architectures show that 2sED correlates well with the training error.
arXiv Detail & Related papers (2024-01-17T12:50:50Z) - Universal Domain Adaptation from Foundation Models: A Baseline Study [58.51162198585434]
We make empirical studies of state-of-the-art UniDA methods using foundation models.
We introduce textitCLIP distillation, a parameter-free method specifically designed to distill target knowledge from CLIP models.
Although simple, our method outperforms previous approaches in most benchmark tasks.
arXiv Detail & Related papers (2023-05-18T16:28:29Z) - Dataless Knowledge Fusion by Merging Weights of Language Models [51.8162883997512]
Fine-tuning pre-trained language models has become the prevalent paradigm for building downstream NLP models.
This creates a barrier to fusing knowledge across individual models to yield a better single model.
We propose a dataless knowledge fusion method that merges models in their parameter space.
arXiv Detail & Related papers (2022-12-19T20:46:43Z) - Dynamically-Scaled Deep Canonical Correlation Analysis [77.34726150561087]
Canonical Correlation Analysis (CCA) is a method for feature extraction of two views by finding maximally correlated linear projections of them.
We introduce a novel dynamic scaling method for training an input-dependent canonical correlation model.
arXiv Detail & Related papers (2022-03-23T12:52:49Z) - DSEE: Dually Sparsity-embedded Efficient Tuning of Pre-trained Language
Models [152.29364079385635]
As pre-trained models grow bigger, the fine-tuning process can be time-consuming and computationally expensive.
We propose a framework for resource- and parameter-efficient fine-tuning by leveraging the sparsity prior in both weight updates and the final model weights.
Our proposed framework, dubbed Dually Sparsity-Embedded Efficient Tuning (DSEE), aims to achieve two key objectives: (i) parameter efficient fine-tuning and (ii) resource-efficient inference.
arXiv Detail & Related papers (2021-10-30T03:29:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.