CorDEL: A Contrastive Deep Learning Approach for Entity Linkage
- URL: http://arxiv.org/abs/2009.07203v3
- Date: Thu, 3 Dec 2020 00:30:33 GMT
- Title: CorDEL: A Contrastive Deep Learning Approach for Entity Linkage
- Authors: Zhengyang Wang, Bunyamin Sisman, Hao Wei, Xin Luna Dong, Shuiwang Ji
- Abstract summary: Entity linkage (EL) is a critical problem in data cleaning and integration.
With the ever-increasing growth of new data, deep learning (DL) based approaches have been proposed to alleviate the high cost of EL associated with the traditional models.
We argue that the twin-network architecture is sub-optimal to EL, leading to inherent drawbacks of existing models.
- Score: 70.82533554253335
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Entity linkage (EL) is a critical problem in data cleaning and integration.
In the past several decades, EL has typically been done by rule-based systems
or traditional machine learning models with hand-curated features, both of
which heavily depend on manual human inputs. With the ever-increasing growth of
new data, deep learning (DL) based approaches have been proposed to alleviate
the high cost of EL associated with the traditional models. Existing
exploration of DL models for EL strictly follows the well-known twin-network
architecture. However, we argue that the twin-network architecture is
sub-optimal to EL, leading to inherent drawbacks of existing models. In order
to address the drawbacks, we propose a novel and generic contrastive DL
framework for EL. The proposed framework is able to capture both syntactic and
semantic matching signals and pays attention to subtle but critical
differences. Based on the framework, we develop a contrastive DL approach for
EL, called CorDEL, with three powerful variants. We evaluate CorDEL with
extensive experiments conducted on both public benchmark datasets and a
real-world dataset. CorDEL outperforms previous state-of-the-art models by 5.2%
on public benchmark datasets. Moreover, CorDEL yields a 2.4% improvement over
the current best DL model on the real-world dataset, while reducing the number
of training parameters by 97.6%.
Related papers
- Numerical Literals in Link Prediction: A Critical Examination of Models and Datasets [2.5999037208435705]
Link Prediction models that incorporate numerical literals have shown minor improvements on existing benchmark datasets.
It is unclear whether a model is actually better in using numerical literals, or better capable of utilizing the graph structure.
We propose a methodology to evaluate LP models that incorporate numerical literals.
arXiv Detail & Related papers (2024-07-25T17:55:33Z) - A Two-Scale Complexity Measure for Deep Learning Models [2.7446241148152257]
We introduce a novel capacity measure 2sED for statistical models based on the effective dimension.
The new quantity provably bounds the generalization error under mild assumptions on the model.
simulations on standard data sets and popular model architectures show that 2sED correlates well with the training error.
arXiv Detail & Related papers (2024-01-17T12:50:50Z) - Federated Learning with Projected Trajectory Regularization [65.6266768678291]
Federated learning enables joint training of machine learning models from distributed clients without sharing their local data.
One key challenge in federated learning is to handle non-identically distributed data across the clients.
We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
arXiv Detail & Related papers (2023-12-22T02:12:08Z) - Universal Domain Adaptation from Foundation Models: A Baseline Study [58.51162198585434]
We make empirical studies of state-of-the-art UniDA methods using foundation models.
We introduce textitCLIP distillation, a parameter-free method specifically designed to distill target knowledge from CLIP models.
Although simple, our method outperforms previous approaches in most benchmark tasks.
arXiv Detail & Related papers (2023-05-18T16:28:29Z) - Dataless Knowledge Fusion by Merging Weights of Language Models [51.8162883997512]
Fine-tuning pre-trained language models has become the prevalent paradigm for building downstream NLP models.
This creates a barrier to fusing knowledge across individual models to yield a better single model.
We propose a dataless knowledge fusion method that merges models in their parameter space.
arXiv Detail & Related papers (2022-12-19T20:46:43Z) - Dynamically-Scaled Deep Canonical Correlation Analysis [77.34726150561087]
Canonical Correlation Analysis (CCA) is a method for feature extraction of two views by finding maximally correlated linear projections of them.
We introduce a novel dynamic scaling method for training an input-dependent canonical correlation model.
arXiv Detail & Related papers (2022-03-23T12:52:49Z) - DSEE: Dually Sparsity-embedded Efficient Tuning of Pre-trained Language
Models [152.29364079385635]
As pre-trained models grow bigger, the fine-tuning process can be time-consuming and computationally expensive.
We propose a framework for resource- and parameter-efficient fine-tuning by leveraging the sparsity prior in both weight updates and the final model weights.
Our proposed framework, dubbed Dually Sparsity-Embedded Efficient Tuning (DSEE), aims to achieve two key objectives: (i) parameter efficient fine-tuning and (ii) resource-efficient inference.
arXiv Detail & Related papers (2021-10-30T03:29:47Z) - End-to-End Weak Supervision [15.125993628007972]
We propose an end-to-end approach for directly learning the downstream model.
We show improved performance over prior work in terms of end model performance on downstream test sets.
arXiv Detail & Related papers (2021-07-05T19:10:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.