Semantic Loss Application to Entity Relation Recognition
- URL: http://arxiv.org/abs/2006.04031v2
- Date: Thu, 17 Sep 2020 22:49:32 GMT
- Title: Semantic Loss Application to Entity Relation Recognition
- Authors: Venkata Sasank Pagolu
- Abstract summary: This paper compares two general approaches for the entity relation recognition.
The main contribution of this paper is an end-to-end neural model for joint entity relation extraction.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Usually, entity relation recognition systems either use a pipe-lined model
that treats the entity tagging and relation identification as separate tasks or
a joint model that simultaneously identifies the relation and entities. This
paper compares these two general approaches for the entity relation
recognition. State-of-the-art entity relation recognition systems are built
using deep recurrent neural networks which often does not capture the symbolic
knowledge or the logical constraints in the problem. The main contribution of
this paper is an end-to-end neural model for joint entity relation extraction
which incorporates a novel loss function. This novel loss function encodes the
constraint information in the problem to guide the model training effectively.
We show that addition of this loss function to the existing typical loss
functions has a positive impact over the performance of the models. This model
is truly end-to-end, requires no feature engineering and easily extensible.
Extensive experimentation has been conducted to evaluate the significance of
capturing symbolic knowledge for natural language understanding. Models using
this loss function are observed to be outperforming their counterparts and
converging faster. Experimental results in this work suggest the use of this
methodology for other language understanding applications.
Related papers
- RESOLVE: Relational Reasoning with Symbolic and Object-Level Features Using Vector Symbolic Processing [1.3049516752695616]
We propose RESOLVE, a neuro-vector symbolic architecture that combines object-level features with relational representations in high-dimensional spaces.
By leveraging this design, the model achieves both low compute latency and memory efficiency.
arXiv Detail & Related papers (2024-11-13T02:17:03Z) - Semantic Loss Functions for Neuro-Symbolic Structured Prediction [74.18322585177832]
We discuss the semantic loss, which injects knowledge about such structure, defined symbolically, into training.
It is agnostic to the arrangement of the symbols, and depends only on the semantics expressed thereby.
It can be combined with both discriminative and generative neural models.
arXiv Detail & Related papers (2024-05-12T22:18:25Z) - Root Causing Prediction Anomalies Using Explainable AI [3.970146574042422]
We present a novel application of explainable AI (XAI) for root-causing performance degradation in machine learning models.
A single feature corruption can cause cascading feature, label and concept drifts.
We have successfully applied this technique to improve the reliability of models used in personalized advertising.
arXiv Detail & Related papers (2024-03-04T19:38:50Z) - Multi-modal Causal Structure Learning and Root Cause Analysis [67.67578590390907]
We propose Mulan, a unified multi-modal causal structure learning method for root cause localization.
We leverage a log-tailored language model to facilitate log representation learning, converting log sequences into time-series data.
We also introduce a novel key performance indicator-aware attention mechanism for assessing modality reliability and co-learning a final causal graph.
arXiv Detail & Related papers (2024-02-04T05:50:38Z) - A Novel Neural-symbolic System under Statistical Relational Learning [50.747658038910565]
We propose a general bi-level probabilistic graphical reasoning framework called GBPGR.
In GBPGR, the results of symbolic reasoning are utilized to refine and correct the predictions made by the deep learning models.
Our approach achieves high performance and exhibits effective generalization in both transductive and inductive tasks.
arXiv Detail & Related papers (2023-09-16T09:15:37Z) - The Relational Bottleneck as an Inductive Bias for Efficient Abstraction [3.19883356005403]
We show that neural networks are constrained via their architecture to focus on relations between perceptual inputs, rather than the attributes of individual inputs.
We review a family of models that employ this approach to induce abstractions in a data-efficient manner.
arXiv Detail & Related papers (2023-09-12T22:44:14Z) - Sparse Relational Reasoning with Object-Centric Representations [78.83747601814669]
We investigate the composability of soft-rules learned by relational neural architectures when operating over object-centric representations.
We find that increasing sparsity, especially on features, improves the performance of some models and leads to simpler relations.
arXiv Detail & Related papers (2022-07-15T14:57:33Z) - On Neural Architecture Inductive Biases for Relational Tasks [76.18938462270503]
We introduce a simple architecture based on similarity-distribution scores which we name Compositional Network generalization (CoRelNet)
We find that simple architectural choices can outperform existing models in out-of-distribution generalizations.
arXiv Detail & Related papers (2022-06-09T16:24:01Z) - Leveraging Unlabeled Data for Entity-Relation Extraction through
Probabilistic Constraint Satisfaction [54.06292969184476]
We study the problem of entity-relation extraction in the presence of symbolic domain knowledge.
Our approach employs semantic loss which captures the precise meaning of a logical sentence.
With a focus on low-data regimes, we show that semantic loss outperforms the baselines by a wide margin.
arXiv Detail & Related papers (2021-03-20T00:16:29Z) - Causal Inference with Deep Causal Graphs [0.0]
Parametric causal modelling techniques rarely provide functionality for counterfactual estimation.
Deep Causal Graphs is an abstract specification of the required functionality for a neural network to model causal distributions.
We demonstrate its expressive power in modelling complex interactions and showcase applications to machine learning explainability and fairness.
arXiv Detail & Related papers (2020-06-15T13:03:33Z) - A logic-based relational learning approach to relation extraction: The
OntoILPER system [0.9176056742068812]
We present OntoILPER, a logic-based relational learning approach to Relation Extraction.
OntoILPER takes profit of a rich relational representation of examples, which can alleviate the drawbacks.
The proposed relational approach seems to be more suitable for Relation Extraction than statistical ones.
arXiv Detail & Related papers (2020-01-13T12:47:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.