Word Alignment-Based Evaluation of Uniform Meaning Representations

• URL: http://arxiv.org/abs/2603.26401v1
• Date: Fri, 27 Mar 2026 13:25:18 GMT
• Title: Word Alignment-Based Evaluation of Uniform Meaning Representations
• Abstract summary: We propose a node-matching algorithm that allows comparison of multiple Uniform Meaning Representations (UMR) of one sentence.<n> sensitivity to word alignment makes the comparison of meaning representations more intuitive and interpretable.
• Score: 0.581757180555022
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Comparison and evaluation of graph-based representations of sentence meaning is a challenge because competing representations of the same sentence may have different number of nodes, and it is not obvious which nodes should be compared to each other. Existing approaches favor node mapping that maximizes $F_1$ score over node relations and attributes, regardless whether the similarity is intentional or accidental; consequently, the identified mismatches in values of node attributes are not useful for any detailed error analysis. We propose a node-matching algorithm that allows comparison of multiple Uniform Meaning Representations (UMR) of one sentence and that takes advantage of node-word alignments, inherently available in UMR. We compare it with previously used approaches, in particular smatch (the de-facto standard in AMR evaluation), and argue that sensitivity to word alignment makes the comparison of meaning representations more intuitive and interpretable, while avoiding the NP-hard search problem inherent in smatch. A script implementing the method is freely available.

Related papers

• Learnable Weighting of Intra-Attribute Distances for Categorical Data Clustering with Nominal and Ordinal Attributes [60.91548718000255]
  We study the intrinsic difference and connection of nominal and ordinal attribute values from a perspective akin to the graph.<n>We propose a novel distance metric to measure the intra-attribute distances of nominal and ordinal attributes in a unified way.<n>We also propose a new clustering algorithm to make the learning of intra-attribute distance weights and partitions of data objects into a single learning paradigm.
  arXiv  Detail & Related papers  (2026-07-06T04:38:39Z)
• A graph-based analysis of semantic types and coercion in contextualized word embeddings [7.260528078468924]
  This paper introduces a graph-based method to examine how lexical and contextual type information is reflected in word embeddings.<n>We select nouns from ten semantic types, annotate corpus instances for type matching, and construct graphs using BERT and sense-enhanced embeddings.<n>Results show that graphs constructed with sense-enhanced embeddings reflect semantic type information better.
  arXiv  Detail & Related papers  (2026-05-22T14:55:54Z)
• Reliable Node Similarity Matrix Guided Contrastive Graph Clustering [51.23437296378319]
  We introduce a new framework, Reliable Node Similarity Matrix Guided Contrastive Graph Clustering (NS4GC) Our method introduces node-neighbor alignment and semantic-aware sparsification, ensuring the node similarity matrix is both accurate and efficiently sparse.
  arXiv  Detail & Related papers  (2024-08-07T13:36:03Z)
• RE-Matching: A Fine-Grained Semantic Matching Method for Zero-Shot Relation Extraction [40.90544879476107]
  We propose a fine-grained semantic matching method tailored for zero-shot relation extraction. We decompose the sentence-level similarity score into entity and context matching scores. Due to the lack of explicit annotations of the redundant components, we design a feature distillation module to adaptively identify the relation-irrelevant features.
  arXiv  Detail & Related papers  (2023-06-08T06:02:34Z)
• Semantic-aware Contrastive Learning for More Accurate Semantic Parsing [32.74456368167872]
  We propose a semantic-aware contrastive learning algorithm, which can learn to distinguish fine-grained meaning representations. Experiments on two standard datasets show that our approach achieves significant improvements over MLE baselines.
  arXiv  Detail & Related papers  (2023-01-19T07:04:32Z)
• Uncertainty-aware Efficient Subgraph Isomorphism using Graph Topology [10.922787879808576]
  Subgraph isomorphism, also known as subgraph matching, is typically regarded as an NP-complete problem.<n>We propose a method for identifying the node correspondence between a subgraph and a full graph in the inexact case without node labels.
  arXiv  Detail & Related papers  (2022-09-15T02:45:05Z)
• Measuring the Interpretability of Unsupervised Representations via Quantized Reverse Probing [97.70862116338554]
  We investigate the problem of measuring interpretability of self-supervised representations. We formulate the latter as estimating the mutual information between the representation and a space of manually labelled concepts. We use our method to evaluate a large number of self-supervised representations, ranking them by interpretability.
  arXiv  Detail & Related papers  (2022-09-07T16:18:50Z)
• Deep Probabilistic Graph Matching [72.6690550634166]
  We propose a deep learning-based graph matching framework that works for the original QAP without compromising on the matching constraints. The proposed method is evaluated on three popularly tested benchmarks (Pascal VOC, Willow Object and SPair-71k) and it outperforms all previous state-of-the-arts on all benchmarks.
  arXiv  Detail & Related papers  (2022-01-05T13:37:27Z)
• DocAMR: Multi-Sentence AMR Representation and Evaluation [19.229112468305267]
  We introduce a simple algorithm for deriving a unified graph representation, avoiding the pitfalls of information loss from over-merging and lack of coherence from under-merging. We also present a pipeline approach combining the top performing AMR and coreference resolution systems, providing a strong baseline for future research.
  arXiv  Detail & Related papers  (2021-12-15T22:38:26Z)
• Wasserstein Distance Regularized Sequence Representation for Text Matching in Asymmetrical Domains [51.91456788949489]
  We propose a novel match method tailored for text matching in asymmetrical domains, called WD-Match. In WD-Match, a Wasserstein distance-based regularizer is defined to regularize the features vectors projected from different domains. The training process of WD-Match amounts to a game that minimizes the matching loss regularized by the Wasserstein distance.
  arXiv  Detail & Related papers  (2020-10-15T12:52:09Z)
• Graph Structured Network for Image-Text Matching [127.68148793548116]
  We present a novel Graph Structured Matching Network to learn fine-grained correspondence. The GSMN explicitly models object, relation and attribute as a structured phrase. Experiments show that GSMN outperforms state-of-the-art methods on benchmarks.
  arXiv  Detail & Related papers  (2020-04-01T08:20:42Z)
• Ranking a set of objects: a graph based least-square approach [70.7866286425868]
  We consider the problem of ranking $N$ objects starting from a set of noisy pairwise comparisons provided by a crowd of equal workers. We propose a class of non-adaptive ranking algorithms that rely on a least-squares intrinsic optimization criterion for the estimation of qualities.
  arXiv  Detail & Related papers  (2020-02-26T16:19:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.