Strikingness-Aware Evaluation for Temporal Knowledge Graph Reasoning

• URL: http://arxiv.org/abs/2605.13153v1
• Date: Wed, 13 May 2026 08:17:54 GMT
• Title: Strikingness-Aware Evaluation for Temporal Knowledge Graph Reasoning
• Abstract summary: We propose a strikingness-aware evaluation framework to quantify event strikingness.<n>Strikingness is then integrated as a weighting factor into metrics like weighted MRR and Hits@k.<n>Our framework provides a more rigorous evaluation, refocusing the field on predicting outstanding events.
• Score: 10.153165023734532
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Temporal Knowledge Graph Reasoning (TKGR) aims at inferring missing (especially future) events from historical data. Current evaluation in TKGR uniformly weights all events, ignoring that most are trivial repetitions, which overestimate the true reasoning ability. Therefore, the rare outstanding events, whose prediction demands deeper reasoning, should be distinguished and emphasized. To this end, we propose a strikingness-aware evaluation framework, which introduces a rule-based strikingness measuring framework (RSMF) to quantify event strikingness by comparing its expected occurrence with peer events derived from temporal rules. Strikingness is then integrated as a weighting factor into metrics like weighted MRR and Hits@k. Experiments on four TKG benchmarks reveal: 1) All representative models perform worse as event strikingness increases, 2) Path-based methods excel on low-strikingness events and representation-based ones on high-strikingness events, 3) We design an ensemble method whose gains stem from fitting trivial events rather than reasoning improvement. Our framework provides a more rigorous evaluation, refocusing the field on predicting outstanding events.

Related papers

• E.M.Ground: A Temporal Grounding Vid-LLM with Holistic Event Perception and Matching [87.38371267983263]
  Temporal Video Grounding aims to precisely localize time segments corresponding to query events.<n>E.M.Ground is a novel Vid-LLM for TVG that focuses on holistic and coherent event perception.<n>E.M.Ground consistently outperforms state-of-the-art Vid-LLMs by significant margins.
  arXiv  Detail & Related papers  (2026-02-05T02:16:00Z)
• Causal Graph based Event Reasoning using Semantic Relation Experts [56.328115024900725]
  We investigate the generation of causal event graphs as a parallel mechanism to help Large Language Models (LLMs) explicitly represent causality during inference.<n>We propose a collaborative approach to causal graph generation where we use LLMs to simulate experts that focus on specific semantic relations.<n>We also introduce a new explainable event prediction task that requires a causal chain of events in the explanation.
  arXiv  Detail & Related papers  (2025-06-07T20:15:45Z)
• Learning under Commission and Omission Event Outliers [10.4442505961159]
  Event stream is an important data format in real life.<n>In this paper, we adopt the temporal point process framework for learning event stream.<n>We provide a simple-but-effective method to deal with both commission and omission event outliers.
  arXiv  Detail & Related papers  (2025-01-23T12:08:21Z)
• AMCEN: An Attention Masking-based Contrastive Event Network for Two-stage Temporal Knowledge Graph Reasoning [29.68279984719722]
  Temporal knowledge graphs (TKGs) can effectively model the ever-evolving nature of real-world knowledge, and their completeness and enhancement can be achieved by reasoning new events from existing ones. However, reasoning accuracy is adversely impacted due to an imbalance between new and recurring events in the datasets. We propose an attention masking-based contrastive event network (AMCEN) with local-global temporal patterns for the two-stage prediction of future events.
  arXiv  Detail & Related papers  (2024-05-16T01:39:50Z)
• Exploring the Limits of Historical Information for Temporal Knowledge Graph Extrapolation [59.417443739208146]
  We propose a new event forecasting model based on a novel training framework of historical contrastive learning. CENET learns both the historical and non-historical dependency to distinguish the most potential entities. We evaluate our proposed model on five benchmark graphs.
  arXiv  Detail & Related papers  (2023-08-29T03:26:38Z)
• Logic and Commonsense-Guided Temporal Knowledge Graph Completion [9.868206060374991]
  A temporal knowledge graph (TKG) stores the events derived from the data involving time. We propose a Logic and Commonsense-Guided Embedding model (LCGE) to jointly learn the time-sensitive representation involving timeliness and causality of events.
  arXiv  Detail & Related papers  (2022-11-30T10:06:55Z)
• An Ordinal Latent Variable Model of Conflict Intensity [59.49424978353101]
  The Goldstein scale is a widely-used expert-based measure that scores events on a conflictual-cooperative scale. This paper takes a latent variable-based approach to measuring conflict intensity.
  arXiv  Detail & Related papers  (2022-10-08T08:59:17Z)
• Improve Event Extraction via Self-Training with Gradient Guidance [10.618929821822892]
  We propose a Self-Training with Feedback (STF) framework to overcome the main factor that hinders the progress of event extraction. STF consists of (1) a base event extraction model trained on existing event annotations and then applied to large-scale unlabeled corpora to predict new event mentions as pseudo training samples, and (2) a novel scoring model that takes in each new predicted event trigger, an argument, its argument role, as well as their paths in the AMR graph to estimate a compatibility score. Experimental results on three benchmark datasets, including ACE05-E, ACE05-E+, and ERE
  arXiv  Detail & Related papers  (2022-05-25T04:40:17Z)
• CEP3: Community Event Prediction with Neural Point Process on Graph [59.434777403325604]
  We propose a novel model combining Graph Neural Networks and Marked Temporal Point Process (MTPP) Our experiments demonstrate the superior performance of our model in terms of both model accuracy and training efficiency.
  arXiv  Detail & Related papers  (2022-05-21T15:30:25Z)
• ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification [74.6318379374801]
  We propose to pre-train a general Correlation-aware context-to-Event Transformer (ClarET) for event-centric reasoning. The proposed ClarET is applicable to a wide range of event-centric reasoning scenarios.
  arXiv  Detail & Related papers  (2022-03-04T10:11:15Z)
• Bad and good errors: value-weighted skill scores in deep ensemble learning [0.0]
  We introduce a strategy for assessing the severity of forecast errors based on the evidence. We introduce a novel definition of confusion matrix and skill scores giving greater importance to the value of the prediction. We show the performances of this approach in the case of three applications concerned with pollution, space weather and stock prize forecasting.
  arXiv  Detail & Related papers  (2021-03-04T08:05:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.