Transformer-Based Temporal Information Extraction and Application: A Review

• URL: http://arxiv.org/abs/2504.07470v1
• Date: Thu, 10 Apr 2025 05:48:24 GMT
• Title: Transformer-Based Temporal Information Extraction and Application: A Review
• Authors: Xin Su, Phillip Howard, Steven Bethard,
• Abstract summary: Temporal information extraction (IE) aims to extract structured temporal information from unstructured text.<n>This paper systematically summarizing and analyzing the body of work on temporal IE using Transformers.
• Score: 11.34995915076939
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Temporal information extraction (IE) aims to extract structured temporal information from unstructured text, thereby uncovering the implicit timelines within. This technique is applied across domains such as healthcare, newswire, and intelligence analysis, aiding models in these areas to perform temporal reasoning and enabling human users to grasp the temporal structure of text. Transformer-based pre-trained language models have produced revolutionary advancements in natural language processing, demonstrating exceptional performance across a multitude of tasks. Despite the achievements garnered by Transformer-based approaches in temporal IE, there is a lack of comprehensive reviews on these endeavors. In this paper, we aim to bridge this gap by systematically summarizing and analyzing the body of work on temporal IE using Transformers while highlighting potential future research directions.

Related papers

• Unify and Anchor: A Context-Aware Transformer for Cross-Domain Time Series Forecasting [26.59526791215]
  We identify two key challenges in cross-domain time series forecasting: the complexity of temporal patterns and semantic misalignment.<n>We propose the Unify and Anchor" transfer paradigm, which disentangles frequency components for a unified perspective.<n>We introduce ContexTST, a Transformer-based model that employs a time series coordinator for structured representation.
  arXiv  Detail & Related papers  (2025-03-03T04:11:14Z)
• LLMTemporalComparator: A Tool for Analysing Differences in Temporal Adaptations of Large Language Models [17.021220773165016]
  This study addresses the challenges of analyzing temporal discrepancies in large language models (LLMs) trained on data from different time periods. We propose a novel system that compares in a systematic way the outputs of two LLM versions based on user-defined queries.
  arXiv  Detail & Related papers  (2024-10-05T15:17:07Z)
• Analysis of Plan-based Retrieval for Grounded Text Generation [78.89478272104739]
  hallucinations occur when a language model is given a generation task outside its parametric knowledge. A common strategy to address this limitation is to infuse the language models with retrieval mechanisms. We analyze how planning can be used to guide retrieval to further reduce the frequency of hallucinations.
  arXiv  Detail & Related papers  (2024-08-20T02:19:35Z)
• State-Space Modeling in Long Sequence Processing: A Survey on Recurrence in the Transformer Era [59.279784235147254]
  This survey provides an in-depth summary of the latest approaches that are based on recurrent models for sequential data processing. The emerging picture suggests that there is room for thinking of novel routes, constituted by learning algorithms which depart from the standard Backpropagation Through Time.
  arXiv  Detail & Related papers  (2024-06-13T12:51:22Z)
• Towards Effective Time-Aware Language Representation: Exploring Enhanced Temporal Understanding in Language Models [24.784375155633427]
  We introduce BiTimeBERT 2.0, a novel time-aware language model pre-trained on a temporal news article collection.<n>BiTimeBERT 2.0 incorporates temporal information through three innovative pre-training objectives.<n>We show BiTimeBERT 2.0 achieves substantial improvements across a broad range of time-related tasks and excels on datasets spanning extensive temporal ranges.
  arXiv  Detail & Related papers  (2024-06-04T00:30:37Z)
• Explaining Text Similarity in Transformer Models [52.571158418102584]
  Recent advances in explainable AI have made it possible to mitigate limitations by leveraging improved explanations for Transformers. We use BiLRP, an extension developed for computing second-order explanations in bilinear similarity models, to investigate which feature interactions drive similarity in NLP models. Our findings contribute to a deeper understanding of different semantic similarity tasks and models, highlighting how novel explainable AI methods enable in-depth analyses and corpus-level insights.
  arXiv  Detail & Related papers  (2024-05-10T17:11:31Z)
• Analysis of the Evolution of Advanced Transformer-Based Language Models: Experiments on Opinion Mining [0.5735035463793008]
  This paper studies the behaviour of the cutting-edge Transformer-based language models on opinion mining. Our comparative study shows leads and paves the way for production engineers regarding the approach to focus on.
  arXiv  Detail & Related papers  (2023-08-07T01:10:50Z)
• Multivariate Time Series characterization and forecasting of VoIP traffic in real mobile networks [9.637582917616703]
  Predicting the behavior of real-time traffic (e.g., VoIP) in mobility scenarios could help the operators to better plan their network infrastructures. This work proposes a forecasting analysis of crucial/QoE descriptors of VoIP traffic in a real mobile environment.
  arXiv  Detail & Related papers  (2023-07-13T09:21:39Z)
• Temporal Attention for Language Models [24.34396762188068]
  We extend the key component of the transformer architecture, i.e., the self-attention mechanism, and propose temporal attention. temporal attention can be applied to any transformer model and requires the input texts to be accompanied with their relevant time points. We leverage these representations for the task of semantic change detection. Our proposed model achieves state-of-the-art results on all the datasets.
  arXiv  Detail & Related papers  (2022-02-04T11:55:34Z)
• Automated Machine Learning Techniques for Data Streams [91.3755431537592]
  This paper surveys the state-of-the-art open-source AutoML tools, applies them to data collected from streams, and measures how their performance changes over time. The results show that off-the-shelf AutoML tools can provide satisfactory results but in the presence of concept drift, detection or adaptation techniques have to be applied to maintain the predictive accuracy over time.
  arXiv  Detail & Related papers  (2021-06-14T11:42:46Z)
• Pretrained Transformers for Text Ranking: BERT and Beyond [53.83210899683987]
  This survey provides an overview of text ranking with neural network architectures known as transformers. The combination of transformers and self-supervised pretraining has been responsible for a paradigm shift in natural language processing.
  arXiv  Detail & Related papers  (2020-10-13T15:20:32Z)
• Language Guided Networks for Cross-modal Moment Retrieval [66.49445903955777]
  Cross-modal moment retrieval aims to localize a temporal segment from an untrimmed video described by a natural language query. Existing methods independently extract the features of videos and sentences. We present Language Guided Networks (LGN), a new framework that leverages the sentence embedding to guide the whole process of moment retrieval.
  arXiv  Detail & Related papers  (2020-06-18T12:08:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.