ChronosLex: Time-aware Incremental Training for Temporal Generalization of Legal Classification Tasks

• URL: http://arxiv.org/abs/2405.14211v1
• Date: Thu, 23 May 2024 06:09:16 GMT
• Title: ChronosLex: Time-aware Incremental Training for Temporal Generalization of Legal Classification Tasks
• Authors: T. Y. S. S Santosh, Tuan-Quang Vuong, Matthias Grabmair,
• Abstract summary: This study investigates the challenges posed by the dynamic nature of legal multi-label text classification tasks, where legal concepts evolve over time. Existing models often overlook the temporal dimension in their training process, leading to suboptimal performance of those models over time. We introduce ChronosLex, an incremental training paradigm that trains models on chronological splits, preserving the temporal order of the data.
• Score: 1.3723120574076126
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study investigates the challenges posed by the dynamic nature of legal multi-label text classification tasks, where legal concepts evolve over time. Existing models often overlook the temporal dimension in their training process, leading to suboptimal performance of those models over time, as they treat training data as a single homogeneous block. To address this, we introduce ChronosLex, an incremental training paradigm that trains models on chronological splits, preserving the temporal order of the data. However, this incremental approach raises concerns about overfitting to recent data, prompting an assessment of mitigation strategies using continual learning and temporal invariant methods. Our experimental results over six legal multi-label text classification datasets reveal that continual learning methods prove effective in preventing overfitting thereby enhancing temporal generalizability, while temporal invariant methods struggle to capture these dynamics of temporal shifts.

Related papers

• Understanding the Limits of Deep Tabular Methods with Temporal Shift [28.738848567072004]
  We introduce a plug-and-play temporal embedding method based on Fourier series expansion to learn and incorporate temporal patterns. Our experiments demonstrate that this temporal embedding, combined with the improved training protocol, provides a more effective and robust framework for learning from temporal data.
  arXiv  Detail & Related papers  (2025-02-27T16:48:53Z)
• CSTA: Spatial-Temporal Causal Adaptive Learning for Exemplar-Free Video Class-Incremental Learning [62.69917996026769]
  A class-incremental learning task requires learning and preserving both spatial appearance and temporal action involvement. We propose a framework that equips separate adapters to learn new class patterns, accommodating the incremental information requirements unique to each class. A causal compensation mechanism is proposed to reduce the conflicts during increment and memorization for between different types of information.
  arXiv  Detail & Related papers  (2025-01-13T11:34:55Z)
• On the Regularization of Learnable Embeddings for Time Series Forecasting [18.069747511100132]
  We investigate methods to regularize the learning of local learnable embeddings for time series processing. We show that methods attempting to prevent the co-adaptation of local and global parameters by means of embeddings perturbation are particularly effective in this context.
  arXiv  Detail & Related papers  (2024-10-18T17:30:20Z)
• Self-Supervised Learning of Disentangled Representations for Multivariate Time-Series [10.99576829280084]
  TimeDRL is a framework for multivariate time-series representation learning with dual-level disentangled embeddings. TimeDRL features: (i) timestamp-level and instance-level embeddings using a [] token strategy; (ii) timestamp-predictive and instance-contrastive tasks for representation learning; and (iii) avoidance of augmentation methods to eliminate inductive biases. Experiments on forecasting and classification datasets show TimeDRL outperforms existing methods, with further validation in semi-supervised settings with limited labeled data.
  arXiv  Detail & Related papers  (2024-10-16T14:24:44Z)
• Temporal-Difference Variational Continual Learning [89.32940051152782]
  A crucial capability of Machine Learning models in real-world applications is the ability to continuously learn new tasks. In Continual Learning settings, models often struggle to balance learning new tasks with retaining previous knowledge. We propose new learning objectives that integrate the regularization effects of multiple previous posterior estimations.
  arXiv  Detail & Related papers  (2024-10-10T10:58:41Z)
• Contrastive Learning Is Not Optimal for Quasiperiodic Time Series [4.2807943283312095]
  We introduce Distilled Embedding for Almost-Periodic Time Series (DEAPS) in this paper. DEAPS is a non-contrastive method tailored for quasiperiodic time series, such as electrocardiogram (ECG) data. We demonstrate a notable improvement of +10% over existing SOTA methods when just a few annotated records are presented to fit a Machine Learning (ML) model.
  arXiv  Detail & Related papers  (2024-07-24T08:02:41Z)
• Learning Discriminative Spatio-temporal Representations for Semi-supervised Action Recognition [23.44320273156057]
  We propose an Adaptive Contrastive Learning(ACL) strategy and a Multi-scale Temporal Learning(MTL) strategy. ACL strategy assesses the confidence of all unlabeled samples by the class prototypes of the labeled data, and adaptively selects positive-negative samples from a pseudo-labeled sample bank to construct contrastive learning. MTL strategy could highlight informative semantics from long-term clips and integrate them into the short-term clip while suppressing noisy information.
  arXiv  Detail & Related papers  (2024-04-25T08:49:08Z)
• TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling [67.02157180089573]
  Time series pre-training has recently garnered wide attention for its potential to reduce labeling expenses and benefit various downstream tasks. This paper proposes TimeSiam as a simple but effective self-supervised pre-training framework for Time series based on Siamese networks.
  arXiv  Detail & Related papers  (2024-02-04T13:10:51Z)
• Unsupervised Temporal Action Localization via Self-paced Incremental Learning [57.55765505856969]
  We present a novel self-paced incremental learning model to enhance clustering and localization training simultaneously. We design two (constant- and variable- speed) incremental instance learning strategies for easy-to-hard model training, thus ensuring the reliability of these video pseudolabels.
  arXiv  Detail & Related papers  (2023-12-12T16:00:55Z)
• Time-series Generation by Contrastive Imitation [87.51882102248395]
  We study a generative framework that seeks to combine the strengths of both: Motivated by a moment-matching objective to mitigate compounding error, we optimize a local (but forward-looking) transition policy. At inference, the learned policy serves as the generator for iterative sampling, and the learned energy serves as a trajectory-level measure for evaluating sample quality.
  arXiv  Detail & Related papers  (2023-11-02T16:45:25Z)
• Revisiting the Temporal Modeling in Spatio-Temporal Predictive Learning under A Unified View [73.73667848619343]
  We introduce USTEP (Unified S-TEmporal Predictive learning), an innovative framework that reconciles the recurrent-based and recurrent-free methods by integrating both micro-temporal and macro-temporal scales.
  arXiv  Detail & Related papers  (2023-10-09T16:17:42Z)
• Prompting-based Temporal Domain Generalization [10.377683220196873]
  This paper presents a novel prompting-based approach to temporal domain generalization. Our method adapts a trained model to temporal drift by learning global prompts, domain-specific prompts, and drift-aware prompts. Experiments on classification, regression, and time series forecasting tasks demonstrate the generality of the proposed approach.
  arXiv  Detail & Related papers  (2023-10-03T22:40:56Z)
• Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
  Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information. We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
  arXiv  Detail & Related papers  (2023-09-15T17:10:51Z)
• Training for the Future: A Simple Gradient Interpolation Loss to Generalize Along Time [26.261277497201565]
  In several real world applications, machine learning models are deployed to make predictions on data whose distribution changes gradually along time. We propose a simple method that starts with a model with time-sensitive parameters but regularizes its temporal complexity using a Gradient Interpolation (GI) loss.
  arXiv  Detail & Related papers  (2021-08-15T11:20:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.