The GELATO Dataset for Legislative NER

• URL: http://arxiv.org/abs/2603.14130v1
• Date: Sat, 14 Mar 2026 21:32:36 GMT
• Title: The GELATO Dataset for Legislative NER
• Abstract summary: This paper introduces GELATO, a dataset of U.S. House and Senate bills from the 118th Congress.<n>We fine-tune transformer-based models (BERT, RoBERTa) of different architectures and sizes on this dataset for first-level prediction.
• Score: 0.08496348835248901
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper introduces GELATO (Government, Executive, Legislative, and Treaty Ontology), a dataset of U.S. House and Senate bills from the 118th Congress annotated using a novel two-level named entity recognition ontology designed for U.S. legislative texts. We fine-tune transformer-based models (BERT, RoBERTa) of different architectures and sizes on this dataset for first-level prediction. We then use LLMs with optimized prompts to complete the second level prediction. The strong performance of RoBERTa and relatively weak performance of BERT models, as well as the application of LLMs as second-level predictors, support future research in legislative NER or downstream tasks using these model combinations as extraction tools.

Related papers

• ANNOTARES: A Dataset for Extracting Logical Structures from German Statutory Texts [4.897636783430679]
  We introduce the task of identifying and segmenting legal conditions (Tatbestand) and legal consequences (Rechtsfolge) within German statutory texts.<n>To support this task, we present ANNOTARES, a novel dataset comprising German law texts with span-level annotations.
  arXiv  Detail & Related papers  (2026-08-04T16:28:53Z)
• Lexicon-Enriched Graph Modeling for Arabic Document Readability Prediction [0.2333246928962179]
  We present a graph-based approach enriched with lexicons to predict document-level readability in Arabic.<n>Our system models each document as a sentence-level graph, where nodes represent sentences and lemmas.<n>For document-level prediction, sentence-level outputs are aggregated using max pooling to reflect the most difficult sentence.
  arXiv  Detail & Related papers  (2025-09-26T19:29:32Z)
• Graph-Structured Speculative Decoding [52.94367724136063]
  Speculative decoding has emerged as a promising technique to accelerate the inference of Large Language Models. We introduce an innovative approach utilizing a directed acyclic graph (DAG) to manage the drafted hypotheses. We observe a remarkable speedup of 1.73$times$ to 1.96$times$, significantly surpassing standard speculative decoding.
  arXiv  Detail & Related papers  (2024-07-23T06:21:24Z)
• CALRec: Contrastive Alignment of Generative LLMs for Sequential Recommendation [18.986613405565514]
  Large Language Models (LLMs) are pretrained on vast corpora of text for sequential recommendation. We propose a two-stage LLM finetuning framework that finetunes a pretrained LLM in a two-tower fashion using a mixture of two contrastive losses and a language modeling loss. Our model significantly outperforms many state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-05-03T18:51:19Z)
• Exploring Large Language Models and Hierarchical Frameworks for Classification of Large Unstructured Legal Documents [0.6349503549199403]
  We explore the classification of large legal documents and their lack of structural information with a deep-learning-based hierarchical framework. Specifically, we divide a document into parts to extract their embeddings from the last four layers of a custom fine-tuned Large Language Model. Our approach achieves a minimum total performance gain of approximately 2 points over previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-03-11T16:24:08Z)
• Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaboration [52.57055162778548]
  Legal Judgment Prediction (LJP) has become an increasingly crucial task in Legal AI. Precedents are the previous legal cases with similar facts, which are the basis for the judgment of the subsequent case in national legal systems. Recent advances in deep learning have enabled a variety of techniques to be used to solve the LJP task.
  arXiv  Detail & Related papers  (2023-10-13T16:47:20Z)
• The Languini Kitchen: Enabling Language Modelling Research at Different Scales of Compute [66.84421705029624]
  We introduce an experimental protocol that enables model comparisons based on equivalent compute, measured in accelerator hours. We pre-process an existing large, diverse, and high-quality dataset of books that surpasses existing academic benchmarks in quality, diversity, and document length. This work also provides two baseline models: a feed-forward model derived from the GPT-2 architecture and a recurrent model in the form of a novel LSTM with ten-fold throughput.
  arXiv  Detail & Related papers  (2023-09-20T10:31:17Z)
• Evaluating and Explaining Large Language Models for Code Using Syntactic Structures [74.93762031957883]
  This paper introduces ASTxplainer, an explainability method specific to Large Language Models for code. At its core, ASTxplainer provides an automated method for aligning token predictions with AST nodes. We perform an empirical evaluation on 12 popular LLMs for code using a curated dataset of the most popular GitHub projects.
  arXiv  Detail & Related papers  (2023-08-07T18:50:57Z)
• A Comparative Study of Transformers on Word Sense Disambiguation [0.0]
  This paper presents a comparative study on the contextualization power of neural network-based embedding systems. We evaluate their contextualization power using two sample Word Sense Disambiguation (WSD) tasks, SensEval-2 and SensEval-3. Experimental results show that the proposed techniques also achieve superior results over the current state-of-the-art on both the WSD tasks.
  arXiv  Detail & Related papers  (2021-11-30T14:10:22Z)
• Few-Shot Named Entity Recognition: A Comprehensive Study [92.40991050806544]
  We investigate three schemes to improve the model generalization ability for few-shot settings. We perform empirical comparisons on 10 public NER datasets with various proportions of labeled data. We create new state-of-the-art results on both few-shot and training-free settings.
  arXiv  Detail & Related papers  (2020-12-29T23:43:16Z)
• CodeBERT: A Pre-Trained Model for Programming and Natural Languages [117.34242908773061]
  CodeBERT is a pre-trained model for programming language (PL) and nat-ural language (NL) We develop CodeBERT with Transformer-based neural architecture. We evaluate CodeBERT on two NL-PL applications by fine-tuning model parameters.
  arXiv  Detail & Related papers  (2020-02-19T13:09:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.