Arce: Augmented Roberta with Contextualized Elucidations for Ner in Automated Rule Checking

• URL: http://arxiv.org/abs/2508.07286v2
• Date: Wed, 10 Sep 2025 02:25:40 GMT
• Title: Arce: Augmented Roberta with Contextualized Elucidations for Ner in Automated Rule Checking
• Authors: Jian Chen, Jinbao Tian, Yankui Li, Yuqi Lu, Zhou Li,
• Abstract summary: ARCE (augmented RoBERTa with contextualized elucidations) is a novel approach that systematically explores and optimize this generation process.<n>ARCE establishes a new state-of-the-art on a benchmark AEC dataset, achieving a Macro-F1 score of 77.20%.<n>This result also reveals a key finding: simple, explanation-based knowledge proves surprisingly more effective than complex, role-based rationales for this task.
• Score: 5.58730646214246
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate information extraction from specialized texts is a critical challenge, particularly for named entity recognition (NER) in the architecture, engineering, and construction (AEC) domain to support automated rule checking (ARC). The performance of standard pre-trained models is often constrained by the domain gap, as they struggle to interpret the specialized terminology and complex relational contexts inherent in AEC texts. Although this issue can be mitigated by further pre-training on large, human-curated domain corpora, as exemplified by methods like ARCBERT, this approach is both labor-intensive and cost-prohibitive. Consequently, leveraging large language models (LLMs) for automated knowledge generation has emerged as a promising alternative. However, the optimal strategy for generating knowledge that can genuinely enhance smaller, efficient models remains an open question. To address this, we propose ARCE (augmented RoBERTa with contextualized elucidations), a novel approach that systematically explores and optimizes this generation process. ARCE employs an LLM to first generate a corpus of simple, direct explanations, which we term Cote, and then uses this corpus to incrementally pre-train a RoBERTa model prior to its fine-tuning on the downstream task. Our extensive experiments show that ARCE establishes a new state-of-the-art on a benchmark AEC dataset, achieving a Macro-F1 score of 77.20%. This result also reveals a key finding: simple, explanation-based knowledge proves surprisingly more effective than complex, role-based rationales for this task. The code is publicly available at:https://github.com/nxcc-lab/ARCE.

Related papers

• Multi-hop Reasoning via Early Knowledge Alignment [68.28168992785896]
  Early Knowledge Alignment (EKA) aims to align Large Language Models with contextually relevant retrieved knowledge.<n>EKA significantly improves retrieval precision, reduces cascading errors, and enhances both performance and efficiency.<n>EKA proves effective as a versatile, training-free inference strategy that scales seamlessly to large models.
  arXiv  Detail & Related papers  (2025-12-23T08:14:44Z)
• ARC-GEN: A Mimetic Procedural Benchmark Generator for the Abstraction and Reasoning Corpus [3.553493344868413]
  This paper introduces ARC-GEN, an open-source procedural generator aimed at extending the original ARC-AGI training dataset.<n>Unlike prior efforts, our generator is both exhaustive (covering all four-hundred tasks) and mimetic.<n>We also discuss the use of this generator in establishing a static benchmark suite to verify the correctness of programs submitted to the 2025 Google Code Golf Championship.
  arXiv  Detail & Related papers  (2025-10-31T18:10:05Z)
• MARAG-R1: Beyond Single Retriever via Reinforcement-Learned Multi-Tool Agentic Retrieval [50.30107119622642]
  Large Language Models (LLMs) excel at reasoning and generation but are inherently limited by static pretraining data.<n>Retrieval-Augmented Generation (RAG) addresses this issue by grounding LLMs in external knowledge.<n>MarAG-R1 is a reinforcement-learned multi-tool RAG framework that enables LLMs to dynamically coordinate multiple retrieval mechanisms.
  arXiv  Detail & Related papers  (2025-10-31T15:51:39Z)
• Is Implicit Knowledge Enough for LLMs? A RAG Approach for Tree-based Structures [0.5352699766206808]
  Large Language Models (LLMs) are adept at generating responses based on information within their context.<n>Retrieval-Augmented Generation (RAG) retrieves relevant documents to augment the model's in-context learning.<n>We propose a novel method to linearize knowledge from tree-like structures by generating implicit, aggregated summaries at each hierarchical level.
  arXiv  Detail & Related papers  (2025-10-12T20:52:43Z)
• CIR-CoT: Towards Interpretable Composed Image Retrieval via End-to-End Chain-of-Thought Reasoning [93.05917922306196]
  Composed Image Retrieval (CIR) aims to find a target image from a reference image and a modification text.<n>CIR-CoT is the first end-to-end retrieval-oriented MLLM designed to integrate explicit Chain-of-Thought (CoT) reasoning.
  arXiv  Detail & Related papers  (2025-10-09T09:41:45Z)
• Towards Open-World Retrieval-Augmented Generation on Knowledge Graph: A Multi-Agent Collaboration Framework [21.896955284099334]
  Large Language Models (LLMs) have demonstrated strong capabilities in language understanding and reasoning.<n>Retrieval-Augmented Generation (RAG) addresses this limitation by incorporating external knowledge sources.<n>We propose AnchorRAG, a novel multi-agent collaboration framework for open-world RAG without the predefined anchor entities.
  arXiv  Detail & Related papers  (2025-09-01T08:26:12Z)
• LLM-based Triplet Extraction for Automated Ontology Generation in Software Engineering Standards [0.0]
  Software engineering standards (SES) consist of long, unstructured text (with high noise) and paragraphs with domain-specific terms.<n>This work proposes an open-source large language model (LLM)-assisted approach to RTE for SES.
  arXiv  Detail & Related papers  (2025-08-29T17:14:54Z)
• Tree-Based Text Retrieval via Hierarchical Clustering in RAGFrameworks: Application on Taiwanese Regulations [0.0]
  We propose a hierarchical clustering-based retrieval method that eliminates the need to predefine k.<n>Our approach maintains the accuracy and relevance of system responses while adaptively selecting semantically relevant content.<n>Our framework is simple to implement and easily integrates with existing RAG pipelines, making it a practical solution for real-world applications under limited resources.
  arXiv  Detail & Related papers  (2025-06-16T15:34:29Z)
• GenKI: Enhancing Open-Domain Question Answering with Knowledge Integration and Controllable Generation in Large Language Models [75.25348392263676]
  Open-domain question answering (OpenQA) represents a cornerstone in natural language processing (NLP)<n>We propose a novel framework named GenKI, which aims to improve the OpenQA performance by exploring Knowledge Integration and controllable Generation.
  arXiv  Detail & Related papers  (2025-05-26T08:18:33Z)
• Generalising from Self-Produced Data: Model Training Beyond Human Constraints [0.0]
  This paper introduces a novel framework in which AI models autonomously generate and validate new knowledge.<n>Central to this approach is an unbounded, ungamable numeric reward that guides learning without requiring human benchmarks.
  arXiv  Detail & Related papers  (2025-04-07T03:48:02Z)
• New Dataset and Methods for Fine-Grained Compositional Referring Expression Comprehension via Specialist-MLLM Collaboration [49.180693704510006]
  Referring Expression (REC) is a cross-modal task that evaluates the interplay of language understanding, image comprehension, and language-to-image grounding.<n>It serves as an essential testing ground for Multimodal Large Language Models (MLLMs)
  arXiv  Detail & Related papers  (2025-02-27T13:58:44Z)
• Failing Forward: Improving Generative Error Correction for ASR with Synthetic Data and Retrieval Augmentation [73.9145653659403]
  We show that Generative Error Correction models struggle to generalize beyond the specific types of errors encountered during training. We propose DARAG, a novel approach designed to improve GEC for ASR in in-domain (ID) and OOD scenarios. Our approach is simple, scalable, and both domain- and language-agnostic.
  arXiv  Detail & Related papers  (2024-10-17T04:00:29Z)
• Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction [12.455647753787442]
  We propose a three-phase framework named Extract-Define-Canonicalize (EDC) EDC is flexible in that it can be applied to settings where a pre-defined target schema is available and when it is not. We demonstrate EDC is able to extract high-quality triplets without any parameter tuning and with significantly larger schemas compared to prior works.
  arXiv  Detail & Related papers  (2024-04-05T02:53:51Z)
• ProgGen: Generating Named Entity Recognition Datasets Step-by-step with Self-Reflexive Large Language Models [25.68491572293656]
  Large Language Models fall short in structured knowledge extraction tasks such as named entity recognition. This paper explores an innovative, cost-efficient strategy to harness LLMs with modest NER capabilities for producing superior NER datasets.
  arXiv  Detail & Related papers  (2024-03-17T06:12:43Z)
• REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answering [115.72130322143275]
  REAR is a RElevance-Aware Retrieval-augmented approach for open-domain question answering (QA) We develop a novel architecture for LLM-based RAG systems, by incorporating a specially designed assessment module. Experiments on four open-domain QA tasks show that REAR significantly outperforms previous a number of competitive RAG approaches.
  arXiv  Detail & Related papers  (2024-02-27T13:22:51Z)
• Enriching Relation Extraction with OpenIE [70.52564277675056]
  Relation extraction (RE) is a sub-discipline of information extraction (IE) In this work, we explore how recent approaches for open information extraction (OpenIE) may help to improve the task of RE. Our experiments over two annotated corpora, KnowledgeNet and FewRel, demonstrate the improved accuracy of our enriched models.
  arXiv  Detail & Related papers  (2022-12-19T11:26:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.