R1-RE: Cross-Domain Relation Extraction with RLVR

• URL: http://arxiv.org/abs/2507.04642v2
• Date: Wed, 06 Aug 2025 17:29:40 GMT
• Title: R1-RE: Cross-Domain Relation Extraction with RLVR
• Authors: Runpeng Dai, Tong Zheng, Run Yang, Kaixian Yu, Hongtu Zhu,
• Abstract summary: We introduce R1-RE, the first reinforcement learning with verifiable reward (RLVR) framework for Relation extraction.<n>Our method elicits the reasoning abilities of small language models for annotation tasks, resulting in significantly improved OOD robustness.<n>The R1-RE-7B model attains an average OOD accuracy of approximately 70%, on par with leading proprietary models such as GPT-4o.
• Score: 8.049972902718046
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Relation extraction (RE) is a core task in natural language processing. Traditional approaches typically frame RE as a supervised learning problem, directly mapping context to labels-an approach that often suffers from poor out-of-domain (OOD) generalization. Inspired by the workflow of human annotators, we reframe RE as a reasoning task guided by annotation guidelines and introduce R1-RE, the first reinforcement learning with verifiable reward (RLVR) framework for RE tasks. Our method elicits the reasoning abilities of small language models for annotation tasks, resulting in significantly improved OOD robustness. We evaluate our approach on the public Sem-2010 dataset and a private MDKG dataset. The R1-RE-7B model attains an average OOD accuracy of approximately 70%, on par with leading proprietary models such as GPT-4o. Additionally, our comprehensive analysis provides novel insights into the training dynamics and emergent reasoning behaviors of the RLVR paradigm for RE.

Related papers

• RuleReasoner: Reinforced Rule-based Reasoning via Domain-aware Dynamic Sampling [25.12721060984898]
  Rule-based reasoning has been acknowledged as one of the fundamental problems in reasoning.<n>We introduce Reinforced Rule-based Reasoning, a.k.a. RuleReasoner, a simple yet effective method to conduct rule-based reasoning.<n>Specifically, RuleReasoner resamples each training batch by updating the sampling weights of different domains based on historical rewards.
  arXiv  Detail & Related papers  (2025-06-10T10:31:21Z)
• RewardAnything: Generalizable Principle-Following Reward Models [82.16312590749052]
  Reward models are typically trained on fixed preference datasets.<n>This prevents adaptation to diverse real-world needs-from conciseness in one task to detailed explanations in another.<n>We introduce generalizable, principle-following reward models.<n>We present RewardAnything, a novel RM designed and trained to explicitly follow natural language principles.
  arXiv  Detail & Related papers  (2025-06-04T07:30:16Z)
• Table-R1: Inference-Time Scaling for Table Reasoning [25.481170375825812]
  We develop and evaluate two post-training strategies to enable inference-time scaling.<n>For distillation, we introduce a large-scale dataset of reasoning traces generated by DeepSeek-R1.<n>For RLVR, we propose task-specific verifiable reward functions and apply the GRPO algorithm to obtain the Table-R1-Zero model.
  arXiv  Detail & Related papers  (2025-05-29T16:28:50Z)
• RAG-Zeval: Towards Robust and Interpretable Evaluation on RAG Responses through End-to-End Rule-Guided Reasoning [64.46921169261852]
  RAG-Zeval is a novel end-to-end framework that formulates faithfulness and correctness evaluation as a rule-guided reasoning task.<n>Our approach trains evaluators with reinforcement learning, facilitating compact models to generate comprehensive and sound assessments.<n>Experiments demonstrate RAG-Zeval's superior performance, achieving the strongest correlation with human judgments.
  arXiv  Detail & Related papers  (2025-05-28T14:55:33Z)
• RM-R1: Reward Modeling as Reasoning [81.50471199906738]
  Reasoning Reward Models (ReasRMs) formulate reward modeling as a reasoning task.<n>We propose a reasoning-oriented training pipeline and train a family of ReasRMs, RM-R1.<n>Our models achieve state-of-the-art performance across three reward model benchmarks on average.
  arXiv  Detail & Related papers  (2025-05-05T06:11:12Z)
• Reinforcement Learning for Reasoning in Large Language Models with One Training Example [133.018487956408]
  We show that reinforcement learning with verifiable reward using one training example (1-shot RLVR) is effective in incentivizing the mathematical reasoning capabilities of large language models (LLMs)<n>We identify some interesting phenomena during 1-shot RLVR, including cross-domain generalization, increased frequency of self-reflection, and sustained test performance improvement even after the training accuracy has saturated.
  arXiv  Detail & Related papers  (2025-04-29T09:24:30Z)
• Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains [92.36624674516553]
  Reinforcement learning with verifiable rewards (RLVR) has demonstrated significant success in enhancing mathematical reasoning and coding performance of large language models (LLMs)<n>We investigate the effectiveness and scalability of RLVR across diverse real-world domains including medicine, chemistry, psychology, economics, and education.<n>We utilize a generative scoring technique that yields soft, model-based reward signals to overcome limitations posed by binary verifications.
  arXiv  Detail & Related papers  (2025-03-31T08:22:49Z)
• R-PRM: Reasoning-Driven Process Reward Modeling [53.06844294668382]
  Process Reward Models (PRMs) have emerged as a promising solution by evaluating each reasoning step.<n>Existing PRMs typically output evaluation scores directly, limiting both learning efficiency and evaluation accuracy.<n>We propose Reasoning-Driven Process Reward Modeling (R-PRM)<n>R-PRM generates seed data from limited annotations, effectively bootstrapping our model's reasoning capabilities.
  arXiv  Detail & Related papers  (2025-03-27T09:23:08Z)
• Continual Contrastive Finetuning Improves Low-Resource Relation Extraction [34.76128090845668]
  Relation extraction has been particularly challenging in low-resource scenarios and domains. Recent literature has tackled low-resource RE by self-supervised learning. We propose to pretrain and finetune the RE model using consistent objectives of contrastive learning.
  arXiv  Detail & Related papers  (2022-12-21T07:30:22Z)
• Automatically Generating Counterfactuals for Relation Exaction [18.740447044960796]
  relation extraction (RE) is a fundamental task in natural language processing. Current deep neural models have achieved high accuracy but are easily affected by spurious correlations. We develop a novel approach to derive contextual counterfactuals for entities.
  arXiv  Detail & Related papers  (2022-02-22T04:46:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.