CMNIE: An Information Extraction Benchmark for Chinese Military News

• URL: http://arxiv.org/abs/2609.10722v1
• Date: Wed, 09 Sep 2026 18:16:08 GMT
• Title: CMNIE: An Information Extraction Benchmark for Chinese Military News
• Abstract summary: We present CMNIE, an information extraction benchmark for Chinese military news.<n>The dataset contains 13,000 instances collected from public Chinese military news.<n> CMNIE remains chal?lenging, especially for relation extraction and exact matching of event?argument spans.
• Score: 6.838012562057879
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Structured extraction from Chinese military news supports intelligence analysis, decision-making, and knowledge base construction. However, existing resources provide limited support for joint informa?tion extraction in this domain, especially when events, event arguments, entities, and relations must be modeled together. We present CMNIE, an information extraction benchmark for Chinese military news. Extend?ing military-domain resources beyond document-level event annotations, CMNIE jointly annotates event triggers, event arguments, named enti?ties, and entity relations under a unified domain schema. The dataset contains 13,000 instances collected from public Chinese military news, with manual annotations for 7 event types, 10 argument roles, 7 entity types, and 8 relation types. We evaluate supervised IE models, zero-shot large language models, and fine-tuned LLM-based extraction methods on a shared test set. Experimental results show that CMNIE remains chal?lenging, especially for relation extraction and exact matching of event?argument spans; zero-shot LLMs often identify relevant semantic units but fail to match gold span boundaries exactly. CMNIE provides a stan?dardized benchmark for studying schema adherence, exact span match?ing, and joint structured extraction in specialized Chinese news.

Related papers

• MODF-SIR: A Multi-agent Omni-modal Distilled Framework for Social Intelligence Reasoning [80.41987492855971]
  We propose a multi-agent collaborative framework built upon a lightweight Multimodal Large Language Model (MLLM)<n>Within this architecture, multi-modal data pertinent to social intelligence is precisely localized.<n>With around 30% of training data from IntentTrain, we achieve state-of-the-art results.
  arXiv  Detail & Related papers  (2026-06-10T12:44:02Z)
• Beyond Isolated Dots: Benchmarking Structured Table Construction as Deep Knowledge Extraction [80.88654868264645]
  Arranged and Organized Extraction Benchmark designed to evaluate ability of large language models to comprehend fragmented documents.<n>AOE includes 11 carefully crafted tasks across three diverse domains, requiring models to generate context-specific schema tailored to varied input queries.<n>Results show that even the most advanced models struggled significantly.
  arXiv  Detail & Related papers  (2025-07-22T06:37:51Z)
• Less is More: Making Smaller Language Models Competent Subgraph Retrievers for Multi-hop KGQA [51.3033125256716]
  We model the subgraph retrieval task as a conditional generation task handled by small language models. Our base generative subgraph retrieval model, consisting of only 220M parameters, competitive retrieval performance compared to state-of-the-art models. Our largest 3B model, when plugged with an LLM reader, sets new SOTA end-to-end performance on both the WebQSP and CWQ benchmarks.
  arXiv  Detail & Related papers  (2024-10-08T15:22:36Z)
• Multi-Grained Query-Guided Set Prediction Network for Grounded Multimodal Named Entity Recognition [9.506482334842293]
  Grounded Multimodal Named Entity Recognition (GMNER) is an emerging information extraction (IE) task.<n>Recent unified methods employing machine reading comprehension or sequence generation-based frameworks show limitations in this difficult task.<n>We propose a novel unified framework named Multi-grained Query-guided Set Prediction Network (MQSPN) to learn appropriate relationships at intra-entity and inter-entity levels.
  arXiv  Detail & Related papers  (2024-07-17T05:42:43Z)
• MultiADE: A Multi-domain Benchmark for Adverse Drug Event Extraction [11.458594744457521]
  Active adverse event surveillance monitors Adverse Drug Events (ADE) from different data sources. Most datasets or shared tasks focus on extracting ADEs from a particular type of text. Domain generalisation - the ability of a machine learning model to perform well on new, unseen domains (text types) - is under-explored. We build a benchmark for adverse drug event extraction, which we named MultiADE.
  arXiv  Detail & Related papers  (2024-05-28T09:57:28Z)
• CMNEE: A Large-Scale Document-Level Event Extraction Dataset based on Open-Source Chinese Military News [4.8309547228489125]
  We propose CMNEE, a large-scale, document-level open-source Chinese Military News Event Extraction dataset. It contains 17,000 documents and 29,223 events, which are all manually annotated based on a pre-defined schema for the military domain. We reproduce several state-of-the-art event extraction models with a systematic evaluation.
  arXiv  Detail & Related papers  (2024-04-18T15:02:35Z)
• Rules still work for Open Information Extraction [0.0]
  This paper presents an innovative open information extraction model, APRCOIE, tailored for Chinese text.<n>To train the model, we manually annotated a large-scale Chinese OIE dataset.<n>In the comparative evaluation, we demonstrate that APRCOIE outperforms state-of-the-art Chinese OIE models.
  arXiv  Detail & Related papers  (2024-03-16T01:40:36Z)
• Enriching Relation Extraction with OpenIE [70.52564277675056]
  Relation extraction (RE) is a sub-discipline of information extraction (IE) In this work, we explore how recent approaches for open information extraction (OpenIE) may help to improve the task of RE. Our experiments over two annotated corpora, KnowledgeNet and FewRel, demonstrate the improved accuracy of our enriched models.
  arXiv  Detail & Related papers  (2022-12-19T11:26:23Z)
• A Closer Look at Debiased Temporal Sentence Grounding in Videos: Dataset, Metric, and Approach [53.727460222955266]
  Temporal Sentence Grounding in Videos (TSGV) aims to ground a natural language sentence in an untrimmed video. Recent studies have found that current benchmark datasets may have obvious moment annotation biases. We introduce a new evaluation metric "dR@n,IoU@m" that discounts the basic recall scores to alleviate the inflating evaluation caused by biased datasets.
  arXiv  Detail & Related papers  (2022-03-10T08:58:18Z)
• Syntactic-GCN Bert based Chinese Event Extraction [2.3104000011280403]
  We propose an integrated framework to perform Chinese event extraction. The proposed approach is a multiple channel input neural framework that integrates semantic features and syntactic features. Experimental results show that the proposed method outperforms the benchmark approaches significantly.
  arXiv  Detail & Related papers  (2021-12-18T14:07:54Z)
• GATE: Graph Attention Transformer Encoder for Cross-lingual Relation and Event Extraction [107.8262586956778]
  We introduce graph convolutional networks (GCNs) with universal dependency parses to learn language-agnostic sentence representations. GCNs struggle to model words with long-range dependencies or are not directly connected in the dependency tree. We propose to utilize the self-attention mechanism to learn the dependencies between words with different syntactic distances.
  arXiv  Detail & Related papers  (2020-10-06T20:30:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.