PatenTEB: A Comprehensive Benchmark and Model Family for Patent Text Embedding

• URL: http://arxiv.org/abs/2510.22264v1
• Date: Sat, 25 Oct 2025 12:01:46 GMT
• Title: PatenTEB: A Comprehensive Benchmark and Model Family for Patent Text Embedding
• Abstract summary: Patent text embeddings enable prior art search, technology landscaping, and patent analysis, yet existing benchmarks inadequately capture patent-specific challenges.<n>We introduce PatenTEB, a comprehensive benchmark comprising 15 tasks across retrieval, classification, paraphrase, and clustering, with 2.06 million examples.<n>We develop the patembed model family through multi-task training, spanning 67M to 344M parameters with context lengths up to 4096 tokens.
• Score: 1.0742675209112622
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Patent text embeddings enable prior art search, technology landscaping, and patent analysis, yet existing benchmarks inadequately capture patent-specific challenges. We introduce PatenTEB, a comprehensive benchmark comprising 15 tasks across retrieval, classification, paraphrase, and clustering, with 2.06 million examples. PatenTEB employs domain-stratified splits, domain specific hard negative mining, and systematic coverage of asymmetric fragment-to-document matching scenarios absent from general embedding benchmarks. We develop the patembed model family through multi-task training, spanning 67M to 344M parameters with context lengths up to 4096 tokens. External validation shows strong generalization: patembed-base achieves state-of-the-art on MTEB BigPatentClustering.v2 (0.494 V-measure vs. 0.445 previous best), while patembed-large achieves 0.377 NDCG@100 on DAPFAM. Systematic ablations reveal that multi-task training improves external generalization despite minor benchmark costs, and that domain-pretrained initialization provides consistent advantages across task families. All resources will be made available at https://github.com/iliass-y/patenteb. Keywords: patent retrieval, sentence embeddings, multi-task learning, asymmetric retrieval, benchmark evaluation, contrastive learning.

Related papers

• MultAttnAttrib: Training-Free Multimodal Attribution in Long Document Question Answering [73.47440561592556]
  We introduce MultAttnAttrib, a training-free attribution-generation method.<n>We show that MultAttnAttrib consistently outperforms a variety of attribution-generation methods.<n>Our method produces attributions at up to one-seventh of the direct inference latency compared to prompting.
  arXiv  Detail & Related papers  (2026-07-01T19:29:19Z)
• Arena-T2I Hard: Benchmarking and Improving Faithfulness with Dependency-Aware Checklist [69.81453995715673]
  Existing faithfulness benchmarks rely on simple atomic instructions.<n>We introduce Arena-T2I Hard, a 310-prompt stress benchmark drawn from real arena T2I logs.<n>We propose a dependency-aware checklist reward that decomposes each prompt into a DAG of yes/no questions and zeroes descendants of failed parents.
  arXiv  Detail & Related papers  (2026-06-30T14:17:05Z)
• TTPrint: Evidence-Grounded TTP Extraction via Diverge-then-Converge Verification [12.412754988937644]
  Extracting MITRE ATT&CK techniques from cyber threat intelligence (CTI) reports is an open-set, multi-label problem.<n>Existing methods lack generalizability across diverse attack descriptions.<n>We propose TTPrint, which addresses this challenge through a diverge-then-converge design inspired by how human analysts work.
  arXiv  Detail & Related papers  (2026-05-25T13:31:49Z)
• TAME: Test-Time Adversarial Prompt Tuning via Mixture-of-Experts for Vision-Language Models [76.92890872338235]
  Large-scale pre-trained Vision-Language models (VLMs) exhibit strong zero-shot generalization, yet are highly vulnerable to imperceptible adversarial perturbations.<n>To enhance robustness without requiring downstream task-specific retraining, we propose TAME, a novel test-time defense.
  arXiv  Detail & Related papers  (2026-05-17T18:07:08Z)
• BIT.UA-AAUBS at ArchEHR-QA 2026: Evaluating Open-Source and Proprietary LLMs via Prompting in Low-Resource QA [65.22695574492265]
  This paper presents the joint participation of the BIT.UA and AAUBS groups in the ArchEHR-QA 2026 shared task.<n>Due to the absence of training data and the strict data privacy constraints inherent to the healthcare domain, we investigate the capabilities of Large Language Models (LLMs) without weight updates.<n>We evaluate several state-of-the-art proprietary models and locally deployable open-source alternatives using various prompt engineering strategies.<n>Our results demonstrate that while proprietary models exhibit strong resilience to prompt variations, domain-adapted open-source models (such as MedGemma 3 27B) achieve highly competitive
  arXiv  Detail & Related papers  (2026-05-05T10:43:56Z)
• Citation-Driven Multi-View Training for Patent Embeddings: QaECTER and Sophia-Bench [1.7299359373305296]
  We introduce Sophiabench, a large-scale patent retrieval benchmark.<n>We also introduce QaECTER, a compact embedding model trained on patent citation graphs.<n>Both benchmarks are designed for practical deployment in large-scale patent search systems.
  arXiv  Detail & Related papers  (2026-04-24T12:18:00Z)
• Beyond Raw Detection Scores: Markov-Informed Calibration for Boosting Machine-Generated Text Detection [105.14032334647932]
  Machine-generated texts (MGTs) pose risks such as disinformation and phishing, highlighting the need for reliable detection.<n> Metric-based methods, which extract statistically distinguishable features of MGTs, are often more practical than complex model-based methods that are prone to overfitting.<n>We propose a Markov-informed score calibration strategy that models two relationships of context detection scores that may aid calibration.
  arXiv  Detail & Related papers  (2026-02-08T16:06:12Z)
• PILOT-Bench: A Benchmark for Legal Reasoning in the Patent Domain with IRAC-Aligned Classification Tasks [5.777437566146122]
  We introduce PILOT-Bench, the first PTAB-centric benchmark that aligns PTAB decisions with USPTO patent data at the case-level.<n>We evaluate a diverse set of closed-source (commercial) and open-source LLMs and conduct analyses across multiple perspectives.<n> PILOT-Bench establishes a foundation for the systematic evaluation of patent-domain legal reasoning.
  arXiv  Detail & Related papers  (2026-01-08T09:26:05Z)
• Towards Automated Quality Assurance of Patent Specifications: A Multi-Dimensional LLM Framework [0.5534764384104999]
  Although AI drafting tools have gained prominence in patent writing, the systematic evaluation of AI-generated patent content quality represents a significant research gap.<n>We propose to evaluate patents using regulatory compliance, technical coherence, and figure-reference consistency detection modules.<n>The framework is validated on a comprehensive dataset comprising 80 human-authored and 80 AI-generated patents from two patent drafting tools.
  arXiv  Detail & Related papers  (2025-10-29T11:20:18Z)
• UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAG [82.84014669683863]
  Multimodal retrieval-augmented generation (MM-RAG) is a key approach for applying large language models to real-world knowledge bases.<n>UniDoc-Bench is the first large-scale, realistic benchmark for MM-RAG built from 70k real-world PDF pages.<n>Our experiments show that multimodal text-image fusion RAG systems consistently outperform both unimodal and jointly multimodal embedding-based retrieval.
  arXiv  Detail & Related papers  (2025-10-04T04:30:13Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• DetectAnyLLM: Towards Generalizable and Robust Detection of Machine-Generated Text Across Domains and Models [60.713908578319256]
  We propose Direct Discrepancy Learning (DDL) to optimize the detector with task-oriented knowledge.<n>Built upon this, we introduce DetectAnyLLM, a unified detection framework that achieves state-of-the-art MGTD performance.<n>MIRAGE samples human-written texts from 10 corpora across 5 text-domains, which are then re-generated or revised using 17 cutting-edge LLMs.
  arXiv  Detail & Related papers  (2025-09-15T10:59:57Z)
• DAPFAM: A Domain-Aware Family-level Dataset to benchmark cross domain patent retrieval [1.1852406625172218]
  We introduce DAPFAM, a family-level benchmark with explicit IN-domain and OUT-domain partitions defined by a new IPC3 overlap scheme.<n>The dataset contains 1,247 query families and 45,336 target families aggregated at the family level to reduce international redundancy.<n>We conduct 249 controlled experiments spanning lexical (BM25) and dense (transformer) backends, document and passage level retrieval, multiple query and document representations, aggregation strategies, and hybrid fusion.
  arXiv  Detail & Related papers  (2025-06-27T11:34:51Z)
• PatentMind: A Multi-Aspect Reasoning Graph for Patent Similarity Evaluation [35.13558856456741]
  Patent similarity evaluation plays a critical role in intellectual property analysis.<n>We introduce PatentMind, a novel framework for patent similarity assessment based on a Multi-Aspect Reasoning Graph (MARG)<n>Our framework provides a structured and semantically grounded foundation for real-world decision-making.
  arXiv  Detail & Related papers  (2025-05-25T22:28:27Z)
• OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning [72.57452266982642]
  OCRBench v2 is a large-scale bilingual text-centric benchmark.<n>It covers 31 diverse scenarios, 10,000 human-verified question-answering pairs, and thorough evaluation metrics.<n>We find that most LMMs score below 50 (100 in total) and suffer from five-type limitations.
  arXiv  Detail & Related papers  (2024-12-31T07:32:35Z)
• AutoPatent: A Multi-Agent Framework for Automatic Patent Generation [16.862811929856313]
  We introduce a novel and practical task known as Draft2Patent, along with its corresponding D2P benchmark, which challenges Large Language Models to generate full-length patents averaging 17K tokens based on initial drafts.<n>We propose a multi-agent framework called AutoPatent which leverages the LLM-based planner agent, writer agents, and examiner agent with PGTree and RRAG to generate lengthy, intricate, and high-quality complete patent documents.
  arXiv  Detail & Related papers  (2024-12-13T02:27:34Z)
• Revisiting BPR: A Replicability Study of a Common Recommender System Baseline [78.00363373925758]
  We study the features of the BPR model, indicating their impact on its performance, and investigate open-source BPR implementations. Our analysis reveals inconsistencies between these implementations and the original BPR paper, leading to a significant decrease in performance of up to 50% for specific implementations. We show that the BPR model can achieve performance levels close to state-of-the-art methods on the top-n recommendation tasks and even outperform them on specific datasets.
  arXiv  Detail & Related papers  (2024-09-21T18:39:53Z)
• PaECTER: Patent-level Representation Learning using Citation-informed Transformers [0.1957338076370071]
  PaECTER is an open-source document-level encoder specific for patents.<n>We fine-tune BERT for Patents with examiner-added citation information to generate numerical representations for patent documents.<n>PaECTER performs better in similarity tasks than current state-of-the-art models used in the patent domain.
  arXiv  Detail & Related papers  (2024-02-29T18:09:03Z)
• DPIC: Decoupling Prompt and Intrinsic Characteristics for LLM Generated Text Detection [56.513637720967566]
  Large language models (LLMs) can generate texts that pose risks of misuse, such as plagiarism, planting fake reviews on e-commerce platforms, or creating inflammatory false tweets. Existing high-quality detection methods usually require access to the interior of the model to extract the intrinsic characteristics. We propose to extract deep intrinsic characteristics of the black-box model generated texts.
  arXiv  Detail & Related papers  (2023-05-21T17:26:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.