PaECTER: Patent-level Representation Learning using Citation-informed Transformers

• URL: http://arxiv.org/abs/2402.19411v1
• Date: Thu, 29 Feb 2024 18:09:03 GMT
• Title: PaECTER: Patent-level Representation Learning using Citation-informed Transformers
• Authors: Mainak Ghosh, Sebastian Erhardt, Michael E. Rose, Erik Buunk, Dietmar Harhoff
• Abstract summary: PaECTER is a publicly available, open-source document-level encoder specific for patents. We fine-tune BERT for Patents with examiner-added citation information to generate numerical representations for patent documents. PaECTER performs better in similarity tasks than current state-of-the-art models used in the patent domain.
• Score: 0.16785092703248325
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: PaECTER is a publicly available, open-source document-level encoder specific for patents. We fine-tune BERT for Patents with examiner-added citation information to generate numerical representations for patent documents. PaECTER performs better in similarity tasks than current state-of-the-art models used in the patent domain. More specifically, our model outperforms the next-best patent specific pre-trained language model (BERT for Patents) on our patent citation prediction test dataset on two different rank evaluation metrics. PaECTER predicts at least one most similar patent at a rank of 1.32 on average when compared against 25 irrelevant patents. Numerical representations generated by PaECTER from patent text can be used for downstream tasks such as classification, tracing knowledge flows, or semantic similarity search. Semantic similarity search is especially relevant in the context of prior art search for both inventors and patent examiners. PaECTER is available on Hugging Face.

Related papers

• Pap2Pat: Towards Automated Paper-to-Patent Drafting using Chunk-based Outline-guided Generation [13.242188189150987]
  We present PAP2PAT, a new challenging benchmark of 1.8k patent-paper pairs with document outlines. Our experiments with current open-weight LLMs and outline-guided generation show that they can effectively use information from the paper but struggle with repetitions, likely due to the inherent repetitiveness of patent language.
  arXiv  Detail & Related papers  (2024-10-09T15:52:48Z)
• A comparative analysis of embedding models for patent similarity [0.0]
  This paper makes two contributions to the field of text-based patent similarity. It compares the performance of different kinds of patent-specific pretrained embedding models.
  arXiv  Detail & Related papers  (2024-03-25T11:20:23Z)
• Leveraging Large Language Models to Improve REST API Testing [51.284096009803406]
  RESTGPT takes as input an API specification, extracts machine-interpretable rules, and generates example parameter values from natural-language descriptions in the specification. Our evaluations indicate that RESTGPT outperforms existing techniques in both rule extraction and value generation.
  arXiv  Detail & Related papers  (2023-12-01T19:53:23Z)
• Unveiling Black-boxes: Explainable Deep Learning Models for Patent Classification [48.5140223214582]
  State-of-the-art methods for multi-label patent classification rely on deep opaque neural networks (DNNs) We propose a novel deep explainable patent classification framework by introducing layer-wise relevance propagation (LRP) Considering the relevance score, we then generate explanations by visualizing relevant words for the predicted patent class.
  arXiv  Detail & Related papers  (2023-10-31T14:11:37Z)
• Adaptive Taxonomy Learning and Historical Patterns Modelling for Patent Classification [26.85734804493925]
  We propose an integrated framework that comprehensively considers the information on patents for patent classification. We first present an IPC codes correlations learning module to derive their semantic representations. Finally, we combine the contextual information of patent texts that contains the semantics of IPC codes, and assignees' sequential preferences to make predictions.
  arXiv  Detail & Related papers  (2023-08-10T07:02:24Z)
• Event-based Dynamic Graph Representation Learning for Patent Application Trend Prediction [45.0907126466271]
  We propose an event-based graph learning framework for patent application trend prediction. In particular, our method is founded on the memorable representations of both companies and patent classification codes.
  arXiv  Detail & Related papers  (2023-08-04T05:43:32Z)
• A Novel Patent Similarity Measurement Methodology: Semantic Distance and Technological Distance [0.0]
  Patent similarity analysis plays a crucial role in evaluating the risk of patent infringement. Recent advances in natural language processing technology offer a promising avenue for automating this process. We propose a hybrid methodology that takes into account similarity, measures the similarity between patents by considering the semantic similarity of patents.
  arXiv  Detail & Related papers  (2023-03-23T07:55:31Z)
• Enriching Relation Extraction with OpenIE [70.52564277675056]
  Relation extraction (RE) is a sub-discipline of information extraction (IE) In this work, we explore how recent approaches for open information extraction (OpenIE) may help to improve the task of RE. Our experiments over two annotated corpora, KnowledgeNet and FewRel, demonstrate the improved accuracy of our enriched models.
  arXiv  Detail & Related papers  (2022-12-19T11:26:23Z)
• A Survey on Sentence Embedding Models Performance for Patent Analysis [0.0]
  We propose a standard library and dataset for assessing the accuracy of embeddings models based on PatentSBERTa approach. Results show PatentSBERTa, Bert-for-patents, and TF-IDF Weighted Word Embeddings have the best accuracy for computing sentence embeddings at the subclass level.
  arXiv  Detail & Related papers  (2022-04-28T12:04:42Z)
• Fast and Effective Biomedical Entity Linking Using a Dual Encoder [48.86736921025866]
  We propose a BERT-based dual encoder model that resolves multiple mentions in a document in one shot. We show that our proposed model is multiple times faster than existing BERT-based models while being competitive in accuracy for biomedical entity linking.
  arXiv  Detail & Related papers  (2021-03-08T19:32:28Z)
• Few-Shot Named Entity Recognition: A Comprehensive Study [92.40991050806544]
  We investigate three schemes to improve the model generalization ability for few-shot settings. We perform empirical comparisons on 10 public NER datasets with various proportions of labeled data. We create new state-of-the-art results on both few-shot and training-free settings.
  arXiv  Detail & Related papers  (2020-12-29T23:43:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.