Hybrid Model for Patent Classification using Augmented SBERT and KNN

• URL: http://arxiv.org/abs/2103.11933v1
• Date: Mon, 22 Mar 2021 15:23:19 GMT
• Title: Hybrid Model for Patent Classification using Augmented SBERT and KNN
• Authors: Hamid Bekamiri, Daniel S. Hain, Roman Jurowetzki
• Abstract summary: This study aims to provide a hybrid approach for patent claim classification with Sentence-BERT (SBERT) and K Nearest Neighbours (KNN) The proposed framework predicts individual input patent class and subclass based on finding top k semantic similarity patents.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Purpose: This study aims to provide a hybrid approach for patent claim classification with Sentence-BERT (SBERT) and K Nearest Neighbours (KNN) and explicitly focuses on the patent claims. Patent classification is a multi-label classification task in which the number of labels can be greater than 640 at the subclass level. The proposed framework predicts individual input patent class and subclass based on finding top k semantic similarity patents. Design/Methodology/Approach: The study uses transformer models based on Augmented SBERT and RoBERTa. We use a different approach to predict patent classification by finding top k similar patent claims and using the KNN algorithm to predict patent class or subclass. Besides, in this study, we just focus on patent claims, and in the future study, we add other appropriate parts of patent documents. Findings: The findings suggest the relevance of hybrid models to predict multi-label classification based on text data. In this approach, we used the Transformer model as the distance function in KNN, and proposed a new version of KNN based on Augmented SBERT. Practical Implications: The presented framework provides a practical model for patent classification. In this study, we predict the class and subclass of the patent based on semantic claims similarity. The end-user interpretability of the results is one of the essential positive points of the model. Originality/Value: The main contribution of the study included: 1) Using the Augmented approach for fine-tuning SBERT by in-domain supervised patent claims data. 2) Improving results based on a hybrid model for patent classification. The best result of F1-score at the subclass level was > 69%) Proposing the practical model with high interpretability of results.

Related papers

• ClaimBrush: A Novel Framework for Automated Patent Claim Refinement Based on Large Language Models [3.3427063846107825]
  ClaimBrush is a novel framework for automated patent claim refinement that includes a dataset and a rewriting model. We constructed a dataset for training and evaluating patent claim rewriting models by collecting a large number of actual patent claim rewriting cases. Our proposed rewriting model outperformed baselines and zero-shot learning in state-of-the-art large language models.
  arXiv  Detail & Related papers  (2024-10-08T00:20:54Z)
• A comparative analysis of embedding models for patent similarity [0.0]
  This paper makes two contributions to the field of text-based patent similarity. It compares the performance of different kinds of patent-specific pretrained embedding models.
  arXiv  Detail & Related papers  (2024-03-25T11:20:23Z)
• PaECTER: Patent-level Representation Learning using Citation-informed Transformers [0.16785092703248325]
  PaECTER is a publicly available, open-source document-level encoder specific for patents. We fine-tune BERT for Patents with examiner-added citation information to generate numerical representations for patent documents. PaECTER performs better in similarity tasks than current state-of-the-art models used in the patent domain.
  arXiv  Detail & Related papers  (2024-02-29T18:09:03Z)
• Unveiling Black-boxes: Explainable Deep Learning Models for Patent Classification [48.5140223214582]
  State-of-the-art methods for multi-label patent classification rely on deep opaque neural networks (DNNs) We propose a novel deep explainable patent classification framework by introducing layer-wise relevance propagation (LRP) Considering the relevance score, we then generate explanations by visualizing relevant words for the predicted patent class.
  arXiv  Detail & Related papers  (2023-10-31T14:11:37Z)
• Adaptive Taxonomy Learning and Historical Patterns Modelling for Patent Classification [26.85734804493925]
  We propose an integrated framework that comprehensively considers the information on patents for patent classification. We first present an IPC codes correlations learning module to derive their semantic representations. Finally, we combine the contextual information of patent texts that contains the semantics of IPC codes, and assignees' sequential preferences to make predictions.
  arXiv  Detail & Related papers  (2023-08-10T07:02:24Z)
• Towards Better Certified Segmentation via Diffusion Models [62.21617614504225]
  segmentation models can be vulnerable to adversarial perturbations, which hinders their use in critical-decision systems like healthcare or autonomous driving. Recently, randomized smoothing has been proposed to certify segmentation predictions by adding Gaussian noise to the input to obtain theoretical guarantees. In this paper, we address the problem of certifying segmentation prediction using a combination of randomized smoothing and diffusion models.
  arXiv  Detail & Related papers  (2023-06-16T16:30:39Z)
• Learning Classifiers of Prototypes and Reciprocal Points for Universal Domain Adaptation [79.62038105814658]
  Universal Domain aims to transfer the knowledge between datasets by handling two shifts: domain-shift and categoryshift. Main challenge is correctly distinguishing the unknown target samples while adapting the distribution of known class knowledge from source to target. Most existing methods approach this problem by first training the target adapted known and then relying on the single threshold to distinguish unknown target samples.
  arXiv  Detail & Related papers  (2022-12-16T09:01:57Z)
• Parametric Classification for Generalized Category Discovery: A Baseline Study [70.73212959385387]
  Generalized Category Discovery (GCD) aims to discover novel categories in unlabelled datasets using knowledge learned from labelled samples. We investigate the failure of parametric classifiers, verify the effectiveness of previous design choices when high-quality supervision is available, and identify unreliable pseudo-labels as a key problem. We propose a simple yet effective parametric classification method that benefits from entropy regularisation, achieves state-of-the-art performance on multiple GCD benchmarks and shows strong robustness to unknown class numbers.
  arXiv  Detail & Related papers  (2022-11-21T18:47:11Z)
• MACE: An Efficient Model-Agnostic Framework for Counterfactual Explanation [132.77005365032468]
  We propose a novel framework of Model-Agnostic Counterfactual Explanation (MACE) In our MACE approach, we propose a novel RL-based method for finding good counterfactual examples and a gradient-less descent method for improving proximity. Experiments on public datasets validate the effectiveness with better validity, sparsity and proximity.
  arXiv  Detail & Related papers  (2022-05-31T04:57:06Z)
• A Survey on Sentence Embedding Models Performance for Patent Analysis [0.0]
  We propose a standard library and dataset for assessing the accuracy of embeddings models based on PatentSBERTa approach. Results show PatentSBERTa, Bert-for-patents, and TF-IDF Weighted Word Embeddings have the best accuracy for computing sentence embeddings at the subclass level.
  arXiv  Detail & Related papers  (2022-04-28T12:04:42Z)
• Certified Robustness to Label-Flipping Attacks via Randomized Smoothing [105.91827623768724]
  Machine learning algorithms are susceptible to data poisoning attacks. We present a unifying view of randomized smoothing over arbitrary functions. We propose a new strategy for building classifiers that are pointwise-certifiably robust to general data poisoning attacks.
  arXiv  Detail & Related papers  (2020-02-07T21:28:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.