Large Language Model Informed Patent Image Retrieval

• URL: http://arxiv.org/abs/2404.19360v1
• Date: Tue, 30 Apr 2024 08:45:16 GMT
• Title: Large Language Model Informed Patent Image Retrieval
• Authors: Hao-Cheng Lo, Jung-Mei Chu, Jieh Hsiang, Chun-Chieh Cho,
• Abstract summary: We propose a language-informed, distribution-aware multimodal approach to patent image feature learning. Our proposed method achieves state-of-the-art or comparable performance in image-based patent retrieval with mAP +53.3%, Recall@10 +41.8%, and MRR@10 +51.9%.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In patent prosecution, image-based retrieval systems for identifying similarities between current patent images and prior art are pivotal to ensure the novelty and non-obviousness of patent applications. Despite their growing popularity in recent years, existing attempts, while effective at recognizing images within the same patent, fail to deliver practical value due to their limited generalizability in retrieving relevant prior art. Moreover, this task inherently involves the challenges posed by the abstract visual features of patent images, the skewed distribution of image classifications, and the semantic information of image descriptions. Therefore, we propose a language-informed, distribution-aware multimodal approach to patent image feature learning, which enriches the semantic understanding of patent image by integrating Large Language Models and improves the performance of underrepresented classes with our proposed distribution-aware contrastive losses. Extensive experiments on DeepPatent2 dataset show that our proposed method achieves state-of-the-art or comparable performance in image-based patent retrieval with mAP +53.3%, Recall@10 +41.8%, and MRR@10 +51.9%. Furthermore, through an in-depth user analysis, we explore our model in aiding patent professionals in their image retrieval efforts, highlighting the model's real-world applicability and effectiveness.

Related papers

• Towards Effective User Attribution for Latent Diffusion Models via Watermark-Informed Blending [54.26862913139299]
  We introduce a novel framework Towards Effective user Attribution for latent diffusion models via Watermark-Informed Blending (TEAWIB) TEAWIB incorporates a unique ready-to-use configuration approach that allows seamless integration of user-specific watermarks into generative models. Experiments validate the effectiveness of TEAWIB, showcasing the state-of-the-art performance in perceptual quality and attribution accuracy.
  arXiv  Detail & Related papers  (2024-09-17T07:52:09Z)
• Knowledge Fused Recognition: Fusing Hierarchical Knowledge for Image Recognition through Quantitative Relativity Modeling and Deep Metric Learning [18.534970504136254]
  We propose a novel deep metric learning based method to fuse hierarchical prior knowledge about image classes. Existing deep metric learning incorporated image classification mainly exploits qualitative relativity between image classes. A new triplet loss function term that exploits quantitative relativity and aligns distances in model latent space with those in knowledge space is also proposed and incorporated in the proposed dual-modality fusion method.
  arXiv  Detail & Related papers  (2024-07-30T07:24:33Z)
• InstructPatentGPT: Training patent language models to follow instructions with human feedback [0.9790236766474201]
  This research aims to increase the likelihood for a language model to generate patent claims that have a higher chance of being granted. To showcase the controllability of the language model, the system learns from granted patents and pre-grant applications with different rewards.
  arXiv  Detail & Related papers  (2024-05-25T11:48:50Z)
• A Dataset and Benchmark for Copyright Infringement Unlearning from Text-to-Image Diffusion Models [52.49582606341111]
  Copyright law confers creators the exclusive rights to reproduce, distribute, and monetize their creative works. Recent progress in text-to-image generation has introduced formidable challenges to copyright enforcement. We introduce a novel pipeline that harmonizes CLIP, ChatGPT, and diffusion models to curate a dataset.
  arXiv  Detail & Related papers  (2024-01-04T11:14:01Z)
• Introspective Deep Metric Learning [91.47907685364036]
  We propose an introspective deep metric learning framework for uncertainty-aware comparisons of images. The proposed IDML framework improves the performance of deep metric learning through uncertainty modeling.
  arXiv  Detail & Related papers  (2023-09-11T16:21:13Z)
• Learning Efficient Representations for Image-Based Patent Retrieval [16.323708969088557]
  We present a simple and lightweight model for content-based patent retrieval. Our approach significantly outperforms other counterparts on a large-scale benchmark. Our model can be elaborately scaled up to achieve a surprisingly high mAP of 93.5%.
  arXiv  Detail & Related papers  (2023-08-26T03:19:14Z)
• Classification of Visualization Types and Perspectives in Patents [9.123089032348311]
  We adopt state-of-the-art deep learning methods for the classification of visualization types and perspectives in patent images. We derive a set of hierarchical classes from a dataset that provides weakly-labeled data for image perspectives.
  arXiv  Detail & Related papers  (2023-07-19T21:45:07Z)
• Learning from Multi-Perception Features for Real-Word Image Super-resolution [87.71135803794519]
  We propose a novel SR method called MPF-Net that leverages multiple perceptual features of input images. Our method incorporates a Multi-Perception Feature Extraction (MPFE) module to extract diverse perceptual information. We also introduce a contrastive regularization term (CR) that improves the model's learning capability.
  arXiv  Detail & Related papers  (2023-05-26T07:35:49Z)
• A Survey on Sentence Embedding Models Performance for Patent Analysis [0.0]
  We propose a standard library and dataset for assessing the accuracy of embeddings models based on PatentSBERTa approach. Results show PatentSBERTa, Bert-for-patents, and TF-IDF Weighted Word Embeddings have the best accuracy for computing sentence embeddings at the subclass level.
  arXiv  Detail & Related papers  (2022-04-28T12:04:42Z)
• Proactive Pseudo-Intervention: Causally Informed Contrastive Learning For Interpretable Vision Models [103.64435911083432]
  We present a novel contrastive learning strategy called it Proactive Pseudo-Intervention (PPI) PPI leverages proactive interventions to guard against image features with no causal relevance. We also devise a novel causally informed salience mapping module to identify key image pixels to intervene, and show it greatly facilitates model interpretability.
  arXiv  Detail & Related papers  (2020-12-06T20:30:26Z)
• Joint Deep Learning of Facial Expression Synthesis and Recognition [97.19528464266824]
  We propose a novel joint deep learning of facial expression synthesis and recognition method for effective FER. The proposed method involves a two-stage learning procedure. Firstly, a facial expression synthesis generative adversarial network (FESGAN) is pre-trained to generate facial images with different facial expressions. In order to alleviate the problem of data bias between the real images and the synthetic images, we propose an intra-class loss with a novel real data-guided back-propagation (RDBP) algorithm.
  arXiv  Detail & Related papers  (2020-02-06T10:56:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.