EIVEN: Efficient Implicit Attribute Value Extraction using Multimodal LLM

• URL: http://arxiv.org/abs/2404.08886v1
• Date: Sat, 13 Apr 2024 03:15:56 GMT
• Title: EIVEN: Efficient Implicit Attribute Value Extraction using Multimodal LLM
• Authors: Henry Peng Zou, Gavin Heqing Yu, Ziwei Fan, Dan Bu, Han Liu, Peng Dai, Dongmei Jia, Cornelia Caragea,
• Abstract summary: EIVEN is a data- and parameter-efficient generative framework for implicit attribute value extraction. We introduce a novel Learning-by-Comparison technique to reduce model confusion. Our experiments reveal that EIVEN significantly outperforms existing methods in extracting implicit attribute values.
• Score: 52.016009472409166
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In e-commerce, accurately extracting product attribute values from multimodal data is crucial for improving user experience and operational efficiency of retailers. However, previous approaches to multimodal attribute value extraction often struggle with implicit attribute values embedded in images or text, rely heavily on extensive labeled data, and can easily confuse similar attribute values. To address these issues, we introduce EIVEN, a data- and parameter-efficient generative framework that pioneers the use of multimodal LLM for implicit attribute value extraction. EIVEN leverages the rich inherent knowledge of a pre-trained LLM and vision encoder to reduce reliance on labeled data. We also introduce a novel Learning-by-Comparison technique to reduce model confusion by enforcing attribute value comparison and difference identification. Additionally, we construct initial open-source datasets for multimodal implicit attribute value extraction. Our extensive experiments reveal that EIVEN significantly outperforms existing methods in extracting implicit attribute values while requiring less labeled data.

Related papers

• EAVE: Efficient Product Attribute Value Extraction via Lightweight Sparse-layer Interaction [94.22610101608332]
  We propose an Efficient product Attribute Value Extraction (EAVE) approach via lightweight sparse-layer interaction. We employ a heavy encoder to separately encode the product context and attribute. The resulting non-interacting heavy representations of the context can be cached and reused for all attributes. Our method achieves significant efficiency gains with neutral or marginal loss in performance when the context is long and number of attributes is large.
  arXiv  Detail & Related papers  (2024-06-10T23:06:38Z)
• ImplicitAVE: An Open-Source Dataset and Multimodal LLMs Benchmark for Implicit Attribute Value Extraction [67.86012624533461]
  ImplicitAVE is the first, publicly available multimodal dataset for implicit attribute value extraction. dataset includes 68k training and 1.6k testing data across five domains. We also explore the application of multimodal large language models (MLLMs) to implicit AVE.
  arXiv  Detail & Related papers  (2024-04-24T01:54:40Z)
• Enhanced E-Commerce Attribute Extraction: Innovating with Decorative Relation Correction and LLAMA 2.0-Based Annotation [4.81846973621209]
  We propose a pioneering framework that integrates BERT for classification, a Conditional Random Fields (CRFs) layer for attribute value extraction, and Large Language Models (LLMs) for data annotation. Our approach capitalizes on the robust representation learning of BERT, synergized with the sequence decoding prowess of CRFs, to adeptly identify and extract attribute values. Our methodology is rigorously validated on various datasets, including Walmart, BestBuy's e-commerce NER dataset, and the CoNLL dataset.
  arXiv  Detail & Related papers  (2023-12-09T08:26:30Z)
• AE-smnsMLC: Multi-Label Classification with Semantic Matching and Negative Label Sampling for Product Attribute Value Extraction [42.79022954630978]
  Product attribute value extraction plays an important role for many real-world applications in e-Commerce such as product search and recommendation. Previous methods treat it as a sequence labeling task that needs more annotation for position of values in the product text. We propose a classification model with semantic matching and negative label sampling for attribute value extraction.
  arXiv  Detail & Related papers  (2023-10-11T02:22:28Z)
• OA-Mine: Open-World Attribute Mining for E-Commerce Products with Weak Supervision [93.26737878221073]
  We study the attribute mining problem in an open-world setting to extract novel attributes and their values. We propose a principled framework that first generates attribute value candidates and then groups them into clusters of attributes. Our model significantly outperforms strong baselines and can generalize to unseen attributes and product types.
  arXiv  Detail & Related papers  (2022-04-29T04:16:04Z)
• MAVE: A Product Dataset for Multi-source Attribute Value Extraction [10.429320377835241]
  We introduce MAVE, a new dataset to better facilitate research on product attribute value extraction. MAVE is composed of a curated set of 2.2 million products from Amazon pages, with 3 million attribute-value annotations across 1257 unique categories. We propose a novel approach that effectively extracts the attribute value from the multi-source product information.
  arXiv  Detail & Related papers  (2021-12-16T06:48:31Z)
• AdaTag: Multi-Attribute Value Extraction from Product Profiles with Adaptive Decoding [55.89773725577615]
  We present AdaTag, which uses adaptive decoding to handle attribute extraction. Our experiments on a real-world e-Commerce dataset show marked improvements over previous methods.
  arXiv  Detail & Related papers  (2021-06-04T07:54:11Z)
• Multimodal Joint Attribute Prediction and Value Extraction for E-commerce Product [40.46223408546036]
  Product attribute values are essential in many e-commerce scenarios, such as customer service robots, product recommendations, and product retrieval. While in the real world, the attribute values of a product are usually incomplete and vary over time, which greatly hinders the practical applications. We propose a multimodal method to jointly predict product attributes and extract values from textual product descriptions with the help of the product images.
  arXiv  Detail & Related papers  (2020-09-15T15:10:51Z)
• Automatic Validation of Textual Attribute Values in E-commerce Catalog by Learning with Limited Labeled Data [61.789797281676606]
  We propose a novel meta-learning latent variable approach, called MetaBridge. It can learn transferable knowledge from a subset of categories with limited labeled data. It can capture the uncertainty of never-seen categories with unlabeled data.
  arXiv  Detail & Related papers  (2020-06-15T21:31:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.