Can Large Language Models be Effective Online Opinion Miners?

• URL: http://arxiv.org/abs/2505.15695v1
• Date: Wed, 21 May 2025 16:09:44 GMT
• Title: Can Large Language Models be Effective Online Opinion Miners?
• Authors: Ryang Heo, Yongsik Seo, Junseong Lee, Dongha Lee,
• Abstract summary: We introduce Online Opinion Mining Benchmark (OOMB), a novel dataset and evaluation protocol designed to assess the ability of large language models to mine opinions effectively.<n>OOMB provides extensive (entity, feature, opinion) annotations and a comprehensive opinion-centric summary that highlights key opinion topics within each content.<n>We conduct a comprehensive analysis of which aspects remain challenging and where LLMs exhibit adaptability, to explore whether they can effectively serve as opinion miners in realistic online scenarios.
• Score: 10.58478659755799
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The surge of user-generated online content presents a wealth of insights into customer preferences and market trends. However, the highly diverse, complex, and context-rich nature of such contents poses significant challenges to traditional opinion mining approaches. To address this, we introduce Online Opinion Mining Benchmark (OOMB), a novel dataset and evaluation protocol designed to assess the ability of large language models (LLMs) to mine opinions effectively from diverse and intricate online environments. OOMB provides extensive (entity, feature, opinion) tuple annotations and a comprehensive opinion-centric summary that highlights key opinion topics within each content, thereby enabling the evaluation of both the extractive and abstractive capabilities of models. Through our proposed benchmark, we conduct a comprehensive analysis of which aspects remain challenging and where LLMs exhibit adaptability, to explore whether they can effectively serve as opinion miners in realistic online scenarios. This study lays the foundation for LLM-based opinion mining and discusses directions for future research in this field.

Related papers

• Aspect-Based Opinion Summarization with Argumentation Schemes [0.13654846342364307]
  It is impractical for customers to go over the vast number of reviews and manually conclude the prominent opinions.<n>Previous approaches, either extractive or abstractive, face challenges in automatically producing grounded aspect-centric summaries.<n>We propose a novel summarization system that not only captures predominant opinions from an aspect perspective with supporting evidence, but also adapts to varying domains without relying on a pre-defined set of aspects.
  arXiv  Detail & Related papers  (2025-06-11T16:38:10Z)
• Aspect-Aware Decomposition for Opinion Summarization [82.38097397662436]
  We propose a modular approach guided by review aspects which separates the tasks of aspect identification, opinion consolidation, and meta-review synthesis.<n>We conduct experiments across datasets representing scientific research, business, and product domains.<n>Results show that our method generates more grounded summaries compared to strong baseline models.
  arXiv  Detail & Related papers  (2025-01-27T09:29:55Z)
• LFOSum: Summarizing Long-form Opinions with Large Language Models [7.839083566878183]
  This paper introduces (1) a new dataset of long-form user reviews, each entity comprising over a thousand reviews, (2) two training-free LLM-based summarization approaches that scale to long inputs, and (3) automatic evaluation metrics. Our dataset of user reviews is paired with in-depth and unbiased critical summaries by domain experts, serving as a reference for evaluation. Our evaluation reveals that LLMs still face challenges in balancing sentiment and format adherence in long-form summaries, though open-source models can narrow the gap when relevant information is retrieved in a focused manner.
  arXiv  Detail & Related papers  (2024-10-16T20:52:39Z)
• Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based Interactions [62.0123588983514]
  Large Language Models (LLMs) have demonstrated wide-ranging applications across various fields. We reformulate the peer-review process as a multi-turn, long-context dialogue, incorporating distinct roles for authors, reviewers, and decision makers. We construct a comprehensive dataset containing over 26,841 papers with 92,017 reviews collected from multiple sources.
  arXiv  Detail & Related papers  (2024-06-09T08:24:17Z)
• CriticEval: Evaluating Large Language Model as Critic [110.29766259843453]
  CriticEval is a novel benchmark designed to comprehensively and reliably evaluate critique ability of Large Language Models. To ensure the comprehensiveness, CriticEval evaluates critique ability from four dimensions across nine diverse task scenarios. To ensure the reliability, a large number of critiques are annotated to serve as references.
  arXiv  Detail & Related papers  (2024-02-21T12:38:59Z)
• Exploring the Potential of Large Language Models in Computational Argumentation [54.85665903448207]
  Large language models (LLMs) have demonstrated impressive capabilities in understanding context and generating natural language. This work aims to embark on an assessment of LLMs, such as ChatGPT, Flan models, and LLaMA2 models, in both zero-shot and few-shot settings.
  arXiv  Detail & Related papers  (2023-11-15T15:12:15Z)
• Generative Judge for Evaluating Alignment [84.09815387884753]
  We propose a generative judge with 13B parameters, Auto-J, designed to address these challenges. Our model is trained on user queries and LLM-generated responses under massive real-world scenarios. Experimentally, Auto-J outperforms a series of strong competitors, including both open-source and closed-source models.
  arXiv  Detail & Related papers  (2023-10-09T07:27:15Z)
• AaKOS: Aspect-adaptive Knowledge-based Opinion Summarization [5.4138734778206]
  The rapid growth of information on the Internet has led to an overwhelming amount of opinions and comments on various activities, products, and services. This makes it difficult and time-consuming for users to process all the available information when making decisions. We propose an Aspect-adaptive Knowledge-based Opinion Summarization model for product reviews.
  arXiv  Detail & Related papers  (2023-05-26T03:44:35Z)
• Sentiment Analysis in the Era of Large Language Models: A Reality Check [69.97942065617664]
  This paper investigates the capabilities of large language models (LLMs) in performing various sentiment analysis tasks. We evaluate performance across 13 tasks on 26 datasets and compare the results against small language models (SLMs) trained on domain-specific datasets.
  arXiv  Detail & Related papers  (2023-05-24T10:45:25Z)
• Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models [115.7508325840751]
  The recent success of large language models (LLMs) has shown great potential to develop more powerful conversational recommender systems (CRSs) In this paper, we embark on an investigation into the utilization of ChatGPT for conversational recommendation, revealing the inadequacy of the existing evaluation protocol. We propose an interactive Evaluation approach based on LLMs named iEvaLM that harnesses LLM-based user simulators.
  arXiv  Detail & Related papers  (2023-05-22T15:12:43Z)
• A Framework of Customer Review Analysis Using the Aspect-Based Opinion Mining Approach [0.0]
  The paper presents a framework of aspect-based opinion mining based on the concept of transfer learning. The model has yielded quite satisfactory results in its task of aspect-based opinion mining.
  arXiv  Detail & Related papers  (2022-12-20T07:54:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.