Sustainability Evaluation Metrics for Recommender Systems

• URL: http://arxiv.org/abs/2507.22520v1
• Date: Wed, 30 Jul 2025 09:46:56 GMT
• Title: Sustainability Evaluation Metrics for Recommender Systems
• Authors: Alexander Felfernig, Damian Garber, Viet-Man Le, Sebastian Lubos, Thi Ngoc Trang Tran,
• Abstract summary: Sustainability-oriented evaluation metrics can help to assess the quality of recommender systems.<n>We discuss different basic sustainability evaluation metrics for recommender systems and analyze their applications.
• Score: 44.41797739774634
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sustainability-oriented evaluation metrics can help to assess the quality of recommender systems beyond wide-spread metrics such as accuracy, precision, recall, and satisfaction. Following the United Nations`s sustainable development goals (SDGs), such metrics can help to analyse the impact of recommender systems on environmental, social, and economic aspects. We discuss different basic sustainability evaluation metrics for recommender systems and analyze their applications.

Related papers

• Contextual Metric Meta-Evaluation by Measuring Local Metric Accuracy [52.261323452286554]
  We introduce a method for contextual metric meta-evaluation by comparing the local metric accuracy of evaluation metrics.<n>Across translation, speech recognition, and ranking tasks, we demonstrate that the local metric accuracies vary both in absolute value and relative effectiveness as we shift across evaluation contexts.
  arXiv  Detail & Related papers  (2025-03-25T16:42:25Z)
• Evaluating Step-by-step Reasoning Traces: A Survey [3.895864050325129]
  Step-by-step reasoning is widely used to enhance the reasoning ability of large language models (LLMs) in complex problems.<n>Existing evaluation practices are highly inconsistent, resulting in fragmented progress across evaluator design and benchmark development.<n>This survey proposes a taxonomy of evaluation criteria with four top-level categories (factuality, validity, coherence, and utility)
  arXiv  Detail & Related papers  (2025-02-17T19:58:31Z)
• Recommender Systems for Social Good: The Role of Accountability and Sustainability [0.21756081703275998]
  recommender systems must go beyond personalization to support responsible consumption, reduce environmental impact, and foster social good.<n>We explore strategies to mitigate the carbon footprint of recommendation models, ensure fairness, and implement accountability mechanisms.
  arXiv  Detail & Related papers  (2025-01-10T13:46:23Z)
• Advancing Sustainability via Recommender Systems: A Survey [23.364932316026973]
  Human behavioral patterns and consumption paradigms have emerged as pivotal determinants in environmental degradation and climate change. There exists an imperative need for sustainable recommender systems that incorporate sustainability principles to foster eco-conscious and socially responsible choices. This comprehensive survey addresses this critical research gap by presenting a systematic analysis of sustainable recommender systems.
  arXiv  Detail & Related papers  (2024-11-12T09:19:32Z)
• Pessimistic Evaluation [58.736490198613154]
  We argue that evaluating information access systems assumes utilitarian values not aligned with traditions of information access based on equal access. We advocate for pessimistic evaluation of information access systems focusing on worst case utility.
  arXiv  Detail & Related papers  (2024-10-17T15:40:09Z)
• Revisiting Reciprocal Recommender Systems: Metrics, Formulation, and Method [60.364834418531366]
  We propose five new evaluation metrics that comprehensively and accurately assess the performance of RRS. We formulate the RRS from a causal perspective, formulating recommendations as bilateral interventions. We introduce a reranking strategy to maximize matching outcomes, as measured by the proposed metrics.
  arXiv  Detail & Related papers  (2024-08-19T07:21:02Z)
• Literature Review of Current Sustainability Assessment Frameworks and Approaches for Organizations [10.045497511868172]
  This systematic literature review explores sustainability assessment frameworks (SAFs) across diverse industries. The review focuses on SAF design approaches including the methods used for Sustainability Indicator (SI) selection, relative importance assessment, and interdependency analysis.
  arXiv  Detail & Related papers  (2024-03-07T18:14:52Z)
• A Comprehensive Survey of Evaluation Techniques for Recommendation Systems [0.0]
  This paper introduces a comprehensive suite of metrics, each tailored to capture a distinct aspect of system performance. We identify the strengths and limitations of current evaluation practices and highlight the nuanced trade-offs that emerge when optimizing recommendation systems across different metrics.
  arXiv  Detail & Related papers  (2023-12-26T11:57:01Z)
• A Survey on Fairness-aware Recommender Systems [59.23208133653637]
  We present concepts of fairness in different recommendation scenarios, comprehensively categorize current advances, and introduce typical methods to promote fairness in different stages of recommender systems. Next, we delve into the significant influence that fairness-aware recommender systems exert on real-world industrial applications.
  arXiv  Detail & Related papers  (2023-06-01T07:08:22Z)
• Evaluating Evaluation Metrics: A Framework for Analyzing NLG Evaluation Metrics using Measurement Theory [46.06645793520894]
  MetricEval is a framework for conceptualizing and evaluating the reliability and validity of NLG evaluation metrics. We aim to promote the design, evaluation, and interpretation of valid and reliable metrics to advance robust and effective NLG models.
  arXiv  Detail & Related papers  (2023-05-24T08:38:23Z)
• FEBR: Expert-Based Recommendation Framework for beneficial and personalized content [77.86290991564829]
  We propose FEBR (Expert-Based Recommendation Framework), an apprenticeship learning framework to assess the quality of the recommended content. The framework exploits the demonstrated trajectories of an expert (assumed to be reliable) in a recommendation evaluation environment, to recover an unknown utility function. We evaluate the performance of our solution through a user interest simulation environment (using RecSim)
  arXiv  Detail & Related papers  (2021-07-17T18:21:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.