A Co-design Study for Multi-Stakeholder Job Recommender System Explanations

• URL: http://arxiv.org/abs/2309.05507v1
• Date: Mon, 11 Sep 2023 14:51:20 GMT
• Title: A Co-design Study for Multi-Stakeholder Job Recommender System Explanations
• Authors: Roan Schellingerhout, Francesco Barile, Nava Tintarev
• Abstract summary: We find that different stakeholder types indeed have strongly differing explanation preferences. Candidates indicated a preference for brief, textual explanations that allow them to quickly judge potential matches. On the other hand, hiring managers preferred visual graph-based explanations that provide a more technical and comprehensive overview at a glance.
• Score: 2.6681297407122577
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent legislation proposals have significantly increased the demand for eXplainable Artificial Intelligence (XAI) in many businesses, especially in so-called `high-risk' domains, such as recruitment. Within recruitment, AI has become commonplace, mainly in the form of job recommender systems (JRSs), which try to match candidates to vacancies, and vice versa. However, common XAI techniques often fall short in this domain due to the different levels and types of expertise of the individuals involved, making explanations difficult to generalize. To determine the explanation preferences of the different stakeholder types - candidates, recruiters, and companies - we created and validated a semi-structured interview guide. Using grounded theory, we structurally analyzed the results of these interviews and found that different stakeholder types indeed have strongly differing explanation preferences. Candidates indicated a preference for brief, textual explanations that allow them to quickly judge potential matches. On the other hand, hiring managers preferred visual graph-based explanations that provide a more technical and comprehensive overview at a glance. Recruiters found more exhaustive textual explanations preferable, as those provided them with more talking points to convince both parties of the match. Based on these findings, we describe guidelines on how to design an explanation interface that fulfills the requirements of all three stakeholder types. Furthermore, we provide the validated interview guide, which can assist future research in determining the explanation preferences of different stakeholder types.

Related papers

• Multi-Interest Recommendation: A Survey [67.28277752101006]
  Multi-interest recommendation addresses the challenge of extracting multiple interest representations from users' historical interactions.<n>It has drawn broad interest in recommendation research.<n>We systematically review the progress, solutions, challenges, and future directions of multi-interest recommendation.
  arXiv  Detail & Related papers  (2025-06-18T09:05:32Z)
• How to Elicit Explainability Requirements? A Comparison of Interviews, Focus Groups, and Surveys [2.30929645503432]
  This study examines the efficiency and effectiveness of three commonly used elicitation methods - focus groups, interviews, and online surveys.<n>The results show that interviews were the most efficient, capturing the highest number of distinct needs per participant per time spent.<n>We recommend a hybrid approach combining surveys and interviews to balance efficiency and coverage.
  arXiv  Detail & Related papers  (2025-05-29T17:23:14Z)
• Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning (v1) [66.51642638034822]
  Reasoning is central to human intelligence, enabling structured problem-solving across diverse tasks. Recent advances in large language models (LLMs) have greatly enhanced their reasoning abilities in arithmetic, commonsense, and symbolic domains. This paper offers a concise yet insightful overview of reasoning techniques in both textual and multimodal LLMs.
  arXiv  Detail & Related papers  (2025-04-04T04:04:56Z)
• The explanation dialogues: an expert focus study to understand requirements towards explanations within the GDPR [47.06917254695738]
  We present the Explanation Dialogues, an expert focus study to uncover the expectations, reasoning, and understanding of legal experts and practitioners towards XAI. The study consists of an online questionnaire and follow-up interviews, and is centered around a use-case in the credit domain. We find that the presented explanations are hard to understand and lack information, and discuss issues that can arise from the different interests of the data controller and subject.
  arXiv  Detail & Related papers  (2025-01-09T15:50:02Z)
• Creating Healthy Friction: Determining Stakeholder Requirements of Job Recommendation Explanations [2.373992571236766]
  We evaluate an explainable job recommender system using a realistic, task-based, mixed-design user study. We find that providing stakeholders with real explanations does not significantly improve decision-making speed and accuracy.
  arXiv  Detail & Related papers  (2024-09-24T11:03:17Z)
• Understanding Before Recommendation: Semantic Aspect-Aware Review Exploitation via Large Language Models [53.337728969143086]
  Recommendation systems harness user-item interactions like clicks and reviews to learn their representations. Previous studies improve recommendation accuracy and interpretability by modeling user preferences across various aspects and intents. We introduce a chain-based prompting approach to uncover semantic aspect-aware interactions.
  arXiv  Detail & Related papers  (2023-12-26T15:44:09Z)
• Clash of the Explainers: Argumentation for Context-Appropriate Explanations [6.8285745209093145]
  There is no single approach that is best suited for a given context. For AI explainability to be effective, explanations and how they are presented needs to be oriented towards the stakeholder receiving the explanation. We propose a modular reasoning system consisting of a given mental model of the relevant stakeholder, a reasoner component that solves the argumentation problem generated by a multi-explainer component, and an AI model that is to be explained suitably to the stakeholder of interest.
  arXiv  Detail & Related papers  (2023-12-12T09:52:30Z)
• PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization [60.00631098364391]
  PromptAgent is an optimization method that crafts expert-level prompts equivalent in quality to those handcrafted by experts. Inspired by human-like trial-and-error exploration, PromptAgent induces precise expert-level insights and in-depth instructions. We apply PromptAgent to 12 tasks spanning three practical domains.
  arXiv  Detail & Related papers  (2023-10-25T07:47:01Z)
• Generative Judge for Evaluating Alignment [84.09815387884753]
  We propose a generative judge with 13B parameters, Auto-J, designed to address these challenges. Our model is trained on user queries and LLM-generated responses under massive real-world scenarios. Experimentally, Auto-J outperforms a series of strong competitors, including both open-source and closed-source models.
  arXiv  Detail & Related papers  (2023-10-09T07:27:15Z)
• ExpertQA: Expert-Curated Questions and Attributed Answers [51.68314045809179]
  We conduct human evaluation of responses from a few representative systems along various axes of attribution and factuality. We collect expert-curated questions from 484 participants across 32 fields of study, and then ask the same experts to evaluate generated responses to their own questions. The output of our analysis is ExpertQA, a high-quality long-form QA dataset with 2177 questions spanning 32 fields, along with verified answers and attributions for claims in the answers.
  arXiv  Detail & Related papers  (2023-09-14T16:54:34Z)
• Explainable Predictive Maintenance [6.274171448205146]
  This position paper highlights the gap between existing XAI methodologies and the specific requirements for explanations within industrial applications. We provide an overview of predictive maintenance tasks and accentuate the need and varying purposes for corresponding explanations. We then list and describe XAI techniques commonly employed in the literature, discussing their suitability for PdM tasks.
  arXiv  Detail & Related papers  (2023-06-08T11:42:47Z)
• Getting MoRE out of Mixture of Language Model Reasoning Experts [71.61176122960464]
  We propose a Mixture-of-Reasoning-Experts (MoRE) framework that ensembles diverse specialized language models. We specialize the backbone language model with prompts optimized for different reasoning categories, including factual, multihop, mathematical, and commonsense reasoning. Our human study confirms that presenting expert predictions and the answer selection process helps annotators more accurately calibrate when to trust the system's output.
  arXiv  Detail & Related papers  (2023-05-24T02:00:51Z)
• Explanation Selection Using Unlabeled Data for Chain-of-Thought Prompting [80.9896041501715]
  Explanations that have not been "tuned" for a task, such as off-the-shelf explanations written by nonexperts, may lead to mediocre performance. This paper tackles the problem of how to optimize explanation-infused prompts in a blackbox fashion.
  arXiv  Detail & Related papers  (2023-02-09T18:02:34Z)
• Selective Explanations: Leveraging Human Input to Align Explainable AI [40.33998268146951]
  We propose a general framework for generating selective explanations by leveraging human input on a small sample. As a showcase, we use a decision-support task to explore selective explanations based on what the decision-maker would consider relevant to the decision task. Our experiments demonstrate the promise of selective explanations in reducing over-reliance on AI.
  arXiv  Detail & Related papers  (2023-01-23T19:00:02Z)
• Experiential Explanations for Reinforcement Learning [15.80179578318569]
  Reinforcement Learning systems can be complex and non-interpretable. We propose a technique, Experiential Explanations, to generate counterfactual explanations.
  arXiv  Detail & Related papers  (2022-10-10T14:27:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.