Expert-sourcing Domain-specific Knowledge: The Case of Synonym Validation

• URL: http://arxiv.org/abs/2309.16798v1
• Date: Thu, 28 Sep 2023 19:02:33 GMT
• Title: Expert-sourcing Domain-specific Knowledge: The Case of Synonym Validation
• Authors: Michael Unterkalmsteiner, Andrew Yates
• Abstract summary: We illustrate tool support that we adopted and extended to source domain-specific knowledge from experts. We provide insight in design decisions that aim at motivating experts to dedicate their time at performing the labelling task. We foresee that the approach of expert-sourcing is applicable to any data labelling task in software engineering.
• Score: 14.51095331294056
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: One prerequisite for supervised machine learning is high quality labelled data. Acquiring such data is, particularly if expert knowledge is required, costly or even impossible if the task needs to be performed by a single expert. In this paper, we illustrate tool support that we adopted and extended to source domain-specific knowledge from experts. We provide insight in design decisions that aim at motivating experts to dedicate their time at performing the labelling task. We are currently using the approach to identify true synonyms from a list of candidate synonyms. The identification of synonyms is important in scenarios were stakeholders from different companies and background need to collaborate, for example when defining and negotiating requirements. We foresee that the approach of expert-sourcing is applicable to any data labelling task in software engineering. The discussed design decisions and implementation are an initial draft that can be extended, refined and validated with further application.

Related papers

• SEKE: Specialised Experts for Keyword Extraction [5.8908163351315075]
  Keywords extraction involves identifying the most descriptive words in a document. We propose a novel supervised keyword extraction approach based on the mixture of experts (MoE) technique. MoE uses a learnable routing sub-network to direct information to specialised experts, allowing them to specialize in distinct regions of the input space.
  arXiv  Detail & Related papers  (2024-12-18T17:34:32Z)
• Leveraging Open Knowledge for Advancing Task Expertise in Large Language Models [36.172093066234794]
  We introduce few human-annotated samples (i.e., K-shot) for advancing task expertise of large language models with open knowledge. A mixture-of-expert (MoE) system is built to make the best use of individual-yet-complementary knowledge between multiple experts.
  arXiv  Detail & Related papers  (2024-08-28T16:28:07Z)
• Causal Discovery with Language Models as Imperfect Experts [119.22928856942292]
  We consider how expert knowledge can be used to improve the data-driven identification of causal graphs. We propose strategies for amending such expert knowledge based on consistency properties. We report a case study, on real data, where a large language model is used as an imperfect expert.
  arXiv  Detail & Related papers  (2023-07-05T16:01:38Z)
• Toward a traceable, explainable, and fairJD/Resume recommendation system [10.820022470618234]
  Development of an automatic recruitment system is still one of the main challenges. Our aim is to explore how modern language models can be combined with knowledge bases and datasets to enhance the JD/Resume matching process.
  arXiv  Detail & Related papers  (2022-02-02T18:17:05Z)
• Extracting Semantics from Maintenance Records [0.2578242050187029]
  We develop three approaches to extracting named entity recognition from maintenance records. We develop a syntactic rules and semantic-based approach and an approach leveraging a pre-trained language model. Our evaluations on a real-world aviation maintenance records dataset show promising results.
  arXiv  Detail & Related papers  (2021-08-11T21:23:10Z)
• KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction [111.74812895391672]
  We propose a Knowledge-aware Prompt-tuning approach with synergistic optimization (KnowPrompt) We inject latent knowledge contained in relation labels into prompt construction with learnable virtual type words and answer words.
  arXiv  Detail & Related papers  (2021-04-15T17:57:43Z)
• Streaming Self-Training via Domain-Agnostic Unlabeled Images [62.57647373581592]
  We present streaming self-training (SST) that aims to democratize the process of learning visual recognition models. Key to SST are two crucial observations: (1) domain-agnostic unlabeled images enable us to learn better models with a few labeled examples without any additional knowledge or supervision; and (2) learning is a continuous process and can be done by constructing a schedule of learning updates.
  arXiv  Detail & Related papers  (2021-04-07T17:58:39Z)
• Leveraging Expert Consistency to Improve Algorithmic Decision Support [62.61153549123407]
  We explore the use of historical expert decisions as a rich source of information that can be combined with observed outcomes to narrow the construct gap. We propose an influence function-based methodology to estimate expert consistency indirectly when each case in the data is assessed by a single expert. Our empirical evaluation, using simulations in a clinical setting and real-world data from the child welfare domain, indicates that the proposed approach successfully narrows the construct gap.
  arXiv  Detail & Related papers  (2021-01-24T05:40:29Z)
• Reasoning over Vision and Language: Exploring the Benefits of Supplemental Knowledge [59.87823082513752]
  This paper investigates the injection of knowledge from general-purpose knowledge bases (KBs) into vision-and-language transformers. We empirically study the relevance of various KBs to multiple tasks and benchmarks. The technique is model-agnostic and can expand the applicability of any vision-and-language transformer with minimal computational overhead.
  arXiv  Detail & Related papers  (2021-01-15T08:37:55Z)
• Expertise Style Transfer: A New Task Towards Better Communication between Experts and Laymen [88.30492014778943]
  We propose a new task of expertise style transfer and contribute a manually annotated dataset. Solving this task not only simplifies the professional language, but also improves the accuracy and expertise level of laymen descriptions. We establish the benchmark performance of five state-of-the-art models for style transfer and text simplification.
  arXiv  Detail & Related papers  (2020-05-02T04:50:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.