Conceptual Engineering Using Large Language Models

• URL: http://arxiv.org/abs/2312.03749v2
• Date: Sat, 02 Nov 2024 01:36:11 GMT
• Title: Conceptual Engineering Using Large Language Models
• Authors: Bradley P. Allen,
• Abstract summary: We use data from the Wikidata knowledge graph to evaluate stipulative definitions related to two conceptual engineering projects. Our results show that classification procedures built using our approach can exhibit good classification performance. We consider objections to this work for three aspects of theory and practice of conceptual engineering.
• Score: 0.0
• License:
• Abstract: We describe a method, based on Jennifer Nado's proposal for classification procedures as targets of conceptual engineering, that implements such procedures by prompting a large language model. We apply this method, using data from the Wikidata knowledge graph, to evaluate stipulative definitions related to two paradigmatic conceptual engineering projects: the International Astronomical Union's redefinition of PLANET and Haslanger's ameliorative analysis of WOMAN. Our results show that classification procedures built using our approach can exhibit good classification performance and, through the generation of rationales for their classifications, can contribute to the identification of issues in either the definitions or the data against which they are being evaluated. We consider objections to this method, and discuss implications of this work for three aspects of theory and practice of conceptual engineering: the definition of its targets, empirical methods for their investigation, and their practical roles. The data and code used for our experiments, together with the experimental results, are available in a Github repository.

Related papers

• A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation [18.600321051705482]
  Concept-based approaches have emerged as some of the most promising explainability methods. We introduce a unifying theoretical framework that comprehensively defines and clarifies these two steps. We show how to efficiently identify clusters of data points that are classified based on a similar shared strategy.
  arXiv  Detail & Related papers  (2023-06-11T23:28:02Z)
• Evaluation of Self-taught Learning-based Representations for Facial Emotion Recognition [62.30451764345482]
  This work describes different strategies to generate unsupervised representations obtained through the concept of self-taught learning for facial emotion recognition. The idea is to create complementary representations promoting diversity by varying the autoencoders' initialization, architecture, and training data. Experimental results on Jaffe and Cohn-Kanade datasets using a leave-one-subject-out protocol show that FER methods based on the proposed diverse representations compare favorably against state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-04-26T22:48:15Z)
• Visual Ground Truth Construction as Faceted Classification [4.7590051176368915]
  Key novelty of our approach lies in the fact that we construct the classification hierarchies from visual properties exploiting visual genus-differentiae. The proposed approach is validated by a set of experiments on the ImageNet hierarchy of musical experiments.
  arXiv  Detail & Related papers  (2022-02-17T08:35:23Z)
• Discovering Concepts in Learned Representations using Statistical Inference and Interactive Visualization [0.76146285961466]
  Concept discovery is important for bridging the gap between non-deep learning experts and model end-users. Current approaches include hand-crafting concept datasets and then converting them to latent space directions. In this study, we offer another two approaches to guide user discovery of meaningful concepts, one based on multiple hypothesis testing, and another on interactive visualization.
  arXiv  Detail & Related papers  (2022-02-09T22:29:48Z)
• Semantic Search for Large Scale Clinical Ontologies [63.71950996116403]
  We present a deep learning approach to build a search system for large clinical vocabularies. We propose a Triplet-BERT model and a method that generates training data based on semantic training data. The model is evaluated using five real benchmark data sets and the results show that our approach achieves high results on both free text to concept and concept to searching concept vocabularies.
  arXiv  Detail & Related papers  (2022-01-01T05:15:42Z)
• On Sample Based Explanation Methods for NLP:Efficiency, Faithfulness, and Semantic Evaluation [23.72825603188359]
  We can improve the interpretability of explanations by allowing arbitrary text sequences as the explanation unit. We propose a semantic-based evaluation metric that can better align with humans' judgment of explanations.
  arXiv  Detail & Related papers  (2021-06-09T00:49:56Z)
• Visualization of Supervised and Self-Supervised Neural Networks via Attribution Guided Factorization [87.96102461221415]
  We develop an algorithm that provides per-class explainability. In an extensive battery of experiments, we demonstrate the ability of our methods to class-specific visualization.
  arXiv  Detail & Related papers  (2020-12-03T18:48:39Z)
• Interpretable Multi-dataset Evaluation for Named Entity Recognition [110.64368106131062]
  We present a general methodology for interpretable evaluation for the named entity recognition (NER) task. The proposed evaluation method enables us to interpret the differences in models and datasets, as well as the interplay between them. By making our analysis tool available, we make it easy for future researchers to run similar analyses and drive progress in this area.
  arXiv  Detail & Related papers  (2020-11-13T10:53:27Z)
• A Diagnostic Study of Explainability Techniques for Text Classification [52.879658637466605]
  We develop a list of diagnostic properties for evaluating existing explainability techniques. We compare the saliency scores assigned by the explainability techniques with human annotations of salient input regions to find relations between a model's performance and the agreement of its rationales with human ones.
  arXiv  Detail & Related papers  (2020-09-25T12:01:53Z)
• Concept Learners for Few-Shot Learning [76.08585517480807]
  We propose COMET, a meta-learning method that improves generalization ability by learning to learn along human-interpretable concept dimensions. We evaluate our model on few-shot tasks from diverse domains, including fine-grained image classification, document categorization and cell type annotation.
  arXiv  Detail & Related papers  (2020-07-14T22:04:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.