Teaching and Critiquing Conceptualization and Operationalization in NLP

• URL: http://arxiv.org/abs/2512.18505v1
• Date: Sat, 20 Dec 2025 20:47:01 GMT
• Title: Teaching and Critiquing Conceptualization and Operationalization in NLP
• Authors: Vagrant Gautam,
• Abstract summary: I outline a seminar I created for students to explore these questions of conceptualization and operationalization.<n>Each subfield has a shared understanding or conceptualization of what these terms mean and how we should treat them.
• Score: 1.6063240324252115
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: NLP researchers regularly invoke abstract concepts like "interpretability," "bias," "reasoning," and "stereotypes," without defining them. Each subfield has a shared understanding or conceptualization of what these terms mean and how we should treat them, and this shared understanding is the basis on which operational decisions are made: Datasets are built to evaluate these concepts, metrics are proposed to quantify them, and claims are made about systems. But what do they mean, what should they mean, and how should we measure them? I outline a seminar I created for students to explore these questions of conceptualization and operationalization, with an interdisciplinary reading list and an emphasis on discussion and critique.

Related papers

• Insight: Interpretable Semantic Hierarchies in Vision-Language Encoders [52.94006363830628]
  Language-aligned vision foundation models perform strongly across diverse downstream tasks.<n>Recent works decompose these representations into human-interpretable concepts, but provide poor spatial grounding and are limited to image classification tasks.<n>We propose Insight, a language-aligned concept foundation model that provides fine-grained concepts, which are human-interpretable and spatially grounded in the input image.
  arXiv  Detail & Related papers  (2026-01-20T09:57:26Z)
• A Geometric Unification of Concept Learning with Concept Cones [58.70836885177496]
  Two traditions of interpretability have evolved side by side but seldom spoken to each other: Concept Bottleneck Models (CBMs) and Sparse Autoencoders (SAEs)<n>We show that both paradigms instantiate the same geometric structure.<n>CBMs provide human-defined reference geometries, while SAEs can be evaluated by how well their learned cones approximate or contain those of CBMs.
  arXiv  Detail & Related papers  (2025-12-08T09:51:46Z)
• Rethinking Saliency Maps: A Cognitive Human Aligned Taxonomy and Evaluation Framework for Explanations [16.045822460720363]
  We introduce the Reference-Frame $times$ Granularity (RFxG) taxonomy.<n>RFxG organizes saliency explanations along two essential axes:Reference-Frame: Distinguishing between pointwise ("Why this prediction?") and contrastive ("Why this and not an alternative?") explanations.<n>Our comprehensive evaluation framework applies these metrics to ten state-of-the-art saliency methods, four model architectures, and three datasets.
  arXiv  Detail & Related papers  (2025-11-17T07:29:25Z)
• FaCT: Faithful Concept Traces for Explaining Neural Network Decisions [56.796533084868884]
  Deep networks have shown remarkable performance across a wide range of tasks, yet getting a global concept-level understanding of how they function remains a key challenge.<n>We put emphasis on the faithfulness of concept-based explanations and propose a new model with model-inherent mechanistic concept-explanations.<n>Our concepts are shared across classes and, from any layer, their contribution to the logit and their input-visualization can be faithfully traced.
  arXiv  Detail & Related papers  (2025-10-29T13:35:46Z)
• On the Role of Entity and Event Level Conceptualization in Generalizable Reasoning: A Survey of Tasks, Methods, Applications, and Future Directions [62.06913340614293]
  This paper proposes a categorization of different types of conceptualizations into four levels based on the types of instances being conceptualized.<n>We present the first comprehensive survey of over 150 papers, surveying various definitions, resources, methods, and downstream applications related to conceptualization.
  arXiv  Detail & Related papers  (2024-06-16T10:32:41Z)
• The Empty Signifier Problem: Towards Clearer Paradigms for Operationalising "Alignment" in Large Language Models [18.16062736448993]
  We address the concept of "alignment" in large language models (LLMs) through the lens of post-structuralist socio-political theory. We propose a framework that demarcates: 1) which dimensions of model behaviour are considered important, then 2) how meanings and definitions are ascribed to these dimensions. We aim to foster a culture of transparency and critical evaluation, aiding the community in navigating the complexities of aligning LLMs with human populations.
  arXiv  Detail & Related papers  (2023-10-03T22:02:17Z)
• ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models [79.10890337599166]
  We introduce ConceptBed, a large-scale dataset that consists of 284 unique visual concepts and 33K composite text prompts. We evaluate visual concepts that are either objects, attributes, or styles, and also evaluate four dimensions of compositionality: counting, attributes, relations, and actions. Our results point to a trade-off between learning the concepts and preserving the compositionality which existing approaches struggle to overcome.
  arXiv  Detail & Related papers  (2023-06-07T18:00:38Z)
• Interpretable Neural-Symbolic Concept Reasoning [7.1904050674791185]
  Concept-based models aim to address this issue by learning tasks based on a set of human-understandable concepts. We propose the Deep Concept Reasoner (DCR), the first interpretable concept-based model that builds upon concept embeddings.
  arXiv  Detail & Related papers  (2023-04-27T09:58:15Z)
• COPEN: Probing Conceptual Knowledge in Pre-trained Language Models [60.10147136876669]
  Conceptual knowledge is fundamental to human cognition and knowledge bases. Existing knowledge probing works only focus on factual knowledge of pre-trained language models (PLMs) and ignore conceptual knowledge. We design three tasks to probe whether PLMs organize entities by conceptual similarities, learn conceptual properties, and conceptualize entities in contexts. For the tasks, we collect and annotate 24k data instances covering 393 concepts, which is COPEN, a COnceptual knowledge Probing bENchmark.
  arXiv  Detail & Related papers  (2022-11-08T08:18:06Z)
• Discovering Concepts in Learned Representations using Statistical Inference and Interactive Visualization [0.76146285961466]
  Concept discovery is important for bridging the gap between non-deep learning experts and model end-users. Current approaches include hand-crafting concept datasets and then converting them to latent space directions. In this study, we offer another two approaches to guide user discovery of meaningful concepts, one based on multiple hypothesis testing, and another on interactive visualization.
  arXiv  Detail & Related papers  (2022-02-09T22:29:48Z)
• Theoretical foundations and limits of word embeddings: what types of meaning can they capture? [0.0]
  Measuring meaning is a central problem in cultural sociology. I theorize the ways in which word embeddings model three core premises of a structural linguistic theory of meaning.
  arXiv  Detail & Related papers  (2021-07-22T00:40:33Z)
• Separating Skills and Concepts for Novel Visual Question Answering [66.46070380927372]
  Generalization to out-of-distribution data has been a problem for Visual Question Answering (VQA) models. "Skills" are visual tasks, such as counting or attribute recognition, and are applied to "concepts" mentioned in the question. We present a novel method for learning to compose skills and concepts that separates these two factors implicitly within a model.
  arXiv  Detail & Related papers  (2021-07-19T18:55:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.