AI-enhanced semantic feature norms for 786 concepts

• URL: http://arxiv.org/abs/2505.10718v1
• Date: Thu, 15 May 2025 21:43:34 GMT
• Title: AI-enhanced semantic feature norms for 786 concepts
• Authors: Siddharth Suresh, Kushin Mukherjee, Tyler Giallanza, Xizheng Yu, Mia Patil, Jonathan D. Cohen, Timothy T. Rogers,
• Abstract summary: We introduce a novel approach that augments a dataset of human-generated feature norms with responses from large language models (LLMs)<n>We find that our AI-enhanced feature norm dataset, NOVA: Norms Optimized Via AI, shows much higher feature density and overlap among concepts.
• Score: 8.68405554675708
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Semantic feature norms have been foundational in the study of human conceptual knowledge, yet traditional methods face trade-offs between concept/feature coverage and verifiability of quality due to the labor-intensive nature of norming studies. Here, we introduce a novel approach that augments a dataset of human-generated feature norms with responses from large language models (LLMs) while verifying the quality of norms against reliable human judgments. We find that our AI-enhanced feature norm dataset, NOVA: Norms Optimized Via AI, shows much higher feature density and overlap among concepts while outperforming a comparable human-only norm dataset and word-embedding models in predicting people's semantic similarity judgments. Taken together, we demonstrate that human conceptual knowledge is richer than captured in previous norm datasets and show that, with proper validation, LLMs can serve as powerful tools for cognitive science research.

Related papers

• Preference Learning for AI Alignment: a Causal Perspective [55.2480439325792]
  We frame this problem in a causal paradigm, providing the rich toolbox of causality to identify persistent challenges.<n>Inheriting from the literature of causal inference, we identify key assumptions necessary for reliable generalisation.<n>We illustrate failure modes of naive reward models and demonstrate how causally-inspired approaches can improve model robustness.
  arXiv  Detail & Related papers  (2025-06-06T10:45:42Z)
• When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration [79.69935257008467]
  We introduce Knowledge Integration and Transfer Evaluation (KITE), a conceptual and experimental framework for Human-AI knowledge transfer capabilities.<n>We conduct the first large-scale human study (N=118) explicitly designed to measure it.<n>In our two-phase setup, humans first ideate with an AI on problem-solving strategies, then independently implement solutions, isolating model explanations' influence on human understanding.
  arXiv  Detail & Related papers  (2025-06-05T20:48:16Z)
• Empirically evaluating commonsense intelligence in large language models with large-scale human judgments [4.7206754497888035]
  We propose a novel method for evaluating common sense in artificial intelligence.<n>We measure the correspondence between a model's judgment and that of a human population.<n>Our framework contributes to the growing call for adapting AI models to human collectivities that possess different, often incompatible, social stocks of knowledge.
  arXiv  Detail & Related papers  (2025-05-15T13:55:27Z)
• EgoNormia: Benchmarking Physical Social Norm Understanding [52.87904722234434]
  We present dataset $|epsilon|$, consisting of 1,853 challenging, multi-stage MCQ questions based on ego-centric videos of human interactions.<n>The normative actions encompass seven categories: safety, privacy, proxemics, politeness, cooperation, coordination/proactivity, and communication/legibility.
  arXiv  Detail & Related papers  (2025-02-27T19:54:16Z)
• Learning to Generate and Evaluate Fact-checking Explanations with Transformers [10.970249299147866]
  Research contributes to the field of Explainable Artificial Antelligence (XAI) We develop transformer-based fact-checking models that contextualise and justify their decisions by generating human-accessible explanations. We emphasise the need for aligning Artificial Intelligence (AI)-generated explanations with human judgements.
  arXiv  Detail & Related papers  (2024-10-21T06:22:51Z)
• Interpreting Pretrained Language Models via Concept Bottlenecks [55.47515772358389]
  Pretrained language models (PLMs) have made significant strides in various natural language processing tasks. The lack of interpretability due to their black-box'' nature poses challenges for responsible implementation. We propose a novel approach to interpreting PLMs by employing high-level, meaningful concepts that are easily understandable for humans.
  arXiv  Detail & Related papers  (2023-11-08T20:41:18Z)
• Semantic Feature Verification in FLAN-T5 [0.0]
  We show that machine-verified norms capture aspects of conceptual structure beyond what is expressed in human norms alone. The results suggest that LLMs can greatly enhance traditional methods of semantic feature norm verification.
  arXiv  Detail & Related papers  (2023-04-12T03:37:57Z)
• Beyond Interpretable Benchmarks: Contextual Learning through Cognitive and Multimodal Perception [0.0]
  This study contends that the Turing Test is misinterpreted as an attempt to anthropomorphize computer systems. It emphasizes tacit learning as a cornerstone of general-purpose intelligence, despite its lack of overt interpretability.
  arXiv  Detail & Related papers  (2022-12-04T08:30:04Z)
• NormSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations On-the-Fly [61.77957329364812]
  We introduce a framework for addressing the novel task of conversation-grounded multi-lingual, multi-cultural norm discovery. NormSAGE elicits knowledge about norms through directed questions representing the norm discovery task and conversation context. It further addresses the risk of language model hallucination with a self-verification mechanism ensuring that the norms discovered are correct.
  arXiv  Detail & Related papers  (2022-10-16T18:30:05Z)
• Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation [47.102566259034326]
  We propose conditional delegation as an alternative paradigm for human-AI collaboration. We develop novel interfaces to assist humans in creating conditional delegation rules. Our study demonstrates the promise of conditional delegation in improving model performance.
  arXiv  Detail & Related papers  (2022-04-25T17:00:02Z)
• WenLan 2.0: Make AI Imagine via a Multimodal Foundation Model [74.4875156387271]
  We develop a novel foundation model pre-trained with huge multimodal (visual and textual) data. We show that state-of-the-art results can be obtained on a wide range of downstream tasks.
  arXiv  Detail & Related papers  (2021-10-27T12:25:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.