Neo-Grounded Theory: A Methodological Innovation Integrating High-Dimensional Vector Clustering and Multi-Agent Collaboration for Qualitative Research

• URL: http://arxiv.org/abs/2509.25244v1
• Date: Fri, 26 Sep 2025 16:26:33 GMT
• Title: Neo-Grounded Theory: A Methodological Innovation Integrating High-Dimensional Vector Clustering and Multi-Agent Collaboration for Qualitative Research
• Authors: Shuide Wen, Beier Ku, Teng Wang, Mingyang Zou, Yang Yang,
• Abstract summary: Neo Grounded Theory (NGT) integrates vector clustering with multi agent systems to resolve qualitative research's scale depth paradox.<n>NGT achieved 168-fold speed improvement (3 hours vs 3 weeks), superior quality (0.904 vs 0.883), and 96% cost reduction.
• Score: 5.848041907318412
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Purpose: Neo Grounded Theory (NGT) integrates vector clustering with multi agent systems to resolve qualitative research's scale depth paradox, enabling analysis of massive datasets in hours while preserving interpretive rigor. Methods: We compared NGT against manual coding and ChatGPT-assisted analysis using 40,000 character Chinese interview transcripts. NGT employs 1536-dimensional embeddings, hierarchical clustering, and parallel agent-based coding. Two experiments tested pure automation versus human guided refinement. Findings: NGT achieved 168-fold speed improvement (3 hours vs 3 weeks), superior quality (0.904 vs 0.883), and 96% cost reduction. Human AI collaboration proved essential: automation alone produced abstract frameworks while human guidance yielded actionable dual pathway theories. The system discovered patterns invisible to manual coding, including identity bifurcation phenomena. Contributions: NGT demonstrates computational objectivity and human interpretation are complementary. Vector representations provide reproducible semantic measurement while preserving meaning's interpretive dimensions. Researchers shift from mechanical coding to theoretical guidance, with AI handling pattern recognition while humans provide creative insight. Implications: Cost reduction from \$50,000 to \$500 democratizes qualitative research, enabling communities to study themselves. Real-time analysis makes qualitative insights contemporaneous with events. The framework shows computational methods can strengthen rather than compromise qualitative research's humanistic commitments. Keywords: Grounded theory; Vector embeddings; Multi agent systems; Human AI collaboration; Computational qualitative analysis

Related papers

• The AI Research Assistant: Promise, Peril, and a Proof of Concept [0.0]
  We provide empirical evidence through a detailed case study.<n>The collaboration revealed both remarkable capabilities and critical limitations.<n>Our experience suggests that, when used with appropriate skepticism and verification protocols, AI tools can meaningfully accelerate mathematical discovery.
  arXiv  Detail & Related papers  (2026-02-26T10:29:05Z)
• Accelerating Scientific Research with Gemini: Case Studies and Common Techniques [105.15622072347811]
  Large language models (LLMs) have opened new avenues for accelerating scientific research.<n>We present a collection of case studies demonstrating how researchers have successfully collaborated with advanced AI models.
  arXiv  Detail & Related papers  (2026-02-03T18:56:17Z)
• Sci-Reasoning: A Dataset Decoding AI Innovation Patterns [14.720475159371361]
  Sci-Reasoning is the first dataset capturing the intellectual synthesis behind high-quality AI research.<n>Our analysis identifies 15 distinct thinking patterns, with three dominant strategies accounting for 52.7%.<n>This dataset enables quantitative studies of scientific progress and provides structured reasoning trajectories for training the next generation AI research agents.
  arXiv  Detail & Related papers  (2026-01-08T04:12:47Z)
• Development and Benchmarking of a Blended Human-AI Qualitative Research Assistant [1.170789976854236]
  We benchmark Muse, an interactive, AI-powered qualitative research system.<n>We find an inter-rater reliability between Muse and humans of Cohen's $$ = 0.71 for well-specified codes.<n>We also conduct robust error analysis to identify failure mode, guide future improvements, and demonstrate the capacity to correct for human bias.
  arXiv  Detail & Related papers  (2025-10-14T21:17:34Z)
• Understanding the Process of Human-AI Value Alignment [1.6799377888527687]
  Value alignment in computer science research is often used to refer to the process of aligning artificial intelligence with humans, but the way the phrase is used often lacks precision.<n>We conduct a systematic literature review to advance the understanding of value alignment in artificial intelligence.
  arXiv  Detail & Related papers  (2025-09-17T09:39:38Z)
• What Makes a Good Natural Language Prompt? [72.3282960118995]
  We conduct a meta-analysis surveying more than 150 prompting-related papers from leading NLP and AI conferences from 2022 to 2025.<n>We propose a property- and human-centric framework for evaluating prompt quality, encompassing 21 properties categorized into six dimensions.<n>We then empirically explore multi-property prompt enhancements in reasoning tasks, observing that single-property enhancements often have the greatest impact.
  arXiv  Detail & Related papers  (2025-06-07T23:19:27Z)
• A Computational Method for Measuring "Open Codes" in Qualitative Analysis [44.39424825305388]
  This paper presents a theory-informed computational method for measuring inductive coding results from humans and Generative AI (GAI)<n>It measures each coder's contribution against the merged result using four novel metrics: Coverage, Overlap, Novelty, and Divergence.<n>Our work provides a reliable pathway for ensuring methodological rigor in human-AI qualitative analysis.
  arXiv  Detail & Related papers  (2024-11-19T00:44:56Z)
• A Multimodal Automated Interpretability Agent [63.8551718480664]
  MAIA is a system that uses neural models to automate neural model understanding tasks.<n>We first characterize MAIA's ability to describe (neuron-level) features in learned representations of images.<n>We then show that MAIA can aid in two additional interpretability tasks: reducing sensitivity to spurious features, and automatically identifying inputs likely to be mis-classified.
  arXiv  Detail & Related papers  (2024-04-22T17:55:11Z)
• Human-Centric Multimodal Machine Learning: Recent Advances and Testbed on AI-based Recruitment [66.91538273487379]
  There is a certain consensus about the need to develop AI applications with a Human-Centric approach. Human-Centric Machine Learning needs to be developed based on four main requirements: (i) utility and social good; (ii) privacy and data ownership; (iii) transparency and accountability; and (iv) fairness in AI-driven decision-making processes. We study how current multimodal algorithms based on heterogeneous sources of information are affected by sensitive elements and inner biases in the data.
  arXiv  Detail & Related papers  (2023-02-13T16:44:44Z)
• Human-Algorithm Collaboration: Achieving Complementarity and Avoiding Unfairness [92.26039686430204]
  We show that even in carefully-designed systems, complementary performance can be elusive. First, we provide a theoretical framework for modeling simple human-algorithm systems. Next, we use this model to prove conditions where complementarity is impossible.
  arXiv  Detail & Related papers  (2022-02-17T18:44:41Z)
• KANDINSKYPatterns -- An experimental exploration environment for Pattern Analysis and Machine Intelligence [0.0]
  We present KANDINSKYPatterns, named after the Russian artist Wassily Kandinksy, who made theoretical contributions to compositivity, i.e. that all perceptions consist of geometrically elementary individual components. KANDINSKYPatterns have computationally controllable properties on the one hand, bringing ground truth, they are also easily distinguishable by human observers, i.e., controlled patterns can be described by both humans and algorithms.
  arXiv  Detail & Related papers  (2021-02-28T14:09:59Z)
• Explainability in Deep Reinforcement Learning [68.8204255655161]
  We review recent works in the direction to attain Explainable Reinforcement Learning (XRL) In critical situations where it is essential to justify and explain the agent's behaviour, better explainability and interpretability of RL models could help gain scientific insight on the inner workings of what is still considered a black box.
  arXiv  Detail & Related papers  (2020-08-15T10:11:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.