AI Coding with Few-Shot Prompting for Thematic Analysis

• URL: http://arxiv.org/abs/2504.07408v1
• Date: Thu, 10 Apr 2025 03:02:15 GMT
• Title: AI Coding with Few-Shot Prompting for Thematic Analysis
• Authors: Samuel Flanders, Melati Nungsari, Mark Cheong Wing Loong,
• Abstract summary: This paper explores the use of large language models (LLMs) to perform coding for a thematic analysis.<n>We utilize few-shot prompting with higher quality codes generated on semantically similar passages to enhance the quality of the codes.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper explores the use of large language models (LLMs), here represented by GPT 3.5-Turbo to perform coding for a thematic analysis. Coding is highly labor intensive, making it infeasible for most researchers to conduct exhaustive thematic analyses of large corpora. We utilize few-shot prompting with higher quality codes generated on semantically similar passages to enhance the quality of the codes while utilizing a cheap, more easily scalable model.

Related papers

• Automatic deductive coding in discourse analysis: an application of large language models in learning analytics [5.606202114848633]
  The emergence of large language models such as GPT has opened a new avenue for automatic deductive coding. We employed three different classification methods driven by different artificial intelligence technologies. We found that GPT with prompt engineering outperformed the other two methods on both datasets with limited number of training samples.
  arXiv  Detail & Related papers  (2024-10-02T05:04:06Z)
• A Thorough Examination of Decoding Methods in the Era of LLMs [72.65956436513241]
  Decoding methods play an indispensable role in converting language models from next-token predictors into practical task solvers. This paper provides a comprehensive and multifaceted analysis of various decoding methods within the context of large language models. Our findings reveal that decoding method performance is notably task-dependent and influenced by factors such as alignment, model size, and quantization.
  arXiv  Detail & Related papers  (2024-02-10T11:14:53Z)
• Scalable Qualitative Coding with LLMs: Chain-of-Thought Reasoning Matches Human Performance in Some Hermeneutic Tasks [0.0]
  We show that GPT-4 is capable of human-equivalent interpretations, whereas GPT-3.5 is not. Our results indicate that for certain codebooks, state-of-the-art LLMs are already adept at large-scale content analysis.
  arXiv  Detail & Related papers  (2024-01-26T19:25:43Z)
• Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding [46.485363806259265]
  Speculative Decoding has emerged as a novel decoding paradigm for Large Language Models (LLMs) inference. In each decoding step, this method first drafts several future tokens efficiently and then verifies them in parallel. This paper presents a comprehensive overview and analysis of this promising decoding paradigm.
  arXiv  Detail & Related papers  (2024-01-15T17:26:50Z)
• LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Coding [0.3149883354098941]
  Large language models (LLMs) are AI tools that can perform a range of natural language processing and reasoning tasks. In this study, we explore the use of LLMs to reduce the time it takes for deductive coding while retaining the flexibility of a traditional content analysis. We find that GPT-3.5 can often perform deductive coding at levels of agreement comparable to human coders.
  arXiv  Detail & Related papers  (2023-06-23T20:57:32Z)
• Tram: A Token-level Retrieval-augmented Mechanism for Source Code Summarization [76.57699934689468]
  We propose a fine-grained Token-level retrieval-augmented mechanism (Tram) on the decoder side to enhance the performance of neural models. To overcome the challenge of token-level retrieval in capturing contextual code semantics, we also propose integrating code semantics into individual summary tokens.
  arXiv  Detail & Related papers  (2023-05-18T16:02:04Z)
• Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Coding [45.5690960017762]
  This study explores the use of large language models (LLMs) in supporting deductive coding. Instead of training task-specific models, a pre-trained LLM could be used directly for various tasks without fine-tuning through prompt learning. Using a curiosity-driven questions coding task as a case study, we found, by combining GPT-3 with expert-drafted codebooks, our proposed approach achieved fair to substantial agreements with expert-coded results.
  arXiv  Detail & Related papers  (2023-04-17T04:52:43Z)
• ConTextual Mask Auto-Encoder for Dense Passage Retrieval [49.49460769701308]
  CoT-MAE is a simple yet effective generative pre-training method for dense passage retrieval. It learns to compress the sentence semantics into a dense vector through self-supervised and context-supervised masked auto-encoding. We conduct experiments on large-scale passage retrieval benchmarks and show considerable improvements over strong baselines.
  arXiv  Detail & Related papers  (2022-08-16T11:17:22Z)
• Enhancing Semantic Code Search with Multimodal Contrastive Learning and Soft Data Augmentation [50.14232079160476]
  We propose a new approach with multimodal contrastive learning and soft data augmentation for code search. We conduct extensive experiments to evaluate the effectiveness of our approach on a large-scale dataset with six programming languages.
  arXiv  Detail & Related papers  (2022-04-07T08:49:27Z)
• Question Answering Infused Pre-training of General-Purpose Contextualized Representations [70.62967781515127]
  We propose a pre-training objective based on question answering (QA) for learning general-purpose contextual representations. We accomplish this goal by training a bi-encoder QA model, which independently encodes passages and questions, to match the predictions of a more accurate cross-encoder model. We show large improvements over both RoBERTa-large and previous state-of-the-art results on zero-shot and few-shot paraphrase detection.
  arXiv  Detail & Related papers  (2021-06-15T14:45:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.