Structured Exploration vs. Generative Flexibility: A Field Study Comparing Bandit and LLM Architectures for Personalised Health Behaviour Interventions

• URL: http://arxiv.org/abs/2603.06330v1
• Date: Fri, 06 Mar 2026 14:42:25 GMT
• Title: Structured Exploration vs. Generative Flexibility: A Field Study Comparing Bandit and LLM Architectures for Personalised Health Behaviour Interventions
• Abstract summary: Behaviour Change Techniques (BCTs) are central to digital health interventions, yet selecting and delivering effective techniques remains challenging.<n>Contextual bandits enable statistically grounded optimisation of BCT selection, while Large Language Models (LLMs) offer flexible, context-sensitive message generation.<n>We contribute design suggestions for reflective AI health behaviour change systems that address a trade-off between structured exploration and generative autonomy.
• Score: 4.562136669058948
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Behaviour Change Techniques (BCTs) are central to digital health interventions, yet selecting and delivering effective techniques remains challenging. Contextual bandits enable statistically grounded optimisation of BCT selection, while Large Language Models (LLMs) offer flexible, context-sensitive message generation. We conducted a 4-week study on physical activity motivation (N=54; 9 post-study interviews) that compared five daily messaging approaches: random templates, contextual bandit with templates, LLM generation, hybrid bandit+LLM, and LLM with interaction history. LLM-based approaches were rated substantially more helpful than templates, but no significant differences emerged among LLM conditions. Unexpectedly, bandit optimisation for BCTs selection yielded no additional perceived helpfulness compared with LLM-only approaches. Unconstrained LLMs focused heavily on a single BCT, whereas bandit systems enforced systematic exploration-exploitation across techniques. Quantitative and qualitative findings suggest contextual acknowledgement of user input drove perceived helpfulness. We contribute design suggestions for reflective AI health behaviour change systems that address a trade-off between structured exploration and generative autonomy.

Related papers

• A Component-Based Survey of Interactions between Large Language Models and Multi-Armed Bandits [2.969473917919491]
  Large language models (LLMs) have become powerful and widely used systems for language understanding and generation.<n>Multi-armed bandit (MAB) algorithms provide a principled framework for adaptive decision-making under uncertainty.<n>This survey explores the potential at the intersection of these two fields.
  arXiv  Detail & Related papers  (2026-01-19T10:53:57Z)
• Reasoning with Preference Constraints: A Benchmark for Language Models in Many-to-One Matching Markets [13.111181135818184]
  Large language models (LLMs) have shown strong performance on complex mathematical tasks, including optimization.<n>Applying LLMs to matching problems, which require reasoning under preferential and structural constraints, remains underexplored.<n>We employ a novel benchmark of 369 instances of the College Admission Problem to evaluate LLMs across key dimensions: feasibility, stability, and optimality.
  arXiv  Detail & Related papers  (2025-09-16T14:48:46Z)
• When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness Methods for LLMs [55.20230501807337]
  We present the first systematic evaluation of 5 methods for improving prompt robustness within a unified experimental framework.<n>We benchmark these techniques on 8 models from Llama, Qwen and Gemma families across 52 tasks from Natural Instructions dataset.
  arXiv  Detail & Related papers  (2025-08-15T10:32:50Z)
• Survey: Multi-Armed Bandits Meet Large Language Models [6.718566736462752]
  Bandit algorithms and Large Language Models (LLMs) have emerged as powerful tools in artificial intelligence.<n>We first examine the role of bandit algorithms in optimizing LLM fine-tuning, prompt engineering, and adaptive response generation.<n>We then explore how LLMs can augment bandit algorithms through advanced contextual understanding, dynamic adaptation, and improved policy selection using natural language reasoning.
  arXiv  Detail & Related papers  (2025-05-19T16:57:57Z)
• Adaptive Distraction: Probing LLM Contextual Robustness with Automated Tree Search [76.54475437069395]
  Large Language Models (LLMs) often struggle to maintain their original performance when faced with semantically coherent but task-irrelevant contextual information.<n>We propose a dynamic distraction generation framework based on tree search, where the generation process is guided by model behavior.
  arXiv  Detail & Related papers  (2025-02-03T18:43:36Z)
• Large Language Model-Enhanced Multi-Armed Bandits [43.34246396804588]
  Large language models (LLMs) have been adopted to solve sequential decision-making tasks such as multi-armed bandits (MAB)<n>We propose an alternative approach which combines the strengths of classical MAB and LLMs.<n>We conduct empirical evaluations using both synthetic MAB tasks and experiments designed using real-world text datasets.
  arXiv  Detail & Related papers  (2025-02-03T07:19:05Z)
• RA-BLIP: Multimodal Adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-training [55.54020926284334]
  Multimodal Large Language Models (MLLMs) have recently received substantial interest, which shows their emerging potential as general-purpose models for various vision-language tasks. Retrieval augmentation techniques have proven to be effective plugins for both LLMs and MLLMs. In this study, we propose multimodal adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-training (RA-BLIP), a novel retrieval-augmented framework for various MLLMs.
  arXiv  Detail & Related papers  (2024-10-18T03:45:19Z)
• EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration [76.66831821738927]
  Large language models (LLMs) remain under-studied in scenarios requiring optimal decision-making under uncertainty.<n>We measure LLMs' (in)ability to make optimal decisions in bandits, a state-less reinforcement learning setting relevant to many applications.<n>Motivated by the existence of optimal exploration algorithms, we propose efficient ways to integrate this algorithmic knowledge into LLMs.
  arXiv  Detail & Related papers  (2024-10-08T17:54:03Z)
• A Multi-LLM Debiasing Framework [85.17156744155915]
  Large Language Models (LLMs) are powerful tools with the potential to benefit society immensely, yet, they have demonstrated biases that perpetuate societal inequalities. Recent research has shown a growing interest in multi-LLM approaches, which have been demonstrated to be effective in improving the quality of reasoning. We propose a novel multi-LLM debiasing framework aimed at reducing bias in LLMs.
  arXiv  Detail & Related papers  (2024-09-20T20:24:50Z)
• Beyond Numeric Rewards: In-Context Dueling Bandits with LLM Agents [25.825941077332182]
  In-Context Reinforcement Learning (ICRL) is a frontier paradigm to solve Reinforcement Learning (RL) problems in the foundation model era.<n>This paper investigates whether Large Language Models (LLMs) can generalize cross-domain to perform ICRL under the problem of Dueling Bandits (DB)<n>We show that LEAD has theoretical guarantees inherited from classic DB algorithms on both weak and strong regret.
  arXiv  Detail & Related papers  (2024-07-02T02:18:14Z)
• A + B: A General Generator-Reader Framework for Optimizing LLMs to Unleash Synergy Potential [20.1396255995056]
  Retrieval-Augmented Generation (RAG) is an effective solution to supplement necessary knowledge to large language models (LLMs) "generate-then-read" pipeline is proposed to replace the retrieval stage with generation from the LLM itself. This paper formalizes a general "A + B" framework with varying combinations of foundation models and types for systematic investigation.
  arXiv  Detail & Related papers  (2024-06-06T11:14:27Z)
• Can large language models explore in-context? [87.49311128190143]
  We deploy Large Language Models as agents in simple multi-armed bandit environments. We find that the models do not robustly engage in exploration without substantial interventions.
  arXiv  Detail & Related papers  (2024-03-22T17:50:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.