MIThinker: A Plug-and-Play Policy-Optimized Thinker For Motivational Interviewing Counseling
Abstract Overview
This paper presents MIThinker, a lightweight reasoning model designed to generate explicit therapeutic thoughts for motivational interviewing (MI) counseling before a response is produced. The method separates thought generation from response generation, allowing the produced thoughts to guide a general-purpose backbone LLM in selecting strategies and composing MI-consistent replies. To create supervision despite the lack of annotated counselor thoughts, the authors build AugR1-MI, an automated pipeline that reverse-engineers counselor reasoning from observed responses in the AnnoMI dataset and iteratively refines those thoughts. MIThinker is then trained with supervised fine-tuning followed by reinforcement learning using rewards for format, alignment, and reasonableness. Evaluations include automatic MI competency measures, success rates in simulated sessions, and expert assessment of both generated thoughts and full counseling sessions.
Novelty
The paper's main novelty is a plug-and-play, domain-specialized thought policy for motivational interviewing that generates therapist-style reasoning separately from the final response. It also introduces AugR1-MI, a reverse-engineering pipeline for constructing large-scale counselor-thought supervision without direct expert thought annotations.
Results
Across automatic and expert evaluations, the MIThinker-based agent MindfulMI achieves MI competency comparable to strong prior systems such as CAMI while using substantially less computation and lower per-turn latency. Reinforcement learning improves MI behavior metrics over supervised fine-tuning alone, and expert evaluation finds no significant difference between oracle thoughts and MIThinker-generated thoughts.
Key Points
- MIThinker generates structured counselor thoughts grounded in theory of mind, motivational interviewing, and stage-of-change reasoning, then uses those thoughts to guide a separate response model.
- The AugR1-MI pipeline derives 31,444 oracle thoughts from 2,322 context-response pairs in 110 AnnoMI sessions through reverse engineering and iterative refinement.
- MindfulMI shows strong MI behavior scores, competitive global MI ratings and session success rates, and markedly better efficiency than heavier modular approaches, though CAMI remains stronger on some success-rate settings and topic exploration.