Conversational Chatbot for Cigarette Smoking Cessation: Report of the User-Centered Design Eleven Step Development Process.

Conversational chatbots are an emerging digital intervention for smoking cessation. No studies have reported on the entire development process of a cessation chatbot.To describe the user-centered design development process for a novel and comprehensive quit smoking conversational chatbot called "QuitBot."The four years of formative research for developing QuitBot followed an eleven-step process: (1) specifying a conceptual model, (2) conducting content analysis of existing interventions (63 hours of intervention transcripts), (3) assessing user needs, (4) developing the chat's persona ("personality"), (5) prototyping content and persona, (6) developing full functionality, (7) programming the QuitBot, (8) conducting a diary study, (9) conducting a pilot randomized trial, (10) reviewing results of the trial, and (11) adding a free-form question and answer (QnA) function, based on user feedback from pilot trial results. The process of adding a QnA function itself involved a three-step process: (a) generating QnA pairs, (b) fine tuning Large Language Models (LLMs) on QnA pairs, and (c) evaluating the LLM model outputs.A quit smoking program spanning 42 days of 2 to 3-minute conversations covering topics ranging from motivations to quit, setting a quit date, choosing FDA-approved cessation medications, coping with triggers, and recovering from lapses/relapses. In a pilot randomized trial with 96% three-month outcome data retention, QuitBot demonstrated high user engagement and promising cessation rates compared to the National Cancer Institute's SmokefreeTXT (SFT) text messaging program-particularly among those who viewed all 42 days of program content: 30-day complete-case, point prevalence abstinence (PPA) rates at three-month follow-up were 63% (39/62) for QuitBot vs. 38% (45/117) for SFT (OR = 2.58; 95% CI: 1.34, 4.99; P =.005). However, Facebook Messenger (FM) intermittently blocked participants' access to QuitBot so we transitioned from FM to a standalone smartphone app as the communication channel. Participants' frustration with QuitBot's inability to answer their open-ended questions lead to us develop a core conversational feature enabling users to ask open-ended questions about quitting cigarette smoking and for the QuitBot to respond with accurate and professional answers. To support this functionality, we developed a library of 11,000 QnA pairs on topics associated with quitting cigarette smoking. Model testing results showed that Microsoft's Azure-based QnA maker effectively handled questions that matched our library of 11,000 QnA pairs. A fine-tuned, contextualized GPT3.5 responds to questions that are not within our library of QnA pairs.The development process yielded the first LLM-based quit smoking program delivered as a conversational chatbot. Iterative testing led to significant enhancements, including improvements to the delivery channel. A pivotal addition was the inclusion of a core LLM-supported conversational feature allowing users to ask open-ended questions.ClinicalTrials.gov Identifier, NCT03585231.