ProKnow: Efficacy of Process Knowledge in Safety Constrained and Explainable Question Generation for Mental Health Diagnostic Assistance

Anxiety Disorder (AD), are prevalent, affecting 20.6% and 4.3% of the U.S. population, respectively, prior to the pandemic. Current Virtual Mental Health Assistants (VMHAs) primarily offer counseling and suggestive care but do not assist with patient diagnosis due to their lack of training in safety-constrained and specialized clinical process knowledge, referred to as ProKnow. This research aims to demonstrate a method for creating ProKnow-data and a viable ProKnow algorithm for a safety-constrained and explainable mental health diagnostic assistant. In this work, ProKnow was defined as an ordered set of information aligned with evidence-based guidelines or categories of conceptual understanding used by domain experts. This study also introduces a new dataset of diagnostic conversations guided by safety constraints and ProKnow, known as ProKnow-data. A method was developed for natural language question generation (NLG) designed to interactively gather diagnostic information from patients, termed ProKnow-algo. This study's findings highlight the limitations of state-of-the-art large-scale language models (LMs) when applied to this dataset. The ProKnow-algo method models process knowledge by explicitly incorporating safety, knowledge capture, and explainability. When used with ProKnow-algo, LMs generated 89% safer questions in the context of depression and anxiety. In contrast, without ProKnow-algo, the generated questions failed to adhere to the clinical process knowledge outlined in ProKnow-data. Furthermore, questions generated using ProKnow-algo exhibited a 96% reduction in average squared rank error. The explainability of the generated questions was assessed by measuring their similarity to concepts within depression and anxiety knowledge bases. Overall, regardless of the type of LM used, ProKnow-algo achieved an average improvement of 82% over simple pre-trained LMs in terms of safety, explainability, and process-guided question generation. The efficacy of ProKnow-algo was evaluated qualitatively and quantitatively by introducing three new evaluation metrics for safety, explainability, and adherence to process knowledge. Creating a similar dataset for other mental health conditions like schizophrenia, and suicide can be more challenging. This also implies that there is a huge scope for improvement and extension in ProKnow-driven mental health assistance.