Using Natural Language Processing to Enable Quality Improvement and Future Research for Patients at Risk of Suicide

Authors

Jennifer M. Boggs, Institute for Health Research, Kaiser Permanente Colorado, Denver, COFollow
J. David Powers, Institute for Health Research, Kaiser Permanente Colorado, Denver, CO
LeeAnn M. Rohm, Institute for Health Research, Kaiser Permanente Colorado, Denver, CO
Steve Hochberg, Institute for Health Research, Kaiser Permanente Colorado, Denver, CO
Arne Beck, Institute for Health Research, Kaiser Permanente Colorado, Denver, COFollow

Publication Date

8-10-2017

Keywords

behavioral and mental health, clinical practice patterns/guidelines, natural language processing, quality improvement

Abstract

Background: Policy research demonstrates that lethal means restriction is an effective strategy for preventing suicide. This evidence has informed national suicide prevention recommendations for medical/behavioral providers to assess patient access to lethal means. The clinical uptake and effectiveness of this practice is unknown. Behavioral health providers document means assessment and restriction counseling in progress notes, but not in structured form for easy extraction. We developed a natural language processing (NLP) query to identify lethal means assessment and restriction counseling within semi-structured clinical notes.

Methods: A total cohort of 11,259 adult Kaiser Permanente Colorado patients who had either a suicide attempt or positive suicide item on the Patient Health Questionnaire (PHQ-9) depression measure from 2010 to 2015 was queried. To identify key terms for the NLP query, we used manual chart review and text mining. All encounters for 1 month following index event (attempt or ideation) were reviewed for 100 patients. Using manual chart review as the gold standard, supervised text mining identified terminology indicating evidence of means documentation. Text mining, clinical consultation and chart review results informed query criteria that were implemented using open source NLTK Python package for NLP. The query was tested and modified throughout three iterations. Negative/positive hits were analyzed on stratified random samples of 40 charts/round. The final query was validated using manual review on a hold-out sample of 200 charts.

Results: We will present a description of the final query, including terms/phrases used, qualifiers, stop words and synonyms. Sensitivity, specificity and positive/negative predictive values for assessment of lethal means and means restriction counseling will be reported on the development and hold-out samples from the final query.

Conclusion: The query will allow us to identify the proportion of high-risk patients who receive recommended assessment of lethal means following suicide attempts/ideation. This query could be used for operational quality improvement or to inform future research on the effectiveness of lethal means assessment/counseling to restrict means in preventing suicide outcomes. Detailed specifications on the methods used to create this NLP query will be made available as a resource for other systems.

Recommended Citation

Boggs JM, Powers JD, Rohm LM, Hochberg S, Beck A. Using natural language processing to enable quality improvement and future research for patients at risk of suicide. J Patient Cent Res Rev. 2017;4:188-9.