A methodological approach to validate pneumonia encounters from radiology reports using Natural Language Processing (NLP)
Panny A, Hegde H, Glurich I, et al. A methodological approach to validate pneumonia encounters from radiology reports using Natural Language Processing (NLP) [published online ahead of print, 2022 Apr 5]. Methods Inf Med. 2022;10.1055/a-1817-7008. doi:10.1055/a-1817-7008
Introduction: Pneumonia is caused by microbes that establish an infectious process in the lungs. The gold standard for pneumonia diagnosis is radiologist-documented pneumonia-related features in radiology notes that are captured in electronic health records in an unstructured format.
Objective: The study objective was to develop a methodological approach for assessing validity of a pneumonia diagnosis based on identifying presence or absence of key radiographic features in radiology reports with subsequent rendering of diagnostic decisions into a structured format.
Methods: A pneumonia-specific Natural Language Processing (NLP) pipeline was strategically developed applying cTAKES to validate pneumonia diagnoses following development of a pneumonia feature-specific lexicon. Radiographic reports of study-eligible subjects identified by International Classification of Diseases (ICD) codes were parsed through the NLP pipeline. Classification rules were developed to assign each pneumonia episode into one of three categories: "positive", "negative" or "not classified: requires manual review" based on tagged concepts that support or refute diagnostic codes.
Results: A total of 91,998 pneumonia episodes diagnosed in 65,904 patients were retrieved retrospectively. Approximately 89% (81,707/91,998) of the total pneumonia episodes were documented by 225,893 chest x-ray reports. NLP classified and validated 33% (26,800/81,707) of pneumonia episodes classified as 'Pneumonia-positive', 19% as (15401/81,707) as 'Pneumonia-negative' and 48% (39,209/81,707) as ''episode classification pending further manual review'. NLP pipeline performance metrics included accuracy (76.3%), sensitivity (88%), and specificity (75%).
Conclusion: The pneumonia-specific NLP pipeline exhibited good performance comparable to other pneumonia-specific NLP systems developed to date.