Recommended Citation
Spencer C, Lee K, Sefen J, et al. Increasing AI-Related Patient Care Discussions in Family Medicine Residency Clinics. Oral presentation at: Alliance of Independent Academic Medical Centers (AIAMC) – National Initiative X: Meeting #3; October 9, 2026; Virtual.
Presentation Notes
Oral presentation at: Alliance of Independent Academic Medical Centers (AIAMC) – National Initiative X: Meeting #3; October 9, 2026; Virtual.
Abstract
Introduction: Artificial Intelligence (AI) tools, such as Large Language Model (LLM) powered medical search engines, are rapidly becoming integrated into clinical medicine, yet adoption and correct-use education remain inconsistent among residents and faculty. Building on previous findings that highlighted the need for curriculum refinement and faculty champions, this performance improvement project focused supervision discussions. Actively creating an open dialogue discussing AI utilization during staffing encounters presents a crucial teachable moment to model appropriate use, clear misconceptions, and guide residents in the strategic leveraging of these tools for patient care.
Aims: To increase by 20% the number of discussions occurring between supervising faculty and residents regarding the utilization of AI during staffing. This increase was measured from a baseline pre-period (June 11 - July 10, 2026) compared to the post-period (July 11 - September 11, 2026).
Methods: We conducted a targeted performance improvement initiative within our family medicine residency program. Interventions included dedicated discussions at residency meetings and focused programming during an annual academic retreat to encourage AI-related staffing conversations. Data was collected using an Epic Reporting Workbench report to pull all staffing notes containing a specific dot-phrase, “While staffing this patient with the resident, we reviewed the use of AI in the context of Medical Education: Yes/No”. The data was exported to an Excel file. We utilized AI tools (Copilot/Claude within system-approved programs) to systematically "mine" the qualitative data markers which had been stored as plain text in staffing note attestations. One high-performing faculty member was excluded from the final dataset as a confounder due to an unusually high baseline rate (>50%) and absence during the post-intervention period.
Results: A total of 1,168 staffing notes were analyzed between June 10, 2026, and September 11, 2026. The baseline AI discussion rate during the pre-period (June 11 - July 10) was 1.9%. Following the interventions, the discussion rate in the post-period (July 11 - September 11) surged to 9.2%. This represents a 384% relative increase, significantly exceeding the 20% project aim. The overall AI discussion rate across the entire period was 8.3%. A weekly review demonstrated peak discussion rates of 14% to 15% between weeks 33 and 35, corresponding directly with the residency retreat and meetings.
Discussion: We successfully and substantially increased faculty-resident discussions regarding AI use during outpatient staffing, demonstrating that targeted reminders and leadership engagement can drive immediate behavioral changes in clinical education. Additionally, this project highlighted an administrative utility of Large Language Models in the extraction of non-discrete data from EHR text to significantly streamline QI data analysis. Moving forward, ongoing efforts are needed to sustain intentional dialogue around the use of AI integration into clinical medicine within residency programs. Clinical AI tools are sure to become a permanent fixture of residency training, and programs have a responsibility to identify the best teaching strategies to encompass the broad complexity of these powerful tools.
Type
Oral/Podium Presentation
Affiliations
Aurora Health Care
Aurora Family Medicine Residency