Episode #7
The Promises and Pitfalls of AI in Residency Assessment with John Hayes, DO, and Karim Hanna, MD
How is artificial intelligence reshaping the way we train and assess the next generation of family physicians? In this episode, Dr John Hayes and Dr Karim Hanna explore the promises and pitfalls of AI in residency assessment, from automating milestone tracking and grading robust essay exams to getting started with your first automated workflow. They also discuss how when AI takes on more of the data analysis and synthesis tasks, faculty will need to increase their coaching skills. On the other side, our guests offer an honest look at the risks of over-reliance on technology, cautioning educators to guard against "un-skilling" learners who may already lean too heavily on AI for clinical reasoning. Learn how to incorporate AI as a powerful supplemental tool while preserving the human connection that lies at the heart of family medicine. Hosted by Omari A. Hodge, MD, FAAFP and Jay-Sheree Allen Akambase, MD Copyright © Society of Teachers of Family Medicine, 2026 Resources: Artificial Intelligence and Machine Learning for Primary Care Curriculum (AiMPC) Course Ethical Use of AI in the Family Medicine Clinic - STFM Webinar An Opportunity to Thrive - AI in Family Medicine - STFM Podcast AI Deep Dive Summer Series Episode 1: Integrating AI into Family Medicine Curriculum Design - STFM Podcast Learning after Training: The Master Adaptive Learner - STFM Podcast STFM Competency-based Medical Education Toolkit STFM Collaborative List - Artificial Intelligence in Medical Education Collaborative John Hayes DO Prior to his current roles, Dr Hayes served as a Family Medicine Clerkship Director, where he led a major curriculum modernization effort that enhanced the clerkship experience for medical students. Dr Hayes is now pioneering the integration of Artificial Intelligence in medical assessments, specifically in Objective Structured Clinical Exams (OSCEs). Additionally, he is developing a summative clinical reasoning examination designed to move away from traditional multiple-choice testing and into essay prompts with AI assisted grading. He is also exploring the use of ambient AI to enhance residency education and improve communication skills for both residents and faculty, aiming to foster a collaborative learning environment. He definitely used AI to help generate this blurb. Karim Hanna, MD Dr Karim Hanna is an Associate Professor at the University of South Florida Morsani College of Medicine and the founding Program Director for the USF TGH Family Medicine Residency. His work integrates primary care education and AI advancements. Dr Hanna's interests in health tech spurred his writing of a blog - MedEd+AI - with a focus on AI's role in medical education and clinical implications. His scholarly work includes research on LLMs for clinical applications and improving education through AI.
