Quantitative Analysis of Surgical Residency Reform; are case logs a valid tool for assessment?
*Sarah B Cairo1,2, *Wendy Craig3, *Caitlin Gutheil3, *Paul K. J. Han3,4, *Kristiina Hyrkas3, *Hannah Kay3, *Lynda Macken3, James F Whiting2,5
1Women and Children's Hospital of Buffalo, Buffalo, NY;2Maine Medical Center, Department of Surgery, Portland, ME;3Center for Outcomes Research and Evaluation and Maine Medical Center Research Institute, Portland, ME;4Palliative Medicine, Hospice of Southern Maine, Scarborough, ME;5Tufts University School of Medicine at Maine Medical Center, Portland, ME
Objective: Curricular changes to surgical residency, developed to rebalance clinical rotations, optimize education over service, decrease the size of service teams, and integrate apprenticeship-type experiences, were implemented in June 2015. This study quantifies the operative experience before and after implementation as part of a mixed-methods program evaluation. Design: Retrospective review of the Accreditation Council for Graduate Medical Education (ACGME) case-log data entered by categorical general surgery residents and data from the American College of Surgeons National Surgical Quality Improvement Program: Quality In-Training Initiative (NSQIP-QITI) pre- and post-intervention at a mid-sized surgical training program. Results: Case logs were reviewed for all categorical residents pre-intervention (2014-15) and post-intervention (2015-16). A total of 11,365 cases, excluding “first-assistant” and “endoscopic” cases were logged for an average of 291 and 263 cases/resident pre- and post-intervention, respectively. The average increased significantly for PGY 3 residents and decreased significantly for PGY 4 residents (table 1). Significant variability was observed between residents at the same PGY level and appeared greatest for PGY 1 and senior (PGY 4, PGY 5) trainees. At the senior level, variability is attributed to resident preference/career choice. When compared to data from QITI, the variability at the PGY 1 level dissipated and was attributed to errors in resident case log behavior. Conclusion: Large variability within PGY level suggests that resident preference may be as influential as curriculum design in dictating operative experience. Statistically significant differences in total average cases/resident for PGY 3/4 trainees were observed and are of uncertain clinical significance.
Table 1. ACGME Log case distribution before and after curriculum change, by post-graduate year (PGY) | |||||||||||
2014 – 2015 (pre-curriculum change) | 2015 – 2016 (post-curriculum change) | ||||||||||
Total | Average cases per resident | Median cases per resident | Min | Max | Total | Average cases per resident | Median cases per resident | Min | Max | t-testa | |
Total cases excluding first assist, endoscopy | |||||||||||
PGY 1 | 375 | 94 | 95.5 | 55 | 129 | 396 | 99 | 85.5 | 35 | 190 | p = 0.891 |
PGY 2 | 978 | 244 | 221 | 218 | 318 | 746 | 186 | 181.5 | 146 | 237 | p = 0.135 |
PGY 3 | 1,047 | 262 | 256.5 | 250 | 284 | 1,413 | 353 | 363 | 301 | 382 | p = 0.005 |
PGY 4 | 1,469 | 367 | 355 | 340 | 424 | 1,131 | 283 | 278 | 253 | 341 | p = 0.037 |
PGY 5 | 2,241 | 448 | 474 | 303 | 551 | 1,569 | 392 | 367.5 | 361 | 480 | p = 0.387 |
Colectomy | |||||||||||
PGY 1 | 1 | 0 | 0 | 0 | 1 | 9 | 2.3 | 1 | 0 | 7 | p = 0.276 |
PGY 2 | 14 | 3.5 | 3 | 1 | 7 | 16 | 4 | 2 | 0 | 12 | p = 0.877 |
PGY 3 | 26 | 6.5 | 5.5 | 4 | 11 | 37 | 9.3 | 8 | 5 | 16 | p = 0.386 |
PGY 4 | 115 | 28.8 | 29 | 20 | 37 | 131 | 32.8 | 35 | 20 | 41 | p = 0.542 |
PGY 5 | 197 | 39.4 | 40 | 29 | 55 | 182 | 45.5 | 42.5 | 30 | 67 | p = 0.543 |
All Hernia Repairs (pediatric and adult) | |||||||||||
PGY 1 | 51 | 12.8 | 12 | 2 | 25 | 50 | 12.5 | 7 | 1 | 35 | p = 0.979 |
PGY 2 | 97 | 24.3 | 24 | 20 | 29 | 81 | 20.3 | 19 | 3 | 40 | p = 0.627 |
PGY 3 | 120 | 30 | 29.5 | 18 | 43 | 135 | 33.8 | 37.5 | 16 | 44 | p = 0.671 |
PGY 4 | 94 | 23.5 | 20.5 | 18 | 35 | 134 | 33.5 | 31 | 29 | 43 | p = 0.103 |
PGY 5 | 279 | 55.8 | 45 | 14 | 107 | 185 | 46.3 | 46 | 32 | 61 | p = 0.625 |
aStudent’s t-test to compare mean pre- and post-curriculum change. Due to small sample size, high risk type I error |
Back to 2017 Program