A Proposed Synthesis for Surgeon Intuition and Machine Learning: A Prospective Study Quantifying the Prognostic Value of Surgeon Intuition
Jayson S. Marwaha1, Brendin R. Beaulieu-Jones1, Hao Wei Chen1, William Yuan2, Chris Kennedy3, Charles H. Cook1, Gabriel A. Brat1
1Surgery, Beth Israel Deaconess Medical Center Department of Surgery, Boston, Massachusetts, United States, 2Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States, 3Department of Psychiatry, Massachusetts General Hospital, Boston, Massachusetts, United States
Objective: Surgical risk prediction models typically use measures of patient physiology (e.g., laboratory values) to make predictions about patient outcomes. However, the surgeon's assessment of the patient may be a valuable predictor as well, given the surgeon's ability to detect things that a statistical model cannot capture. The purpose of this study was to quantify the predictive utility of surgeon intuition and incorporate it in risk prediction models. Design: From October 2021 - February 2022, surgeons at an academic medical center were surveyed immediately before and after surgical procedures on their perceived risk of the patient developing postoperative complications (intuition data). 26 clinical variables were also collected on these patients from an institutional registry of surgical patients (physiology data). Both sources of data were used to build models to predict the likelihood of experiencing any 30-day postoperative complications as defined by the National Surgical Quality Improvement Program (NSQIP) of the American College of Surgeons (ACS). Setting: Single academic medical center. Patients: All patients undergoing surgery at our institution. Main Outcome Measures: Likelihood of any NSQIP-defined 30-day postoperative complication. Results: 7,143 patients with retrospectively collected physiology data were included who underwent surgery at our institution from January 1, 2017 - January 20, 2022. 102 of these patients also had prospectively collected surgeon intuition data. A model trained on only physiology data had an AUC of 0.78 in predicting any postoperative complication. A model trained on only preoperative intuition data had an AUC of 0.73. A model trained on postoperative intuition data had an AUC of 0.86. A model trained on physiology and intuition combined had an AUC >0.95. Conclusions: Postoperative surgeon intuition alone is a strong predictor of patient outcomes and may be an effective way to incorporate intraoperative events into risk prediction models. Models trained on both patient physiology and intuition exhibit the highest performance. Researchers should consider training future surgical risk prediction models on both sources of data.
Table 1. Performance of models trained on physiology data and intuition data in predicting 30-day postoperative complications.
| Model Training Data Source | AUC (95% CI) |
| Physiology data | 0.78 (0.75-0.81) |
| Preoperative intuition data | 0.73 (0.58-0.88) |
| Postoperative intuition data | 0.86 (0.74-0.98) |
| Physiology, preoperative intuition, and postoperative intuition combined | >0.95 (--) |
Back to 2022 Abstracts