A Multicenter Assessment of the Accuracy of Claims Data in Appendicitis Research
Brendin R. Beaulieu-Jones MD, MBA1, *Aksel D. Laudon 1, *Swetha Duraiswamy 1, *Frank Yang 2, *Elizabeth Chen 3, *David R. Flum 2, *Kasey Lerner 4, *Heather Evans 4, *Alex Charboneau 5, *Vlad Simianu 5, *Lauren Thompson 6, *Faris Azar 6, *Victoria Valdes 7, Chaitan Narsule 7, *Sabrina Sanchez 1, Frederick T. Drake 1
1General Surgery, Boston Medical Center, Boston, MA; 2General Surgery, University of Washington, Seattle, WA; 3Western University of Health Services, Pomona, CA; 4Medical University of South Carolina, Charleston, SC; 5Virginia Mason Medical Center, Seattle, WA; 6General Surgery, St. Mary's Medical Center, West Palm Beach, FL; 7General Surgery, Lahey Health Medical Center, Burlington, MA
Background: Health services research on appendicitis often relies on administrative databases and claims data; however, the ability of billing codes to accurately identify complicated appendicitis may be limited. In a single institution study, we showed that ICD-9/10 codes performed poorly in identifying complicated appendicitis, leading to significant misclassification in research as well as inaccuracies in hospital billing (with tendency toward underpayment). To validate these findings, we performed a nationwide, multicenter study to investigate accuracy of appendicitis ICD-9/10 codes.
Study Design: We conducted a multicenter cohort analysis of adult patients with acute appendicitis who presented to one of 6 US medical centers during 2012-2015 (ICD-9 era) and 2018-2021 (ICD-10 era). Potentially eligible patients were randomly selected from a cohort identified based on ICD-9/10 codes. Diagnosis was then confirmed via chart review based on pre-specified clinical and/or radiographic criteria. Each patient was characterized as complicated or uncomplicated based on American Association for the Surgery of Trauma system for disease severity, which we considered our clinical gold standard. Radiologic characteristics were used for non-operatively managed patients and the operative description for surgical patients. Test parameters (e.g., sensitivity) and 95% confidence intervals were calculated.
Results: 2061 patients were screened and 1832 met inclusion criteria. 54.1% of patients were male, 25% non-white and 44% publicly insured or uninsured. In total, 21.1% (387/1832) of patients had complicated appendicitis based on our gold standard definition: 18.8% (312/1661) of surgical patients and 43.9% (75/171) of non-operative patients (p<0.001). Among all patients, 17.3% had a billing code for complicated appendicitis (12.5% true positives and 4.8% false positives). Overall accuracy was 86.6% and ranged from 81.0% to 93.8% across the 6 study sites. Sensitivity and positive predictive value (PPV) for complicated appendicitis were 0.59 (95%CI: 0.54-0.64) and 0.72 (0.67-0.77) respectively [Table 1]. Payor status was associated with differences in rate of non-operative management (commercial 7.2%, public 9.9%, p=0.05); however, PPV did not vary (commercial 70.5%, public 78.2%, p=0.22).
Conclusion: Billing codes have a poor sensitivity and PPV for distinguishing complicated from uncomplicated appendicitis. This multicenter study validates results from our prior single-center study, and these results have significant implications for how we should interpret data from administrative database studies of appendicitis. Going forward, these data can enable researchers to objectively quantify misclassification inherent to studies on appendicitis—and its impact on study endpoints. Inaccuracies in diagnosis codes may also negatively impact hospital reimbursement for appendicitis care, which should align stakeholders in promoting efforts to improve accuracy of billing codes.
Table 1 - Diagnostic Accuracy of ICD-9/10 Codes to Detect Complicated Appendicitis (2x2 Table)
| Gold Standard | |||||
| Complicated (n=387) | Uncomplicated (n=1445) | ||||
| Discharge ICD Code | Complicated (n=317) | 229 (12.5) | 88 (4.8) | 0.722 (0.671 to 0.769) | PPV (95% CI) |
| Uncomplicated (n=1515) | 158 (8.6) | 1357 (74.1) | 0.896 (0.879 to 0.910) | NPV (95% CI) | |
| 0.592 (0.542 to 0.640) | 0.939 (0.926 to 0.950) | ||||
| Sensitivity (95% CI) | Specificity (95% CI) | ||||
Back to 2024 Posters