Fig. 2

Receiver operating characteristic curves (ROC) for predicting the composite of death or organ failure at 30 days after hospitalization for COVID-19. (A) development and (B) internal validation datasets stratified according to individual machine learning models; Fig. 2C shows ROC for predicting outcome stratified by the new CORE-COVID-19 and 4 existing risk prediction tools. CORE-COVI-19 model consistently outperformed each existing risk prediction tools. Fig. 2D and E showed decision curve analysis stratified according to machine learning models in development (D) and validation (E) data sets. Fig. 2F illustrate decision curve analysis stratified by CORE-COVID-19 and other existing risk prediction tools for outcome prediction with net benefit of CORE-COVID-19 exceeding that of other models at wide range of thresholds. The "intervention for all" indicated net benefit from 0 to 0.15 below 20% of threshold probability. The ML models achieved the best net benefit at around .07–.08 when the threshold probability approached the minimum in the training dataset. The models still showed net benefit when the threshold probability rose to approximately 75%; the GBM even showed net benefit at above 80% of threshold probability. On the validation data set, the best net benefit ranged between .05–.07, and the models offered net benefit at around 70% of threshold probability at most. The maximum net benefit for CORE-COVID-19 model was best at 0.1 threshold and continued to show net benefit at above 55% of threshold probability which was higher than existing prediction tools. ISARIC-4C had its best net benefit, which was comparable to ML models in training, but the maximum threshold probability showing net benefit was only around 35%. The qSOFA presents net benefit at above 50% of threshold probability but its best net benefit was only approximately 0.03. Abbreviations: AUC, area under receiver operating characteristic curve; CORE-COVID-19, Collaboration for Risk Evaluation in COVID-19; CURB-65, confusion, urea, respiratory rate, blood pressure, and age ≥ 65 years; ISARIC-4C, International Severe Acute Respiratory and emerging Infections Consortium Coronavirus Clinical Characterization Consortium; qSOFA, quick sequential organ failure assessment; MEWS, modified early warning score