External validation of the QCovid 2 and 3 risk prediction algorithms for risk of COVID-19 hospitalisation and mortality in adults : national cohort study in Scotland

Abstract

Objective The QCovid 2 and 3 algorithms are risk prediction tools developed during the second wave of the COVID-19 pandemic that can be used to predict the risk of COVID-19 hospitalisation and mortality, taking vaccination status into account. In this study, we assess their performance in Scotland. Methods We used the Early Pandemic Evaluation and Enhanced Surveillance of COVID-19 national data platform consisting of individual-level data for the population of Scotland (5.4 million residents). Primary care data were linked to reverse-transcription PCR virology testing, hospitalisation and mortality data. We assessed the discrimination and calibration of the QCovid 2 and 3 algorithms in predicting COVID-19 hospitalisations and deaths between 8 December 2020 and 15 June 2021. Results Our validation dataset comprised 465 058 individuals, aged 19–100. We found the following performance metrics (95% CIs) for QCovid 2 and 3: Harrell’s C 0.84 (0.82 to 0.86) for hospitalisation, and 0.92 (0.90 to 0.94) for death, observed-expected ratio of 0.24 for hospitalisation and 0.26 for death (ie, both the number of hospitalisations and the number of deaths were overestimated), and a Brier score of 0.0009 (0.00084 to 0.00096) for hospitalisation and 0.00036 (0.00032 to 0.0004) for death. Conclusions We found good discrimination of the QCovid 2 and 3 algorithms in Scotland, although performance was worse in higher age groups. Both the number of hospitalisations and the number of deaths were overestimated.