ABSTRACT
Many popular disease transmission models have helped nations respond to the COVID-19 pandemic by informing decisions about pandemic planning, resource allocation, implementation of social distancing measures and other non-pharmaceutical interventions. We compare five epidemiological models for forecasting and assessing the course of the pandemic. We compare how the models analyze case-recovery-death count data in India, the country with second highest reported case-counts in a world where a large proportion of infections remain undetected. A baseline curve-fitting model is introduced, in addition to three compartmental models: an extended SIR (eSIR) model, an expanded SEIR model developed to account for infectiousness of asymptomatic and pre-symptomatic cases (SAPHIRE), another SEIR model to handle high false negative rate and symptom-based administration of tests (SEIR-fansy). A semi-mechanistic Bayesian hierarchical model developed at the Imperial College London (ICM) is also examined. Using COVID-19 data for India from March 15 to June 18 to train the models, we generate predictions from each of the five models from June 19 to July 18. To compare prediction accuracy with respect to reported cumulative and active case counts and cumulative death counts, we compute the symmetric mean absolute prediction error (SMAPE) and mean squared relative prediction error (MSRPE) for each of the five models.
For active case counts, SEIR-fansy yields an SMAPE value of 0.72, and the eSIR model yields a value of 33.83. For cumulative case counts, SMAPE values are 1.76 for baseline model, 23.10 for eSIR, 2.07 for SAPHIRE and 3.20 for SEIR-fansy. For cumulative death counts, the SEIR-fansy model performs the best, with an SMAPE of 7.13, as compared to 26.30 for the eSIR model. Using Pearson’s correlation coefficient and Lin’s concordance correlation coefficient, for cumulative case counts, the baseline model exhibits highest correlation (both Pearson’s as well as Lin’s coefficient), while for cumulative death counts, projections from SEIR-fansy exhibit the best performance: For cumulative cases, correlation coefficients computed for the baseline model are 1 (Pearson) and 0.991 (Lin). For eSIR, those values are 0.985 (Pearson) and 0.316 (Lin). For SAPHIRE, we compute 1 (Pearson) and 0.975 (Lin). Finally, for SEIR-fansy we have those values at 1 (Pearson) and 0.965 (Lin). Similarly, for cumulative deaths, correlation coefficients computed for eSIR is 0.978 (Pearson) and 0.206 (Lin), and for SEIR-fansy we have those values at 0.999 (Pearson) and 0.742 (Lin). Three models (SAPHIRE, SEIR-fansy and ICM) return total (sum of reported and unreported) counts as well. We compute underreporting factors on two specific dates (June 30 and July 10) and note that on both dates, the SEIR-fansy model reports the highest underreporting factor for active cases (June 30: 6.10 and July 10: 6.24) and cumulative deaths (June 30: 3.62 and July 10: 3.99) for both dates, while the SAPHIRE model reports the highest underreporting factor for cumulative cases (June 30: 27.79 and July 10: 26.74).
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
The authors would like to thank the Center for Precision Health Data Sciences at the University of Michigan School of Public Health, The University of Michigan Rogel Cancer Center and the Michigan Institute of Data Science for internal funding that supported this research.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Not applicable
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
All data and code available at covind19.org