Constant Specificity Surveillance for Real-Time Outbreak Detection

Shannon C. Wieland,^1,2 B. Berger,^1,3 K. Mandl^2,4
1Massachusetts Institute of Technology, Cambridge, Massachusetts; ²Children's Hospital, Boston, Massachusetts;
³Harvard-MIT Division of Health Sciences and Technology, Cambridge, Massachusetts; ⁴Harvard Medical School, Boston, Massachusetts

Corresponding author: Shannon C. Wieland, Massachusetts Institute of Technology, 290 Massachusetts Avenue, Cambridge, MA 02139. Telephone: 617-645-6875; E-mail: shann@mitt.edu.

Disclosure of relationship: The contributors of this report have disclosed that they have no financial interest, relationship, affiliation, or other association with any organization that might represent a conflict of interest. In addition, this report does not contain any discussion of unlabeled use of commercial products or products for investigational use.

Abstract

Introduction: Certain modeling techniques have been implemented to detect unusually high emergency department (ED) visit rates on the basis of historical data, including parametric regression, autoregression, multiresolution wavelet analysis, and additive modeling. An outbreak-detection strategy is informative only to the extent that its specificity is known because the probability of a false alarm in the absence of an outbreak should be understood to allocate resources appropriately in the event of an alert that triggers a public health investigation.

Objectives: This study demonstrated that the specificity of outbreak detection using current methods varies substantially on multiple timescales. A modeling approach that provides constant specificity surveillance was developed.

Methods: Autoregressive, Serfling, trimmed seasonal, and wavelet-based outbreak detection models were evaluated for changes in specificity over time. All model simulations used 12 years of historical respiratory syndrome ED visits at a major pediatric hospital in an urban setting. Changes in specificity were detected by using error analyses modified for binomial data and chi-squared analysis. Sensitivity was evaluated by adding synthetic 1-day outbreaks among 10 patients to the historical data. A new outbreak- detection method was developed that used generalized additive models of both the ED visit mean and variance.

Results: The specificity of four previously published models (i.e., autoregressive, Serfling, trimmed seasonal, and wavelet-based) was a nonconstant function of the day of the week, the month of the year, or the year of the study (p<0.05). The seasonal changes in specificity led to a paradoxical increase in sensitivity to simulated outbreaks during winter months when compared with summer months. The new method had constant specificity over all three time scales (p<0.05) without a loss of sensitivity compared with previous models.

Conclusion: Interpretation of alarms using outbreak detection strategies is difficult because the specificity is extremely variable. The fluctuations in specificity are caused by changes on the same time scales in the variance of the ED use signal. For example, false alarms are more frequent during winter months when signal variance peaks. Previous models adjust for changes with time of the expected number of ED visits but fail to adjust for the changing variance of visits with the season, day-of-week, and longterm trend. The developed model accounts for changes with time of both the expected number of ED visits and the variance of the number of visits. This enables surveillance of known, constant specificity, enhancing the ability of public health practitioners to interpret the meaning of an alarm.

Use of trade names and commercial sources is for identification only and does not imply endorsement by the U.S. Department of Health and Human Services. References to non-CDC sites on the Internet are provided as a service to MMWR readers and do not constitute or imply endorsement of these organizations or their programs by CDC or the U.S. Department of Health and Human Services. CDC is not responsible for the content of pages found at these sites. URL addresses listed in MMWR were current as of the date of publication.

Disclaimer   All MMWR HTML versions of articles are electronic conversions from ASCII text into HTML. This conversion may have resulted in character translation or format errors in the HTML version. Users should not rely on this HTML document, but are referred to the electronic PDF version and/or the original MMWR paper copy for the official text, figures, and tables. An original paper copy of this issue can be obtained from the Superintendent of Documents, U.S. Government Printing Office (GPO), Washington, DC 20402-9371; telephone: (202) 512-1800. Contact GPO for current prices.

Date last reviewed: 8/5/2005