Linked Analysis for Definition of Nurse Advice Line Syndrome Groups, and Comparison to Encounters

Persons using assistive technology might not be able to fully access information in this file. For assistance, please send e-mail to: mmwrq@cdc.gov. Type 508 Accommodation and the title of the report in the subject line of e-mail.

Steven F. Magruder,¹ J. Henry,² M. Snyder^{2

1}Johns Hopkins University, Laurel, Maryland; ²Kaiser Permanente of the Mid-Atlantic States, Rockville, Maryland

Corresponding Author: Steven F. Magruder, Johns Hopkins University Applied Physics Laboratory, 11100 Johns Hopkins Rd., Laurel, MD 20723-6099. Telephone: 443-778-6537; Fax: 443-778-6885; E-mail: steve.magruder@jhuapl.edu.

Disclosure of relationship: The contributors of this report have disclosed that they have no financial interest, relationship, affiliation, or other association with any organization that might represent a conflict of interest. In addition, this report does not contain any discussion of unlabeled use of commercial products or products for investigational use.

Abstract

Introduction: Nurse advice call centers are a potentially important source of data for syndromic surveillance purposes. For this reason, researchers at Johns Hopkins University Applied Physics Laboratory and Kaiser Permanente of the Mid-Atlantic States (KPMAS) have been collaborating to develop methods to use this data within the ESSENCE II Syndromic Surveillance System in the National Capital Region.

Objective: The objective of this report is to present a general method for finding syndrome groups in data sources that can be linked to physician encounters and to determine effective advice call syndrome groups for use with KPMAS advice data.

Methods: Advice calls are linked to physician encounters and stratified by patient age. They are placed in groups according to a maximum positive predictive value criterion. The groups are evaluated by correlating the resulting syndrome time series against physician encounter data.

Results: Potentially useful advice syndrome groups are found for respiratory, lower gastrointestinal (GI), and total GI syndromes for each age stratum.

Conclusion: The time series of the advice data for respiratory, lower GI, and upper GI syndromes accurately predict the physician encounter time series for the corresponding syndromes for each age stratum.

Introduction

Advice lines are facilities such as nurse call centers that receive telephone calls from persons requiring information, triage, or immediate assistance. Government agencies, health-care systems (especially HMOs), or private contractors can operate these facilities. Advice lines gather information about a caller's complaints that enables them to allocate appropriate clinical resources to that patient with appropriate urgency. This practice results in an electronic record carrying syndromic information.

Nurse advice data for syndromic surveillance purposes have several potential advantages. The syndromic information is immediately captured in an electronic format. The information has been interpreted by a medical professional. The advice call event might occur either before or instead of a physician encounter (1). Nurse advice call volume does not decline as sharply on weekends and holidays as does physician-encounter volume. For these reasons, researchers at Johns Hopkins University Applied Physics Laboratory and Kaiser Permanente of the Mid-Atlantic States (KPMAS) have been collaborating to develop methods to use this data within the ESSENCE II Syndromic Surveillance System in the National Capital Region (2). The potential use of this type of data for surveillance has also received attention in the United Kingdom (3).

At KPMAS, appointment scheduling and the nurse advice hotline function together within the KPMAS call center, which serves as a major entry point into the delivery system. Nurses operate the advice hotline, administering protocol-driven, medically appropriate advice and scheduling acute-care office visits when necessary. By this process, each call is assigned one of 527 possible advice guidelines (e.g., diarrhea, adult, pinworms, or contact exposure) indicating syndromic information. This report addresses how best to aggregate this information to characterize trends in broad syndrome groups, particularly those defined by CDC relating to critical biologic terrorism--associated agents (4). A previous paper on the use of advice guideline groupings for syndromic surveillance evaluated the performance of groupings chosen on the basis of names of the advice protocols and their presumed usage (1). For this study, a method was used for finding empirical groupings.

Methods

The KPMAS information system captures the date and time a patient was seen for an outpatient office visit and the date and time the patient called to schedule that appointment. A third record contains the date and time the patient contacted the nurse advice hotline. For this study, a nurse advice hotline call and an outpatient office visit are defined as linked if all of the following criteria were met: 1) the patient identification number assigned to the nurse advice hotline call is the same as the patient identification number assigned to the office visit; 2) the calendar date of the nurse advice hotline call is the same as the calendar date when the patient called for the appointment; and 3) the time of the nurse advice hotline call was equal to, or earlier, than the time the patient called for an appointment.

Using the KPMAS information system, a database was created that contained all nurse advice calls made during calendar year 2002 that could be linked to corresponding physician encounters and International Classification of Diseases, Ninth Revision (ICD-9)-coded physician diagnoses. Approximately 38% of all advice calls were linked in this way for approximately 570,500 linked call-encounter pairs. These physician diagnoses provide a standard for classifying a case into a particular syndrome group. Visits were grouped according to the CDC diagnosis-based biologic terrorism syndrome groups (including categories 1 and 3 only) (4) (Table 1). Approximately 26% of the linked call-encounter pairs were classified into one of these CDC groups.

For each advice guideline, the fraction of the calls linked to diagnoses in each syndrome group and the fraction linked to diagnoses outside of all the groups (i.e., falls in the nonsyndromic group) were determined. These fractions represent the historic positive predictive value (PPV) of each advice guideline for predicting each syndrome type. Advice syndrome groups were then formed by including in a given group only those guidelines whose PPV is greater for that group than for any other group (including the nonsyndromic group). If PPV was equal for two syndrome groups, the same guideline was assigned to both groups.

The advice syndrome groups formed by this procedure are then evaluated by comparing time series for advice calls (daily counts in each syndrome group) against physician diagnoses time series in each syndrome group, using a different period than the one used to define the groups. The period used for this analysis was January 2003--November 2004. Rather than estimating marginal probabilities that a call will fall into a given syndrome group, time series were compared because the identification of disease anomalies typically depends on observations of temporal trends and because this allowed inclusion of both linked and unlinked advice calls and physician visits in the evaluation.

For each syndrome group, r² (i.e., the fraction of the variance in the physician diagnosis daily counts explained by the nurse advice daily counts) was calculated. This comparison of same-day counts was conducted because of the short observed median time lags (Table 1). Nevertheless, the actual correspondence between calls and encounters might be higher than the same-day comparisons suggest. Except for day of week, holiday, and snow corrections described below, r²is calculated directly from the time series of daily counts. No corrections are made for serial autocorrelation or for seasonality; all time scales are assumed to be equally valid for evaluating the correspondence between these two data sources.

For some comparisons, the data by patient age (which is available both in the nurse advice records and in the physician encounter records) was stratified. The age classes included infant (aged 0--1 years), pediatric (aged 2--17 years), and adult. For some comparisons, data were stratified by day of week (i.e., weekends and holidays, Mondays and days after holidays, and other weekdays). When day of week was not stratified, each daily data count was divided by the mean value for its day-of-week class and then the data was smoothed with a 7-day moving average before calculating r². This step was necessary to correct for differing day-of-week patterns in the advice and encounter time series. The 7-day average was used to improve the statistical strength of the data when stratifying by age (i.e., daily counts for a given age and syndrome group were sometimes too small to provide meaningful results). The use of 7-day averages also mitigates any loss of correlation caused by small lags between the time of an advice call and the time of a physician encounter.

Before any other processing steps, 8 days were culled from the data set because of large snowfalls, which apparently reduced the numbers of physician encounters without noticeably affecting the numbers of advice calls.

Use of Positive Predictive Value

The goal in defining syndrome groups is to provide a good estimate for any given day of the number of advice calls that will eventually result in physician diagnoses falling in a given syndrome group. Denoting the number of advice calls using guideline i as a_i, the number of those that would be diagnosed in the syndrome group (given a physician examination) as s_i , and the sum of all calls that would be diagnosed into the syndrome group as S, the object is to find coefficients, f_i that minimize the mean squared error, E given by

In this case, the mean is taken over an ensemble of days on which advice call and physician encounter counts are measured. Ignoring correlations between numbers of advice calls following different guidelines, the condition (obtained by differentiating Equation 1 with respect to f_i) for minimizing the squared error can be written as

where overbars denote mean value. If it is further assumed that the number of advice calls that would not be diagnosed in the syndrome group are uncorrelated with those that would, and also that fluctuations both in a and in s follow a Poisson distribution (so that variances equal means), the equation would be

In words, f_i is the positive predictive value of guideline i for predicting the syndrome in question.

Advice syndrome groups could be formed as weighted sums, as implied by Equation 1. However, assigning each advice guideline to a single group adds clarity to the interpretation of the data. When unique assignments are desired, each guideline should be assigned to the group for which it would be most heavily weighted, the one for which it has the greatest PPV.

Results

Using the method described, empirical nurse advice syndrome groups were identified for fever, lower gastrointestinal (GI), upper GI, total GI, rash, and respiratory syndromes. All advice guidelines in the upper and lower GI groups also fall in the total GI group. When estimating whether a guideline belongs in upper or lower GI, the total GI group was not considered as an alternative diagnosis. No advice guidelines had maximum PPV for any of the other CDC syndrome groups. The advice calls falling into the fever, upper GI, and rash syndromes were too infrequent to be useful as a predictor of the encounter data (Table 2). Potentially useful advice guideline groupings were found for the lower GI, total GI, and respiratory syndrome groups (Tables 3--5). R-squared values calculated for these groupings and for the various subpopulations described previously fell in the range 63%--95% (Table 6). These values were determined by correlating the counts of a given syndrome group constructed on the nurse advice guidelines with the (same day) counts of the same syndrome obtained from physician encounters, as explained in detail in the methods section.

Discussion

Certain syndrome group definitions contain entries that seem to be spurious coincidences (e.g., "ADMINISTRATIVE NOTE" in the infant GI and Respiratory syndromes or "FEEDING, NEONATAL" in Adult GI). These advice guidelines were rare and have negligible effect. They were left in the list to keep the methodology clear; no ad hoc editing of the syndrome groups was performed.

Certain entries seem inconsistent with respect to age groups (e.g., some guidelines labeled "adult" are included in the infant syndrome groups, and some labeled "peds" are included in the adult groups). This might indicate some inconsistent use of the age designations in the guideline names, but the actual age of the patient is recorded in the nurse advice call records; therefore, these ambiguities do not interfere with the ability in practice to stratify guideline syndrome groups by age. These age-inconsistent guideline names should not pose a problem.

One type of finding in these results consists of advice guidelines that are unexpectedly missing from these groupings. For example, adult nausea and adult vomiting are not included because it was discovered that these complaints most often result in diagnoses falling outside of the CDC syndrome groups. However, if a particular public health threat of concern could present itself as adult nausea and vomiting only, then it would be important to monitor this channel also, but perhaps not aggregated with the other complaints, which are more specific to the GI syndrome as it usually occurs. Other similar examples are adult upper respiratory infection and respiratory distress in adults. The absence of acute laryngitis in the adult list has a different type of explanation; 4-digit ICD-9 codes were sometimes recorded in the database, while the CDC syndrome definitions only include the 5-digit codes for acute laryngitis. For methodological clarity, no attempt was made to "fix" this type of problem.

The most marginal case is in respiratory syndrome for infants; this case has a relatively low number of identifiable nurse advice calls (Table 2).

Conclusion

A patient-linked mapping of nurse advice guidelines to physician diagnoses has yielded advice syndrome groups that accurately track the temporal behavior of the CDC diagnosis-based groups for respiratory, lower GI, and total GI syndromes separately for infants, pediatrics, and adults in approximately 2 years of data collected in the National Capital Region. This same technique indicates that advice syndrome groups that might be created for other single CDCdefined syndromes will be substantially less specific than the diagnosis-based syndrome groups; more cases would probably be determined to fall outside of the syndrome groups than in them on examination by a physician.

References

Use of trade names and commercial sources is for identification only and does not imply endorsement by the U.S. Department of Health and Human Services.

References to non-CDC sites on the Internet are provided as a service to MMWR readers and do not constitute or imply endorsement of these organizations or their programs by CDC or the U.S. Department of Health and Human Services. CDC is not responsible for the content of pages found at these sites. URL addresses listed in MMWR were current as of the date of publication.

Disclaimer All MMWR HTML versions of articles are electronic conversions from ASCII text into HTML. This conversion may have resulted in character translation or format errors in the HTML version. Users should not rely on this HTML document, but are referred to the electronic PDF version and/or the original MMWR paper copy for the official text, figures, and tables. An original paper copy of this issue can be obtained from the Superintendent of Documents, U.S. Government Printing Office (GPO), Washington, DC 20402-9371; telephone: (202) 512-1800. Contact GPO for current prices.