Using Technologies for Data Collection and Management
Janet J. Hamilton and Richard S. Hopkins
- Definitions and Assumptions
- Guiding Principles for Selecting and Using Technologies
- Evolving Approach and Conceptual Shift in Field Deployments
- Packing Equipment and Preparing for Deployment
- Establishing Working Relationships and Initial Arrival
- Using Technologies Across Outbreak Investigation Phases
- Tracking and Managing Other Types of Information
- Evaluating the Role Workforce Plays in Technology Use
- Using Public Health Informatics
- Using Traditional or Widely Used Applications
- Understanding the Future of Technology in Field Investigations
- References
Technologies and surveillance systems play an integral, increasing, and evolving role in supporting public health responses to outbreaks or other urgent public health events. The functions supported might include event detection, event characterization, enhanced surveillance, situational awareness, formal epidemiologic investigations, identification and management of exposed persons, and monitoring of the response itself and its effectiveness. In any field investigation, decisions need to be made early and strategically regarding methods, data sources, systems, and technologies. Skillful initial selection of optimal tools and approaches improves the investigation.
To the extent possible, anticipate whether an investigation will be a low-profile and localized or might result in a large, possibly multicentric investigation of considerable public health importance and public interest. This early forecast guides system and technology selections. Anticipate that methods or technologies can change or evolve during the investigation as the scope or direction of the investigation changes. Plan regular reviews of the adequacy of the methods in use, and, if needed, make a transition from one data collection platform or process to another.
Previously, all components of a field investigation were likely to be performed actually in the field. Developments in information systems, data integration, and system interoperability have now made possible and sometimes desirable for some components (e.g., data collection, data cleaning, data analysis) of “field” investigations to be performed off-site (e.g., by central office staff). Access by both office and field staff to systematically collected data often simultaneously or in near– real time, improves support of the field investigation. Broader investments in health information technology (IT) and widespread adoption of electronic health records (EHRs), spurred by the Health Information Technology for Economic and Clinical Health Act in the United States enacted as part of the American Recovery and Reinvestment Act of 2009, have expanded the role technology can play in supporting a public health response (1).
For the purposes of this chapter, the terms outbreak and field investigation represent any acute public health problem requiring urgent epidemiologic investigation, including
- Infectious disease outbreaks;
- Clusters of cancers, birth defects, or poisonings;
- Environmental exposures;
- Diseases or conditions of unknown etiology;
- Natural disasters; or
- Threats arising from events elsewhere in the world.
During larger and higher profile investigations, the field response most likely will occur in the context of the country’s organized approach to emergency management—for example, in the US the National Incident Management System (https://www.fema.gov/pdf/emergency/nims/nimsfaqs.pdf) or the country’s equivalent approach to emergency management.
In this chapter, the term technology refers broadly to
- Computers,
- Software applications,
- Mobile devices,
- Personal health status monitoring devices,
- Laboratory equipment,
- Environmental monitors and sensors, and
- EHRs.
Technology is also used in regard to
- Public health surveillance systems;
- Ongoing public health databases;
- Purpose-built databases for specific investigations; and
- Technologies that enable storing, managing, and querying data and sharing data among these devices and databases.
Emergency situations typically create increased demands for epidemiologic and laboratory resources. Important factors that affect data collection and management during an event response—compared with business as usual—include time constraints; immediate pressure to both collect and instantaneously summarize substantial amounts of data, typically in fewer than 24 hours; limited human resources; often insufficient data preparedness infrastructure; and unfamiliar field deployment locations and logistics (see also Chapter 2).
Two guiding principles for selecting and using technologies during a field response are:
- Technologies for data collection and management should streamline and directly support the workflow of field investigations rather than disrupt or divert resources and staff time away from epidemiologic investigations and related laboratory testing activities (2).
- Technologies should facilitate more time for epidemiologists to be epidemiologists—to find better data, acquire them, clean them, and use data to better characterize the event, monitor its progress, or monitor the implementation or effectiveness of control measures—and more time for laboratorians to perform testing.
The choice of technology platforms should be driven by the
- Goals of the investigation;
- Training and skills of available staff;
- Existing infrastructure for gathering and managing case reports and other surveillance data;
- Number of geographically distinct data collection sites or teams expected and the number of jurisdictions involved;
- Speed and frequency with which interim summaries or situation reports are needed;
- Types of formal or analytic epidemiologic investigations expected (e.g., surveys, longitudinal studies, or additional human or environmental laboratory testing); and
- Other factors that will be evident in the situation.
The chosen technologies and systems should be subjected to periodic review as the investigation continues.
Technologic devices (e.g., mobile and smart devices, personal monitoring devices), EHRs, social media and other apps, automated information systems, and improved public health informatics practices have opened exciting opportunities for more effective and efficient public health surveillance. They are transforming how field teams approach the collection, management, and sharing of data during a field response.
Traditionally in field investigations, a public health agency deploys personnel to the geographic area where the investigation is centered, and the investigation is largely led and managed in the field, with periodic reports sent to headquarters. Although site visits are necessary to identify crucial information and establish relationships necessary for the investigation, a shift is occurring to a new normal in which field response data collection is integrated with existing infrastructure, uses jurisdictional surveillance and informatics staff, and uses or builds on existing surveillance systems, tools, and technologies.
Field data collection can be supported by management and analysis performed off-site or by others not part of the on-site team. Data collection, management, and analysis procedures often can be performed by highly skilled staff without spending the additional resources for them to be on-site. For example, active case finding by using queries in an established syndromic surveillance system (e.g., ESSENCE, which is part of the National Syndromic Surveillance Program [https://www.cdc.gov/nssp/news.html#ISDS]) or reviewing and entering case and laboratory data in a state electronic reportable disease surveillance system can be performed from any location where a computer or smart device and Internet connectivity are available. Data collected in the field electronically can be uploaded to central information systems. When data are collected by using paper forms, these forms can be scanned and sent to a separate data entry location where they can be digitized and rapidly integrated into a surveillance information system.
This approach enables the field team to focus on establishing relationships necessary for supporting epidemiologic investigation and data collection activities or on laboratory specimen collection that can only be accomplished on-site. Specialized staff can be assigned to the team; these staff remain at their desks to collect, manage, or analyze data in support of the field investigation. Staff might include data entry operators, medical record abstractors, data analysts, or statistical programmers. Implementing coordinated field and technology teams also enables more and highly skilled staff across multiple levels (local, state, or federal) to contribute effectively to an investigation. How to coordinate data activities in multiple locations needs to be planned for early in the response.
Field investigations often are led by personnel with extensive epidemiologic, disease, and scientific subject-matter expertise who are not necessarily expert in informatics and surveillance strategies. From a data perspective, such leadership can result in the establishment of ineffective data collection and management strategies. To support effective data collection and management, for all outbreaks, field investigators should
- Identify a role (e.g., chief data scientist or chief surveillance and informatics officer) that reports to a position at a senior level in the incident command structure (e.g., incident commander or planning section chief); and
- Identify and establish the role at the start of the response.
Whatever title is assigned to the role, the person filling the role should have clearly delineated duties and responsibilities, including
- Coordinating the full spectrum of data collection and management processes and systems used during the response;
- Being familiar with existing surveillance systems, processes, procedures, and infrastructure and how they are used currently;
- Identifying when and where existing systems can be modified to support the response or if temporary systems or processes need to be established;
- Anticipating that data collection methods or technologies might need to evolve or change during the investigation as the scope or direction of the investigation changes;
- Preventing creation of divergent, one-off, or disconnected data collection, management, and storage;
- Meeting regularly with response staff to identify additional system needs or modifications and ensuring that data collection and management activities support the progressing response;
- Regularly reviewing the adequacy of the surveillance systems, methods, and technology in use during the response and, if needed, plan for and implement a transition from one data collection strategy or platform to another; and
- Communicating surveillance system needs to the incident commander so that decisions and adequate resources for supporting surveillance efforts can be secured.
When preparing and packing for field deployments, two technology items are essential for each investigator: a portable laptop-style computer and a smartphone (essentially a pocket computer providing access to a camera, video, geolocating and mapping services, and data collection capacities). Depending on power availability and Wi-Fi or network connectivity, extra batteries or battery packs/mobile charging stations capable of charging multiple devices, such as laptops and phones, can be crucial. A mobile hotspot device to create an ad hoc wireless access point, separate from the smartphone, can be useful in certain situations. For example, after Hurricane Irma made landfall in Florida during September 2017, widespread power losses lasted for days. Deployed epidemiology staff were housed in locations without consistent power and had to travel to established command centers to charge phones, laptops, and rechargeable batteries once a day. Portable printers or scanners are other optional items to consider. Car-chargers for laptops and phones are also useful, although gas shortages can be a constraint and make car-chargers less optimal during certain types of responses.
As far as possible, responders should be deployed with items similar to ones they have been using on a regular basis. This will ensure that the investigator is familiar with the equipment and how it functions in different settings (e.g., how it accesses the Internet and device battery life), that it has all the expected and necessary software installed, and, perhaps most importantly, that it can be connected to the network (e.g., how it accesses the deploying agency’s intranet). Equipment caches of laptops, tablets, or other devices purchased only for use during events can lead to considerable deployment problems (e.g., lack of training in how to use the specialized equipment, network compatibility, or obsolescence of either hardware or software).
Field investigators responding to out-of-jurisdiction locations most likely will need to be issued temporary laptops from within the response jurisdiction to ensure network and software compatibility, connectivity, and adherence to jurisdictional security requirements. Temporary access (log-ins and passwords) to key surveillance system applications or updates to an investigator’s existing role-based access to these applications will also need to be considered.
Data often move at the speed of trust. A field team should establish strong working relationships at the start of the response with those who invited the epidemiologic assistance. On-site visit time should be used to ensure that the relationship will, among other tasks, facilitate gathering data and meet the needs of local authorities. Plans need to be made at the outset for sharing regular, timely data summaries and reports with local partners.
Upon initial arrival, the field team should assess existing surveillance systems and the processes for data submission to these systems. The assessment should address
- Data types already collected and available,
- Data timeliness,
- Data completeness,
- How easily and rapidly systems or processes can be modified or changed,
- Equipment available (e.g., laptops and phones),
- Available surveillance system staffing, and
- Known or anticipated problems and concerns with data quality, availability, and timeliness.
If the team is deploying out of its own jurisdiction, the team leader should seek assistance and consultation from someone at the jurisdictional level who fills a role like that of the chief surveillance and informatics officer (see previous section).
An outbreak investigation and response has defined steps and phases (see Chapter 3), and each has specific technology and information needs. In recent years, public health agencies have benefitted from technologic advances that support outbreak detection—whether the outbreak is caused by a known or unknown agent. For example, to detect reportable disease clusters effectively, the New York City Department of Health and Mental Hygiene each day prospectively applies automated spatiotemporal algorithms to reportable disease data by using SaTScan (Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA). This system enabled detection of the second largest US outbreak of community-acquired legionellosis by identifying a cluster of eight cases centered in the South Bronx days before any human public health monitor noticed it and before healthcare providers recognized the increase in cases (3). The identification led to an extensive epidemiologic, environmental, and laboratory investigation to identify the source—a water cooling tower—and then implement measures to remediate it. Although technology is revolutionizing approaches to cluster detection, this chapter assumes the field team will be responding after a known event or outbreak has been detected; thus, the following discussion focuses on using technologies for conducting initial characterization, active case finding, enhanced surveillance, supporting and evaluating control measures, and situational awareness, and for monitoring the response and its effectiveness.
Conducting Initial Characterization, Active Case Finding, and Monitoring
In an outbreak setting, routine data management often changes because of new stressors or novel circumstances, particularly the need to almost immediately gather data, produce reports, and inform decision makers and the public (see also Chapters 2 and 3). To assess population groups at highest risk, geographic extent, and upward or downward trends of disease incidence throughout a confirmed outbreak, investigators can use existing surveillance mechanisms. However, such mechanisms might need to be enhanced; for example, investigators might need to
- Create a new syndrome or add new queries to an existing syndromic surveillance system;
- Ask physicians and laboratorians to report suspected and probable, as well as confirmed, cases;
- Conduct active case finding; and
- Provide laboratories with diagnostic direction or reagents or ask them to send specimens meeting certain criteria to the state public health laboratory.
Regardless of whether case detection is enhanced, the technology used should support production of a line-listing for tracking cases that are part of the investigation. The system should also document what changes are made to individual cases and when those changes are made, including changes that result from new information gathered or learned or from epidemiologic findings. The system should ensure that laboratory data are easily made relational (Box 5.1). Even if investigation data are collected entirely or partially on paper, those data usually are keypunched into electronic data systems for further analysis, and the paper forms are scanned and stored electronically. As stated in a review of the 2003 severe acute respiratory syndrome (SARS) outbreak in Toronto, an important step in achieving seamless outbreak management is “uniform adoption of highly flexible and interoperable data platforms that enable sharing of public health information, capture of clinical information from hospitals, and integration into an outbreak management database platform” (5).
On May 9, 2014, the Florida Department of Health in Orange County received notification from a hospital infection preventionist about a man with suspected Middle East respiratory syndrome (MERS). Specimens tested at CDC confirmed this was the second reported, confirmed US MERS case. (4)
The investigation determined that the patient possibly exposed others in four general settings: airplanes during travel from Saudi Arabia to Orlando, Florida; at home (household contacts and visiting friends); a hospital outpatient waiting room while accompanying a relative for an unrelated medical reason; and later, an emergency department waiting room where he sought care for his illness. Multiple levels of contacts were tracked by the four exposure settings and risk for exposure (e.g., healthcare workers at high or low risk depending on procedures performed). The Epi Info 7 (CDC, Atlanta, Georgia) database that was created supported easy generation of line listings for tracking contacts and linking contact and laboratory information, including associated exposure settings, tracking isolation periods, contact method, attempts, signs and symptoms, final outcome, persons who should provide a clinical specimen, number and types of specimens collected (multiple and over time), whether specimens were received for testing, and laboratory results. The novel nature of the investigation required that additional data fields be captured as the scope of the investigation shifted. Because field investigators can control Epi Info 7 database management, these needed shifts were able to be met rapidly with no technical support. As a result, the progress of the contact investigation was able to be monitored in real time to identify priorities, optimally use personnel resources, and ensure leadership had current information on which to base decisions. Because of the ongoing reported MERS cases in the Middle East, CDC used BioMosaic, a big-data analytics application, to analyze International Air Transport Association travel volume data to assess potential high-exposure areas in the United States on the basis of US-bound travel. Effective database management and linking of epidemiologic and laboratory information in a single location supported the investigation.
ITs can be used to improve the quality, completeness, and speed of information obtained in a field investigation and the speed and sophistication of reports that can be generated from that information at the individual or aggregate level. To ensure that the full benefits of these technologies are realized, investigators need to perform the following actions:
- Begin with the type of output desired, create mock reports, and work backward to define the necessary input elements, ideally at the outset before any data collection begins; however, in reality this process often is iterative.
- Test the data export features and ensure the analytic software can easily access the necessary data.
- Carefully consider the questions leadership will need to have answered and ensure that the collected data elements answer the overarching questions. Completing this step may directly affect the underlying table structure of the database.
- Develop a flowchart detailing the steps associated with data gathering, information sharing, data management, and data technology. This flow chart will also help to identify processes that must be or should be manual and to identify and remove duplications of data transfer and entry (Figure 5.1).
- Recognize that field setups typically need rapid creation and modification of the database and to allow for creation of case records from laboratory results and for addition of multiple laboratory results to case records.
- Ensure the database can track both cumulative data (total cases) and temporal data changes (what occurred during the previous 24 hours or previous week). Collect status changes (i.e., change-history status and date-time stamps when the data changed) for priority data elements to ensure accurate reporting of information that changed and when (e.g., the number of new cases, the number of cases that changed from probable to confirmed, or the number of suspected cases that have been ruled out).
- Ensure the database supports user-defined data extraction and query capabilities. Do not underestimate the need for easy access to the data by the field investigators for data entry and rapid data summary and for planning the next day’s field operations (e.g., completed interviews, number of houses to revisit, number of non-English interviewers, number of persons with specimens collected, and number of specimen collection containers or other laboratory supplies on hand and needed). Field investigators should not have to be experts in formulating relational database queries.
- Plan to test field data collection equipment and applications (see also Chapter 2). For example, if interviews and data collection are to be performed in a door-to-door sampling effort, are the laptop computers too heavy to hold while completing an interview? Can the screens be viewed in direct sunlight? Does the system screen navigation match the flow of the interview? Will Internet connectivity be available?
ITs can be used to improve the quality, completeness, and speed of information obtained in a field investigation and the speed and sophistication of reports that can be generated from that information at the individual or aggregate level. To ensure that the full benefits of these technologies are realized, investigators need to perform the following actions:
- Begin with the type of output desired, create mock reports, and work backward to define the necessary input elements, ideally at the outset before any data collection begins; however, in reality this process often is iterative.
- Test the data export features and ensure the analytic software can easily access the necessary data.
- Carefully consider the questions leadership will need to have answered and ensure that the collected data elements answer the overarching questions. Completing this step may directly affect the underlying table structure of the database.
- Develop a flowchart detailing the steps associated with data gathering, information sharing, data management, and data technology. This flow chart will also help to identify processes that must be or should be manual and to identify and remove duplications of data transfer and entry (Figure 5.1).
- Recognize that field setups typically need rapid creation and modification of the database and to allow for creation of case records from laboratory results and for addition of multiple laboratory results to case records.
- Ensure the database can track both cumulative data (total cases) and temporal data changes (what occurred during the previous 24 hours or previous week). Collect status changes (i.e., change-history status and date-time stamps when the data changed) for priority data elements to ensure accurate reporting of information that changed and when (e.g., the number of new cases, the number of cases that changed from probable to confirmed, or the number of suspected cases that have been ruled out).
- Ensure the database supports user-defined data extraction and query capabilities. Do not underestimate the need for easy access to the data by the field investigators for data entry and rapid data summary and for planning the next day’s field operations (e.g., completed interviews, number of houses to revisit, number of non-English interviewers, number of persons with specimens collected, and number of specimen collection containers or other laboratory supplies on hand and needed). Field investigators should not have to be experts in formulating relational database queries.
- Plan to test field data collection equipment and applications (see also Chapter 2). For example, if interviews and data collection are to be performed in a door-to-door sampling effort, are the laptop computers too heavy to hold while completing an interview? Can the screens be viewed in direct sunlight? Does the system screen navigation match the flow of the interview? Will Internet connectivity be available?
- Use programmed data quality and validity checks to identify and resolve discrepancies at the time of data collection. For example, date fields should only accept valid dates within a given range, or pregnancy should be available as a valid value for women only.
- Be aware that, typically, the more complex the data entry checks or programmed skip patterns in place, the more time that is needed to set up the form itself; during field responses, setup time can be an important tradeoff against other uses of investigators’ time and against data quality concerns.
- Recognize that structured data collection techniques and standardization processes can minimize data quality problems, although even highly structured data collection techniques do not eliminate data errors. The standardization process that facilitates computer-readable data forms risks losing the richness of information identified within unstructured documents (i.e., clinicians’ notes or field observations). How data elements are collected (e.g., structured drop-down lists, free text, check boxes when multiple selections are possible, or radio buttons for single-choice selections) dictates data storage format and table structures and can dictate how the data can be analyzed (e.g., symptoms being reported by interviewees can be stored in a comma-delimited string, or each symptom can be stored as yes/no choices in separate columns).
Note: DMAT: Disaster Medical Assistance Team; ED: Emergency Department
Source: Reference 6.
Using Routine Surveillance Data and Systems
The value and use of routine surveillance data systems should not be underestimated during outbreak investigations and ideally will be managed within a data preparedness framework. Many state reportable disease surveillance systems (both commercially available or state-or in-house– designed) now have outbreak management components (7). To avoid duplicating efforts or processes, field investigators should understand and assess existing surveillance systems that support outbreak management before determining which technologies to use (Box 5.2).
In addition to public health electronic disease surveillance systems supporting outbreak management components, reportable disease electronic laboratory reporting (ELR) is now a mainstay of reportable disease surveillance. Every state health department has operational ELR systems (9). Although ELR was designed for supporting individual identification and reporting of disease events, it can also be used to support outbreak response activities. Using existing surveillance systems, including ELR processes, supports outbreak detection, characterization, outbreak identification, and control measure evaluation (Box 5.3).
Following the identification of the initial case of Zika virus infection attributed to likely local mosquito-borne transmission in Florida, the Florida Department of Health conducted active surveillance in selected areas of the state to identify locally acquired Zika virus infections and to assess whether ongoing transmission was occurring (8). Data collected during these field surveys were managed in the outbreak module (OM) of the state health department–developed reportable disease surveillance application, Merlin. Three types of OM events were used: index, cases and their contacts; urosurvey (i.e., survey administration paired with urine sample collection), participants of residential, business, or clinic urosurveys; and other, nonindex cases. Data regarding residential urosurveys (persons within a 150-meter radius of a locally acquired index case), business urosurveys (employees at a business or worksite), and clinic urosurveys (persons who lived or worked in the area of interest) were collected and analyzed. For each urosurvey OM event, an event-specific survey was generated in real time and used to capture the collected data. In 2016, door-to-door survey data were collected on a simple paper form. Surveys were faxed nightly to the central office staff in Tallahassee, where existing reportable disease data entry staff entered all the survey data collected; the digitized information was made available to local-and state-level investigators within 24 hours.
In 2016, 87 OM events (49 index, 32 urosurvey, and 6 other) were initiated. These events comprised approximately 2,400 persons, of whom approximately 2,200 (92%) had participated in any urosurvey event. Managing the data within the Merlin system OM was also useful for immediately linking the laboratory data received electronically from the state public health laboratory information system via the state’s existing electronic laboratory reporting (ELR) infrastructure. Modifications were made to the ELR feed (e.g., new Zika virus test codes were added) with these changes able to be completed before the first urosurvey was launched, thus ensuring rapid data receipt. For positive laboratory results, case records were created immediately, and those records were linked between the case record and separate survey data collection areas. Merlin continued to support routine case reporting, and the OM facilitated flexible group-specific, event-level investigations. Event surveys comprised core questions and site-or setting-specific questions. Use of core questions enabled comparability within and between urosurvey event data, improving the ability to conduct ad hoc analysis. Managing data electronically within Merlin and OM facilitated easy access to data for export, event-specific analysis, and linking for mapping. The seamless management of case and survey data eliminated duplicate data entry. During the response, modifications were made to automate sending reportable disease case data from Merlin to CDC for national reporting in the ArboNet database (replacing a previously manual data entry process).
Identification of influenza outbreaks can be challenging, often relying on a healthcare provider to recognize and report that information to the jurisdiction’s health department. Influenza infections among certain populations at high risk (e.g., older persons, particularly those in nursing homes or other long-term care facilities) can have more severe outcomes especially if there are delays implementing appropriate antiviral treatment or chemoprophylaxis. To more effectively identify outbreaks in this setting, the Florida Department of Health (FDOH) implemented regulations to require reporting of influenza results through electronic laboratory reporting (ELR).
Following an approach first described by the New York Department of Health and Mental Hygiene (10), FDOH obtained a list of all licensed nursing homes and other long-term care facilities from the state licensing agency. Addresses of these facilities were then matched to the patient address received on the ELR form to determine whether a person in these facilities had an influenza-positive specimen. A single positive result within these high-risk settings triggers an outbreak investigation. Previously unreported outbreaks have been identified through this approach.
As another example, during the FDOH’s 2016 response to locally acquired Zika virus infections (8), ELR was vital for evaluating public health recommendations. In Miami-Dade County, FDOH recommended that all pregnant women be tested for Zika virus infection after active local transmission was identified. Laboratories obtaining testing capacity for Zika virus were asked to send all Zika laboratory test results to FDOH. The FDOH birth defects program determined the estimated number of live births and pregnant women living in Miami-Dade, and the ELR data (negative and positive results) were used to assess what proportion of pregnant women had actually been tested and where the public or healthcare providers needed additional outreach or education. With the high volume of testing performed (approximately 65,000 results in 2016), using technology (established ELR processes and advanced analytic software) made such an approach feasible.
Syndromic surveillance uses data about symptoms or health behaviors (e.g., substantial increases in over-the-counter medication sales) and statistical tools to detect, monitor, and characterize unusual activity for further public health investigation or response and situational awareness. The most recognized and largest syndromic surveillance data source is patient encounter data from emergency departments and urgent care centers. These data can be monitored in near–real time as potential indicators of an event, a disease, or an outbreak of public health significance or to provide event characterization and monitoring after initial detection. ESSENCE, an established syndromic surveillance system, was used to quickly facilitate active case finding when Zika virus was introduced in the US in 2016 and 2017 (8) (Box 5.4).
Zika virus disease (Zika) became a widespread public health problem in Brazil in 2015 and quickly spread to other South and Central American countries and eventually to the United States. Zika is associated with increased probability of severe birth defects in babies when their mothers are infected with the virus during pregnancy. Zika also has been associated with Guillain-Barré syndrome.
The primary vector for Zika is the Aedes aegypti mosquito, which is present in Florida. With large numbers of tourists visiting Florida annually, including from many of the countries with Zika outbreaks, Florida instituted measures to minimize introduction of this disease into the state (8). Identification of persons infected with Zika early in the course of their illness allows for a twofold public health intervention: (1) patient education about how to avoid mosquito bites while viremic to help prevent spread to others and (2) mosquito control efforts that are targeted to the areas where the patient has been (e.g., home or work).
Florida’s syndromic surveillance system (ESSENCE-FL) has nearly complete coverage in hospitals that have emergency departments (245/250 hospitals). Queries were created to search the chief complaint, discharge diagnosis, and triage notes field for Zika terms (including misspellings of the words Zika and microcephaly) and clusters of symptoms (e.g., rash, fever, conjunctivitis, or joint pain) in individuals who had travel to countries of concern. Dashboards were created by state-level staff and shared with county epidemiologists to facilitate daily review of emergency department visits for which Zika was suspected.
A total of 19 Zika cases (10 in 2016, 7 in 2017, and 2 in 2018) were identified by using ESSENCE-FL. These visits were not reported to public health by using traditional reporting mechanisms and would not have been identified without active case finding using ESSENCE-FL by the public health agency. These identifications were completed by using an existing surveillance system and helped to reduce the probability of introducing locally spread Zika in Florida.
Building New Surveillance Systems Versus Modifying Existing Systems
There is a danger that data management in the context of a field investigation can create more, rather than better, data systems. Condition-specific, event-specific, or stand-alone systems that are not integrated or interoperable require burdensome, post hoc coordination that is difficult and time-consuming, if not impossible.
Rather than setting up new stand-alone systems,
- Work to modify existing systems. Making an urgent system modification is typical, and modifying systems often is more sustainable than designing and developing separate, nonintegrated data management approaches.
- Consider stand-alone systems only when no other options are available. If used, immediately implement a plan to retrieve and share the data with other systems.
- Look for opportunities in which the event response can help catalyze surveillance system modifications that will strengthen future surveillance activities.
Using EHRs
With broad implementation of EHRs, opportunities exist for improving links between healthcare providers and public health departments, making data collection during field investigations more effective and timely (11). Increasingly, public health agencies have been able to establish agreements with healthcare facilities, often at the local level, to support remote access to EHRs for day-to-day surveillance activities. With such access to EHRs, staff can review medical records remotely to gather additional clinical, exposure, or demographic data about a case whose case report has been received through other channels.
Routine use of such access by local or state health department staff before an event can reduce public health learning curves when EHRs need to be accessed during a response event. Even without routine access, field investigators have been able to get time-limited system-specific EHR access during such response events, as happened during the response to the multistate outbreak of fungal meningitis in 2012 (Box 5.5). This benefitted the outbreak team as they conducted active case finding, completed case abstraction after case identification, and characterized the cases. Medical records abstraction can be done remotely by technical experts who are not on the deployed field team. Familiarity with EHR systems and direct contact with vendors can be helpful. Healthcare provider office staff might be knowledgeable about conducting record-level retrieval in the EHR product, but they might be less skilled at producing system extracts or querying across records (e.g., all persons receiving a specific procedure during a specific time frame) in ways that clinical users of the EHR have little occasion to do.
On September 18, 2012, a clinician alerted the Tennessee Department of Health about a patient with culture-confirmed Aspergillus fumigatus meningitis diagnosed after epidural steroid injection. This case was the first in a multistate outbreak of fungal infections linked to methylprednisolone acetate injections produced by the New England Compounding Center (Framingham, MA) (12). Three lots of methylprednisolone acetate distributed to 75 medical facilities in 23 states were implicated. Medical record abstraction is a common practice during outbreak investigations, but it typically requires on-site abstraction. The Tennessee Department of Health used remote desktop access to electronic health records (EHRs) to review data regarding known affected patients and identify the background rate of adverse events from the procedures of concern. Remote EHR access enabled abstraction of past, current, and follow-up visits and review of medical histories, clinical course of the disease, laboratory test results, imaging results, and treatment data. This was critically important to inform the real-time development and dissemination of CDC guidelines for patient care that evolved with the constantly changing clinical manifestations. Remote EHR access saved health department and facility staff time, enabled staff to return to their offices to complete case ascertainment, and supported multiple highly skilled staff working simultaneously. Assistance from facility information technology staff was needed in certain instances to obtain remote desktop access and provide guidance on using the EHR.
During the investigation, public health authorities needed a substantial amount of information quickly on an ongoing basis and from multiple, disparate institutions, and traveling to obtain the information was impractical. To remedy the challenges of accessing EHRs remotely, areas for improvement include better understanding of privacy policies, increased capability for data sharing, and links between jurisdictions to alleviate data entry duplication.
When data to support an event response might be in an EHR, field teams should
- Use on-site time to establish necessary relationships and agreements to support remote or desk EHR access;
- Have a low threshold for requesting remote EHR access;
- Elevate resolution of any barriers to EHR access that are encountered to jurisdictional leadership and request assistance from privacy and legal teams (see also Chapter 13);
- Expand the response team to include experts in medical data abstraction who can support the response remotely;
- Contact EHR vendors or use health department surveillance and informatics staff to facilitate coordination with vendors and to help with gaining remote access or to performing data extractions or queries across records; and
- Ensure quality EHR data integration into existing surveillance systems.
Using EHRs is new to some public health workers and can present challenges. For example, public health users require time to learn how to access, connect, and navigate systems. Where in the EHR the needed data are stored depends in part on how healthcare facilities use their EHRs; for example, data ideally stored as coded elements or in available system-designated fields might instead be located in free-text boxes. The more system users exist, the more likely the same data element is recorded in different ways or in different places. Data important to the response might even be stored on paper outside the EHR system. Ideally, public health personnel have access to an institution’s entire EHR system, but some facilities still require that those personnel request specific records, to whom the facility assigns specific record access; the latter approach slows the process. The benefits of timely data and data access have proved to be worth the effort to overcome these challenges.
Improving Analysis, Visualization, and Reporting
During outbreaks and response events in recent years, demand has increased for rapid turnaround of easily consumable information. This demand is in part driven by cultural changes and expectations, where people now have powerful computers in their pockets (smartphones) and easy access to social media, the Internet, and 24- hour news cycles. The field team must meaningfully summarize the data and produce reports rapidly, turning collected data into information useful for driving public health action.
Regardless of collection method, after data are digitized, analytic and statistical software can be used to manipulate the data set in multiple ways to answer diverse questions. Additionally, advanced analytic software enables use of other types of data (e.g., electronic real-time data about air or water quality or data acquisition or remote sensing systems, such as continual or automated collection and transmission). Combining these data with geographic information system data can facilitate overlay of environmental and person-centric information by time and place (11).
The following principles apply to facilitating effective analysis and visualization:
- Data must be easily exportable to other systems for analysis; often this process can be automated. Even when data collection occurs in one primary database, completing data analyses may require use of other, more sophisticated tools or merger of outbreak data with data from other sources.
- Establish a report schedule (e.g., every day at 9 am) early during the investigation. In larger outbreaks for which data input is managed in multiple or disparate locations, communicating explicitly when data should be entered or updated in the system and what time the daily report will be run is imperative for ensuring that the most up-to-date information is available for analysis. Reports summarizing the cumulative information known, as well as daily or even twice-daily data summaries (i.e., situation reports) (Handout 5.1), might be necessary.
- Use software to automate report production to run at specific times. This function is useful during larger events where situation reports might be needed multiple times each day.
Transitioning from Field Investigations to Ongoing Surveillance
New systems or processes at the local, state, and federal levels often have been developed for supporting outbreak responses. Because of time and resource constraints in outbreak settings, surveillance systems or processes initiated during outbreaks can partially duplicate other processes. They may be time-consuming or staff-intensive in ways that are acceptable during the response but not as part of a routine system and may present integration problems when the outbreak is over. To minimize this potential, field investigators should ensure that processes for reviewing data collection are strictly followed throughout the outbreak. Field investigators should begin transition planning for sustainability with the goal of transitioning as soon as possible to existing mechanisms, keeping in mind related data collection activities that may be needed in future, long-term records management and storage, and continued analyses.
Determining Security, Standards, and Database Backups
Data security is paramount in any uses of technology in a field response. Computers, tablets, and other mobile devices taken into the field must be protected against data loss and unauthorized access. Determinations must be made regarding what types of equipment can interact with the public health agency’s internal network. Confidential data storage on a local machine should be discouraged, and, if unavoidable, address the need early and through the public health agency’s privacy and security standards (see also Chapter 13).
Before data collection or device selection,
- Understand at a high level the public health agency’s privacy and security standards;
- Assess whether data security in the field meets the jurisdiction’s standards, which might require meeting with the health department’s IT director;
- Determine how data collection (and mobile or off-site data collection) will interact with potential firewall and network problems;
- For field deployments where Internet connections or department of health network availability might not be consistently accessible (as was the situation during deployments after Hurricane Irma struck Florida in 2017 and during the Zika response in Miami– Dade County where door-to-door surveying was done inside large apartment buildings), ensure strict security standards are followed; and
- Implement effective database management and rigor by establishing regular and automated backup procedures.
A situation report summarizing epidemiologic activities was produced daily by the Florida Department of Health and provided to the incident commander for decision-making and resource prioritization purposes. Data were extracted from the state reportable disease surveillance system into analytic software, where the report production was automated.
Zika Fever Summary Points
-112 Zika fever cases have been reported in Florida as of 2:30 pm on 05/12/16
– No cases are new since 2:30 pm on 50/11/16
– 7 cases have been in pregnant women
– 0 cases were acquired in Florida
– 13 cases have been hospitalized during their illiness
– 4 cases are currently ill
– 0 cases have been associated with microcephaly, fetal intercranial calcifications, or poor fetal outcomes (after 1st trimester)
– 1 case has been associated with Guillain-Barré syndrome
– 1117 people have been tested by the Bureau of Public Health Laboratories for Zika virus (634 were pregnant women)
Focus first on the case information associated with the event. Most often, the field team will also track or manage much other data as well (Box 5.6). For example, in Illinois during a measles outbreak in 2015, in addition to tracking cases, the field team needed to track the number of persons placed in quarantine (14) (Box 5.7).
Technology will not solve insufficient levels of physical and human resources. Technology decisions to support field deployments often are based, not on the best technology to support the response, but rather on the knowledge and comfort level among staff. The location of the data collection (e.g., from a desk telephone, at a clinic or hospital, or through door-to-door interviews) is a key factor in determining the acceptable technology, as is the acceptance of technology by those under investigation who might be asked to use the instruments or tools directly. If large amounts of data need to be collected manually, not having adequate data entry staff can be a limitation. Necessary data entry staffing is often underestimated, and the data entry process can become a bottleneck. Field team staffing concerns to consider include the following:
- The number of persons collecting, entering or digitizing data, deployment lengths (from rotating staff leads to the need for multiple trainings), deployment locations, and familiarity with and knowledge of technical tools.
- Available training time. Some level of in-event training is often needed. Training staff for data collection and management during the field response can make introducing new technology incredibly difficult and can lead to a lack of acceptance or perceived acceptability of the new tools.
- Interviewing skill sets, languages spoken, and interview locations.
- Team member personal safety and equipment safety.
In addition to epidemiologic, scientific, and disease knowledge, a highly skilled data manager with a firm understanding of public health informatics might be needed on-site as a member of the field team. To be most effective, this person should be familiar with existing surveillance systems, practices, and procedures. Field investigators should have a low threshold for requesting such support if not part of the initial team.
Field investigators might need to manage noncase data stored in different systems, requiring field teams to have access or collect and aggregate the information; in other situations, the field team might need to create a means of tracking data from multiple sources. For example,
- Current and projected need for laboratory materials, including the number of reagents or laboratory specimen-collection materials (e.g., swabs); specific types of transport media (e.g., stool cans for ova and parasite analysis and bacteriology); or sterile containers.
- Turnaround time from receipt of specimen at the laboratory to availability of results (and by laboratory test type, e.g., culture, polymerase chain reaction [PCR]).
- Availability of crucial supplies (e.g., antiviral drugs, oxygen, or sterile gloves) and equipment (e.g., respirators) and services (e.g., dialysis) in relation to current and projected needs.
- Number of completed surveys conducted each day, number of homes visited, specific residences, and persons needing to be revisited.
- Availability of vaccine, locations offering the vaccine, and doses of vaccine administered (and demographic characteristics of those vaccinated).
- Availability of and access to prophylactic treatments.
- School and workplace attendance and closures.
- Hospital bed availability by bed type, as well as staffing of available beds or staffing issues.
- Documentation of control measures implemented (e.g., number and status of persons isolated or quarantined, number of persons prophylaxed, notification of mosquito control authorities, and if and when spraying occurred).
- Number of persons exposed and exposure location.
- Adequacy of staff for monitoring exposed and isolated persons, status of exclusions, and follow-up testing dates.
- Number of households visited and number of attempts, number of exposed persons contacted, and number of persons responding to contact.
- Distribution of letters and other education materials.
- Status of recalls of food or medical products, continued availability of product at retail or treatment locations, status of contact with facilities receiving implicated products, and assurance products are no longer in use.
- Travel restrictions.
In January 2015, the Illinois Department of Public Health (IDPH) began investigating a large US measles outbreak comprising 15 confirmed cases and many exposed contacts. The customizable function in REDCap enabled IDPH to rapidly modify the existing Ebola virus disease module to create a measles-specific questionnaire for supporting the measles contact investigation. Within 72 hours, the IDPH measles module in REDCap was ready for use by multiple local health departments. The REDCap survey instrument was offered to 33 (52%) of 63 contacts as a monitoring option alternative to daily telephone calls for reporting body temperature and symptoms, with 17 (52%) of the 33 contacts completing one or more surveys. Postevaluation found REDCap simplified follow-up by reducing staff time and effort for monitoring contacts identified as being at low risk for developing infection. Moreover, the system supported rapid prioritization of persons who needed further follow-up among those contacts failing to report their symptoms daily. To enhance the tool for future use, Spanish and Polish language translation options, a vaccination history data collection tool, and the ability to manage multiple contacts within one household were requested (14).
As an emerging field, informatics is only vaguely familiar to some professionals in public health. Public health informatics specialists design and implement public health– related systems that efficiently handle data crucial to public health practice. Informatics tools and approaches if applied well can find an appropriate balance between the ideal of public health informatics practice and the reality of field data collection (15).
Public health informaticians are trained to understand public health programs and their data needs as well as information system design—it is this dual training that distinguishes them from most public health agency IT workers. Health IT service professionals are often confused with public health informatics specialists. Health IT service professionals should be able to resolve infrastructure problems such as network connections, whereas trained public health informaticians should be able to support public health decisions by facilitating the availability of timely, relevant, and high-quality information by calling on a broad array of disciplines, including IT architecture and security, statistics, data management science, and systems theory (11).
Because field investigations evolve rapidly, description of specific technologies or programs to support outbreaks, surveillance, and data collections can become outdated quickly (Table 5.1). Ideally, public health agencies should use modern technologies to facilitate public health practice. In reality, public health agencies may struggle to incorporate new technology, in part because of the lack of resources and availability of savvy informatics staff in the field (15). Many health departments have restrictive lists of approved software, although exceptions or new approvals can often be expedited during outbreak responses if the need or role the desired software will serve can be demonstrated.
Widely used data collection applications include the Centers for Disease Control and Prevention’s (CDC) Epi Info (CDC, Atlanta, Georgia) and, increasingly, REDCap (Research Electronic Data Capture; Vanderbilt University, Nashville, Tennessee). Many state-based reportable disease surveillance systems have integrated outbreak management modules (7). Some other free software packages (e.g., SurveyMonkey [San Mateo, CA]) are inadequate for storing and collecting confidential data, and their use must be avoided (see also Chapter 13). Numerous survey tools are designed for handling onetime data collection (e.g., launching a single-use questionnaire), but do not support saving and reusing the same survey.
Equally important are data management and analysis of collected data. Such software as SPSS (IBM Corporation, Armonk, NY), SAS (Statistical Analysis System; SAS Institute, Inc., Cary, NC), and R (R Foundation, Vienna, Austria) are invaluable analysis and data management applications. Google Maps (Google Inc., Mountain View, CA), and geographic information system data (see Chapter 17) are valuable mapping tools. Be aware, however, that confidentiality can be a serious problem when creating point maps of people with disease, exposure, or injury.
Worldwide adoption of the Internet has enabled a new class of participatory systems that enable people to contribute and share information and collaborate in real time (16). Social media applications (e.g., Facebook [Facebook, Inc., Menlo Park, CA], Yelp [Yelp, Inc., San Francisco, CA], and Twitter [Twitter, Inc., San Francisco, CA]) are increasingly used to conduct surveillance and crowdsourcing, serving both to push out and pull in health-related information. Using social media for both pushing and pulling information can be helpful during outbreaks to support distribution of public health messaging and to support active case finding. These types of data have been used to derive signals of important health trends faster and more broadly than more traditional case reporting systems (11). For example, New York City has used social media for active case finding, contact identification, and evaluation of education and prevention messaging during a community-based outbreak of Neisseria meningitidis (17,18) (Box 5.8). Rigorous evaluation of the reproducibility, reliability, and utility of data derived from these new data sources is an area of active research (19).
Application | Comments |
---|---|
Epi Info (CDC, Atlanta, GA) |
|
Microsoft Access (Microsoft Corp., Redmond, WA) |
|
REDCap (Vanderbilt University, Nashville, TN) |
|
Outbreak management components of reportable disease surveillance systems |
|
Application | Comments |
---|---|
SAS (Statistical Analysis System; SAS Institute, Inc., Cary, NC) |
|
SPSS (IBM Corporation, Armonk, NY) |
|
R (R Foundation, Vienna, Austria) |
|
ArcGIS (Esri, Redlands, CA) |
|
ESSENCE (Electronic Surveillance System for the Early Notification of Community- based Epidemics) |
|
SaTScan (Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA) |
|
BioMosaic (CDC, Atlanta, GA) |
|
HealthMap (Boston Children’s Hospital, Boston, MA) |
|
Application | Comments |
---|---|
EpiCollect (Imperial College London, UK) |
|
Apps |
Note: Field investigators should check with the investigating agency’s authorities and use extreme caution regarding privacy and confidentiality requirements before using a new application (see also Chapter 13). |
Mobile devices |
|
Single- use online forms |
|
Social media |
|
CDC, Centers for Disease Control and Prevention; REDCap, Research Electronic Data Capture.
In September 2012, the New York City Department of Health and Mental Hygiene (NYC DOHMH) identified an outbreak of Neisseria meningitidis serogroup C invasive meningococcal disease among men who have sex with men (MSM). The final tally of cases that occurred during August 2010–February 2013 was 22 cases (7 deaths), of which only 7 cases were in people who were not MSM. Although the attack rate of N. meningitidis among MSM in New York City had increased, identifying links among patients and among potentially exposed persons was difficult (17).
One approach during the investigation was to use patients’ cell phones to identify contacts and links among cases. NYC DOHMH obtained cell phone logs to identify who the case-patients had called, who had called them, and incoming and outgoing text messages. The list of phone numbers was analyzed by a matching program written in SAS code to identify numbers in common (SAS Institute, Inc., Cary, NC). One phone number was common among three persons, and investigators discovered these three men had attended events together (18).
Review of cell phones themselves proved effective in identifying common apps among patients and exploring links among them (i.e., mini-networks). NYC DOHMH used social media apps to disperse information to the public regarding getting tested or vaccinated. Information was distributed through Twitter (Twitter, Inc., San Francisco, CA) and approximately 100 Internet sites and blogs that had high MSM viewership, and through NYC DOHMH–sponsored banner and pop-up online advertisements and meet-up apps targeting MSM.
During this outbreak, use of technology and social media was evaluated to assess the effectiveness of education and control measures. “In November 2012, a total of 40,116 (8.2%) of 488,000 pop-ups ads and 2,782 (0.6%) of 463,645 banner ads were clicked on, compared with 87 (14.4%) of 605 e-mail blasts to users of a popular hookup online site. Of 266 users surveyed, 118 (44%) recalled having received an e-mail about the outbreak; only 77 (29%) users recalled having seen one of the banner ads on the site” (17). During this outbreak, use of social media supported the outbreak response through active case finding, contact tracing, and communication of education and prevention messages.
The pervasive use of technology in healthcare and in everyday life will continue to propel and transform public health surveillance and data collection and management during field investigations. Mobile devices hold particular promise for data collection because they can be used as point-of-care devices, perform exposure monitoring, conduct health status monitoring, function in remote locations, and are readily carried and used at any time (20,21). In the future, maturation of data interoperability standards will facilitate more immediate information sharing. The evolution of public health informatics and use of technologies in field responses will continue to require ingenuity and adaptation and provide exciting opportunities during response events.
Acknowledgments
The authors thank Aaron Kite-Powell, Surveillance and Data Branch, Division of Health Informatics and Surveillance Center for Surveillance, Epidemiology, and Laboratory Services, Office of Public Health Scientific Services, Centers for Disease Control and Prevention, for his honest feedback and helpful conversations. For their writing and contributions to field examples presented, the authors thank David Atrubin, Leah Eisenstein, Nicole Kikuchi, Bureau of Epidemiology, Florida Department of Health; Benjamin G. Klekamp, Florida Department of Health—Orange County; Marion A. Kainer, Healthcare Associated Infections and Antimicrobial Resistance Program, Tennessee Department of Health; and Don Weiss, Bureau of Communicable Disease, New York City Department of Health and Mental Hygiene. For her collaboration and expertise, the authors thank Marcella Layton, Bureau of Communicable Disease, New York City Department of Health and Mental Hygiene. Janet Hamilton is grateful for the understanding, encouragement, and love of her husband, Eric I. Hamilton and her children Jackson and Elaine who allowed mom more weekend and night computer time.
References
- American Recovery and Reinvestment Act of 2009, Pub. L. No. 111-5, 123 Stat. 226. February 17, 2009.
- Martin SM, Bean NH. Data management issues for emerging diseases and new tools for managing surveillance and laboratory data. Emerg Infect Dis. 1995;1:124– 8.
- Greene SK, Peterson ER, Kapell D, Fine AD, Kulldorff M. Daily reportable disease spatiotemporal cluster detection, New York City, New York, USA, 2014–2015. Emerg Infect Dis. 2016;22:1808–12.
- Bialek MD, Allen D, Alvarado-Ramy F, et al. First confirmed cases of Middle East respiratory syndrome coronavirus (MERS-CoV) infection in the United States, updated information on the epidemiology of MERS-CoV infection, and guidance for the public, clinicians, and public health authorities—May 2014. MMWR. 2014;63;431–6. Erratum in: MMWR. 2014;63:554.
- Public Health Agency of Canada. Learning from SARS: Renewal of public health in Canada. http://www.phac-aspc.gc.ca/publicat/sars-sras/naylor/index-eng.php
- Florida Department of Health, Bureau of Epidemiology. County health department epidemiology hurricane response toolkit. Updated April 20, 2015, p. 19.
- Public Health Informatics Institute. Electronic Disease Surveillance System (EDSS) vendor analysis. http://www.phii.org/resources/view/4409/electronic-disease-surveillance-system-edss-vendor-analysis
- Likos A, Griffin I, Bingham AM, et al. Local mosquito-borne transmission of Zika virus—Miami–Dade and Broward counties, Florida, June–August 2016. MMWR. 2016;65:1032–8.
- Centers for Disease Control and Prevention. Progress in increasing electronic reporting of laboratory results to public health agencies—United States, 2013. MMWR. 2013;61;797–9.
- Levin-Rector A, Nivin B, Yeung A, Fine A, Greene S. Building-level analyses to prospectively detect influenza outbreaks in long-term care facilities: New York City, 2013–2014. Am J Infect Control. 2015;43:839–43.
- Centers for Disease Control and Prevention. CDC’s vision for public health surveillance in the 21st century. MMWR Suppl. 2012;61(Suppl):1–44.
- Centers for Disease Control and Prevention. Multistate outbreak of fungal infection associated with injection of methylprednisolone acetate solution from a single compounding pharmacy—United States, 2012. MMWR. 2012;61;839–42.
- Florida Department of Health, Bureau of Epidemiology. Daily situation report from the field response to Zika virus, 2016. Production Date May 12, 2016, 2:30 PM.
- Vahora, J, Hoferka, S. How Illinois used REDCap to support contact monitoring for the 2015 measles outbreak. June 11, 2015. http://www.cste.org/blogpost/1084057/219374/How-Illinois-Used-REDCap-to-Support-Contact-Monitoring-for-the-2015.Measles-Outbreak.
- Fond M, Volmert A, Kendall-Taylor N. Making public health informatics visible: communicating an emerging field. A FrameWorks Strategic Map the Gaps Report. Washington, DC: FrameWorks Institute; 2015. https://frameworksinstitute.org/toolkits/informatics/elements/pdfs/informatics_phiistrategicmtgfinalseptember2015.pdf
- Eysenbach G. Medicine 2.0: social networking, collaboration, participation, apomediation, and openness. J Med Internet Res. 2008;10:e22.
- Kratz MM, Weiss D, Ridpath A, et al. Community-based outbreak of Neisseria meningitidis serogroup C infection in men who have sex with men, New York City, New York, USA, 2010−2013. Emerg Infect Dis. 2015;21:1379–86.
- Grounder P, Del Rosso P, Adelson S, Rivera C, Middleton K, Weiss D. Using the Internet to trace contacts of a fatal meningococcemia case—New York City. 2010. J Public Health Manag Pract. 2012;18:379–81.
- Hopkins RS, Tong CC, Burkom HS, et al. A practitioner-driven research agenda for syndromic surveillance. Public Health Rep. 2017;132 Suppl: 116S–26S.
- Waegemann CP. mHealth: the next generation of telemedicine? Telemed J E Health. 2010;16:23–5.
- Gerber T, Olazabal V, Brown K, Pablos-Mendez A. An agenda for action on global e-health. Health Aff (Millwood). 2010;29:233–6.