Evaluation of NIOSH Programs

What to know

NIOSH uses program reviews to evaluate the relevance and impact of its research programs. Most recently, NIOSH contracted independent panels of experts to conduct its reviews. Using documented evidence, these reviews assessed the contributions our programs have made to safety and health outcomes. Prior to that, reviews were conducted by the National Academies of Science, Engineering, and Medicine (National Academies).

Overview

In 2017-2019, NIOSH conducted reviews of five of its programs using independent panels of experts. Each panel reviewed an evidence package and heard presentations from program staff describing the program's activities and accomplishments over the past 10 years. Panelists provided individual scores for relevance and impact, using a scale of one to five. They also made recommendations for future activities. The panel chair averaged the scores and compiled the recommendations into a final report. All programs received a score of 8 out of 10 or better.

NIOSH used a modified version of contribution analysis to guide the review process. This approach emphasizes demonstrating a program's contribution to an outcome rather than establishing causal attribution. Learn more about this process as described in the American Journal of Evaluation.

Findings

The five program reviews were:

Construction

Emergency Preparedness and Response

  • Evidence package – Focuses on Emergency Responder Health Monitoring and Surveillance (ERHMS); anthrax preparedness and response capabilities; and other emergency preparedness activities and responses.
  • Expert panel report

Exposure Assessment

Healthcare and Social Assistance

  • Evidence package – Focuses on safe and healthy workplaces, musculoskeletal disorders, hazardous drugs and other chemicals, and infectious disease transmission.
  • Expert panel report

Mining

Next Steps

Currently, NIOSH is focused on building evaluation capacity, rather than conducting additional reviews. Learn more in Evaluation Capacity-Building Plan 2021-2025.

Explore More

From 2005-2008, the National Academies conducted evaluation studies of eight NIOSH programs:

To ensure consistency across reviews, NIOSH charged the National Academies in 2005 to develop a review framework. The framework document included standards and methods used in all eight reviews.