About
The example provided is not intended to be applied directly to any specific TB program. The actual activities implemented as part of TB program evaluation should be identified by state or local TB program officials in collaboration with other program interest holders.
Background and Rationale
Please provide a background, purpose or rationale and expected outcome of your project.
Planned Start Date: October 1, 2020 – Planned Completion Date: August 1, 2021
Scenario: Completing treatment among patients with latent tuberculosis infection (LTBI) is needed to prevent development of active tuberculosis (TB) disease; and promote progress towards TB elimination. Despite these important goals, over the past 3 years our program has not been making progress towards meeting the 2025 national target of at least 93% of contacts to sputum acid-fast bacilli smear-positive TB patients newly diagnosed with latent TB infection completing treatment. While almost all patients with latent TB infection begin treatment, a third do not complete treatment. Understanding why this occurs will assist in designing interventions that promote completion of treatment among persons with latent TB infection.
Long-Term Goal
Meet the 2025 National Performance Target (93%) of patients with newly diagnosed TB disease who complete treatment within 12 months among those who are eligible to complete treatment within 12 months.
Expected Outcomes and Short-Term Goals
Identify factors affecting success or failure to complete treatment among patients with latent TB infection. Using this information, the TB program will design and implement a plan that addresses these factors.
Objectives
Please provide one or more objectives for each goal of the evaluation topic. Objectives should be Specific, Measurable, Achievable, Realistic, and Time-bound (SMART). Each objective must have at least 1 evaluation question.
- Identify facilitators, barriers, and potential solutions to barriers expressed by patients in complying with latent TB infection treatment during October 1, 2020–May 1, 2021.
- Identify facilitators, barriers, and potential solutions to barriers expressed by public health staff in promoting completion of latent TB infection treatment during October 1, 2020–May 1, 2021.
- Identify characteristics of patients succeeding or failing to comply with latent TB infection treatment during October 1, 2020–May 1, 2021.
- Identify public health system factors that might be related to patients succeeding or failing to comply with latent TB infection treatment and possible solutions to barriers during October 1, 2020–May 1, 2021.
Corresponding National TB Indicators Project Indicator
Program evaluation should ideally reflect the National Tuberculosis Indicator Project (NTIP). Please specify the NTIP indicator addressed by this evaluation. If the evaluation is not linked to an NTIP indicator, please provide an explanation regarding how the evaluation relates to the scope of work in your cooperative agreement with the Centers for Disease Control and Prevention (CDC).
Contact Investigations: Latent TB infection Completion of Treatment
Examination of Immigrants and Refugees: Completion of latent TB infection treatment among contacts of persons with latent TB infection
Evaluation Plan Questions, Indicators, Analysis and Timeline
| Evaluation Question | Measure | Data Source | Method of Analysis | Target | Timeline/Persons Responsible |
|---|---|---|---|---|---|
| Each evaluation question or activity must correspond to an objective | Each evaluation question should have its own deliverable Having multiple deliverables per question is permissible |
Each deliverable needs a data source | Each deliverable should have a way to be calculated or measured objectively | How will you know your deliverable was met? | What is your anticipated timeline and who is responsible for different aspects of the evaluation? |
| 1. What are facilitators, barriers, and potential solutions to barriers expressed by patients in complying with LTBI treatment? | 75% of LTBI patients will identify facilitators, barriers, and potential solutions to barriers | Medical records of LTBI patients receiving treatment during November 1, 2020–May 1, 2021 | Each patient will be asked by nursing staff about facilitators, barriers, and potential solutions to barriers during ≥2 treatment sessions | 75% of all LTBI patients receiving treatment during November 1, 2020–May 1, 2021, will be asked to identify facilitators, barriers, and potential solutions to barriers to treatment | Develop questions, a database for responses, and instruct nurses: October 1–November 1, 2020
Analysis: May 1–July 1, 2021 |
| 2. What are facilitators, barriers, and potential solutions to barriers expressed by public health staff in promoting completion of LTBI treatment? | 90% of health care staff identify and record facilitators, barriers, and potential solutions to barriers to patient adherence with LTBI treatment | Health care staff conducting examinations, administering medication, or overseeing directly observed treatment (DOT) or electronic DOT (eDOT) to LTBI patients November 1, 2020–May 1, 2021 | Each health care staff member will complete a questionnaire regarding facilitators, barriers, and possible solutions to barriers during November 1, 2020–May 1, 2021 | 90% of health care staff will identify and record facilitators and barriers to providing treatment | Develop questions, database for responses: October 1-November 1, 2020
Analysis: June 1–August 1, 2021 |
| 3. What are patient characteristics among those succeeding or failing to comply with LTBI treatment? | Using the electronic surveillance system, review 50% of patients receiving treatment for LTBI November 1, 2020–May 1, 2021, and record treatment adherence and selected variables | Electronic surveillance system | The TB coordinator will conduct a review by using the electronic surveillance system for a random sample of 50% of eligible patients (patients receiving LTBI treatment during November 1, 2020–May 1, 2021) | 75% of eligible patients will have variables extracted and recorded | Select primary and secondary variables: October 1–November 1, 2020
Develop database to input abstracted data: November 1–December 1, 2020
Analysis: June 1–July 1, 2021 |
| 4. What public health system factors do clinic administrative and management staff express as affecting whether patients succeed or fail to complete LTBI treatment, and what solutions to possible barriers to LTBI treatment completion are expressed? | Staff involved in clinic administration or management will participate in a structured meeting to discuss analyses and brainstorm about related health system factors affecting LTBI treatment completion and strategies for addressing those factors | Structured meeting | Clinic administrative and management staff will discuss perceived facilitators, barriers, and potential solutions to barriers to LTBI treatment completion | At least four staff involved in clinic administration or management will participate in the structured meeting | Develop structured meeting questions: May 1–July 1, 2021 Conduct meeting: July 1–15, 2021Write up meeting summary: July 15–August 1, 2021 |
Evaluation Findings
Evaluation Question or Indicator Outcomes
| Evaluation Question or Indicator Outcomes (Listed results are by evaluation question and per NTIP indicator, if appropriate.) |
Benchmark Met or Not Met |
|---|---|
| 1.What facilitators, barriers, and possible solutions to barriers were expressed by patients in complying with treatment for LTBI?
20 eligible patients; 19 interviewed Facilitators
Possible Solutions to Barriers eDOT; evening clinic hours |
Met — 90% of patients asked about barriers and facilitators |
| 2.What facilitators, barriers, and possible solutions were expressed by public health staff in promoting completion of LTBI treatment? 10 eligible staff; 7 interviewed. Facilitators Calling patients who are late to remind them (4 nurses); having a regular schedule so that staff can develop a relationship with the patients (2 respondents) Barriers Constantly being assigned to different areas of the public health clinic so that staff do not develop a relationship with patients (3 respondents); too little time allotted for visits so that staff are unable to have a meaningful conversation with the patient receiving DOT (2 respondents) Possible Solutions to Barriers Nursing managers improve consistent assignment of staff, increase time allotted to visits by 5 minutes each, and development of prompting questions that can guide health care staff in identifying patients’ barriers to treatment completion |
Not met — 70% of eligible staff interviewed instead of 90% because of staff turnover |
| 3. What are patient characteristics among those succeeding or failing to adhere with the LTBI treatment regimen? 20 eligible patients; all electronic records reviewed Characteristics of Adherent Patients (12 Patients) Ages 30–60 years; ability to speak English fluently; or being US-born Characteristics of Nonadherent Patients (8 Patients) Ages 20–30 years or ≥60 years; being male; not speaking English well; or being homeless for part of the treatment period |
Met — all records reviewed |
| 4. What public health system factors do clinic administrative and management staff express as affecting whether patients succeed or fail to complete LTBI treatment, and what possible solutions to barriers are proposed?
Successful Adherence Extended clinic hours Unsuccessful Adherence Lack of communication with local homeless shelters
Extend clinic hours; develop association with homeless shelter staff |
Met — 5 staff participated and provided feedback in structured meeting |
Program Facilitators
What helped facilitate the completion of this evaluation? (This is helpful for other programs who might want to try a similar evaluation.)
- Having a timeline and an example to follow
- Attending monthly calls with the CDC program evaluation consultant
- Organized and up-to-date electronic surveillance records
Program Challenges
What major challenges did you face during this evaluation? (If you did not meet your benchmark, explain why here.)
- Not enough time was allotted for the records review and abstraction
- Many variables were missing from the records
- Three staff were unavailable for interview
Use of Findings
How did you use the findings?
We held a meeting with health care staff and clinic management on August 12, 2021, during which we reviewed all of the findings and discussed facilitators, barriers, and possible solutions to barriers. We identified multiple strategies for addressing barriers and expanding facilitators, as follows:
- Provide a bus pass for all eligible and interested patients (i.e., those expressing difficulty getting to the clinic)
- Extend clinic hours from 5:00 to 6:00 pm each day
- Develop and maintain a contact list of patients and a protocol or follow-up if they are ≥30 minutes late for an appointment
- Decrease shifting of TB staff to other clinic areas during hours when the majority of DOT patients come to the clinic
- Adopt use of culturally and linguistically appropriate materials and use of an alternative language telephone line or other interpreter service
- Establish an association with nearby homeless shelter staff
List of Deliverables, Products, and Dissemination Plan
Please provide a list of final deliverables and describe how you plan to disseminate to appropriate stakeholders.
- Include funding for bus passes in the next annual budget (September 2021)
- Revise staffing structure so that the clinic can remain open an additional hour each day (Begins October 2021)
- At next staff meeting, instruct nursing staff to maintain contact sheet for DOT patients and to call them if they are ≥30 minutes late
- Find and use TB materials in patients' first language or develop materials in patients' first language if none are found
- Schedule meeting with director of nearby homeless shelters to discuss areas for collaboration