Patterns of Sustainability Capacity Among Organizations That Deliver the National Diabetes Prevention Program: A Latent Profile Analysis

Lillian Madrigal, PhD, MPH¹; Regine Haardörfer, PhD, MEd¹; Michelle C. Kegler, DrPh, MPH¹; Sarah Piper, MPH¹; Linelle M. Blais, PhD¹; Mary Beth Weber, PhD, MPH¹; Cam Escoffery, PhD, MPH, CHES¹ (View author affiliations)

Suggested citation for this article: Madrigal L, Haardörfer R, Kegler MC, Piper S, Blais LM, Weber MB, et al. Patterns of Sustainability Capacity Among Organizations That Deliver the National Diabetes Prevention Program: A Latent Profile Analysis. Prev Chronic Dis 2023;20:230067. DOI: http://dx.doi.org/10.5888/pcd20.230067.

PEER REVIEWED

Abstract

Introduction

Since the launch of the National Diabetes Prevention Program (DPP) in 2010, more than 3,000 organizations have registered with the Centers for Disease and Control and Prevention to deliver the program; today, however, only approximately 2,000 organizations are registered, indicating challenges with sustainability. We used the Program Sustainability Assessment Tool (PSAT) to explore patterns of sustainability capacity among National DPP delivery organizations.

Methods

We used data from a cross-sectional online survey conducted in August and September 2021 of staff members (N = 440) at National DPP delivery organizations. We conducted a latent profile analysis to identify latent subpopulations on the basis of respondent PSAT domain scores. Regression analyses were used to estimate associations between derived latent classes, PSAT scores, and respondent characteristics.

Results

The 4-class model included 4 groups of capacity for program sustainability, ranging from low to high: low (class 1) with 8.0% of the sample, medium-low (class 2) with 22.0%, medium-high (class 3) with 41.6%, and high (class 4) with 28.4%. Program evaluation (mean score = 5.1 [SD = 1.4]) and adaptation (mean score = 5.3 [SD = 1.3]) were the domains with the highest scores, while funding stability (mean score = 4.0 [SD = 1.6]) and Partnerships (mean score = 4.0 [SD = 1.7]) had the lowest scores. In our sample of National DPP delivery organizations, most reported relatively high capacity for program sustainability, and key indicators associated with sustainability capacity were virtual delivery, location of delivery, funding sources, and organization type.

Discussion

Similar to sustainability capacity findings from other PSAT studies, our study found that funding stability and partnerships are areas to strengthen. This insight is useful in sustainability planning at organizational and national levels across multiple programs.

Top

Introduction

In 2020, an estimated 96 million adults in the US had prediabetes, a diagnosis that indicates a person is at risk for developing type 2 diabetes (1). For 20 years, the Diabetes Prevention Program (DPP), a lifestyle intervention to delay the onset of diabetes among people at high risk for diabetes, has been rigorously tested and adapted in multiple populations and formats (2,3). In 2010, the Centers for Disease Control and Prevention (CDC) launched the National DPP initiative to scale and sustain the intervention to make it widely available to the US population with prediabetes (4). CDC has invested in the program via multiple federal award mechanisms to support the infrastructure, implementation, and scaling of the program with the goal of these programs to become financially self-sustaining through various private and public payer models. Today, more than 2,000 National DPP organizations are registered to implement the year-long lifestyle-change program, a decrease from the more than 3,000 organizations that provided the National DPP from 2012 through 2019; this decrease indicates challenges in sustainability (5,6).

To make a population-level impact, evidence-based interventions need to be scaled and sustained (7). The field of implementation science defines sustainability as “the continued use of program components at sufficient intensity for the sustained achievement of desirable program goals and population outcomes” (7). The longer an intervention remains in place, the greater reach and effect it can have (8). Factors associated with sustainability include, among others, the adaptability of a program, support of champions and other key partners, program fit within an organization, perceived impacts and benefits of a program, and organization capacity (7,9,10).

Although people familiar with the National DPP know of many organizations that have been delivering the program for years, research is limited on the average length of delivery or sustainability in general. A 2019 study of 165 CDC-recognized organizations delivering the National DPP defined sustainability by using the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework’s domain of maintenance (“the extent to which programs had potential for sustainability, measured by the number of delivery sites achieving full CDC recognition, the number of sites continuing to deliver the program without cooperative agreement funding, and organizational and financial support or program reimbursement from private or public payers”) (11). The study found that in 4 years (2012–2016), 132 sites (80%) had at least 12 months of participant data and 33 (25%) of these 132 achieved full CDC recognition. CDC recognition involves demonstrating how the delivery organization has effectively delivered the National DPP and met a set of CDC implementation and outcome standards. Because the National DPP takes 1 year to complete, 12 months of data is considered a short-term indicator of sustainability at best. No other sustainability findings, including factors predicting program sustainability, were reported.

A movement has begun in recent years to better define, operationalize, and measure sustainability of evidence-based public health programs (7,12,13). The Program Sustainability Assessment Tool (PSAT), developed in 2014 through a comprehensive review of tools measuring public health program sustainability, examines 8 domains that affect program sustainability capacity (14). The PSAT defines program sustainability capacity as the ability to maintain programming and its benefits over time. The PSAT has been used primarily as a planning tool at a single point in time and has not been tested in a predictive capacity (12). Understanding patterns in program sustainability capacity across organizations may be useful in understanding the landscape of National DPP delivery.

Latent profile analysis (LPA) is a statistical method that focuses on identifying subpopulations within a population based on a certain set of continuous variables into mutually exclusive groups or classes, called “latent profiles” (15,16). The term “latent” is used to describe the class membership that cannot be directly observed. At the organizational level, LPA has been used to examine patterns in contextual and organizational factors in community-based programs (17), clinician practices (18), and community readiness for programs (19) to better understand evidence-based program implementation and adoption. However, at the organizational level, to our knowledge, no studies have used LPA to examine program sustainability capacity among National DPP delivery organizations.

The objective of this study was to explore patterns of program sustainability capacity among organizations delivering the National DPP by using the Program Sustainability Assessment Tool (PSAT), to understand whether organizations can be categorized into distinct groups based on dimensions of their sustainability capacity and if those groups are associated with specific organizational characteristics. The findings from this research have the potential to support National DPP implementation by providing an understanding of sustainability capacity strengths and weaknesses across different organizations as well as recommendations for capacity building for sustainability at National DPP delivery organizations.

Top

Methods

This study used data from a cross-sectional online survey conducted in August and September 2021 of staff members at National DPP delivery organizations. The sample included staff members in the following key roles: lifestyle coaches, master trainers, and program coordinators. This study was reviewed and determined to be exempt by the Emory University Institutional Review Board (STUDY00002611).

Measures

The survey instrument included 107 items: 23 items about the respondent and the respondent’s organization, 40 items from the Program Sustainability Assessment Tool (https://sustaintool.org/psat), and 38 Likert-scale items from the Consolidated Framework for Implementation Research (https://cfirguide.org/). Of the 23 items on respondent and organization characteristics, we analyzed 4 items of respondent characteristics (role at organization, gender, race and ethnicity, and age) and 12 items on organizational characteristics: 1) recognition in CDC’s Diabetes Prevention Recognition Program (DPRP); 2) length of program delivery, in years; 3) enrollment (the number of program participants enrolled to date in DPP); 4) the number of lifestyle coaches on staff; 5) the number of staff members who are not lifestyle coaches; 6) the number of staff members who are 100% dedicated to the National DPP; 7) type of organization; 8) size of organization; 9) program delivery mode (eg, in person, distance); 10) location (eg, rural, urban), 11) racial and ethnic composition of program participants; and 12) National DPP funding sources. Because of the large number of variables for respondent and organization characteristics and small sample sizes for some categories of race and ethnicity, we aggregated data to compare organizations that serve either White-only populations or non–White-only populations to understand differential reach to these audiences. This study focused on the analysis of the delivery organization characteristics and the PSAT items. We did not analyze data from the Consolidated Framework for Implementation Research.

The PSAT covers 8 sustainability domains: Environmental Support, Funding Stability, Partnerships, Organizational Capacity, Program Evaluation, Program Adaptation, Communications, and Strategic Planning. Items assessed the extent to which a program has or does the following on a Likert scale from 1 to 7 (1 = a little or no extent, 7 = to a very great extent). Domain scores are averaged and provide a PSAT score of 1 to 7, which indicates the level of sustainability capacity. The higher the score, the greater the sustainability capacity. The PSAT has been tested for psychometric properties in trainings and evaluations with more than 550 people and more than 250 unique programs at state and local levels (14,20). Reliability testing determined a high reliability, with Cronbach α for domain subscales ranging from 0.79 to 0.92 (14).

Data collection and study sample

Study participants were recruited from the National DPP implementer population at Emory University’s Diabetes Training and Technical Assistance Center (DTTAC). In the past 10 years, DTTAC has directly trained more than 5,000 lifestyle coaches representing more than 2,000 organizations across all 50 states. The survey was distributed to an email contact list of 6,470 National DPP delivery organizations that have participated in Emory’s DTTAC programs or subscribed to the DTTAC newsletter. The first 336 respondents received a $15 Amazon gift card for their participation. After data cleaning for completion, we included 586 responses (9% response rate) in the analysis. Of those 586 respondents, 440 (75%) had a calculable PSAT score. According to PSAT instructions, averages are totaled and nonresponse items are excluded; in other words, we excluded respondents who did not respond to PSAT items. PSAT scores can be calculated by answering one or more of the items. Of those who responded to the PSAT questions, an average 35 (SD, 8.9) of 40 questions were answered, and response completion ranged from 1 to 40 items.

Descriptive analysis

Data were exported from Qualtrics and analyzed by using SAS software version 9.4 (SAS Institute Inc). After data were cleaned, we ran standard descriptive statistics (eg, frequencies, distributions, means). We scaled (divided by 100) the enrollment-to-date variable to assist with comparisons across other variables after removing 4 outliers above the 99th percentile. We compared respondents with and without a PSAT score; we used χ² and t tests to detect significant differences between groups. The level of significance was set at P <.05 for all analyses.

PSAT item internal consistency. We calculated the means and SDs for all 40 PSAT items along with domain averages and the total PSAT score average. We calculated Cronbach α for all domains and the total PSAT score, measuring the internal consistency of the items within each domain scale and the domains together to form the PSAT score.

LPA. LPA is a person-centered approach that classifies respondents into mutually exclusive profiles, groups, classes, or clusters (16). First, the researcher selects numerous groups or classes to assign the respondent to (ie, asks the statistical program to allocate respondents into 2 classes, 3 classes, 4 classes, and so on, by using the data available). After analyzing all respondent data for each PSAT domain score, the respondents are assigned a probability of being part of each derived class and assigned to a class according to their highest posterior probability. The assigned class is treated like an unobserved categorical variable, where its value indicates which profile a respondent belongs to with a certain degree of probability (16).

This LPA included the 8 PSAT domain score variables. We used Mplus version 8.3 (Muthén and Muthén) to run a succession of 8 models. Cases with missing items were estimated in Mplus by using full information maximum likelihood. The selection of the best model included reviewing the fit indices and information criteria; latent class proportions and sizes; and the researchers’ interpretation and entropy of the latent classes. All entropy values in the models run were greater than 0.85. After review of fit indices, class proportions, and the visual profiles of each, we selected the 4-class model as the best fit and most meaningful output of class proportions.

To estimate associations between derived latent classes and organization characteristics, we conducted multivariable multinomial logistic regression using class 1 (low sustainability score) as the reference group. Respondent organization characteristics in the multivariable multinomial logistic regression included the following variables: the number of program participants enrolled to date, staffing (number of lifestyle coaches on staff, number of staff dedicated 100% to National DDP), organization type, organization size, program delivery mode, location, programs with White only and non-White only populations enrolled, and National DPPP funding sources. The final sample for the multivariable multinomial logistic regressions included 259 respondents, which was smaller than our sample of 440 respondents to the PSAT questions because of data missing at random.

Bivariate and multivariable regression

To gain a better understanding of relationships between variables and compare with the LPA results, in addition to the LPA, we conducted bivariate and multivariable regressions for all key organization characteristic variables with the PSAT score as the outcome. We calculated Pearson correlation coefficients for variables that captured similar dimensions (multicollinearity). For the full multivariable regression model, we combined variables (racial and ethnic composition of program participants and program delivery mode) or eliminated variables (recognition in CDC’s DPRP, length of program delivery, staff members who are not DPP lifestyle coaches) on the basis of degree of correlation and theoretical overlap with other variables. The final multivariable regression model included 21 variables for organization characteristics.

After reviewing patterns of missing data, we concluded that data were missing at random, that is, missingness was related only to variables that were collected. We used multiple imputations in SAS software version 9.4 to handle data missing at random for the enrollment (30% missing) and organization size (20% missing). Twenty-one variables on organization characteristics from our main analysis plus 4 auxiliary organization variables related to missingness (DPRP status, years of program delivery, non-lifestyle coach staff, and health care organization as reference group for organization type) were added to the imputation model. We used the fully conditional method to handle the continuous, ordinal, and dichotomous variables. We created a total of 10 imputed data sets and compared these results with the original regression model and the LPA multivariable multinomial logistic regression.

Top

Results

Of the 440 survey respondents with a PSAT score, 399 (90.7%) were lifestyle coaches, 387 (88.0%) were women, 264 (60.0%) were White, and 288 (65.4%) were aged 35 to 64 years (Table 1). Most (52.5%) respondents belonged to organizations with full recognition in CDC’s DPRP. The mean (SD) length of program delivery was 4.6 (3.1) years (median, 4; IQR, 3.0–6.0), and mean (SD) enrollment to date was 1,991 (28,325) program participants (Table 2). The most common types of respondent organizations were health care and hospitals (31.4% of respondents); community-based health care (eg, community health centers, federally qualified health centers, Indian Health Service; 23.6% of respondents); health insurers, employers, and other (eg, private businesses; 15.5% of respondents); and government agencies (13.4% of respondents).

Most (78.0%) respondents reported offering programs in some type of virtual mode (distance, 57.3%; online, 17.3%; hybrid, 23.9%; no response, 1.5%). Respondents reported mostly enrolling White (67.7% of respondents), Black (48.4% of respondents), and Hispanic or Latino (33.4% of respondents) populations. Respondents indicated that their organization was primarily funded or supported by grants (41.6% of respondents), state or local government (23.4% of respondents), or the federal government or CDC (22.5% of respondents) (Table 2).

When we compared the 440 respondents with a PSAT score with the 146 respondents who did not answer the PSAT items, one significant difference was that respondents with a PSAT score were significantly more likely to be program coordinators (41.1% vs 27.4%; P =.003) for the National DPP (a role more heavily involved with the program at the organizational level). Almost all (93.8%; 137 of 146) respondents who did not complete the PSAT also did not answer the demographic questions, which were at the end of the survey, indicating they simply did not complete the survey.

PSAT scores

Mean (SD) scores for individual items in the PSAT instrument ranged from 3.6 (1.9) to 5.6 (1.4) (Table 3). All items and domain means were slightly positively skewed. By domain, program evaluation (5.1 [1.4]) and adaptation (5.3 [1.3]) were rated highest, whereas funding stability (4.0 [1.6]) and partnerships (4.0 [1.7]) were rated lowest. The mean (SD) PSAT score was 4.6 (1.3).

Latent profile analysis

The model consisted of 4 classes with mean domain scores that ranged from low to high: class 1 had low program sustainability (8.0% of the sample), class 2 had medium-low program sustainability (22.0% of the sample), class 3 had medium-high program sustainability (41.6% of the sample), and class 4 had high program sustainability (28.4% of the sample) (Figure).

Figure.
Four-class model of Program Sustainability Assessment Tool domains in a latent profile analysis of patterns of sustainability capacity among organizations that deliver the National Diabetes Prevention Program. Class 1 had low program sustainability (8.0% of the sample), class 2 had medium-low program sustainability (22.0% of the sample), class 3 had medium-high program sustainability (41.6% of the sample), and class 4 had high program sustainability (28.4% of the sample). [A tabular version of this figure is available.]

That we found no distinct patterns among classes indicates that all 8 PSAT domains were consistently scored across capacity levels, strengthening the evidence for internal consistency of the PSAT score, which is the average of all 8 domain scores. All organizations, despite capacity level, tended to have the same areas of strength and weakness. Program evaluation and adaptation had the highest scores in all classes across the model, whereas funding stability and partnerships had the lowest scores. This pattern matches the pattern of domain averages in the raw PSAT scores (Table 2), further supporting internal consistency of the PSAT score.

Multivariable multinomial logistic regression

In the regression analyses of data from 259 survey respondents, compared with the class with low program sustainability (class 1), all other classes had on average 5.68 times (95% CI, 1.21–27.07) greater likelihood of having obtained grant funding to support their National DPP (Table 4). Compared with class 1, the class with medium-to-low program sustainability (class 2) was 4.36 times (95% CI, 1.08–17.67) more likely to be supported by state or local government funding, while the class with medium-high program sustainability (class 3) was less likely to have state employee coverage benefits for the National DPP (0.05, 95% CI, 0.003–0.84). Lastly, the class with high program sustainability (class 4) was 7.99 times (95% CI, 1.07–59.70) more likely than class 1 to be a government agency or academic institution.

Multivariable regression analysis

Since our LPA model demonstrated the internal consistency strength of the PSAT score, we also ran a multivariable regression model using PSAT score as the outcome with the multiple imputations data set to compare organizational characteristic predictors. In multivariable regression analysis, the virtual delivery mode (0.49 PSAT points; 95% CI, 0.19–0.79; t = 3.21; P =.001) and rural location (−0.48 PSAT points; 95% CI, −0.80 to −0.16; t = −2.92; P =.004) were significantly associated with PSAT score (Table 5).

Top

Discussion

The PSAT has been used by numerous health promotion programs to improve sustainability planning (21–26). This is the first study to use the PSAT to examine sustainability capacity in a sample of National DPP delivery organizations. The LPA did not identify any distinct patterns among PSAT respondents. Instead, the LPA categorized National DPP delivery organizations into a series of 4 classes that ranged from low PSAT scores to high PSAT scores. Overall, National DPP delivery organizations reported relatively high program sustainability capacity: most (70.0%) organizations were placed in the high (28.4%) or medium-high (41.6%) program sustainability class. These results provide evidence to support the reliability of the PSAT score to identify sustainability capacity.

In our study, similar to other studies using the PSAT, the funding stability domain had the lowest average score, and program adaptation and evaluation had the highest scores (14,25,27). In the multivariable multinomial logistic regressions of the 4-class LPA model, we also found that, compared with the low program sustainability class (class 1), organizations in the higher program sustainability classes (classes 2, 3, and 4) were more likely to have grant funding support for their National DPP efforts. Funding stability is often thought of as one of the most important factors in many sustainability frameworks and can influence other sustainability domains (10,28).

Although our study found external grant funding to be associated with sustainability, the National DPP has focused on reimbursements by insurance providers (employers, Medicare, Medicaid) (6), not on program implementation funding for financial sustainability. Only 15% of our sample indicated having Medicare and Medicaid support for their programs. Nationally, Medicare and Medicaid funding has posed challenges, namely that reimbursement rates are lower than actual organizational costs incurred, which may explain why Medicare and Medicaid were not significantly associated with sustainability capacity in our study (6,29). Future research could examine the extent to which organizations receive various forms of financial support, the duration of funding, and the impact of funding on sustainability outcomes for DPPs and other chronic disease programs.

Program evaluation and adaptation domains had the highest average scores in our sample, which may be explained by the robust standards and guidance provided by CDC’s DPRP. Many delivery organizations are familiar with collecting and analyzing their program data to submit to CDC to maintain their recognition status. Likewise, CDC has promoted various program adaptations to ensure the lifestyle change program curriculum is effective in the communities and populations where they are implemented. The National DPP’s Customer Service Center also provides technical assistance resources for tailoring program elements for various situations.

The LPA classes with higher program sustainability capacity (classes 3 and 4) rated the organizational capacity and communications domains highly. Findings were similar in other sustainability frameworks and studies (28). The organizational capacity domain refers to having the internal support and resources needed to effectively manage a program, while the communications domain focuses on strategic contact with interested parties and the public about the program. It is not surprising that organizations with resources, capacity, and strong communications can maintain their health promotion programming. Other factors related to these domains are characteristics such as effective leadership and support from champions (30), which in the PSAT are captured by the environment support domain, which was also highly rated by the groups in our sample with higher program sustainability capacity. Our previous research using the Consolidated Framework for Implementation Research to examine National DPP implementation also found that organizational capacity in terms of structural characteristics and leadership engagement affected the implementation outcome of reach (31).

One way in which organizational capacity strengths can be seen in our sample is through technology and staffing for virtual delivery. Before the COVID-19 pandemic, only a small number (121 in 2019) of organizations were offering the DPP program virtually (4); today, on the basis of our study findings and the technical assistance provided to delivery organizations, it is clear that many more organizations use virtual delivery methods out of necessity. In our study, 447 of 586 (76.3%) organizations reported virtual delivery. Benefits of virtual delivery include decreasing the cost of delivery; eliminating some logistical challenges, such as finding physical space for the program; and overcoming barriers to participant attendance, such as transportation or scheduling, while maintaining program outcomes on par with the in-person DPP (32). Using the PSAT score as the regression outcome, we found that organizations with any type of virtual delivery mode (online, distance, hybrid) were associated with higher sustainability capacity. In addition, virtual delivery allows for greater geographical reach of participants. In our analysis, the programs in rural locations were significantly associated with lower sustainability capacity scores. One implication of these data is that the National DPP should prioritize improving implementation in rural communities and virtual delivery modes to enhance their adoption and sustainability efforts.

Overall, organization type did not appear to influence sustainability capacity. In our multivariable multinomial logistic regressions of the 4-class LPA model, government and academic organizations were positively associated with the highest capacity class (class 4), which may indicate some benefit of being a government agency or academic institution when considering the long-term delivery and maintenance of the National DPP. Perhaps the staff may have more opportunities and capabilities (ie, grant writing skills) to receive funding at these types of institutions. Government agencies, such as health departments, also receive chronic disease funding and technical assistance, which may also affect their sustainability capacity. However, the lack of significant relationships between organization types and sustainability capacity supports CDC’s vision that a wide variety of organizations can deliver the National DPP. More research with a larger sample of National DPP organizations should be used to examine sustainability outcomes (eg, length of delivery, enrollment growth, maintained participant outcomes, financial self-sustainment) based on organization type.

The National DPP and other chronic disease prevention programs can use the findings of this study to support using the PSAT to assess sustainability capacity of their programs and as a first step toward sustainability planning. Future research can build on this work by using the PSAT to assess sustainability capacity at the point of program adoption and how well the measure can predict long-term sustainability outcomes (eg, length of delivery, sustained health outcomes, program growth). Providers of implementation technical assistance can also use the PSAT as the first step in an intervention to build sustainability capacity to help organizations with low PSAT scores. People providing technical assistance should pay particular attention to low levels of funding stability and help organizations secure new funding before current sources expire. Lastly, our study found that the National DPP program coordinator was more likely than respondents in other staff roles to complete the PSAT assessment. This finding indicates that the staff in the program coordinator role may be most able to complete organization-level assessments and should be engaged in sustainability planning and these types of studies in the future.

Strengths and limitations

Strengths of this study include the use of the PSAT, a reliable measure for sustainability capacity with a large and diverse sample of National DPP delivery organizations. Although we had problems with missing data, we used multiple imputation methods to account for these data and draw meaningful conclusions. Another strength includes the large number of organizational characteristic variables used to study associations with program sustainability capacity scores. One limitation is that we recruited our sample from Emory’s DTTAC contact list; differences between this group and the larger National DPP population of implementers may exist because of resources the Emory group may have received from Emory University. We were also unable to accurately assess whether multiple people from the same organization completed the survey because of inconsistencies in how respondents provided organization names. We acknowledge that this limitation may have resulted in cluster effects. However, upon review of the organization names provided, more than 70% of the data included 1 respondent per organization listed. This large proportion of singletons in the data indicates that clustering cannot be adequately accounted for through methods such as a random effects model because of biases in the random effects estimation (33). An opportunity for future research may be to examine differences in sustainability capacity perception between groups of people with different organizational roles (eg, leadership compared with implementation staff) and how to engage various people in sustainability planning.

The incomplete information on organization name highlights another limitation: the inability to use data directly from the CDC DPRP database due to the federal government’s data use restrictions. Because of these restrictions, we relied on respondents to self-report their organizational information, which was not always complete, even after our attempts to follow up with respondents with missing data. For future studies, we hope researchers will have increased access to CDC DPRP data to better evaluate these implementation science questions at the national level.

Conclusion

We used the PSAT to describe the sustainability capacity of National DPP delivery organizations. Understanding sustainability capacity is important to help program implementers with program planning, delivery, and scaling. Maintaining a program over time allows for growth and evolution within an organization in ways such as increasing program offerings, participant enrollment, and reach to new populations (10). Increased delivery allows for more program impact locally, while understanding sustainability across organizations offers collective learning and best practices to scale-up and scale-out programs in different settings, organizations, and populations. With the limited resources we have in public health promotion, we must strive to understand levers, and develop plans and interventions to scale and sustain effective evidence-based programs as best we can.

Top

Acknowledgments

This work was made possible by the partnership between Emory’s Diabetes Training and Technical Assistance Center and The Georgia Center for Diabetes Translation Research and funding from the Centers for Disease Control and Prevention (U48 DP006377). The Program Sustainability Assessment Tool is copyrighted by Washington University in St Louis and was used with permission.

Top

Author Information

Corresponding Author: Lillian Madrigal, PhD, MPH, Rollins School of Public Health, Emory University, 1518 Clifton Rd, Atlanta, GA, 30322 (lmadrig@emory.edu).

Author Affiliations: ¹Rollins School of Public Health, Emory University, Atlanta, Georgia.

Top

References

1. Centers for Disease Control and Prevention. National Diabetes Statistics Report. Last reviewed June 29, 2022. Accessed February 1, 2023. https://www.cdc.gov/diabetes/data/statistics-report/index.html
2. Ali MK, Echouffo-Tcheugui J, Williamson DF. How effective were lifestyle interventions in real-world settings that were modeled on the Diabetes Prevention Program? Health Aff (Millwood). 2012;31(1):67–75. PubMed doi:10.1377/hlthaff.2011.1009
3. Ackermann RT, Finch EA, Caffrey HM, Lipscomb ER, Hays LM, Saha C. Long-term effects of a community-based lifestyle intervention to prevent type 2 diabetes: the DEPLOY extension pilot study. Chronic Illn. 2011;7(4):279–290. PubMed doi:10.1177/1742395311407532
4. Gruss SM, Nhim K, Gregg E, Bell M, Luman E, Albright A. Public health approaches to type 2 diabetes prevention: the US National Diabetes Prevention Program and beyond. Curr Diab Rep. 2019;19(9):78. PubMed doi:10.1007/s11892-019-1200-z
5. Centers for Disease Control and Prevention. National Diabetes Recognition Program: registry of all recognized organizations. Accessed February 1, 2023. https://dprp.cdc.gov/Registry
6. Ritchie ND, Baucom KJW, Sauder KA. Current perspectives on the impact of the National Diabetes Prevention Program: building on successes and overcoming challenges. Diabetes Metab Syndr Obes. 2020;13:2949–2957. PubMed doi:10.2147/DMSO.S218334
7. Shelton RC, Cooper BR, Stirman SW. The sustainability of evidence-based interventions and practices in public health and health care. Annu Rev Public Health. 2018;39(1):55–76. PubMed doi:10.1146/annurev-publhealth-040617-014731
8. Glasgow RE, Klesges LM, Dzewaltowski DA, Estabrooks PA, Vogt TM. Evaluating the impact of health promotion programs: using the RE-AIM framework to form summary measures for decision making involving complex issues. Health Educ Res. 2006;21(5):688–694. PubMed doi:10.1093/her/cyl081
9. Scheirer MA. Is sustainability possible? A review and commentary on empirical studies of program sustainability. Am J Eval. 2005;26(3):320–347.
10. Shelton RC, Chambers DA, Glasgow RE. An extension of RE-AIM to enhance sustainability: addressing dynamic context and promoting health equity over time. Front Public Health. 2020;8:134. PubMed doi:10.3389/fpubh.2020.00134
11. Nhim K, Gruss SM, Porterfield DS, Jacobs S, Elkins W, Luman ET, et al. . Using a RE-AIM framework to identify promising practices in National Diabetes Prevention Program implementation. Implement Sci. 2019;14(1):81. PubMed doi:10.1186/s13012-019-0928-9
12. Palinkas LA, Spear SE, Mendon SJ, Villamar J, Reynolds C, Green CD, et al. . Conceptualizing and measuring sustainability of prevention programs, policies, and practices. Transl Behav Med. 2020;10(1):136–145. PubMed doi:10.1093/tbm/ibz170
13. Stirman SW, Dearing JW. Sustainability of Cancer Practices and Programs. In: Chambers DA, Vinson CA, Norton WE, eds. Advancing the Science of Implementation Across the Cancer Continuum. Oxford University Press; 2018:312–329.
14. Luke DA, Calhoun A, Robichaux CB, Elliott MB, Moreland-Russell S. The Program Sustainability Assessment Tool: a new instrument for public health programs. Prev Chronic Dis. 2014;11:130184. PubMed doi:10.5888/pcd11.130184
15. McCutcheon AL. Latent class analysis. Sage Publications, Inc; 1987.
16. Spurk D, Hirschi A, Wang M, Valero D, Kauffeld S. Latent profile analysis: a review and “how to” guide of its application within vocational behavior research. J Vocat Behav 2020;120.
17. Smith EP, Witherspoon DP, Lei P-W. The “haves, have some, and have nots:” a latent profile analysis of capacity, quality, and implementation in community-based afterschool programs. Prev Sci. 2021;22(7):971–985. PubMed doi:10.1007/s11121-021-01258-z
18. Becker-Haimes EM, Lushin V, Creed TA, Beidas RS. Characterizing the heterogeneity of clinician practice use in community mental health using latent profile analysis. BMC Psychiatry. 2019;19(1):257. PubMed doi:10.1186/s12888-019-2234-0
19. de Oliveira Corrêa A, Brown EC, Lee TK, Mejía-Trujillo J, Peréz-Gómez A, Eisenberg N. Assessing community readiness for preventing youth substance use in Colombia: a latent profile analysis. Int J Ment Health Addict. 2020;18(2):368–381. PubMed doi:10.1007/s11469-019-00191-1
20. Washington University in St. Louis, Center for Public Health Systems Science. Program Sustainability Assessment Tool. Accessed February 1, 2023. https://sustaintool.org/psat/assess/#about-assessment
21. Adams EE. Providers’ Perspectives of Sustainability in Nutrition Wellness Programmes in the Porirua Community. Master’s thesis. University of Otago; 2017.
22. King DK, Gonzalez SJ, Hartje JA, Hanson BL, Edney C, Snell H, et al. . Examining the sustainability potential of a multisite pilot to integrate alcohol screening and brief intervention within three primary care systems. Transl Behav Med. 2018;8(5):776–784. PubMed doi:10.1093/tbm/ibx020
23. Llauradó E, Aceves-Martins M, Tarro L, Papell-Garcia I, Puiggròs F, Prades-Tena J, et al. . The “Som la Pera” intervention: sustainability capacity evaluation of a peer-led social-marketing intervention to encourage healthy lifestyles among adolescents. Transl Behav Med. 2018;8(5):739–744. PubMed doi:10.1093/tbm/ibx065
24. Reichert J. Fighting the Opioid Crisis Through Substance Use Disorder Treatment: A Study of a Police Program Model in Illinois. Illinois Center for Justice Research and Evaluation. September 9, 2019. Accessed February 1, 2023. https://icjia.illinois.gov/researchhub/articles/fighting-the-opioid-crisis-through-substance-use-disorder-treatment-a-study-of-a-police-program-model-in-illinois
25. Stoll S, Janevic M, Lara M, Ramos-Valencia G, Stephens TB, Persky V, et al. . A mixed-method application of the Program Sustainability Assessment Tool to evaluate the sustainability of 4 pediatric asthma care coordination programs. Prev Chronic Dis. 2015;12:E214. PubMed doi:10.5888/pcd12.150133
26. Tabak RG, Duggan K, Smith C, Aisaka K, Moreland-Russell S, Brownson RC. Assessing capacity for sustainability of effective programs and policies in local health departments. J Public Health Manag Pract. 2016;22(2):129–137. PubMed doi:10.1097/PHH.0000000000000254
27. Moreland-Russell S, Combs T, Polk L, Dexter S. Assessment of the sustainability capacity of a coordinated approach to chronic disease prevention. J Public Health Manag Pract. 2018;24(4):E17–E24. PubMed doi:10.1097/PHH.0000000000000663
28. Bodkin A, Hakimi S. Sustainable by design: a systematic review of factors for health promotion program sustainability. BMC Public Health. 2020;20(1):964. PubMed doi:10.1186/s12889-020-09091-9
29. Meyer H. Medicare diabetes prevention: enrollment short of projections. Health Aff (Millwood). 2021;40(11):1682–1687. PubMed doi:10.1377/hlthaff.2021.01292
30. Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3-4):327–350. PubMed doi:10.1007/s10464-008-9165-0
31. Madrigal L, Manders OC, Kegler M, Haardörfer R, Piper S, Blais LM, et al. . Inner and outer setting factors that influence the implementation of the National Diabetes Prevention Program (National DPP) using the Consolidated Framework for Implementation Research (CFIR): a qualitative study. Implement Sci Commun. 2022;3(1):104. PubMed doi:10.1186/s43058-022-00350-x
32. Katula JA, Dressler EV, Kittel CA, Harvin LN, Almeida FA, Wilson KE, et al. . Effects of a digital diabetes prevention program: an RCT. Am J Prev Med. 2022;62(4):567–577. PubMed doi:10.1016/j.amepre.2021.10.023
33. Bell BA, Ferron JM, Kromrey JD. Cluster size in multilevel models: the impact of sparse data structures on point and interval estimates in two-level models. JSM (Joint Statistical Meetings). 2008. Accessed June 10, 2023. http://www.asasrms.org/Proceedings/y2008/Files/300933.pdf

Top

Tables

Abbreviation: DPP, Diabetes Prevention Program.
^a Of 586 respondents to the survey, 440 (75%) had a calculable score for the Program Sustainability Assessment Tool and were included in the analysis.

Abbreviations: —, does not apply; CDC, Centers for Disease Control and Prevention; DPP, Diabetes Prevention Program.
^a Of 586 respondents to the survey, 440 (75%) had a calculable score for the Program Sustainability Assessment Tool and were included in the analysis.
^b Enrollment-to-date variable was scaled (divided by 100) to assist with comparisons across other variables after removing 4 outliers above the 99th percentile.

Abbreviations: —, does not apply; CDC, Centers for Disease Control and Prevention; DPP, Diabetes Prevention Program; PSAT, Program Sustainability Assessment Tool.
^a Of 586 respondents to the survey, 440 (75%) had a calculable score for the Program Sustainability Assessment Tool and were used in the analysis.
^b Scaled from 1 to 7; the higher the score, the greater the sustainability capacity.

Abbreviations: CDC, Centers for Disease Control and Prevention; DPP, Diabetes Prevention Program.
^a Final sample included 259 respondents, smaller than our sample of 440 respondents to the Program Sustainability Assessment Tool questions because of data missing at random.
^b Reference group is class 1, low program sustainability scores (8.0% of respondents).

Abbreviations: CDC, Centers for Disease Control and Prevention; DPP, Diabetes Prevention Program; PSAT, Program Sustainability Assessment Tool.
^a Estimates from 10 imputations.

Top