Overview and Methodology of the Adolescent Behaviors and Experiences Survey — United States, January–June 2021

Introduction

Beginning March 2020, many U.S. schools closed nationwide to prevent transmission of SARS-CoV-2, the virus that causes COVID-19 (1). Although studies indicate that COVID-19 illness is generally less severe in children than adults (2), children are still at risk for developing severe illness and complications from COVID-19. In addition, the pandemic has had negative effects on adolescents’ mental health (3). Compared with 2019, the proportion of mental health–related emergency department visits in 2020 increased approximately 31% among youths aged 12–17 years (3). On January 13, 2021, approximately 50% of all students in the United States were receiving online-only instruction (4), and research has shown that online-only instruction has had a negative effect on the mental health of adolescents (5). Although these studies provide data on the effects of the COVID-19 pandemic on mental health among youths, little is known about other experiences such as economic and food insecurity, abuse by a parent, and other risk behaviors youths might have engaged in during this time, such as alcohol and drug use. To assess youth behaviors and experiences during the COVID-19 pandemic, CDC developed and conducted the Adolescent Behaviors and Experiences Survey (ABES) among students in grades 9–12.

In spring 2020, CDC used funds from the Coronavirus Aid, Relief, and Economic Security (CARES) Act to launch ABES (https://www.govinfo.gov/app/details/PLAW-116publ136). CDC subject matter experts from the Division of Adolescent and School Health program, in collaboration with CDC’s COVID-19 Response Team, rapidly developed and implemented ABES by adapting methodology from the national Youth Risk Behavior Survey (YRBS), part of the nation’s largest surveillance system dedicated to adolescent health (6). ABES asked students about six categories of behaviors included in a typical YRBS questionnaire: 1) behaviors that contribute to unintentional injury and violence, 2) tobacco product use, 3) alcohol and other drug use, 4) sexual behaviors, 5) dietary behaviors, and 6) physical inactivity. In addition to these questions, ABES included one question about students’ perceptions of racism and 12 questions about the following experiences during the COVID-19 pandemic: 1) increased use of alcohol and drugs, 2) economic and food insecurity, 3) abuse by a parent, 4) poor mental health, 5) school and social engagement, and 6) use of telemedicine. This overview report describes the 2021 ABES methodology, including the online questionnaire and administration, sampling, data collection, weighting, and descriptive analyses. In addition, this MMWR Supplement also includes reports on substance use, mental health and suicidality, perceived racism, and disruptions to student life among high school students. Findings from ABES during the COVID-19 pandemic can help guide parents, teachers, school administrators, community leaders, clinicians, and public health officials in decision-making for student support and school health programs.

ABES Methodology

Overview

ABES was a one-time, online survey conducted during January–June 2021 to assess student behaviors and experiences during the COVID-19 pandemic. ABES surveyed high school students in grades 9–12 attending U.S. public and private schools. Classes were randomly selected to participate within a nationally representative sample of schools. Because of the different instructional models used across the nation during the pandemic (i.e., in-person only, virtual only, and hybrid), ABES was designed as a self-administered, anonymous survey that was administered online. The online administration allowed each school and teacher the flexibility to decide whether students completed the survey during instructional time or on their own time. In addition to student-level data, school-level data (e.g., instructional mode and school-provided equipment) also were collected.

School Characteristics

The COVID-19 pandemic resulted in modifications of the learning environment, and many students faced several challenges, including lack of adequate resources at home (7). Therefore, collecting information on school characteristics during the fielding of ABES was important. School staff reported the school’s instructional model (in-person only, virtual only, or hybrid) at the time of recruitment; hybrid schools had a portion of a school’s student body receiving in-person instruction while others received virtual instruction. School staff members also reported whether the school had reduced class sizes and whether they provided laptops or Chromebooks, tablet computers, and Wi-Fi hotspots to students for home use.

Student Online Questionnaire and Administration

Students completed ABES questionnaires in English or Spanish using a secure URL on any Internet-connected device. They were required to watch a 2-minute video (with an option of an English version, with English audio, or a Spanish version, with English audio and Spanish subtitles) and read brief instructions before starting the questionnaire. On average, students completed the 110-item questionnaire in 30 minutes. Although students were encouraged to finish the survey in one sitting, they were able to save their answers, stop the survey, and resume the survey as needed. Cumulative timing data were captured for the entire time a student was logged in, thus a meaningful range of completion time is not available. The survey included 97 questions from the 2021 national YRBS questionnaire. Six of these questions were modified to allow students who were attending school only virtually to indicate that a question asking about a behavior on school property did not apply. The questionnaire also included 12 new questions (not included on the YRBS questionnaire) assessing COVID-19–related behaviors and experiences and one new question on perceived racism (Box). The questionnaire was designed with no skip patterns; however, students could skip questions they did not want to answer. (Questionnaires in English and Spanish are available at https://www.cdc.gov/healthyyouth/data/abes.htm.)

In participating schools, after parental permission was granted and a student agreed to participate, teachers of selected classes provided students with instructions for accessing the survey and a randomly generated login code that allowed for completely anonymous participation. Among eligible students, 2.6% had a parent who refused to allow participation, and an additional 4.3% did not return a permission form when active consent was required. To help maintain privacy, only one question appeared on the screen at a time. Images were added for questions related to use of tobacco products, substances, and contraceptives to improve respondents’ recognition and understanding of the questions.

Sampling

ABES used the same sampling methods as the national YRBS, except that a larger sample was drawn in anticipation of lower response rates (6). The ABES sampling frame included public and private schools with grades 9–12 in all 50 U.S. states and the District of Columbia. Combined data obtained from MDR (formerly Market Data Retrieval) and the National Center for Education Statistics (NCES) were used to create the sampling frame. Public schools were identified from the Common Core of Data (https://nces.ed.gov/ccd), and private schools were identified from the Private School Survey (https://nces.ed.gov/surveys/pss), both of which are NCES databases. The sampling frame excluded alternative, special education, U.S. Department of Defense, Bureau of Indian Education, and vocational schools that serve students who are also enrolled in another public school. Home-schooled students and students who dropped out of high school were not eligible for ABES if they were not enrolled in a school.

ABES used a stratified, three-stage cluster sampling approach to obtain a nationally representative sample of students. The first sampling stage consisted of primary sampling units (PSUs) encompassing a county, portion of a county, or a group of counties. A total of 81 PSUs were selected across 16 primary strata by urban and nonurban location and percentage of non-Hispanic Black and Hispanic or Latino (Hispanic) students in the PSU. The second sampling stage consisted of 335 secondary sampling units (SSUs), defined as physical schools with grades 9–12 or linked schools combined to provide the four grades. At the first and second stages, the sample was selected with probability proportional to size. Schools with ≥28 students per grade were defined as large, and those with <28 students were defined as small. The third sampling stage selected one or two classes within each grade of an SSU. All students in a selected class were eligible for ABES, unless they were unable to complete the questionnaire independently.

Data Collection Procedures

The ABES study protocol was reviewed and approved by institutional review boards at CDC and ICF International, CDC’s survey contractor.* The survey was administered during January–June 2021; student participation was voluntary and anonymous. ABES was designed to be a self-administered questionnaire completed online during or outside of instructional time on any Internet-connected device. Most participating students (91%) completed ABES using a laptop or desktop computer; 9% of students used a phone or tablet computer.

Data Processing and Response Rates

ABES data were cleaned and edited for inconsistencies. Values that were not plausible or logical were excluded from analysis (e.g., a student who answered that he had never smoked cigarettes but also answered that he had smoked cigarettes during the past 30 days). Before data editing, students in 128 schools submitted 7,998 ABES questionnaires (7,953 in English and 45 in Spanish). Among these 7,998 records, 293 were excluded because <20 questions had been answered, resulting in 7,705 questionnaires with valid data. The school response rate was 38%, the student response rate was 48%, and the overall response rate ([Student response rate] × [School response rate]) was 18%.

Race and ethnicity were ascertained using methods from the 2019 national YRBS (6); however, in this MMWR Supplement, the “other” race and ethnicity category was disaggregated to report the following racial categories (all non-Hispanic): American Indian or Alaska Native, Asian, Native Hawaiian or other Pacific Islander, and multiracial. The response options for sexual identity differed from those in the 2019 national YRBS (Table 1).

Weighting

ABES data were weighted based on student sex and grade to account for school and student nonresponse and the oversampling of non-Hispanic Black students and Hispanic students. Weights were applied to all records and scaled so that weighted counts equaled the total sample size, and weighted student proportions in each grade matched national proportions. As a result, ABES weighted estimates are nationally representative of all students in grades 9–12 attending public and private schools in the United States.

Analytic Methods

To account for the complex sampling design and weighting, ABES data were analyzed using SAS (version 9.4; SAS Institute) and SUDAAN (version 11.0.1 or 11.0.3; RTI International). The methods section of each report in this MMWR Supplement includes details on the analytical methods used in each analysis. For this overview report, a descriptive analysis of ABES student- and school-level data was conducted.

Data Availability and Dissemination

ABES data and documentation describing the data are available to the public (https://www.cdc.gov/healthyyouth/data/abes.htm). Student survey and school characteristics data have been merged into one data set and are available in Access and ASCII formats. SAS and SPSS programs are provided for converting the ASCII data into SAS and SPSS data sets. No approval is needed to use the ABES data set. Data requests and other ABES-related questions can be sent to CDC by using the YRBS data request form (https://www.cdc.gov/healthyyouth/data/yrbs/contact.htm).

Results

Among the high school students surveyed, 26.7% were in 9th grade, 25.5% in 10th grade, 24.3% in 11th grade, and 23.6% in 12th grade (Table 2). In addition, 50.4% of students were female and 49.8% were non-Hispanic White, 25.4% were Hispanic, and 12.9% were non-Hispanic Black. The remaining students reported other races (all non-Hispanic): American Indian or Alaska Native (0.7%), Asian (4.9%), multiracial (5.8%), and Native Hawaiian or other Pacific Islander (0.5%). A majority of students self-identified as heterosexual (77.5%); 13.2% self-identified as gay, lesbian, or bisexual; and 9.3% self-identified as other or questioning.

Of the 128 participating schools, 75.0% reported using a hybrid instructional mode, 21.9% used online instruction only, and 3.1% used in-person instruction only (Table 3). Approximately half of the schools (50.8%) had reduced class sizes. Most schools (94.5%) provided laptops or Chromebooks to students, and 60.3% provided Wi-Fi hotspots.

Discussion

CDC adapted successful school-based surveillance methods from the national YRBS to develop and launch ABES rapidly during the COVID-19 pandemic. In addition to using existing YRBS questions on the ABES survey, agencywide collaboration was used to develop new ABES questions, provide technical assistance on survey development and data collection, and conduct the analyses presented in this MMWR Supplement. ABES was the first national online survey of its kind available in both English and Spanish launched by CDC; online administration allowed students the flexibility to complete the survey during instructional time or on their own time. However, because schools faced numerous COVID-19–related challenges during the 2020–21 school year, certain districts and schools were reluctant to participate in ABES. For example, certain schools were concerned about loss of instructional time and the additional time required for school staff members. In some instances, contacting appropriate district- and school-level decision-makers to discuss participation was difficult because schools were minimally staffed or closed (ICF International, personal communication, 2021). Despite these challenges, ABES was still successfully administered, resulting in data that can be used to assess the impact of COVID-19 on various experiences and behaviors among high school students.

This is the first national school-based survey among high school students to assess a range of behaviors and experiences during the COVD-19 pandemic. A limited number of studies exist using other surveys among youths, but they have either had a narrower focus (e.g., focusing on one behavior such as substance use) (8) or are not nationally representative (9). With the varying school instructional models across the United States, obtaining national benchmark estimates for adolescent behaviors and experiences was imperative. Revised suppression criteria enabled the reporting of ABES findings for American Indian or Alaska Native, Asian, multiracial, and Native Hawaiian or other Pacific Islander students in this MMWR Supplement when data were available. This change allowed for a better understanding of behaviors and experiences among more specific groups of youths. Although the sampling methodology is the same for ABES and YRBS, findings should not be compared (i.e., prepandemic versus during pandemic) because administration modality and settings differed between surveys. YRBS uses a paper and pencil scannable survey that is administered in person in the schools; ABES used an online survey that students could complete during instructional time or on their own time, with access provided by teachers through schools.

Limitations

The findings in this report are subject to at least five limitations. First, ABES data are representative of students enrolled in schools and are not representative of all U.S. adolescents, including those not enrolled in schools; an estimated 5% of adolescents aged 14–17 years were not enrolled in school in 2018 (10). Second, although ABES protocols were designed to protect students’ privacy, responses might have been visible to others during the survey, potentially leading to social desirability or response bias. Third, information on the instructional model being used by schools was collected at the time of school recruitment and might have changed during the study period. Therefore, interpreting findings among students who attended in-person, virtual, and hybrid school conditions is difficult, as is distinguishing in-person versus virtual attendance for the hybrid category. Fourth, because ABES was a one-time cross-sectional survey, causality or directionality of the findings cannot be determined, and changes in behaviors or experiences could not be assessed. Furthermore, the survey was not designed to allow comparisons by survey completion date; therefore, the results cannot be used to assess the impact of changes during the COVID-19 pandemic on responses. Finally, ABES had an 18% overall response rate; low response rates increase the potential for nonresponse bias. However, high response rates alone do not rule out nonresponse bias. Nonresponse bias analyses revealed that weighting adjustments minimized the potential for nonresponse bias because sample weights were adjusted to account for nonresponding schools (AM Roberts, R Iachan, L Harding, X Deng, ICF International, unpublished data, 2021).

Conclusion

Collecting nationally representative data on behaviors and experiences among youths during the COVID-19 pandemic provides information on which behaviors are concerning and most prevalent during this time. ABES findings highlighted in this MMWR Supplement can help determine which risk behaviors are more prevalent among youths to assist parents, teachers, school administrators, community leaders, clinicians, and public health officials with addressing these issues. Additional surveillance is needed to monitor behaviors and experiences that are associated with the pandemic among youths across the United States.