Differences in 2019-2022 COVID-related NAEP Urban District Score Declines in Grade 4 Based on County-level Health and Socioeconomic Data

The COVID-19 pandemic resulted in many schools making difficult decisions about remote instruction, masking, and social distancing policies, as well as how and when to transition students back to classroom learning. In light of these challenges, the 2022 state National Assessment of Educational Progress (NAEP) assessments revealed declines for 4th- and 8th-grade reading and mathematics assessment scores compared to the pre-pandemic levels of 2019. These declines were reflected by significant decreases in 9 of the 26 NAEP Trial Urban District Assessments (TUDAs). This memo explores the relationship of classes of health-related indicators and changes in grade 4 reading and mathematics scores at a time of heightened health and social risk associated with the COVID-19 pandemic.

For this study, NAEP data were merged with data from the 2022 County Health Rankings & Roadmap, provided by the University of Wisconsin, to obtain relevant indicators of community health. Based on health data for the counties in which the TUDA districts were located, a latent class analysis (LCA) model was developed that divided the schools into three classes, or groups, based on their level of community health.

For example, schools in class 1 were the “healthiest” in that their counties had lower rates of infant mortality, child mortality, and low-weight births; fewer years lost due to premature death; and lower rates of frequent mental distress among adults relative to the other classes of schools. Furthermore, schools in class 1 were in counties with the longest life expectancies, the highest median incomes, and the lowest rates of firearm fatalities. In contrast, class 3 schools were located in counties that were notably worse according to the totality of these indicators, and schools in class 2 fell between those in classes 1 and 3 on these indicators.

The LCA model results suggested that TUDA schools in class 1 (i.e., those located in “healthier” surrounding communities with higher median household incomes) had higher grade 4 reading and mathematics scores in 2019 and 2022 compared to schools in less healthy and lower-SES counties (classes 2 and 3). These schools’ scores also fell less during the height of the pandemic compared to others. On the other hand, TUDA schools with less healthy community indicators and lower median household incomes not only performed relatively worse on the grade 4 assessments in both years, but also experienced a greater decline in scores from 2019 to 2022. A student-level analysis of the data provided similar results.

This study highlights the potential value of integrating NAEP with other datasets to inform policies aimed at improving student academic performance.