Evaluations of Massachusetts School Wraparound Zone and School Redesign Grant Programs Show Student Success in State English and Math Test Scores

Washington, D.C. – Evaluations by the American Institutes for Research (AIR) of two federally funded initiatives aimed at revamping chronically low-performing schools in Massachusetts found that students in both programs improved their scores in state tests of English language arts and mathematics.

While both programs resulted in widespread improvement, AIR’s evaluations found particularly strong increases in achievement among students with limited English proficiency.

The evaluations, conducted for the Massachusetts Department of Elementary and Secondary Education, examined Massachusetts Wraparound Zones (WAZ)—an initiative that factors such non-academic issues as mental health and safety, for example, into school reforms—and the impact of state School Redesign Grants (SRG) on low-performing schools in the Commissioner’s Districts (the state’s 10 largest urban districts).

Both programs relied on major federal grants geared toward states with track records of innovation in jump-starting struggling schools. Massachusetts schools relied heavily on a $250-million Race to the Top grant to fund the wraparound zones. The SRG grants were funded by a School Improvement Grant from the U.S. Department of Education.

The results of the wraparound zones study underscore growing attention to how non-academic factors help transform low-performing schools. “Integrated support services,” as they are sometimes called, address the holistic needs of school children and have featured prominently in recent academic reforms in Cleveland and Nashville.

“When students struggle academically, often it is because they have outside stressors that act as barriers to learning,” said Allison Gandhi, an AIR principal researcher and lead author of the report. “If schools do not address such non-academic factors, they are unlikely to see large-scale improvement in academics. The promising results from the Massachusetts Wraparound Zones initiative show that academic and non-academic work can be interwoven to produce improved academic achievement. This mix should be integral to any school turnaround strategy.”

The WAZ initiative, in place from 2011-2014, awarded grants to 32 schools in seven school districts to address non-academic challenges. The goals were to enhance school climate, implement systems to identify students with non-academic needs, integrate social service resources and create a system for district level feedback and improvement. While school programs differed, family volunteer efforts, partnerships with local mental health clinics and other community providers, and an emphasis on school discipline were typical elements.

According to the study, students in WAZ schools experienced greater gains in English language arts and math  achievement than students in similar schools that did not receive the grant. These effects were significant after two and three years of implementation for English language arts and after two years of implementation for math. Gains were particularly strong in the grades 3 and 4 for limited English proficient students—equivalent to nearly a full year of achievement gains one would expect to see for a typical fourth grade student, according to AIR’s evaluation.

The WAZ schools also fared well compared to other schools in Massachusetts, according to state data. Students attending WAZ schools gained 5.8 percentage points on state English language arts tests and 7.9 percentage points on state mathematics tests. During the same period, scores in the state as a whole dropped 0.4 percentage points in English language arts, and gained 1.4 percent in math.

The program also appears to have played a role in school turnaround. Among WAZ schools, those that started 2010 in Level 4 status—the designation the state gives to its most struggling schools—66 percent had exited that status by 2014, compared to 40 percent among non-WAZ schools.

School turnaround was the primary focus of the SRG grants. Evaluating 31 schools in eight districts, AIR  focused on student achievement and attendance beginning in the 2010-2011 school year. Findings include:

  • Students in schools with grant funding performed better than students in similarly situated comparison schools in both math and English language arts. The effects were significant after the first, second and third years of implementation.
  • The grants were associated with a decrease in the achievement gap between limited English proficient and non-LEP students in math and English language arts across all three years.
  • Grant receipt appears to have had no net effect on student attendance.

“The disbursement of SIG grants had consistently positive effects on student academic achievement, particularly among limited English proficient students,” said Christina LiCalsi, a senior researcher at AIR who led  the study. “The results are robust across districts and school levels, even though the grants didn’t affect student attendance.”

Visit the AIR website to view the reports Focusing on the Whole Student: Final Report on the Massachusetts Wraparound Zones and Evaluation of Massachusetts Office of District and School Turnaround Assistance to Commissioner’s Districts and Schools: Impact of School Redesign Grants.

About AIR
Established in 1946, with headquarters in Washington, D.C., the American Institutes for Research (AIR) is a nonpartisan, not-for-profit organization that conducts behavioral and social science research and delivers technical assistance both domestically and internationally in the areas of health, education and workforce productivity. For more information, visit www.air.org