Lessons from Networks for School Improvement: School Year 2020-2021

This paper describes what the foundation is learning across NSI grantees in order to supplement a more formal summative evaluation that is underway, with the first report expected in March 2024. While the summative evaluation will analyze school- and student-level data from participating sites, as well from the organizations that support those sites, this analysis draws on grantee check-in notes, artifacts produced by the organizations, interviews by program officers, and foundation documents to identify early themes and trends. The foundation is purposefully drawing on data from multiple sources and rhythms of data collection and analysis to provide as rich a picture of the NSI work as possible. We want to balance rigorous but slower methodological approaches—that represent the gold standard in program evaluation—with quicker, more action-oriented data reporting and analysis to inform the work of our grantees and the foundation in real time.

To date, we’ve observed four trends, based on the work during the 2019-20 and 2020-21 school years:

• As they’ve gained experience, learned from each other, and responded to the effects of the pandemic, the majority of NSI have begun focusing on common problems and solutions across schools in their network, reducing variation in the evidence-based practices and measures that school teams use. This increases the likelihood that schools can share and learn from each other.
• Increasing the frequency of structured inquiry cycles so that educators can learn more quickly which change practices should be adopted, adapted, or abandoned remains a challenge across networks. This was particularly true given the disruptions to schooling caused by the COVID-19 pandemic. But there are early signals that NSI with shorter, more frequent inquiry cycles are seeing increases in locally defined measures of change.
• All sites are still in the process of developing measurement systems, particularly near-term measures of change, to tell whether the practices they implement are leading to improvements.
• In some sites, changes are extending beyond individual schools to influence the larger system, such as modifications in student grading policies and alignment with other district initiatives.