ANALYSIS OF THE IMPACTS OF CITY YEAR’S WHOLE SCHOOL, WHOLE CHILD
MODEL ON PARTNER SCHOOLS’ PERFORMANCE
City Year is a learning organization committed to the rigorous evaluation of its WHOLE SCHOOL, WHOLE CHILD model, which trains and deploys teams of AmeriCorps members to low-performing, urban schools to empower more students to reach their full potential. A third-party study by Policy Studies Associates (PSA) examined the impact of WHOLE SCHOOL, WHOLE CHILD on City Year’s partner schools’ performance in comparison to similar schools without City Year. The study used publicly available data that local education agencies (LEAs), State and U.S. Departments of Education use to assess school performance.
PSA found that schools with teams of City Year AmeriCorps members were two to three times more likely to increase English Language Arts (ELA) and math proficiency rates compared with similar schools that did not partner with City Year, as measured by state assessments. Schools that partnered with City Year also gained the equivalent of approximately one month of additional learning in math and ELA.
Methodology & Research Questions
PSA used a methodology common in education and social science research - a quasi-experimental comparison group study - of City Year’s impact on its partner elementary, middle and high schools in 22 cities over three school years, from 2012 to 2014. The study was designed to estimate the impact of City Year’s holistic WHOLE SCHOOL, WHOLE CHILD model during the first three years of its national implementation. City Year first piloted the WHOLE SCHOOL, WHOLE CHILD model in 2007 and, by 2011-2012, had achieved greater consistency in its implementation and in the collection of student data. The model is designed to build community-based learning environments that foster the academic and social emotional mind-sets and competencies essential to sustained student performance, and includes school climate work, whole class support,targeted one-on-one and small group tutoring in literacy and math, attendance interventions, social and emotional learning supports, and after school programming. More than 93% of the schools partnered with City Year are high-poverty (i.e. serve populations in which 75% or more students are classified as economically disadvantaged).
Leveraging publicly available data on school performance downloaded from state and district websites and City Year program implementation and participation data collected by City Year staff, PSA examined student performance on state standardized tests in ELA and math between schools that partnered with City Year and schools that did not partner with City Year, but that had similar demographic and performance characteristics. To identify comparison schools, PSA used an established method to select the comparison group called propensity score matching that identified schools that were similar in terms of baseline state standardized test performance data, enrollment, and demographic information, including the percentage of English-language learners and economically disadvantaged students. The analyses focused on approximately 600 schools in 22 cities: 150 schools that partnered with City Year and 460 that did not partner with City Year.
PSA’s analyses of City Year’s impact on school performance were guided by several research questions, including:
- How do the whole school or grade wide outcomes across performance in English Language Arts and math of City Year schools compare to outcomes of other similar schools that do not partner with City Year?
- What is the predicted impact on academic performance by having City Year in a school?
The data on student academic performance available for this study were limited to school-level or school grade-level summary data and did not include the analysis of student-level changes in performance.
The findings of this study suggest that City Year’s WHOLE SCHOOL, WHOLE CHILD model generates positive results for its partner elementary, middle, and high schools. As measured by state assessments:
- Schools that partnered with City Year were approximately two times more likely to improve ELA proficiency rates during the 2011-2012, 2012-2013 and 2013-2014 school years than schools that did not partner with City Year.
- Schools that partnered with City Year were approximately two times more likely to improve math proficiency rates during the 2011-2012 school year, and three times more likely to improve during the 2012-2013 school year, than schools that did not partner with City Year. Schools that partnered with City Year during the 2013-2014 school year were more likely to improve, but the results were not statistically significant this year.
- Schools that partnered with City Year gained the equivalent of approximately one month of additional learning in ELA and math compared to schools that did not partner with City Year.
- On average, during the 2011-2012 and 2012-2013 school years – and across all geographic locations – schools that partnered with City Year were more likely to improve ELA and mathproficiency rates than schools that did not partner with City Year.
Implications & Takeaways
City Year’s effect. While City Year has commissioned third-party studies in the past, and is currently undergoing a randomized control trial, this is the first national, exploratory effort by a third-party evaluator to examine the whole school outcomes of City Year’s model in partnership with schools. The results suggest that the comprehensive services offered to partner schools by WHOLE SCHOOL, WHOLE CHILD have a measurable impact on student performance on state assessments. In every city, schools with City Year were more likely to improve on test scores in ELA and math, suggesting a “City Year effect.”
Future research to support ongoing model refinement. The WHOLE SCHOOL, WHOLE CHILD model is designed to help schools organize their student interventions and whole school initiatives to better meet the high scale and intensity of student need. In support of City Year’s ongoing efforts to continuously refine its model, future research should examine in greater detail the extent to which the City Year partnership enables schools to improve practices, scale and deliver effective reforms, and provide more instructional time to meet the unique needs of students.
In The News
“…because we saw clear and consistently positive differences in CY schools versus their matched comparisons, by subject, by school level, and by site, we believe that these results support the conclusion that City Year is on to something with its WHOLE SCHOOL, WHOLE CHILD school improvement model and that further study is warranted.” –PSA Researcher
About City Year
City Year is an education-focused organization founded in 1988 dedicated to helping students and schools succeed. City Year partners with public schools in 26 urban, high-poverty communities across the U.S. and through international affiliates in the U.K. and Johannesburg, South Africa. Diverse teams of City Year AmeriCorps members provide high-impact student, classroom and school-wide support, to help students stay in school and on track to graduate from high school, ready for college and career success. A proud member of the AmeriCorps national service network, City Year is made possible by support from the Corporation for National and Community Service, school district partnerships, and private philanthropy from corporations, foundations and individuals. Learn more at www.cityyear.org.
About Policy Study Associates
Policy Studies Associates (PSA), founded in 1982, conducts research in education and youth development for clients including federal, state, and local government agencies, foundations, and other organizations that aim to use high-quality data to make policy decisions. The studies PSA conducts for its clients span evaluation, policy analysis, and other forms of systematic inquiry. In these studies, PSA’s trained researchers integrate information from many sources to examine the operations and effects of programs to improve schooling and other services for children and youth. They draw on their expertise in research and evaluation design, data collection (using surveys, interviews, and observations), analysis, and reporting. Learn more at www.policystudies.com.