January 2006

Jay P. Greene, Ph.D.
Endowed Chair and Head, Department of Education Reform
University of Arkansas

Joshua H. Barnett
Distinguished Doctoral Fellow
University of Arkansas

Gary W. Ritter, Ph.D.
Endowed Chair, Department of Education Reform
University of Arkansas

Marcus A. Winters
Doctoral Academy Fellow
University of Arkansas

Executive Summary

Policymakers, practitioners, journalists, and the general public regularly use the results of state accountability tests to assess the quality of schools.  Schools with high scores must be “good schools,” people assume, while schools with low scores must be “bad schools.”  Unfortunately this use of test results is actually a misuse.  Test score results are only partially a reflection of the quality of school instruction; they are also partially a reflection of the advantages and disadvantages that students bring to school. A school with high test scores might actually be of sub-par quality propped up by very advantaged students.  Conversely, a school with low test scores might actually be of high quality masked by the severity of its students’ disadvantages. Unless one isolates the influence of student characteristics and other resources made available to schools, it is impossible to determine school quality simply from test results.

The School Performance Index (SPI) is an attempt to disentangle school quality from the advantages and disadvantages given to a school.  Using a regression model controlling for a host of student characteristics, community characteristics, and resources, we are able to predict how well each school should be performing on standardized tests given those inputs.  The extent to which schools perform better or worse than we would expect given the context in which they operate is our best estimate of the quality of the school itself.  Based on our analyses, we note six key findings:

• After controlling for student characteristics and resources, Arkansas students perform slightly higher than the national average;
• School performance on the Iowa Test of Basic Skills is not substantially affected by the spending of the district;
• School performance on the Iowa Test of Basic Skills is not substantially affected by the size of the school or district;
• School performance on the Iowa Test of Basic Skills is partially affected by the household income, educational attainment of residents, percentage of Hispanic students, and percentage of married families in the district;
• School performance on the Iowa Test of Basic Skills is substantially affected by the percentage of black students and the percentage of students who qualify for free or reduced price school lunch in the school; and
• Some schools and school districts in Arkansas perform substantially better than we would expect given their student characteristics and resources, while others perform substantially worse than expected.

The School Performance Index is certainly not perfect. For example, it may not fully control for all advantages and disadvantages given to each school, thereby confusing school quality with unmeasured student characteristics. Nevertheless, we believe that it is a significant advance in assessing school quality. As the policymakers in the Arkansas Department of Education consider ways of reporting the level of school performance as required in Act 35, we hope they will consider the approach outlined in this report.