by Walter R. Tschinkel

Professor of Biological Science
Florida State University
Tallahassee, FL 32306-4370

email: tschinkel@bio.fsu.edu
phone: 644-4489

The Bush Administration and the legislature, after months of lobbying, arguing, wrangling, dealing and agonizing, has given us the A-Plus Plan with its School Accountability Report (http://www.firn.edu/doe/schoolgrades/account.htm). Upon analysis, this Report turns out to be merely an elaborate and expensive way to grade schools on the poverty or affluence of their students.

The Bush/Brogan School Accountability Report assigns each school a grade primarily on its raw, overall standardized test scores. Because standardized test performance of schools is very reliably predicted by poverty, the poverty-level of a school is the by far the strongest predictor of its grade. In fact, if you tell me the percent of a school’s students that are on supported lunch (an indicator of low family income), I will tell you its Bush/Brogan grade with 80% accuracy. That is, I will be wrong only one out of five times.

If you think I’m bluffing, let me show you that it’s true. Let us simply classify schools by their affluence/poverty makeup: very affluent, moderately affluent, moderately poor, very poor. Next, let us grade them on their affluence or poverty. The most affluent schools get an A, the next group gets a B, and so on, with the poorest schools getting a D. The graph and the table below show how closely the grades based on poverty correspond to those assigned by the Bush/Brogan School Accountability Report. Simply by considering school affluence/poverty, we are able to assign the same grade as the Bush/Brogan "performance-based" system in about 80% of the cases, that is 26 of the 33 schools. And we did this without looking at a single test score.

Is this a fair, or even a sensible, way to grade our schools? Only if you think poverty should be punished. Does the Bush/Brogan grade tell us anything new about a schools’ educational performance? Of course it does not. It tells us what proportion of the student body comes from poor families. In this system, both "good" and "bad" schools attain their status not through their efforts and expertise, but mostly through the good or bad fortune of having few or many children from poor families. It is not my purpose here to dwell on the poverty-performance link. Whatever its reasons, it is a reality faced in much greater measure by schools at the low end of the socioeconomic scale than the high end. No school grading system that does not take this socioeconomic factor into account is useful in telling us how well our schools are really doing, given the socioeconomic status of the students they are asked to educate. Would it not be much fairer to adjust school performance for poverty before grading schools?

I think it would, and hereby offer the Prof. Walter’s Level-Playing-Field School-Grading System, which provides a poverty-adjusted estimate of school performance as an alternative to the Bush/Brogan School Accountability Report. We begin with a regression analysis of the school performance data (3 standardized tests) against the poverty level of the student body (% on supported lunch). This statistical method shows about 80% of the test scores are predicted by the poverty-level of the student body. I detailed this relationship in a previous My View column on March 14 (this article can be found on my website at http://www.fsu.edu/~biology/faculty/wrt.html). For every percent that poverty increases, the school’s scores drop by an average of 1.6 points. The most affluent schools, those with fewer than 15% poor students, have scores higher than 230, while the poorest, with more than 75% poor students, have scores below 120, less than about half those of the most affluent schools. This same relationship was equally strong in all seven major school systems in Florida. We can work toward eliminating this effect of poverty, but success will come only when we figure out how to help all students learn equally well.

Next, we take the difference between each school’s actual test scores and the test score predicted by the regression for a school of that socioeconomic condition. These differences are called the residuals, and tell us how much better or worse than average a school tested, given its particular level of poverty. By doing this, we have removed the effect of poverty on test scores. The result is that the maximum difference in test scores has shrunk from 175 points to only about 70 (the lost 105 points are the effect of poverty). Residuals less than zero indicate that (with poverty effects removed) a school did less well than average, and residuals above zero indicate that it did better than average.

Let us now assign letter grades to these residual scores. Here is my scale: above 25 gets an A; between 5 and 25 gets a B; between -20 and 5 gets a C (note that C’s straddle the average, which is zero); between -35 and -20 gets a D; anything below -35 gets an F. Before I assign grades to schools, I want to emphasize that I am doing so only to make a point. I am not advocating such simple data for grading schools.

The table below lists our elementary and middle schools in the order of the grades assigned by the Bush/Brogan Plan.

When graded according to the Level-Field system, there are grade-changes, both up and down, within each Bush/Brogan grade. Now we can recognize that schools like Riley, Hartsfield, and Woodville (to name a few) are doing relatively well compared to other schools of similar socioeconomic makeup. My system recognizes this and rewards them with A’s and B’s instead of the C’s and D’s assigned by the Bush/Brogan system. It might pay us to find out what these schools are doing to rise above the average. On the other hand, my system also shows that schools like Swift Creek, Buck Lake and Griffin do not deserve their Bush/Brogan A’s because they are only average as compared to other schools of similar socioeconomic makeup. Hence, the Level-Field system assigns them a C, because the Level-Field system does not reward schools for being lucky enough to be teaching mostly affluent students.

The case of Griffin highlights another flaw of the Bush/Brogan School Accountability Plan. Griffin received an A, not because of its terrific performance on standardized tests, but because it met several additional criteria. These were: 1) percent of long absences or suspensions among students were below state averages; 2) greater than 95% of the student body was tested; 3) no subgroup test performance fell below minimum criterion; 4) improvement in reading scores without a decline in math and writing over 1998. Only the last two can actually be considered academic performance. The first two are bureaucratic tricks. It is a bit like requiring that an athlete run the 100 yd. dash in 10 seconds, but you credit him with half a second if he wears the right color shorts, and another half second if she pulls her socks up before starting. Neither has anything to do with performance, and both serve to obscure real performance.

You may ask, "Well, how are we supposed to know how our schools are really doing?" I suggest that we insist on a much more sophisticated analysis of school data by the state Department of Education, instead of letting them just plunk it onto their web site or onto a newspaper page so the public can worry about what it means. At the very least, school performance needs to be adjusted for the nature of the student body. Better yet, let us not pretend that a single number can adequately assess the performance of our schools. Performance must be measured, not by any single number, but by the relationship between what goes into a school and what comes out. The large and expensive bureaucracy at DOE can reasonably be expected to explain to the public how the data are related to each other, what they mean and how our schools are really doing. This will allow us to discover what works and what doesn’t work, and thus to spend money more effectively. Only a full statistical analysis of the data followed by publication of the results can lead to an effective plan that enjoys widespread public understanding and support. Then, perhaps well-intentioned but pointless maneuvers such as the Bush/Brogan School Accountability Plan will have a more difficult time in coming to life.