Monday, June 13, 2005

What are we, Arkansas?

Dick Morris tells the story of when Bill Clinton was governor of Arkansas and the teachers had just taken a state-wide proficiency exam. The results were not good. Between a third and half of all teachers failed the test.
The solution? Morris was told to poll Arkansas voters and find out what percentage of the teachers they expected to fail the test. As Gov. Clinton had said to Morris, "I can decide what score is passing and what is failing." Morris' polling revealed that Arkansas voters expected 10 percent of the teachers to fail, rather than the 30–50 percent who actually failed. But when the Clinton Administration released the "results" of the tests to the public, it reported that only 10 percent had failed. In the end, only a handful of Arkansas' incompetent teachers lost their jobs.
In Wisconsin, the No Child Left Behind act requires schools and school districts to meet certain standards. When those standards were not met, did Wisconsin immediately react in a crisis mode to rescue our schoolchildren from failing schools? No, the decision was made to play with the "confidence intervals" of the data, challenging the results and driving the number of failing schools down from 108 schools to 51 schools.

Interestingly, the reporter for the Journal Sentinel spoke with one identified statistician about the practice of using confidence intervals to adjust the numbers. Instead of discussing themathematicss, the UW-Milwaukee statistics expert focused instead on the politics and policy-making of the decision.
Jay Beder, a statistician and associate math professor at the University of Wisconsin-Milwaukee, says Wisconsin's use of confidence intervals appears to be reasonable, given the consequences of being flagged as a school failing to make progress.

"I'd rather see them be cautious," Beder said. "The consequence to a school is tremendous."
The consequences for the students, or even which students, go unmentioned.