(Cross-posted from Think it Through)
Data driven public policy – it is a cherished goal of the post-ideological, bipartisan, totally rationalist America we are supposed to be moving towards. We hear praise in the media for public officials who declare themselves non-ideological and strictly data driven. Beware of such public officials.
A newly released survey of police officers and commanders in New York City reveals that Rudolph Giuliani‘s reputation as a tough law-and-order mayor in the 1990s was built on a foundation of false data. As mayor, Giuliani sent word down the line that crime in the city must be reduced. He wanted statistics to show that crime rates in New York were dropping compared to other cities. According to extensive interviews among police precinct commanders and supervisors in the city, reported in the New York Times on February 6, the police felt considerable pressure from their superiors to alter the crime statistics. When the boss asked for something, they delivered.
Giuliani installed a computer scoring system that precincts used to regularly spew out favorable statistics. The system, called CompStat, has since been franchised to police departments throughout the U.S. and across the globe, according to the Times story. The story quotes one of the researchers who conducted the study as saying, "Those people in the CompStat era felt enormous pressure to downgrade index crime, which determines the crime rate, and at the same time they felt less pressure to maintain the integrity of the crime statistics." What is the sense of having a base of comparison if it’s no good? Did Giuliani intend to say, "My latest computerized update shows that New York is lagging behind other cities—we are experiencing greater numbers of homicides and drug crimes?"
Giuliani’s CompStat does to police officers on the beat what education secretary Arne Duncan wants to do to teachers in the classroom. Continuing the policies of George W. Bush, Duncan insists that teachers be evaluated on how well their students do on statewide standardized tests. The education establishment will compare the tests from one school to another and one state to another to determine where the "bad" schools and "bad" teachers are located. There are at least three things wrong with this.
First, as parents across the country have been telling us for years in focus groups, this causes teachers to teach to the tests because their livelihoods depend on it. This policy victimizes students, as teachers are forced to prepare students to memorize information they will need to know to pass the tests, rather than using an array of teaching techniques to challenge students to expand their minds and gain a love of learning.
Second, tests alone do not give a full measure of a teachers’ talent. Evaluations by mentors, principals, peers, and students all should be considered when judging teachers’ merit.
Third, using test scores as the primary evaluator encourages and even promotes cheaters. During the 1990s, at about the same time that Giuliani was incentivizing creative crime reporting in the precinct houses, the superintendent of Houston public schools was doing the same for classroom teachers – and winning national attention for his data-driven educational achievements. Superintendent Roderick Paige instituted teacher pay incentives tied to student test scores. Under Paige, test scores in Houston’s public schools improved dramatically and its high schools reported the lowest dropout rates for any large city in the nation.
Beaming with pride, President Bush named Paige his first Secretary of Education in 2001. Then the news media exposed the facts that under Paige’s leadership Houston teachers had changed test results to get higher scores, and school administrators counted high school dropouts as students who decided to "transfer" to other schools.
There is nothing wrong with data, but when it becomes the central arbiter of the worth of a person or policy, it is in danger of deceiving. Society’s elevation of data coincides with the current passion for the debasement of policies that are based on values (ideology) or strongly held beliefs (partisanship). This is part of a culture that elevates "bipartisan" to an end in itself. Does it not make you at least a little suspicious that education now seems to be the one issue on which Democrats and Republicans agree – and that both parties win support from the business community, including those companies that produce the tests?
Once again, I think of George Carlin’s comment that "bipartisan usually means that a larger than usual deception is being carried out."
John Russonello is a partner with Belden Russonello & Stewart: Public Opinion Research and Strategic Communications in Washington, DC. He writes the blog "Think it Through."