A little "3-Card Monte' Anyone?"

---

October 9, 2007

Op-Ed Columnist

High-Stakes Flimflam

By BOB HERBERT

It’s time to rein in the test zealots who have gotten such a stranglehold on the public schools in the U.S.

Politicians and others have promoted high-stakes testing as a panacea that would bring accountability to teaching and substantially boost the classroom performance of students.

“Measuring,” said President Bush, in a discussion of his No Child Left Behind law, “is the gateway to success.”

Not only has high-stakes testing largely failed to magically swing open the gates to successful learning, it is questionable in many cases whether the tests themselves are anything more than a shell game.

Daniel Koretz, a professor at Harvard’s Graduate School of Education, told me in a recent interview that it’s important to ask “whether you can trust improvements in test scores when you are holding people accountable for the tests.”

The short answer, he said, is no.

If teachers, administrators, politicians and others have a stake in raising the test scores of students — as opposed to improving student learning, which is not the same thing — there are all kinds of incentives to raise those scores by any means necessary.

“We’ve now had four or five different waves of educational reform,” said Dr. Koretz, “that were based on the idea that if we can just get a good test in place and beat people up to raise scores, kids will learn more. That’s really what No Child Left Behind is.”

The problem is that you can raise scores the hard way by teaching more effectively and getting the students to work harder, or you can take shortcuts and start figuring out ways, as Dr. Koretz put it, to “game” the system.

Guess what’s been happening?

“We’ve had high-stakes testing, really, since the 1970s in some states,” said Dr. Koretz. “We’ve had maybe six good studies that ask: ‘If the scores go up, can we believe them? Or are people taking shortcuts?’ And all of those studies found really substantial inflation of test scores.

“In some cases where there were huge increases in test scores, the kids didn’t actually learn more at all. If you gave them another test, you saw no improvement.”

There is not enough data available to determine how widespread this problem is. “We know it doesn’t always happen,” said Dr. Koretz. “But we know it often does.”

He said his big concern is where this might be happening. “There are a lot of us in the field,” he said, “who think that if we ever really looked under the covers, what we’d find is that the shortcuts are particularly prevalent in lower-achieving schools, just because the pressure is greater, the community supports are less and the kids have more difficulties. But we don’t know.”

One aspect of the No Child Left Behind law that doesn’t get enough attention is that while it requires states to make progress toward student proficiency in reading and math, it leaves it up to the states themselves to define “proficiency” and to create the tests that determine what constitutes progress.

That’s absurd. With no guiding standard, the states’ tests are measurements without meaning.

A study released last week by the Thomas B. Fordham Institute and the Northwest Evaluation Association found that “improvements in passing rates on state tests can largely be explained by declines in the difficulty of those tests.”

The people in charge of most school districts would rather jump from the roof of a tall building than allow an unfettered study of their test practices. But that kind of analysis is exactly what’s needed if we’re to get any real sense of how well students are doing.

Five years ago, President Bush and many others who had little understanding of the best ways to educate children were crowing about the prospects of No Child Left Behind. They were warned then about the dangers of relying too much on test scores.

But those warnings didn’t matter in an era in which reality was left behind.

“No longer is it acceptable to hide poor performance,” said Mr. Bush, as if those who were genuinely concerned about the flaws in his approach were in favor of poor performance.

During my interview with Dr. Koretz, he noted that by not rigorously analyzing the phenomenon of high-stakes testing, “we’re creating an illusion of success that is really nice for everybody in the system except the kids.”

That was a few days before the release of the Fordham Institute Study, which used language strikingly similar to Dr. Koretz’s. The study asserted that the tests used by states to measure student progress under No Child Left Behind were creating “a false impression of success.”

The study was titled, “The Proficiency Illusion.”