Consultant says flawed FCAT was one-time event

A review of the test that inflated the scores for third-graders finds no systemic problem.

By RON MATUS and JEFFREY S. SOLOCHEK, Times Staff Writers
Published December 11, 2007

An independent review of the botched 2006 FCAT has concluded the Department of Education was right about what went wrong.

The Buros Center for Testing said in a 45-page report released Monday that the technical, test-making errors that led to inflated scores on the third-grade reading test - and to the biggest crisis yet for Florida's FCAT-based accountability system - were an aberration.

Both the Education Department and the Buros Center, a highly regarded testing authority at the University of Nebraska, determined a one-time problem was at fault: the misplacement of "anchor" questions that ensure the Florida Comprehensive Assessment Test's difficulty level is consistent from year to year.

"We are firmly convinced that this explanation is by far the most likely explanation," the report said.

The report recommended against rescoring the 2006 test, saying there is "no appropriate formula" for doing so.

At least one frequent Education Department critic said the report was "reassuring" - to a point.

"An objective source providing this kind of information should be reassuring to those who support the FCAT, and to parents and teachers across the state," said state Sen. Don Gaetz, R-Niceville, who chairs the Senate Education Committee. "That doesn't mean that there won't continue to be questions."

The Education Department disclosed in May that the scores of more than 200,000 third-grade tests were wrong in 2006, erasing a historic, 8 percent spike in the number of third-graders reading at grade level that year.

The department then named a high-profile advisory panel of superintendents and testing experts, which in turn decided to hire the Buros Center.

The Buros review, which is ongoing on other FCAT-related matters, "looks good so far," said John Hilderbrand, Hillsborough's director of assessment and a member of the state panel.

It took "a lot of guts" for the state to let national experts put its testing system under the microscope, said Hilderbrand, who has been critical of some aspects of the state's testing regimen.

The Buros report, though, won't end a wide-ranging debate over how the FCAT is used to grade schools, evaluate some teachers and retain some students. And it may add fuel to the fire.

The Buros experts described the FCAT as "professionally developed," called its Education Department overseers high quality and said the steady increases on the third-grade reading test are "valid reflections of increased student learning."

But they also suggested major policy changes, including using more than FCAT scores to evaluate schools and using more than one year's worth of data when making decisions about sanctions and rewards.

Sherman Dorm, a University of South Florida professor and another member of the state oversight panel, said the Education Department did not shy from implementing some of the technical suggestions Buros made in an earlier report.

"It will be very interesting to see if the state of Florida takes the same approach to dealing with the policy recommendations," he said.

Ron Matus can be reached at matus@sptimes.com or 727 893-8873.