FCAT blunder overlooked
The ex-education chief says 2006 scores raised concerns but reviews found no issues.
By RON MATUS
Published May 25, 2007
Florida's former education commissioner said he was so concerned about last year's record spike in FCAT scores that he ordered three groups, including an outside group of testing experts, to "triple check" the numbers.
After further review, John Winn says, he was told that no problems were found.
"Did they do as diligent job as maybe they've done this year, looking back?" Winn told the St. Petersburg Times on Thursday. "I guess, obviously not."
Winn also said the testing errors that came to light this week were "always my greatest fear."
"I just hope it doesn't turn back the clock" on school accountability, he said.
Winn's comments came a day after his replacement acknowledged that last year's scores on the third-grade Florida Comprehensive Assessment Test were bungled, prompting an outcry from teachers and lawmakers and further eroding public confidence in the FCAT-heavy school system that Winn helped build under former Gov. Jeb Bush.
Education Commissioner Jeanine Blomberg, who took over after Winn resigned in February, said last year's test was not properly "equated." The result: inflated scores that may have warped last year's school grades and allowed some students to skate into fourth grade with poor reading skills. In coming weeks, more than 200, 000 tests will be rescored.
Fallout continued Thursday. Pinellas County schools superintendent Clayton Wilcox said he would use the blunder to study whether the state's retention policy is working. And Senate President Ken Pruitt, R-Port St. Lucie, all but gave the green light for the head of the education committee to pursue an inquiry.
"To borrow a phrase, we need to find out what the DOE knew and when they knew it, " said Sen. Don Gaetz, R-Niceville.
Winn said his first reaction to last year's scores was, "This is great." But red flags soon went up, Winn said, not only because the third-grade scores had increased so much - in fact, as much as the previous three years combined - but because the fourth-grade scores had taken a puzzling dive.
Winn said he ordered three groups to take a look - an internal group of DOE officials; a group at Harcourt Assessment, the testing company that develops the FCAT; and a technical advisory group that includes both DOE officials and outside testing experts, including some from Florida school districts. All three groups are a permanent part of the infrastructure that oversees the FCAT, but in this case, Winn gave them a specific task: Find out if there is anything wrong with the test that may have skewed the results.
"I said to Cornelia, 'You need to dig, ' " Winn said, referring to DOE testing ace Cornelia Orr.
The conclusion - that everything was okay - was reasonable, Winn said, given that the sweeping school initiatives put into place by Bush had jelled for the third-graders who took last year's test.
"When you put it into context - the first third-graders to go through the whole Reading First program - it was understandable, it was plausible ... that we would get extraordinary results, " he said.
He said context may also explain why last year's review didn't turn up anything - and this year's did. Two years' worth of squirrelly numbers made it clear there was a problem, and this time, the DOE didn't stop digging until it found an explanation, he said.
"The more clear and compelling the evidence is that there is a problem, the more you go back and look, " he said. "We learned that from disasters we've had, from 9/11, from the space shuttle disaster."
Blomberg said Wednesday that an independent panel of experts will recommend ways to audit future FCAT tests so similar glitches do not happen again. Despite the rescoring, last year's school grades will not be readjusted, and last year's third-graders will not be at risk of retention.
Hillsborough school superintendent MaryEllen Elia credited the DOE for acknowledging the error and taking steps to fix it.
"Mistakes can be made, " Elia said. "It's what happens when you make the mistake that is very important. I think DOE officials have stepped up to the plate."
"It's commendable that the state is willing to look at the results and question their processes, " agreed David Scanga, the research and evaluation director for Pasco schools. "The truth is, creating a test and getting it to be equal year in and year out is a very difficult task."
Scanga said the public needs to understand that "test development is not an exact science."
But the public may not be listening. Wednesday's FCAT bomb went off amid widespread public skepticism about the merits of high-stakes testing, stoked by teachers who object to test scores being used to grade schools and rate educators. With Bush out of office, critics see momentum shifting in their direction.
"We knew we had a lot on the FCAT, so we had to have the absolute best practices in the nation to maintain that credibility, " Winn said. "We did everything we could to maintain that.
"This is something we all wish hadn't happened."
Times staff writers Tom Tobin, Letitia Stein and Jeffrey Solochek contributed to this report. Ron Matus can be reached at firstname.lastname@example.org or (727) 893-8873.