Scoring not FCAT's only fault

Experts call for an overhaul as a group reviews the botched scores.

By Ron Matus
Published June 2, 2007

ORLANDO - A high-profile advisory group began delving into the mind-numbing details of making and scoring the FCAT on Friday, the first step in reviewing last year's botched scores and hopefully preventing future foulups.

But first came sobering comments from superintendents and testing experts alike: Put more safeguards on the Florida Comprehensive Assessment Test, make it more transparent and explain it better to the public.

Or else.

"I think all aspects of state assessment should be looked at," said John Hilderbrand, the recently retired testing director for Hillsborough County schools. "If you don't do that ... there will be an uprising."

State education officials say they're listening.

Education commissioner Jeanine Blomberg told the group that Friday's meeting was the first of many, and that future debates can range as far and wide as members want to go. She also said an independent, outside review should be a new, permanent part of the state's complicated testing system.

"This is the beginning," she said.

The Department of Education formed the advisory group in the wake of last week's bombshell disclosure that more than 200,000 third-grade reading scores from last year's FCAT may have been artificially inflated. The department's theory: It and its testing contractor, Harcourt Assessment, did not properly "equate" the test, meaning they essentially made it too easy.

For now, the ramifications are unclear. But everything from school grades to student retention to how Florida schools are rated under the federal No Child Left Behind Act hinges on FCAT scores. And it's clear the flub has rattled already shaky public support for the test, which then-Gov. Jeb Bush used to ground his polarizing vision of education reform.

"There's a deserved skepticism," said Hillsborough superintendent MaryEllen Elia, another member of the advisory group.

For advice on how to proceed, the Education Department invited a slew of stakeholders -- including six superintendents, six district testing experts and Pasco state Rep. John Legg -- to a Holiday Inn conference room. It also invited an official from the state teachers union -- a stalwart opponent of Bush's education agenda -- and a representative from a rabidly anti-FCAT group called the Florida Coalition for Assessment Reform.

More FCAT oversight "is a step in the right direction," said Marshall Ogletree, the union's chief lobbyist. "It should have been done years ago."

Group members waded into highly technical details about standardized testing for more than three hours, with some members confessing at times that they couldn't keep up. At one point, Cornelia Orr, the Education Department testing director, stopped in the middle of an explanation about how FCAT questions are calibrated for difficulty. "Is everybody asleep on me, Jeanine?" she asked.

Members laughed. But the difficulty in explaining the FCAT is a real and huge obstacle for the department, especially given widespread negative feelings held by many teachers and parents and a steady drumbeat of criticism.

"I have to be able to explain it to my people, so they understand it and buy into it," said Gilchrist County superintendent Buddy Vickers.

Other members said any explanation needs to include this admission: The FCAT isn't infallible.

"We need to acknowledge that assessments aren't perfect," said Lee Baldwin, the testing director for Orange County.

To deal with the bungled third-grade scores, the group agreed to forward suggestions for hiring an independent expert to review the department's theory about the root of the problem. Blomberg said the department will contact the experts to see who can do the work, but that the advisory group will ultimately decide who should be hired.

Members wanted to know how soon the review could be wrapped up. School grades are usually issued in June, and districts need to know, soon, which kids failed the FCAT so they can get remedial help over the summer.

Blomberg said she did not know, but expected to have a clearer picture next week.

Members also raised an array of other concerns about FCAT scoring.

Education Department officials say other recent FCAT scores have been rechecked for accuracy and found to be solid. But "I'm personally not convinced yet," said Baldwin.

Many members also pointed to FCAT scores for higher grades, which continue to show that middle and high school students fare far worse than their elementary school counterparts. Their comments echoed a recent St. Petersburg Times review, which found students in different grades are held to very different standards on the FCAT.

Vickers said students weren't getting dumber as they moved through the system and teachers weren't working any less hard. "There must be something else, " he said. "It must be the process."

Blomberg said that would be a good subject for a future meeting. "It sounds like we'll need more meetings that we initially thought," she said.

Ron Matus can be reached at 727 893-8873 or matus@sptimes.com.