New scoring for FCAT may add to grade gap
By STEPHEN HEGARTY
© St. Petersburg Times, published April 12, 2001
A small change in the way the state's FCAT tests are being scored this year could cause a widening in Florida's achievement gap between white students and the state's African-American and Hispanic students.
The change -- to count only multiple-choice items in test results and exclude extended-answer items -- is likely to result in across-the-board, though slight, improvement in student scores, according to an analysis done for the Florida Department of Education.
Whether those increases will be significant enough to bump large numbers of students into higher performance categories or to boost school grades is unclear. DOE officials think the impact will be small.
"I don't think we're looking at significant differences," said Education Commissioner Charlie Crist. "It looks like the changes would be nominal."
However significant those improvements are, the analysis based on last year's scores shows that they are likely to help certain students more than others.
The report, by researchers with the Human Resources Research Organization in Virginia, states that "for all tests except eighth-grade mathematics, white students would benefit more from elimination of open-ended items than would other ethnicities." Also, English-speaking students would benefit more than those who speak Spanish.
For instance, the report says that if you look at fourth-grade reading test scores from last year and compare them to scores with the open-ended items removed, African-American children's scores would have improved by 0.81 points. White children would have jumped 4.28 points. In eighth-grade reading, African-American children would have lost 2.64 points, while white children would have gained 6.44.
This is just one in a series of changes in the state's all-important FCAT test, which forms the foundation of Florida's school accountability system. The test is only 4 years old, but every year one variable or another has been tinkered with, affecting scores in ways big and small, and making comparisons difficult.
Some of the changes have been in response to requests from teachers, or to solve a problem. For instance, last yearthe results for mobile students were not included in the schools' scores. That change came about at the request of teachers, and it slightly affected scores and boosted some schools' grades.
The change this year has its genesis in the late return of scores last year. Then-Education Commissioner Tom Gallagher wanted to ensure that scores would be returned before the end of school this year and agreed to use only the multiple-choice items for scoring purposes. The extended-answer items, which take longer to score, will be returned later, after school grades have already been calculated.
"We just need to report those changes and be aware of them," said JoAnn Carrin, spokeswoman for DOE. "And you need to be careful making comparisons."
Now that it's clear that the elimination of open-ended items from the scoring calculations will have an impact, the important question is whether the impact will be big enough to affect the state's accountability system.
"That's the question, whether the improvements are real -- because nobody will be bothered if schools look better," said Monty Neill, associate director for the National Center for Fair and Open Testing.
Florida's school accountability system is based not on students' raw test scores, but on where that score falls in a scale of five achievement levels. A student helps his school if he scores at or above the second or third achievement level. So the key is whether a boost of four or five points will be enough to bump a student into a higher achievement level.
"We're only talking about a difference of three or four points; that's not significant unless you're borderline," said Carrin of DOE. "If you're in the lower end of Level 2, and you get another three or four points, you're still going to be in Level 2."
The analysis did not address the question of school grades directly, though it did recommend that DOE "acknowledge that some students and schools will receive higher classifications than they would have in previous years, but accept that change as the new ... system."
Crist said he expects the change to last just one year. He plans to include both multiple-choice and open-ended items in scoring for next year.
© 2006 • All Rights Reserved • Tampa Bay Times
490 First Avenue South St. Petersburg, FL 33701 727-893-8111
From the Times state desk
From the state wire