The public will be the ultimate judge

The ballot review presented today is the most comprehensive classification yet of votes cast but not counted in the 2000 presidential election.

In all, the project reviewed 175,010 ballots.

To do it, a group of media companies including the St. Petersburg Times hired a not-for-profit organization, the National Opinion Research Center at the University of Chicago. That organization, which has no political affiliation, sent teams into Florida to review each of the ballots that did not register a vote in the 2000 presidential contest, or that registered more than one vote.

Three NORC representatives, called coders, looked at each of 61,190 undervoted ballots recovered in the counties and marked what they saw on a coding sheet. They did not speak to each other or consult with NORC supervisors or employees of the elections offices they visited. They were allowed to use small lights, furnished by NORC, to determine whether light could pass through dimpled chad on punch card ballots. They did not touch the ballots, which were handled only by elections officials. For archival purposes, copies were made of all the optical scan ballots after the review was completed. Those copies are now maintained by NORC.

After a test in which it was clear that three coders would almost invariably agree on what overvoted ballots showed, the media group and NORC agreed to use just one coder on each of the 113,820 overvoted ballots reviewed in the counties.

This review comes with an asterisk made necessary by the inability of elections supervisors to accurately recover the ballots that did not register votes or registered two or more votes on election night. In order to compensate for the loss of those ballots, the media group asked for new segregations of the ballots in several counties.

The biggest changes occurred in Orange County, where elections officials decided to manually recount all 280,155 votes cast on optical scan machines in the presidential election, separating them into piles of undervotes, overvotes and piles for each presidential candidate.

That segregation produced new totals for each candidate in Orange County. In essence, Orange County officials did NORC's job, reclassifying ballots and finding votes that were not counted by machines on Nov. 7.

When they were finished, Bush had picked up an additional 184 votes in Orange, and Gore had added 249 to his unofficial total in that county for purposes of this report. Small adjustments were also made after hand segregations from a total of 12 precincts in Baker, Leon and St. Johns counties.

Here are some other questions and answers about the media review:

Q: It's been almost a year since Florida's prolonged recount. Other media reviews of the ballots finished months ago. Why did this take so long?

A: The delay is largely the result of the media group's insistence on accuracy. Though it proved impossible to duplicate the exact numbers of undervoted and overvoted ballots that were reported after the first machine recount on Nov. 8, the group came very close. NORC reviewed 61,190 undervotes -- just nine fewer than the state reported after election night. NORC's review of overvotes missed 1,427 of the ballots classified as overvoted by the supervisors on election night -- a difference of 1.25 percent. By comparison, the ballot review led by the Miami Herald and USA Today looked at 3,627 more undervotes than were reported by the state. That same review missed 3,986 overvotes statewide, a difference of 3.58 percent. In some precincts, both reviews saw more faulty ballots than were reported by the counties on Nov. 7. That was the result of the inability of supervisors of elections to find by hand or machine the same ballots classified as overvotes and undervotes on Nov. 7.

This project was further delayed by the attacks of Sept. 11, at which time the media companies devoted their efforts to covering those events and the aftermath.

Q: So what's the difference between this review and the previous one?

A: Today's review was more comprehensive.

In eight of Florida's largest punch card counties, the Herald/USA Today effort reviewed computer files from the machine recount to determine what was marked on ballots classified as overvotes. That review would not catch ballots where the machines made a mistake, rejecting valid ballots as overvotes.

Today's review counted those.

As for undervotes, the Herald/USA Today effort, staffed by one accountant and one reporter per team for undervotes, saw more of those ballots than actually existed. That is the result of a combination of factors including poor ballot segregation by the supervisors, machine errors that created more undervotes during re-segregation for the newspapers, and, perhaps, errors by the reviewers and supervisors that led to the same ballots being displayed and coded more than once in some cases.

Q: Why are the results different?

A: The number of punch card ballots that registered no presidential votes changed with each trip through the machines. Chad fell off, and ballots that once were considered undervotes suddenly registered a vote. In many cases, supervisors of elections in counties that used optical scan technology sorted the ballots by hand to determine which were undervotes and overvotes. That hand sorting was very inaccurate, and frequently excluded the very ballots that NORC needed to see -- ballots with clearly marked votes that machines failed to count. The media group persuaded many Florida supervisors to re-segregate the ballots using the same machines used on election night. That produced a more reliable, but still imprecise, result.

Q: Who paid for this latest review?

A: The St. Petersburg Times, New York Times, Wall Street Journal, Washington Post, Associated Press, CNN, Palm Beach Post and Tribune Newspapers (which include the Los Angeles Times, Chicago Tribune, Orlando Sentinel and South Florida Sun-Sentinel). The group spent about $71,000 to pay for the time of elections workers who segregated and displayed the ballots for the NORC coders. Additionally, the companies paid NORC about $800,000 for its work.

Q: Who decided which of the uncounted ballots were votes?

A: In a sense, the public will be the ultimate judge. NORC's coders drew no conclusions about which ballots represented valid votes, they simply marked down what the ballot looked like. Was one corner of a presidential chad detached? Were there clear punches made on the chad for the U.S. Senate race? What color ink was used on the optical scan ballot? All of those elements were recorded, then keyed into computer databases. The analysis of those databases was then undertaken independently by each of the media companies, though they shared some computer resources and compared notes on what they saw.

Q: Who were the coders?

A: Some were NORC employees, others were hired from temporary agencies to do the work. They filled out forms and were screened in an effort to weed out anyone who was incapable of doing the work in an unbiased manner. Their work was then checked each day to make sure that they did not consistently see things other NORC coders did not see on the undervoted ballots. One of the 153 coders who worked on this project was dismissed because he displayed coding very different from others looking at the same ballots in Baker County. That coder's work, on 79 ballots there, was not included in the analysis. Another coder later claimed in a missive on a Web site that he was a fervent Democrat, but it did not appear to influence his work as a coder. He saw the same things on ballots as the other coders he worked with.

Q: Were there other Democrats and Republicans who reviewed the ballots?

A: Yes. Of the 153 coders, 152 volunteered their party affiliation. About 36 percent were Democrats, 29 percent Republicans and 30 percent independents. Another 3.9 percent said they were members of smaller parties. They divided more sharply along gender lines, with 70 percent female and 30 percent male. A study on whether party or gender influenced what the coders saw will be published in January by NORC. It will show that men were more slightly likely to see Bush votes and women Gore votes among dimples on undervoted Votomatic ballots. Republicans were also more likely to see Bush votes, and Democrats saw more Gore votes among those dimples. But, according to NORC's researchers, those variabilities were blunted by the use of three coders, and the requirement for the stories published here that two of those coders agree on what they saw before we accepted it as a vote. Those variabilities among gender and party affiliation virtually disappear on optical scan ballots.

Q: Will members of the public get to draw their own conclusions?

A: Yes, today the complete database and the tabulators used by some of the media companies will be made available to the public. Academics, historians, political junkies and anyone who is just plain curious will be able to analyze the data themselves to see what might have occurred in Florida.