Are Optical Scans the Answer?
Today's N.Y. Times published these letters responding to its editorial last week extolling the virtues of optical-scan technology. They include one from Prof. Michael Shamos of Carnegie-Mellon, which is more worthy of reading than the NYT's editorial.

The NYT's editorial page was among the first to drink the paper-trail Kool-Aid, having insistently editorialized for over a year that contemporaneous paper records should be required for electronic voting technology. During this period, they've exhibited an almost willful blindness to the practical difficulties in implementing this technology, as well as the serious questions about whether it will really do much to enhance security -- see here and here for my earlier criticisms of the NYT's simpleminded approach to the problem.

Without renouncing its previous position, the NYT's editorial page has now found religion on something else. Its latest editorial argues that the State of New York should adopt optical scan voting technology, a hand-marked but machine-read ballot similar to SAT tests. The NYT touts the optical scan as the "most reliable and cost-effective of the current technologies." It's true that some types of optical scan equipment do pretty well in terms of accuracy. Yet the NYT's latest editorial continues to present an overly simplified and misleading picture in several respects. Let's take a closer look:

- While optical scans are better than punch cards, the evidence doesn't support the claim that optical scans are the most accurate method of voting. It's true that optical scans did better than first-generation electronic voting machines, but the available evidence indicates that they don't do as well as the present generation. As set forth in this report by MIT's Charles Stewart III, counties that switched from punch cards to optical scans saw their uncounted vote rate decline by 1.12%. That's good, but not as good as counties that went from punch cards to electronic voting, which saw their uncounted vote rate decrease by 1.46%. Strange, isn't it, that we've not seen the NYT acknowledge the dramatic improvement in states like Georgia, which converted to electronic voting statewide and saw a dramatic decrease in uncounted votes, particularly in heavily minority precincts.

- The NYT's editorial makes no attempt to distinguish between two different kinds of optical-scan systems: precinct count and central count. With a precinct-count system, the voter can check for overvotes before casting her ballot, by feeding it through a counter that will alert the voter if she's cast more than the allowed number of votes for a particular race. The counter can also be programmed to notify voters if they've undervoted (i.e., failed to make any choice in a particular race) though this isn't normally done because it would slow down the voting process too much. With a central-count system, there's no mechanism by which to check for overvotes or undervotes. By contrast, contemporary electronic voting systems don't allow overvotes, and have a verification screen that allows voters to check their work -- and to correct mistaken undervotes.

- The NYT presents a misleading picture of the costs associated with optical scan vs. electronic voting technology. It's true that the optical scan system will generally entail smaller up-front costs, because the equipment is less expensive. But the Caltech/MIT project found that those costs even out over time, around 15 - 20 years. That's because optical scan systems have "much higher operating cost" due to the cost of printing out paper ballots in each election.

- Continuing its disdain for the rights of disabled and non-English proficient voters, the NYT makes no mention of the fact that contemporary electronic voting systems are accessible while optical scans are not. DREs have disability access features for those who can't read or use their hands and arms, and are easier for non-English speaking voters to use independently. There is a hybrid system called the Automark being marketed by ES&S (one of the big bad voting machine companies that the NYT has repeatedly attacked), which is supposedly accessible to people with disabilities and non-English speakers. Basically, this system allows voters to make their choices using an electronic voting interface, then prints out a ballot that the voter can place in an optical-scan counter. This system isn't yet certified in many states and, as far as I can tell, hasn't been implemented on any significant scale in any real election. Also, it's not quite clear to me how a blind voter would insert the printout into a counter while preserving the privacy of her ballot -- or whether there's a mechanism on the counter to ensure that her vote was correctly counted (maybe the implicit argument is that voter verfiability is important for able-bodied voters, but can be dispensed with for those with disabilities). The Automark is a promising technology, but not yet a proven one.

- Finally, it's not at all clear that optical scans are any more reliable or secure than electronic voting. Prof. Shamos' letter makes this point, noting that "Despite the fact that the voter personally marks the ballot and has the chance to verify his or her choices, no machine has ever been built that can read a ballot the way a human eye does, and there is no assurance that the machine will count the ballot the way it was marked by the voter." Shamos expands on this point in a post on the VSPR listserv, which he's given me permission to quote:

What we're not discussing is the variant marks that are NOT caught by the machine and which the voter has no opportunity to correct. This occurs most often in straight-party voting where the voter has attempted unsuccessfully to override the straight-party vote in a particular office. If the override mark is not read by the machine, there is no indication of either an overvote or an undervote and the ballot will not be rejected. Even the rejection of an undervoted ballot depends on the scanning equipment not having been tampered with. If the DRE opponents are correct, and we can't rely on any machines, then why couldn't an intruder set the scanner never to reject a ballot that contained a vote for his favorite candidate, even if some other race were undervoted? This would never be caught in a manual audit since there is no indication on the ballot that the voter voluntarily asked for it to be counted with an undervote despite having been rejected initially....

The whole problem with optical scanning is that opscan ballots can be marked in an infinitude of ways, but there are only a finite number of choices solicited. The mapping from the infinite space to the finite space is ambiguous and, in many jursidictions, undocumented. We need a binary method of voting in which each ballot position is either marked on unmarked without ambiguity. DREs, whatever else may be wrong with them, provide this. I'm not saying it's impossible with document ballots, but no one has yet designed one that has this property.... In fact the range of marks is so diverse that the different states have developed no consensus on what actually constitutes a vote, and some states haven't even tried to do so. Why should it ever be necessary to decide what constitutes a vote? Is this the best that technology can offer -- an endless debate about how to blacken an oval?....

With respect to things "going awry" on DREs, the voter review screen is perfectly adequate, provided that the voter looks at it. (Likewise, the reviewability of an opscan ballot is of little value unless the voter reviews it.) For things to go awry on a DRE, the vote has to be recorded incorrectly internally but reflected correctly on the review screen. This is of course possible but there's no evidence that it ever even happened once. Once we add voter verification to DREs (which I'm all for), we then have a non-problem. The difference between me and the paper trail advocates is that I believe there are numerous and better ways of providing verification than relying on unsecured little pieces of paper. If we really want every vote to count, let's design a system that has a chance of achieving it, and optical scan isn't it.

Points well taken, in my opinion. Although precinct-count optical scan is preferable to punch cards, it's a mistake to equate paper with integrity or accuracy. This isn't to say that precinct-count optical scan is a bad system ... only that it's not without its problems and risks. - posted by Dan Tokaji @ 11:27 PM