Auditability (Bruce O'Dell)

What does "auditability" really mean for elections?

How can we know that the output of computerized voting devices can be trusted?
Whenever computers are used to cast or tally votes, there are inherent risks that must be mitigated:

• Computer software is written to perform as specified, but errors still may occur.
• Whenever computer systems process transactions of high value to society, there is temptation for a malicious programmer to include covert functionality that causes software to deliberately perform other than intended, for personal or financial gain.

Limitations of testing computers

It is well known in the information technology profession that computers are ultimately "black boxes" - you cannot actually see what bits are really present and executing; and all methods to attempt to do so require other software that itself has the same problem, in an infinite regress. There is no workaround.

The only way to know what is running in a computer at any given moment is to observe its behavior: give all possible inputs, measure its corresponding outputs, and then check to see if the inputs and outputs you observe match the specification.

If computer software is always tested before use, why bother to produce an “audit trail”?

Unfortunately, you really have no guarantee that a given computer program's behavior as measured, say, at 10:00 AM will have any relationship to the same program's execution at noon. Computers have clocks and can tell time, and can easily be programmed to behave differently at different times, on different dates – or under an endless variety of different circumstances.

When it comes to systems processing high-value transactions of interest to potential criminal embezzlers - like money or votes - the inherent limitations of point-in-time behavioral testing make it unacceptably risky. Some kind of computer behavioral monitoring system is required that can record a vulnerable system's inputs and corresponding outputs while it is processing critical transactions. This would provide all the information needed to enable a human auditor or another automated auditing system to spot processing errors or manipulation of the transactions.

How are computerized financial transactions protected?

One reason why electronic financial transactions are as secure as they are (by which I only mean that embezzlement is the exception and not the rule) is that while financial transactions are private, they are hardly anonymous; you need to prove your identity to all the other counterparties involved. Each counterparty gets and keeps their own independent records of the transaction, all counterparties are strongly motivated to spot discrepancies and compare their records with others, while procedures relating to resolution of financial disputes are mature.

Why are voting systems so difficult to protect?

Unfortunately voting is a private and anonymous transaction, so conventional counterparty-based financial auditing mechanisms are simply impossible. Although some computer scientists feel they've identified some all-electronic means of auditing the accuracy of electronic vote totals, ultimately there is no reliable means to do so for the end-to-end voting process. Essentially, every all-electronic auditing scheme records just the voting software's assertion that Voter X voted for "Smith for Governor". At the moment of creating the electronic audit record, the computer could be programmed to electronically assert you input “Smith for Governor" even though you actually input "Jones for Governor". Every all-electronic auditing scheme, no matter how elaborate, would from that point on then simply record that lie with every appearance of the truth.

The only way you can dispute that kind of an electronic lie is with some kind of independent, tangible, write-one-time-only receipt that could be used as a proof you really voted for Jones. But this last step in the verification chain is prohibited - for a host of good reasons, including voter intimidation/extortion and vote-selling. So the best we can do is create an anonymous receipt that says the equivalent of "Someone Voted for Jones", to have the voter verify the accuracy of that assertion, and then deposit it with the electoral authorities, who must retain that record in support of possible auditing or recounting.

Any all-electronic means of auditing electronic voting is a waste of money, and all-electronic auditing methods that are claimed to be reliable are actually unpatched security vulnerabilities.

How can paper vote records be auditable in practice?

But once you turn to paper vote records - either VVPAT scrolls or optical scan ballots - "auditability" takes on a whole new set of dimensions and must be assessed in terms of its purpose: to detect or deter both vote tallying errors and outright manipulation.

The risks of error and covert manipulation are inherent to the use of computer software. Human nature being what it is, those risks are ever-present in all systems that process high-value transactions - especially those involving money or voting. So for "auditability" to have any meaning in such systems, auditing must always be performed.

To be usable as an audit mechanism, both the accuracy and integrity of any paper record must also be assured.

Accuracy of paper vote records

Accuracy means that every voter has actually checked that the paper record accurately records their intent. Needless to say, this does not always occur; regardless of anything else we do with the ballot paper, the tally can never be known to a greater accuracy than the rate at which a voter accurately verifies their intent. I would expect paper ballots to have much higher accuracy than VVPATS since in that case the audit record is the same thing as the actual vote-casting "device"; ballots inherently require a lot of scrutiny by the voter, almost certainly more than is typically expended in checking a VVPAT.

Integrity of paper vote records

To ensure integrity, no one must be able to alter, delete, or substitute paper ballot records after they are verified by the voter. Immediately after the election, traditional paper-based audit and control concerns take precedence. In general, the more time passes since creation and the further it travels from point of origin, the more risk there is of manipulation or destruction of paper records. The key consideration becomes the integrity of the chain of custody of paper records - who has had access to the ballots, and under what conditions?

How can you prove your belief about the integrity of your paper records, given that paper technology is also vulnerable to manipulation, and there are also very high potential rewards for undetected alteration of paper vote records?

Best practices and practical limitations

Unfortunately, there is no such thing as perfect security; the best we can do is mitigate the risks as best we can.

In recognition of this inherent problem, the Canadian system of counting paper ballots in-precinct on election night - in concert with their absentee/early voting procedure - is highly secure. The paper flow is always under observation, and ballots are immediately counted in front of multiple adversarial counterparties - the political party representatives.

Admittedly, even rigorous paper-handling processes are not perfectly secure - but on the other hand, in the last 600 years of general use of paper records, we have figured out some pretty good paper-based audit procedures. Yet I doubt that many jurisdictions in America handle paper election records with the level of custodial care that we find, say, in handling real estate collateral in the mortgage-backed securities market, much less in Canadian elections.

So as a practical matter, I'd have to conclude that simply having a VVPAT offers ultimately no assurance of practical "auditability" - the records in the field are only as accurate as the rate at which people actually verify them, and with the passage of time are increasingly unlikely to have a clear, secure chain of custody. The same applies to optical scan ballots.

Practical barriers to effective auditing

Worse yet, there major impediments to "auditability" even when VVPATs or optical scan ballots are recounted or audited after the fact - wherever recounts are still allowed, that is. Since paper records are typically not recounted unless margins are very close, brazen theft would be rewarded in practice. No candidate losing by a healthy margin wants to challenge an election and force a recount. Political culture being what it is in America, those candidates would quickly get labeled as "sore losers" who "waste the public's money and the government's time" on pointless recounts, who use "conspiracy theories" to compensate for their inability to admit they lost.

Even when they do occur, recent experiences in Ohio and Washington state clearly reveal fundamental flaws in the both the approach and execution of present-day recounts. Recounts as currently legally chartered are "broken" and existing spot-audit protocols are subject to the same limitations, as well. So what can we do?

Mandatory in-precinct auditing of vote records

If paper vote records are only effectively "auditable" to the extent they are accurate and intact and actually audited, I believe the best approach, short of fully hand-counted paper ballots, is mandatory in-precinct auditing of the paper records with corresponding protocols to secure absentee and early voting.

Recent research shows that an audit of a small percent of the ballots in all precincts is far superior to auditing all of the ballots in a small percentage of precincts.