Deserves a lot of credit. Thats the federal agency that is tasked with helping local election jurisdictions run secure, robust, useable elections. Over time, their testing procedures have gotten better and more sound. The trick is any Computer Security person will tell you testing only gets you a certain level of confidence. There is always going to be ways to get around it. Some of the things im not so confident about are things like tamperevident tape. We put these numbered seals over seams of the Voting Machines and if you try to hack your way in and get in there and mess with the brains inside, you have to pull that piece of tape up. It will it looks like it has been messed with. It says, void, void, void. Typically, a heat gun, something that anybody thats been in a shop class, is all you need to lift that tape without disturbing it. Plenty of examples like bad keys that are not as good. I think the fact that we have been a little rigorous about saying, please dont ever do networking on these machines. Virginia is one of the few places i dont even want to state it. There was something you could do from like across the street to one of these machines. Do you want to talk about that . You could use a pringles can attack, it is an effective antenna for wifi. You can log in from across the street and manipulate the votes. Thats unique. T typically it was the only machine in use in the United States that had wifi. You didnt even need that. As i recall, any voter in the precinct who had a smartphone could connect to the wifi that the voting machine was using and get access. There are these concepts of voting like one we Call Software independence, which is the notion that any undeticketable error in the software of the machine should not result in an indy textable error in the outcome. Thats why we have things like paper trail and a set of crypto graphic voting methods that arent widely used. Some of us worry about them in other ways. Those can provide some sort of hard check against the software being bad ideas. We have two ways of hacking, remote hack if the system isnt connected to the internet, doesnt have to be directly connected and proximity hack, if you can access the ports while you are in the poll booth or if you can access the machines while theyre in storage before theye distributed. Many of these sit in schools and feeders overnight or they sit in an insecured warehouse for years in between elections. You hear people say these Voting Machines arent connected to the internet. That should give you a little bit of comfort but not a whole lot. As dan wall lec said recent letter, there was malicious hacking before we had the internet and the network. There were things called very few people may see or know what they were but they are viruses that are transmitted by floppy discs, media that you can put in and out of the machines. When i was at princeton, people designed a virus for this machine used throughout the entire state of georgia. This was a machine where the default password was 1, 2, 3, 4. Not even spaceballs combination, 1, 2, 3, 4, 5, thats not much better, and they designed a virus that would do this, in one election, say the primary election, you would get access to the back of the machine and stick a usb stick on there and this would install. Between the two elections, that would have the opportunity to go from that machine to the election management system, the one computer that tells all the other machines, here is what the ballots are going to look like for the next election. The general election. Thats a way of installing a piece of malware on a device that over a longer period of time spreads itself. Thats the kind of thing if you think about what kind of attackers would do that kind of thing thats not something two months ago woke up said ooh, man, maybe we want to hack the vote, so to speak. What you are going to have is much more sophisticated and longterm entities like nation states that are more likely to employ techniques like that. Georgia is not a swing state by any stretch of the imagination, but there are platforms that may be susceptible to these kind of slower proximity hacks. It is interesting that georgia did that sort of test. Georgia actually had a problem with Certified Software in i think it was 2008. They had all these touch screens in the warehouse and using officials at georgia tech who were helping them. About two weeks before the election, these helpers went in and upgraded the software on these debolt systems and no one had oversight over the software. They simply said these machines needed to be upgraded and they installed on thousands of machines. Thats sort of a problem of process. You can have an external actor that gets access to these machines and you can have a problem with upgrading machines at the last minute in a way that software is not examined or certified. If you are doing something intentionally, you can design it in such a way it disappears once it has caused its problems so that someone examining the code afterwards will not be able to see the malicious codes presence there. There was an interesting case in iowa a number of years ago thats very much like that where there was an upgrade to the windows operating system on i dont remember which brand of machine. It turned out it had a certain feature which meant that each voter when they stepped up to the voting machine, it would prehighlight whoever the previous voter had worked for. It was a feature of how windows worked, nothing intentional. Nobody recertified it, because they didnt think changing the version of windows would cause a change in the voting bemay have behavior. Doug joans talks about that at doug jones from iowa is the one who talked about that in his book. We are talking about remote and proximity hacking. Internet voting, while it is not a huge problem right you know, Election Officials are very keen for internet voting. Theyll throw out a lot of reasons for why this is crucial for Going Forward and why the young generation of voters with smartphones, this is what they want and they do. They seem to think there is nothing wrong with it, why shouldnt they be able to vote online for more convenience . How widespread is internet voting right now and some of the issues around it . There are more than 30 states that allow internet voting for military and overseas vote certificates. In the case of alaska, any voter can vote online. There are no standards for internet Voting Systems the National Institute of technology that is charged with writing these standards has declined to provide standards saying, we dont know how to do it securely. Therefore, we are not going to provide a standard that says how to do it securely. We dont think it can be done at this point. However, as i say, more than 30 states are doing it, thence states are doing so and doing it on their own. Its unclear what security measures they are bringing to bear in terms of external reviews, anything like that. Theres been nothing public ive seen from anywhere other than here in the district of columbia with the infamous case of the university of michigan hack against a test system that they provided. That was the one where they made all the robots win the election. There was a sample election and not a real election of course, and they played the michigan fight song for every voter. How many votes were accounted from internet voting in this election then . Its pretty hard to find that information out. Most states do not break out the source for the votes. Theyll tell you this many votes from this county. They dont tell you this many came over the internet and this many were mailin, absentee ballots or in person absentee and so on. Its hard to tell. In virginia i was on the commission that looked at this and i think that there were somewhere in the range of 10,000 voters that were eligible if the internet voting passed, there were 10,000 people that would have taken advantage of it out of the voting population of about 4. 5 million. I want to talk about audits and bring you massimo in about election monitoring and influencing. Lets discuss auditing. How many states have auditeding at this point and why is virginia particularly opposed . Lets talk about what exactly an audited involves in terms of 1 and all that. Absolutely. If youre going to capture a paper record, you want to do something with that paper record. Count all of them maybe, which is the standard definition of an audit is basically a recount. These are extremely expensive and timeconsuming. There had been efforts since 1964 to try to get aspects of counting the paper to check the computer results without having to do a full recount. California passed the first statewide paper audit law in i think 1964. Its morphed over the years to be a 1 manual tabulation of Voting Machines from certain polling places, certain precincts. They used random numbers to pick a set of polling places in every county that they recount every ballot in the voting places and compare that to the Election Results. That is the simplest way to think about an audit. This is the number thats put in the statute, 1 , 2 . Some say if it gets below of half of one in the margin, you go. Theres a new flavor of audits. The australian ballot, a secret ballot transformed elections in the lastentury, this thing im about to talk about is going to transform elections around the world in the next century. They are called risklimiting audits. This is a statistical way of counting the paper to assure that you have a way of capturing if you misstated the outcome. If the outcome you reported to the press and the thing everyone waits for Election Night, thats incorrect, they have a high probability correcting that bad outcome. They are very different than 1 . You look at the margin of the race and other factors. And then you tune the sample that you pick to be and to give you the confidence that you need. So for example if the race is extremely close, say the bush v. Gore race in florida, you may have to do a recount. But for most cases where theres 5 margin, 2 or 3 margin, you count as few ballots as you need to to confirm the actual result. If you cant confirm the result, you enlarge your sample or do a full recount which corrects your misstated outcome. So this is what you need to do to make sure. Having the paper is one thing but counting it correctly is another thing. Thats one thing we are lagging behind. For example, theres almost half states, 24 states have no mandatory manual audit of their paper. 13 states have some post election auditing, but it tends to be 1 , 5 , things that arent tuned to the margin of the race. Theyre not very well designed. There are a whole bunch in mexico and california have provisions in the law to bring them in and do a sample and compare them and decide if you need to count them all. David becker said this morning, if jill stein starts winning South Carolina by a wide margin, that might be a sign that somethings off. The ironic thing he didnt say is if jill stein wins South Carolina by 5 , probably theres nothing legally that can be done about it because it wont be within the margin of a recount, so an audit is the only way we would discover it. Not that i have anything against jill stein, but it would be a surprising result. To address the question of virginia, as far as i know, ive talked to a bunch of people, virginia seems to be the only state that its illegal. I say illegal because its with an asterisk. The asterisk is if every race on the ballot was decided by a margin of at least 10 and after all the results had been certified by the state, i. E. , it doesnt make a difference any more, then you can do an audit. You cant do an audit any other time. Its not an audit to have an opportunity to correct the outcome. They will say you have screwed that up. Dont do that again. Right. They tried this out. They did some sample audits after this. It used to be they were totally illegal. This was an improvement, very small improvement. And the outcome of the audit was the results were pretty similar. Of course an audit result will never be exactly the same as the original. You have to know that theres going to be differences because youre going to have human differences in how they look at the color circles or the machine differences. The conclusion was it was very close, it was close enough we were confident in the results, and therefore we never needed to do an audit again. That was the conclusion of the study. Its like if you file your taxes with the irs and they audit you and say, everythings okay. You will never be audited or considered ever again because you obviously know how to do everything perfect. The irs doesnt work that way, yet that was the outcome of that audit examination. There was an election in california in a small county. I think this was 2008. Where the audit wouldnt have caught the problem. A small county. They discovered this was an optical scan machine. They had a paper trail. They discovered that the machine had dropped about 167 ballots. The votes were there after they did the initial canvassing of the ballots after the election. They disappeared some time after that. It would not have been caught in that 1 because the 1 takes a random number of precincts in order to do the audit. And this would not have been included in that 1 . They decided to do this new radical transparency where while they have their diebold optical scanning machines, they bought a fujitsu printer. They scanned all the same ballots simultaneously in their scanner. When they compared the results, they discovered the diebold system dropped 167 ballots and the other caught all of them. What do you trust . The Risk Assessment or Risk Management really quickly. There are materiality audits and process auditing. Materiality audits check the numbers, check that you arrived at the right result. Process auditing checks that everything that goes into making the result was done correctly. Thats where its very hard to do that right. You dont get up to be a process auditor when you go to work and stuff. Its not the best job in the world. Those are the things where we need just as much rigor to that as we do with all the processes that arrive at the actual results themselves. Lets switch it for a moment. I want to come back to security of Voting Machines a little later. Lets talk about influence hacking. We have this sort of unprecedented election in the u. S. Weve never had this situation before, or have we . Its quite common erseas and the cia has been successful influencing elections in the past. Give us an idea of sort of where we are contextually for this kind of situation in the u. S. Is it true that we havent had it before . If we place it in the wider context of electoral processes worldwide, there is a growing experience by election observers in dealing with electronic wording. You mentioned a few examples in the u. S. States that adopted it. Outside of the u. S. There is, i would say, consolidated experience in india, in countries like the philippines and brazil, for example. Youre referring specifically to the machines again. Im referring to machines again, but they are also experiencing internet voting in some cases. Although that is not the rule for the latest political elections in those countries. Talking about audit, i think you can get what does that mean in terms of auditing a system, so to speak . If the focus is on election day, there will be a lot of interest in looking at the numbers and focusing on the statisti techniques if you had thmeans that are basically related to the availability of a paper trail that can be used. But auditing system, auditing the process before election day, three, four months before election day is also practice that Election Observation nations are trying to introduce in their best implementation of the guidelines adopted internationally. There are examples from the osce in europe and also the carter center. Both developed handbooks on observing electronic voting. These are actually drawn from general principles that belong to the Election Observation per se, not necessarily focused on electronic voting. They have specificity that are related to the medium, to the technique. Just to give you an example, you mentioned the risks related to internet voting, but if you assess them, you would look at how data that are produced through electronic Voting Machines, for example, are processed by computers that may be hooked to the internet. So that is a form of indirect access to the internet that is not normally considered part of the label category internet voting, and still presents risks of manipulation. Let me just support the Russian Security firm ceo of kasperski labs that identified last friday on an italian tv he was interviewed. He was asked, what is the biggest threat to democracy in your view . He said it is internet voting, in his view. Unless the environment and procedures and the systems are safe enough there will be a growing tendency towards using these means of voting, and there would be risks, high risks of reaching manipulation, before, during and after elections. In fact, that applies also in different ways to the history of voting, also to paper ballots. So what we had to learn was how observing elections can introduce elements of independent assessment that can help election management bodies to make the environment for voting safer, and to increase also the confidence by voters in the system, which is one of the challenges of the u. S. System now. Just one, not the only one. In the recent paper from harvard university, identified five challenges to the integrity of the elections in the u. S. These ones, the risk of hacking is only one of the five. You have the regulation of campaign financing, issues about the polarization, and therefore trust among Political Parties in the electoral procedures. You have issues of, of course, lack of professional standards in electoral management, especially in highly fragmented environment where elections are managed. I would say the most important one is lack of Public Confidence in the electoral process. All of these are interrelated, although we focus now on just one of them. I think this should address all of them at the same time. Lets talk about some of those latter ones influencing and things. There have been reports and concerns that for example the Associated Press might