As commissioner masterson mentioned, the panel we have here is called election efficiency and integrity, improving the Voter Experience but most of this panel i think will be focussed on data and the importance of data and how we use it in the Elections Community. We have a great panel here for you. You have biographies in front of you. My staff has provided some abbreviated biographies which ill read and then well start hearing from the panel. Ive asked each of them to speak for about five minutes or so from their perspective on this panel and then i have some questions for them. And then we will go to questions from you. So before i get started, i want to thank you all for being here. Really excited to kick off the 2018 elections. Its hard to believe its already 2018. Were kicking off the elections but officials have already started working on this. The day after their last election. So i know those of you who are Election Officials, this is not a kick off for you. As others have said, it is just the kickoff in january. And you all have been working on this election since the last election, and i appreciate all the hard work you put in to making our elections so good in this country. I will do some brief panel introductions, and then we will get going. To my right is secretary barbara sadasky. She has been secretary of state since 2015. She has more than three decades of combined Public Service and prior to becoming secretary of state she represented clark County District eight in Nevada State Senate and before that she was elected to three consecutive terms representing clark County District five and the nevada assembly. To my left is dr. Christian, she is a Senior Researcher at the marsh group where she served as a lead researcher for the Voting Survey or eves and they did a fantastic job for us this year. Lets make that a Public Statement and thats not an easy task and hes worked a number of and our friends at the federal voting agency. So thank you for being here. To my far right is michael. Hes the register of voters for San Bernardino, california which is the largest geographic in the country. Hes worked for elections in three states and his expertise is in improving operations, reporting results quickly and accurately and while michael has made his mark at each post in San Bernardino county, he has made the Election Office into a more Efficient Organization and i know its quite an impressive operation. Including the launch of new applications and redesign of the countys ballot. Consolidation of underused polling places and much more. So were excited to hear about what youre doing in your office in San Bernardino. And to my far left, maybe politically, i dont know. Probably of me. Dr. Stewart iii who am cstel i stewart iii who am i stewart iii who am i saying at the right . Distinguished professor of Political Science at the Massachusetts Institute of technology, mit, where he has taught since 1985. And his Biggest Research includes close looks inside congressional politics, elections and important american political developments. Those of us in this room have come to rely on charles undeniable ability to help us understand voters and how Election Administrators can best serve them. So thank you for taking timeout to be with us today. They will all take about five minutes. I will start to my right with michael sscarvellcarvello. Take it away. Ive got to warn you. Im an elections geek. So im going to refer to my notes to keep you on track. The goal is to conduct elections in fair, accurate, accessible, and transparent manner, so meeting that goal is difficult. Because our budgets are tight, our staff is small. We rely on temp workers and poll workers and the expectation is perfection from the media and the public. So thats very difficult. So were constantly thinking of new ways to improve our processes and one of the ways we can do that is to use data. And so we typically collect raw data, analyze that data and use it to convert it to information to improve our processes. Traditionally the analysis to prepare that data has been simple. We look at the number of registered voters. We look at the historic turnout. We analyze how many ballots and we have put those computations into law. The issue though is that many times those computations are based on averages and averages across the county and thats helpful but it can be problematic because sometimes we underestimate how much supplies we need. And sometimes we overestimate and we waste money. In reality, many Elections Officials like myself have spent time to be more efficient. In my 18 years in this business ive challenged my staff to crunch their numbers. And i always tell them they should have paid attention in math class. I remind them everything we do has to be based on numbers. We have an elections Analytics Department that does nothing but crunch numbers and develop processes all day for our different departments. So we currently analyze data in every area of our organization. In the process kind of goes like this. We collect data, and im going to give you an example about how we look at polling places, for instance. So we collect data about the polling places. We find and inspect accessible polling places in quality locations. I think were going to talk about that this afternoon in one of the panels. We inventory the number of parking spaces. We measure the square footage, then we collect data about voter behavior. We know historically we look at voter turnout and we look at where people live and we look at when they vote. And we measure when people vote by the hour. And this is something thats kind of unique and i dont think too many counties do that. Around the country. We also look at how people vote. Did they vote a regular ballot . Did they vote a provisional ballot . Or did they just drop off a vote by mail ballot at that location and then we also predict how the voter will behave on that election day. We do simulations. So we look at how long it takes a voter to drive to their location. We look at how long it takes to park their car how long it takes to process that voter and we e. Is it a long ballot or short ballot . And then return to their car. So we gather that data. How long does it take for a p visional supervisor to do that . Then we take all the information and crunch it. Take the data and we calculate the capacity of each of our polling places. We input that, then we take that information, and put it into gis software and assign people to polling places. Because the goal is to keep every polling place busy and be efficient but not have any polling place be overwhelmed and have too many voters and then have lines. We also project the newspaperum pollers by the hour in each polling place and use this to determine how many supplies and booths we need and small ballots of each time. Keep in mind, people think of how many ballots. Well, most Election Officials its not a ballot. In our country over 400 different ballot types that we have to distribute. So we have to calculate the number of each of those types. So its very complicated. We also assign the number of poll workers based on projections and do that for the peak hour so that we dont have any problems. So the number of poll workers varies, knonot just three to fi anywhere from 18 to those. We also do it around break times. So we found this Data Analysis is helpful and helped keep things smooth at the polling places. But we also know its not always dependable. So we also, in addition to prior to election day, on election day we actually survey each and every one of our polling places to find out what the turnout is like and the ballot usage. Then we make corrections as necessary. The result of our Data Collection and analysis has been that we have created a lot more convenience for our voters. We have very few lines. And if there are lines, the wait times are short. Weve saved hundreds of thousands of dollars and used that money to not just give taxpayers back the money, but expand our services, like increasing Early Morning sites and drop off locations. This was one example. We did early voting analysis and use this in the future and tapping into amber to see what they are doing in colorado to crunch their numbers. A couple of other examples of how we use data. We use it to speed oup our ballot counting, of our paper ballots. Project the drive times to get ballots back and monitor the progress. We analyze the time it takes to process ballots at each step of the way to process them, to count them, to duplicate ballots and store them. And then we adjust as necessary on election day. We also use numbers to train and assign our poll workers. We have over 3500 poll workers in our county. But to get those 3500 poll workers we have to contact over 10,000 people. We have to schedule over 7,000 to come to training. We have to train over 5,000. And we have to assign over 4,000 to get the 3500 to work on election day. We must recruit and train people in six different launguages and place people at different precincts. Because we are a large area, we train by region. What we found is it in each region people behave differently in different regions. So use the an information from our analysis to look at that behavior to have different projections in each different type of region so we arent short of workers. So thats a quick look at how we use data. Its helped us and increased our efficiency and made the Voter Experience better. We also then pass that information to state and federal officials to compile that data. Then we compare our performance against other jurisdictions. Great. Thank you so much. And i know there is unending number of data points you can collect and do Different Things with, so its very interesting some of the things that you are doing. Just incredible. Thank you from the local level. We go now to the state level with secretary cegavske from nevada. And interested in hearing what you have to say. Well, thank you for having me today. I want to thank you for putting on this summit. Christi, for all of your assistance. In the state of nevada we have 17 counties and of those we have 15 elected clerks and two voter registrars. And we have a very large county in clark, so we understand sometimes the pain that you are going with. But one of the things that i wanted to explain to everybody, because one of the things thats always unusual for me is to hear acronyms. And many of you will hear acronyms and they can mean Something Different. So i want to make sure you are all aware of some of the acronyms ill be using just to talk about what we are required to do. One of them is the Election Administration and Voting Survey that we do get from the aic. Its a biannual survey administered by the aic after every federal election which is every two years, and that vote a Data Collected is absentee voting, military and overseas voting, polling places, poll workers, and provisional voting. We also have what is the vra, which is the Voter Registration agency. And its a Government Agency designated by the federal or a state law that is required to offer its customers or clients in most circumstances the opportunity to register to vote. Vras include the dmv, public assistance agencies like welfare or employment agencies, service to people are disabilities like aging and military recruitment offices. The next one that youll hear a little bit about is the covered transactions and every time a customer or a client with the vra is requesting a service filling out a form or application, the vra is required by law to give the person the opportunity to register to vote. These interactions are called covered transactions. And two examples for us in nevada that is when you are going into do anything with your drivers license at the dmv or a person submitting an application for welfare assistance. And then the national Voter Registration act, the nrva, that is a 1993 federal law also known as the motor voter law that requires the dmv to offer the opportunity to register to vote. So with those, i wanted to make sure everyone knows our acronyms. In nevada, we use data to improve the administration of elections and Voter Experience in several ways. And among other things, the office of the secretary of state is currently focusing on transactional data to monitor the effectiveness of Voter Registration agencies. So regular basis looking at what they give us to find out if its accurate and if there is any issues that we might see. We coordinate routine reporting with Voter Registration agencies and nevada local Election Officials as well. And Voter Registration agencies record and report the number of covered transactions, whether the client chooses to register during the covered transaction or if they are going to send it in later or give it to somebody else, those numbers are tracked. We know that they have them. And thats where they got that application. And then the number of Voter Registration applications each local vra office transmits to the county Election Office. We also, the local officials track and report the number of valid applications received from each Voter Registration agency, as i said, and then send that information to the secretary of States Office. And the secretary of state receives monthly reports from the department of Motor Vehicles. We get also from the health and Human Services. And these reports are made Available Online for the secretary of state nevada at www. Nv str www. Nvsos. Gov. And also have a program that monitors data to look at any complaint or compliance as well as best practices. And ill give you an example. If a significant decrease from previous reporting periods is identified in the number of completed Voter Registration applications being transmitted from a Voter Registration agency to a local Election Office, the secretary of state Program Staff and the Voter Registration agency will investigate to determine the cause and whether any corrective action is necessary. And we do that on a regular basis between the two agencies. Then we are evaluating the disposition of completed Voter Registration applications among offices is another metric Program Staff uses to identify where review and corrective action might be necessary. And an example of that local Voter Registration agencys office that exhibits lower than average voter applications or rejected can see the issues and need for the staff to review training materials. The secretary of States Office puts this Data Collection into charts and graphs, which allows for the review of large amounts of data at a glance as well as identification of significant statistical variations. Visuals also allow for quick comparison of data over a period of time and other data sets. And the example for this is the department of health and Human Services reports multiple figures on a monthly basis. In the raw data format, interpreting this data can be daunting, but by placing this in a scatter plot or line program, staff can easily determine if there are any deviations. And then collecting this data on covered transactions from Voter Registration agencies allows the secretary of state to be proactive during conversations with any of the advocacy groups. And nevada uses Voter Registration and turnout also to identify best practices at the local levels. So we feel very, very confident in the information that we are getting and being able to work with any of the agencies in nevada to see if we have any issues or where we might need to go in a different direction or look at how we might need to change what we are doing. So look forward to the next part. Yeah, i think, you know, both of these little presentations have pointed out, really the goal here is to make the process better. Absolutely. For the voter and serve our taxpayers better as well. You know, public servants, at least this side of the aisle, and collecting that data makes a big difference on how we can make the Voter Experience better and more efficient. So from the west side of the country im now turning to the other side of the country. And the two panelists to my left here are both researchers academics. And interested in hearing what you all have to say on what youve seen with the data and how to make things more efficient and bring more integrity to the voting process. So dr. Gregg. Thank you so much for having me today. My name is krish shah gregorowicz. Im happy i got to follow our fellow colleagues. If we use these processes and really evaluate whether the policies that we have in place, the processes, and the investments that we are making in the election system are having the intended effect. The big challenge that we dealt with as part of the election Voting Survey is how can we learn from each other. They talked about elections create so much data. And we have great examples of harnessing that data to do ones job better and make sure elections are run well. But i think its really important to remember, and lucky im on one of the early panels, there is no one election in the United States. There are thousands of independent elections run at the state and local levels. They are run with their own policies and their own processes. And, frankly, in a lot of ways their own vagocabulary. When we talk about acronyms and defining those, thats important, because without having common definitions what they are and what we can do with them, we struggle to talk to each other. So the Election Administration and Voting Survey for those of you who dont know, it is a large scale only large scale Data Collection instrument for administrative elections in the u. S. Calling it a survey is misnomer its election in the United States covering variety of topics, military and overseas voting, technology, and our processes. And whats really remarkable, so 2004 was the first eaves survey was fielded, and very remarkable to thing fundamental questions about elections in the United States, so for example how many precincinnati and polli precincts are there in the United States . Its mind boggling to see there was no place to find that answer. The answer for that in 2016, there was about 178,000 individual precincts and about 116,000, almost 117,000 individual polling places, additionally about 8500 early voting locations in the u. S. Ill caveat and say those numbers are still complicated, to interpret, but thats a very important piece of information. Its really shocking that we wouldnt have known that before. I think a couple of other highlights that we learned from 2016 eaves. There were about local election and state lexow officia Election Officials did a lot of forms. By far, the department of Motor Vehicle offices, about a third of them were coming from dmvs. One changed from previous elections was increasing use of online Voter Registration systems. So we had about 17. 4 of registration forms coming from online systems. Thats up from in 2012, it was only about 5 . So big jump there. Similarly, epoll books we saw 75 increase in epoll books across the country in 2016 since 2012. 2. 5 million provisional ballots. About half of which came from california. Over all, about 71 counted in part or full. So thats an important way of metric to consider how well that protection is working. Very active overseas population. We saw about 178,000 ballots transmitted to overseas citizen voters, so nonmilitary, about 100,000 more than in 2012. Almost 80 were returned and ultimate ultimately counted. So these are important pieces of information how elections in the United States are doing. And again from all jurisdictions from across the country. So while these are sort of neat in sites, insights ks what insights, whats really important about the eaves since it started, it helped us develop a common language to talk about elections with. And our secretary from nevada kind of mentioned this, defining acronyms. Without a common definition of these basic terms, whats registration, what do we mean by objected ballot or accepted ballot, we really cant. It becomes difficult to learn from each other and make these comparisons. So i think that has been a very important certainly a work in progress. And i think this year the federal Voting Assistance program through their overseas Voting Initiative coordinating with the counsel of state governments ran a section B Working Group and brought together state and local Election Officials from across the country to talk about the section b which is military and overseas of the eaves to talk about the challenges that they face collecting that data and reporting it. And helped refine the definitions. Those were lessons we were able to apply across the eaves to the other sections. And really when we have this common language to talk to each other, it really unlocks the possibility of using all the data that get collected or more of the data. Because it really is copious amounts. In order to report to the eaves, not even necessarily to do this amazing work, evaluating ones processes at individual level, but to administer elections, this is sort of the tools on the ground make sure elections are happening. And so its very exciting, i think, there are so many possibilities out there already moving to projects with common data, data standardization where looking at directly from the data that are being collected at transactional level and how can we take it from multiple jurisdictions, i believe it was 13 involved in that project this year, how can we make these tran Tax Transactional data talk to each other. So very exciting opportunities in the future. And i think the eaves is such an important part of that. Thank you. It really does show us the big picture nationwide of how things are going, the successes and the challenges. And down to the smaller levels, smaller jurisdiction levels which is really great. And as we continue to work on that with our partners in the federal government and on the ground, in the states and local jurisdictions and of course our researchers will make it even better. I think its in valuable information. So with that ill go to dr. Stewart. Thank you for being here. Looking forward to your comments. Great to be here and thanks for inviting me this morning. Actually, i wanted to start with the equip christy made in the introduction about our relative political locations. And i think thats actually a good place to start. Those of us who are in this business of researching elections, especially trying to make them better, oftentimes talk about making it easier to voter and harder to cheat. And i think thats the goal that we all share as we try to make elections better. And this Movement Toward data driven Election Administration is really an effort to try to understand the degree to which we might have certain values that determine maybe how we think about achieving those goals. But oftentimes we can decide, some of those questions based on the facts. And i think that those of us who are geeks and all of us on this panel sound like we are geeks, i think we are driven to see how much we can base our knowledge, our actions on the evidence in front of us. And i think there is it a lot of Success Stories that ill talk about in just a little bit, in a second, where when we focus on the facts, we can really make a lot of progress. And i think thats the important thing here. So the topic today is making the voting experience better for voters, making it better for the nation. And i guess the way that i jump into this as an election geek is thinking back to the year 2,000, and the event that got me actively involved in studying Election Administration and many people around the country interested, at least noticed that, gee, Something Interesting here in Election Administration. And that of course was the florida recount. And being a native floridian, having been registered to vote when i was 18 years old by my second Grade Sunday School teacher, ive always been interested in Election Administration from a young age. And the interesting thing to me as an academic about the florida recount, once we kind of took a deep breath and tried to figure out what was going on was to try to understand whether, first of all, whether what happened in florida was a oneoff situation or was it common. And secondly to try to figure out really how bad is it after all . Even the best of elections from time to time are going to be close. So the fact that you have a recount is not a problem. In fact, thats a good thing, to try to get the answer right. The question is are the problems that emerge during a recount, are they general, are they specific to a place, and how big are they. And it turns out in 2,000, there wasnt a good sense about the answers to these questions is florida oneoff or not . Had these problems persisted for a long time or not . And when help america vote was passed in 2002, then the question became okay well spend a couple of billion to make elections better, how will we know if that money was well spent . All of these come down to questions of fact and data. And in 2000, it was really interesting to me as quantitative social scientist that despite all the data generated by elections, very little of it had been used for management purposes and for the purpose of assessing how well the elections were run. And so that was one of the things that we did in an organization that i continue to lead called formerly cal tech Mit Technology project i like to call it the Mit Technology project. But what we did in this is we looked around to find ways in which we could use data already being collected to answer this question, how bad was it in florida, how does it compare to other states, how does it compare to the past. And in part we discovered, i discovered someone named kim brace who i saw over here who had been collecting for many, many years, turnout data and election return data and voting technologies, discovered that on the Census Bureau had been asking questions about problems they were having when they turned out to vote. We discovered there was a load of data out there that could be used to assess elections and then to chart the improvements. So we discovered in 2,000, for instance, that when we got all this data and looked at it from the perspective of how many lost votes were there in 2000 . By a lost vote i mean, think about it this way, you wake up on election day, intending to vote, you as a voter do everything right, and at the end of the day the vote that you cast is not counted or even you are not allowed to vote because say there was a screw up in your Voter Registration. How many votes was that . Because that was really kind of the nut of the problem in the florida recount. And what we discovered, for instance, from kims data between 1. 5 and 2 million votes on that election day were lost because of failures in Voting Machines, hanging chads, poorly maintained lever machines, almost up to 2 million voters that day in an electorate of 100 million. We also were able to calculate that between 1. 5 and 3 Million People did not get to vote because of problems with Voter Registration. We further discovered that around a Million People walked away from the polls because of long lines or other problems that they faced in the polling places. And then an unknown number at the time problems with absentee ballots which we later discovered to be of a similar magnitude to the numbers i just mentioned. So millions of people in 2000. And also in 1996 and 1992 had done everything right and didnt have their vote counted. Thats where we were in 2000. Fast forward to the present h had those billions of dollars and these efforts had been worthwhile . Well, it turns out using the same methods we see that roughly three quarters of those votes that were lost because of voting machine problems have gone away. About half the Voter Registration lost votes have gone away. About a quarter of the polling place problems have gone away. In other words, we have, in the 2016 election, 2012 election, 2008 election, 2004 election. There have been a couple million more voters in america who had their votes counted purely because we got better at running elections. And we are able to document that. And thats really valuable thing. Both for us to understand how well our democracy runs. To understand that we can make it better. And to understand how to use our limited resources that have been mentioned several times to target where to make things better. So as an academic, im interested in these things just because im curious about everything, kind of perverse way, but i think that the use of data like this that comes from wide variety of sources can help us to pinpoint where problems are. As an aside, oftentimes when one discovers there are problems with Voting Machines or long lines, its very hard to tell, is this a problem every where or is this a problem in a few places . Hand understandi and understanding the magnitude of the problem and where to focus is really important thing. And it turned out 2000 just about everybody had problems with Voting Machines. And there was one state, and i dont want to shame anybody so ill make up a name and say sworz a, guess what they got to the data before we did and they got really good at their voting machine management. So targeting and figuring out where the problems are and going after the problems. There is a similar story from 2012. Back in the simpler days of 2012, all we cared about was people waiting in line a long time, rather than the things that came out in the 2016 election, you know, it was kind of a similar thing. Well, how bad are the lines . How bad is the line problem . Turns out that just about every newspaper or news website in america on election day has a picture of a lot of people standing in line waiting to vote at 7 00 in the morning. Well, is that generally true . Or is it just because there is like one precinct every where and every where else is good . So how extensive is the problem . Where should we concentrate our resources . And how will we know when we did better . So this is a case where its not that Election Officials have been gathering data in this area so much, although it sounds like in San Bernardino they have been, but again we have been able to amass data to first of all assess where the problems are and make the problems better in 2016. So 2012 we were already asking voters how long they were waiting in line to vote. And a dozen states statewide. Every city in america it was probably a problem that needed to be looked at. But a lot of places in america not an endemic problem, more e oneoff situation. So here we had to get data practices. Since universal practice is dont count people in line. So working with bipartisan policy center, we developed protocols for local Election Officials to counting on lines. And to turn those numbers into estimates of line lengths. And i can remember that in 2016, when we went back and asked voters how long they waited in line, those states that had really long lines in 2012, and ill name them, wont make up names, south carolina, virginia, famously, florida, their lines were cut in half or cut by three quarters. And part of that thats a sto start of lot of things going on, but part of that was story to figure out actual polling places to understand where the problems were because every polling place wasnt a problem child. And then to respond. So from my perspective, whenever i give my big talk about how elections are going in america, and those of you who have seen the one hour version of this talk, i oftentimes will start with one of my favorite videos, which is by Charles Ray Eames which many of us saw in high school or college about the powers of ten. And i used that video to illustrate how there are Different Levels of understanding the world, one of which being elections. And we can understand whats happening at a National Level, a state level, county level or at polling places. And in order to get elections right in america, we need to understand all of those levels and whats happening at all those levels. And i think a comprehensive program of assessing elections in america are looking at all those levels. The final thing ill say, just to close up, is that due to the generosity of the Carnegie Corporation i have a year off to write a book about the elections in america. And subtitle of it, working sub title are voting in america, doing better but feeling worse. The sub title there continues to it be oftentimes a sense that america is in certain places just sort of a Banana Republic and we can never quite get it right. I think the story is quite a different one. The evidence is we actually are getting better. And we do have a positive story. And that there are challenges. The trick is to measure where the challenges are to instrument voting so that we can use our limited time and money efficiently to make elections even better and overcome the new challenges that continue to be thrown at us. So thank you. Thank you, dr. Stewart. You know, i joke about our relative political positions, but really the great thing about the Elections Community is it that we all do come together focused on what are the facts, how can we find common ground, and make the experience better both for Election Administrators and those reporting on it and especially the voters. Even though we possibly have different parties all throughout your community, we all do have the same focus, and that is making sure people get out to vote and that their experience is a good one so they continue to vote. So with that, im going to ask a couple of questions of the panel. And then well go to the audience. We dont have a huge amount of time because those were longer than five minute presentations. But well worth listening to. As weve heard elections do create a lot of data. And i would like to ask, and maybe ill start on this side, what are the two or three data points or sets of data you think are the most important for voters to understand how elections work . Well, okay, so i guess i am on this side. We already established that. So when i go around and i talk about elections, it seems to me that kind of voters dont know the basic contours of how we vote. And voters in the west are amazed at how we vote in the east and vizier vceversa. So one line of controversy and policy, things like voter id, people dont know what happens in other states. People just dont know how you register in other states. So just kind of basic facts like that. Kind of how do we vote, how do we register, how do we check in and do things. The other thing that i find really interesting, and krysha pointed on this, is the next fact on precincts, because this can set some of the context of election, between 100 and 200,000 precincts we have about a million poll workers mobilized on election day. And that is a management challenge. Also the greatest mobilization, democratic mobilization in this country. And i think we fail to appreciate the size of that effort and the importance of that effort. And thats a fact i wish more voters knew hand moand more pol makers new. Dr. Gregorowicz. I also agree with that since on the coast we dont get a lunch break. Im definitely for that. Its all about the lunch break. Yeah, echo, 41 of people in the 2016 election voted before election day, by mail, absentee voting tan that is important. Those dynamics, i agree with you 100 . The other thing is we, data are aggregated up, from a voting machine, polling place, up to the precinct level hand county and state level, and i think there is a need for more localism in some of the data that are reported. People are interested in how their neighborhood voted or how many turned out. So that is something that i think there is an interest in and that is lacking, yeah. Okay. Im going to go to Something Different on this side. You talked about a lot of the data that you, both of you used to analyze your elections and how things are going and where you need to fix things. Do you also use that data in Budget Discussions, you know, in resource allocation, things of that nature . Certainly. And i kind of talked about how we you know, the county administrators they just want to save money. But what we do is we say we are going to save money here, and we are going to use it to expand our services there. And so we certainly do use those analysis in our Budget Discussions. But i just wanted to touch base, on the last question, because i was anxious to answer that. If we are talking about educating voters, the number one way that they get educated was through our websites. And so one of the greatest tools ever is Google Analytics. Its wonderful. Because back in 2012, we redesigned our website, and we thought we knew what we were doing and it was great. We looked at Google Analytics and we are seeing people are still confused and having a hard time finding things. So we adjusted our website to that. In fact, we went even further and created an application where we instead of people searching for information we logged in and fed them the information. So that Google Analytics really helped us educate those voters in a better way. You can answer one or both of those questions. Well, to be honest, yes, data does help us in any Budget Discussion when we go before the legislature, we bring them examples, we bring them our statistics. Those are very, very lep fuhelp any of the discussions about making new laws and budget as well. And the effectiveness of the Voter Registration is something i think that we needed to point out as well is not only in the Voter Registration agencies, but we have online. So there is a lot more we could be analyzing and looking at, but we are doing a lot. And i notice california from listening to my colleague here, that they do an awful lot more than i think we do. But thats one of the criteria that i think is a problem is that we all dont do the same. And so we have smaller comparisons. And so if we can be on the same page, and thats along with the acronyms, you know, just being on the same page for those helps us with whatever we are colle collecting and analyzing it from state to state because we are all different. Thats why these conversations are important because we can hear what other people are doing and apply them to our own offices and processes. Im go to the audience because we dont have a whole lot of time and take a couple questions. Well go over here. First, this was excellent. Thank you all very much. I have a question for dr. Stewart and the rest of the panel. When you pointed out those wonderful improvements and more people actually getting their votes counted, did i get it right that the least improvement was problems in the polling place . And if im remembering, if i heard you properly, im wondering if you and the panelists have any ideas about what can be done to improve that area . Thank you. So very quickly, since the gentleman directed the question to me, but im sure the rest of the panelists have better ideas than i do. Yes, you heard me right. Which is that the polling places that have been kind of lagged behind from 2000, the observation i make when you talk at greater length about this is that if you think about the other big problems that were identified in 2000, Voting Machines and voting registration, each of those have magic bullet. You get new machines and provisional ballots. Those are things that can be implemented from the top down usually and just a matter sometimes of spending money. Polling places are about managing those Million People. And thats more like pushing on wet spaghetti. And management of people is a huge problem, in general. So it strikes me that in many ways the polling place problems which is the smallest in number, are the most vexing, because of the nature of the problem. Other panelists . Only comment i have for the poll workers is what i hear from the 17 counties in nevada is that we get volunteers. We have seniors. We have people who sometimes are new or sometimes have been there for a long time and dont know the new laws specifically. But we do training and we try to improve that training. But we have human beings that are at the polling places doing the work. And sometimes that can be an issue. And other times we are really, really happy they are there and they have the knowledge that they do. So its kind of a mixed bag sometimes. Other questions from the audience . Yes . So dr. Stewart you mentioned the difference between quantitative and qualitative. So id like you to talk about that a bit as it relates to how li Election Officials might do that type of analysis. And mr. Scarpello you mentioned you collected a lot of data and analyzed it. How would you encourage local Election Officials to analyze that data . How did you do that . How would they go about analyzing that data . Just quickly because im interested in the answer to the second question. I mean, certainly well, look, the advantage of quantifying things is that its very efficient way of in a mass way identifying problems and moving in wholesale manner in management. The qualitative data by which i think the question implies talking to people about whats up fills in why it is that you might have a problem here or there. And certainly like any sort of management tool, that there is a variety of ways of approaching the data that are certainly going to be powerful. But i want to hear how they actually got to do this great stuff in San Bernardino. And i have to say your office is not small, we only have about 25 people. We are probably the smallest Elections Office per capita in the country. We only have 28 to serve the population of 2 Million People. In 20,000 square miles. But i think the important thing is to get buyin from the management. To get the county to invest in analytics. And to then participate on a National Level in discussions like this to get education from Election Center and doug chapin organization and to gather information from other Election Officials and learn from each other. I think thats the best way to improve processes. And i think we have seen that a lot since 2000. When i got in this business in 2000, in the bush gore year, i was shocked in omaha, nebraska they were using typewriters, they werent using computer. So we have come a long way because people share information. Answer that . I would say on the qualitative end its an important question. Because with all these jurisdictions throughout the country collecting their data in their own way, i think that is really a blind spot a bit that we make some heavy assumptions what it is these quantitative data that we are looking at are. So i think in terms of qualitatively going out and kind of understanding who is in putting data, how are these systems connected or not connected, what really are they and classifying, are dates being put in realistic or thrown in to fill a gap at the end . These are important qualitative things we need to know that would help on the quantitative end make better use of and sense of the data we have. Question from the audience . I have a question around early voting and data around early voting. Obviously early voting is different from state and how much is allotted from a couple days to multiple days. Either from academic National Survey perspective, is there any data or sense of usage rate over time . By that i mean is there the first couple of days huge in flux, then it wanes to election day . Does it pick back up election day . And does that vary by state as states look to either expand or contract their early voting periods to try to see actually when are voters actually using it at sort of highest usage rate . Anybody want to jump on that . Go ahead. If i understand your question right, you are talking about the early voting and trends that we see from the beginning to the end . Yes. We see usually you can just stop and you can see who wins and loses in nevada just because people come out and early vote. The numbers are very, very high. And one of the things that we see is it depends on if there is a weekend or during the week, you no h you know when people are coming. We have the malls and now have the community centers. But there is that trend of seeing high numbers in the beginning and not so much in the middle, but on the weekends its really big turnout. So it just depends on the days. And you can look, and i do think each state is a little different in their number of days that they have for early voting if they have early voting. I dont know if any one has it at this time. But its very popular in nevada. People have habits in voting. So they get used to voting a certain way. And to change those habits sometimes takes time. What we found in colorado years ago is we made early voting much more available. We went from one location to 13 locations. And that increased. And i think in 2008 we had as many people voting early as on election day. And of course amber took that concept and expanded it to have early voting all the time. So we are kind of going through that transition in california right now. We are expanding our early voting. People are starting to catch on. This last president ial year, early voting in our county exploded. And we expect it to continue to grow in the future. Just really quickly. Many states, i mean states vary in the actual data they release. And if one is inclined to kind of dig into voter files, i love giving a shout out to North Carolina because they basically put all their data on the site and you go and suck them dry. And one of the files i have is what they call, its all within the absentee file, but you can get down to the time of day from early voters, and i think thats a great sandbox to learn about these questions for people really interested in crunching numbers. Question from the audience . Im excited about all of the focus on analytics. But weve had major polls showing voters have low could have had ens in the results of the election. And wondering if there is it an emphasis on turning that analytic eye to the actual counting of the ballots to count more ballots by hand or statistically significant number of ballots by hand or releasing digital images to the public and candidates so that analytic analysis could be made of each digital ballot image. I think people are lacking in confidence right now is the measurement and count of the ballots even though i think its great that you are measuring parking lots, i think the ballots is where people would like to see more progress. Thank you. I would take an answer on that from any of you. Thank you for the question. Anybody want to take that on . I have one answer to that, which is the empirical one. Actually the challenge of the premise of the question but then to rotate it. One of the things i do is i study a lot voter confidence and public opinion. And turns out first of all voters are very, very confident about the quality of vote counting locally and the state level. And quite skeptical about nationwide. And thats number one thing. Number two, the thing that moves voter confidence is whether your candidate wins or loses. And there is virtually nothing from all the studies ive done and my colleagues have done, there is nothing that really in Election Administration that appreciably moves voter confidence. Having said that, i think there are very good reasons for doing things like risk audits and other techniques who assures voters are paying attention, Election Officials that votes were counted properly. So my own take on things, it can be if one wants to hang reforms of Election Administration on voter confidence, one may be disappointed. Because voters arent basing their confidence in the system as far as we can tell from the restesh on Election Administration things. And interestingly we dont count ballots on National Level, right, we just dont. We have 51 elections in a general election, federal general election year. And kind of goes back to your comment about process as well. That will is a lack of understanding of this very complicated process. Voter education. Yeah. I would say what you are looking at is a perception problem. So its not an actual problem, its a perception problem. So rather than changing the process, what we have to do is educate people. What we find when we have that candidate that loses, they come in the day after the election and they say had is all crooked. Then we walk them through and we educate them, very transparent, educate them about every single aspect of what weve done in that election. By the time they walk out of there, 99. 9 of the time they are satisfied. And so its the same with the public. We need to do a better job of educating the public on the process. And if we do, i think theyll have a lot more confidence. Other questions from the audience . Yes . How are you guys using data for activities, if we can talk about that, local and statewide . She wanted to know how you use data for list maintenance activities. Michael . Well, list maintenance is something that has changed a lot with the help of american vote act in 2002. County, we had, what, over 3,000 county databases now we have 50 statewide. And california just made this change in 2015 and 2016. And with a statewide list, it becomes a more efficient list. We can look at cross county moves, duplicate voters, et cetera, et cetera. And so that has really comealong ways in the last several ways. And that is going to expand and hopefully we see a consortium hf counties or states that have put their list together and we see cross state moves. And so that data will get even better and better and time goes on with the use of mass databases, et cetera. Well, we have come to the end of our discussion here. We could have gone on for a lot longer. I think as election geeks we could endlessly have discussions how we use them. But that was a fantastic discussion and thank you for the panel ess to be here. I appreciate it. And also thank all of you, our audience for your questions. And well now take a 15 minute break and reconvene in this room at 11 15. Thank you very much. [ applause ] coming up here on cspan3 hearing on what to do with the gang ms 13 and u. S. Response to transnational organized crime. Thats followed by a discussion about innovations in higher he had education. At 8 tonight American History tv prime time with look at president lincoln friends and enemies and how he handled both. The president of the United States. [ applause ] tuesday night, President Trump gives first state of the Union Address to congress and the nation. Join us on cspan for preview of the evening starting at 8 00 p. M. Eastern. Then the state of the union speech live at 9 00 p. M. Following the speech the democratic response from congressman joe kennedy. Also hear your reaction and comments from members of congress. President trumps state of the Union Address, tuesday night, live on cspan. Listen live on the free cspan radio app. And available live or on demand on your desktop, phone or tablet at cspan. Org. The House Homeland Security subcommittee on Counter Terrorism is looking into what it will take for the u. S. To get rid of transnational gangs. Talk with federal Law Enforcement officers and state department officials. This is about an hour. Good morning. The committee on Homeland Security subcommittee on Counter Terrorism and intelligence will come to order. The subcommittee is meeting today to hear testimony from the fbi, the department of homeland secured, and the department of state and federal