Order. For 40 years, cspan has been providing america unfiltered coverage of congress, the white house, the Supreme Court and Public Policy events from washington dc and around the country so you can make up your own mind. Created by cable in 1979, c span is brought to you by your local cable or satellite provider. Cspan, your unfiltered view of government. Next on cspan three, a discussion on federal standards and oversight of Artificial Intelligence. The National Institute of standards and technology and the Information Technology and Innovation Foundation center for data innovation is the host of this discussion. Good morning everybody. I am a senior policy analyst with the center for data innovation. We are a nonpartisan, nonprofit Public Policy think take focusing on the intersection of data, technology and Public Policy. We are based in washington dc. We are affiliated with the Information Technology and Innovation Foundation which has been ranked for several years as a top science and technology think tank. We are happy to be here today to be partnering with this event. Particularly this panel. I am joined by lynn parker, the assistant director of Artificial Intelligence at the White House Office of science and technology policy. Next we have jason, the general manager for the Corporate Standards Group at microsoft. And anthony robbins, the Vice President for north American Public sector. We have a great group of public and private sector experts that can talk about the activities going on in the Sand Development phase and the people helping define u. S. Leadership in ai. Before we get into the discussion, we will have audience q a and i want to do some stage setting here. The term ai is often used to describe two different but related topics. There are technical standards. Things like reliability performance, accuracy. And then there is the oversight. These are very different but related things. Standards are a prerequisite for oversight. The oversight side of the conversation receives a disproportionately larger share of the attention from policymakers. Concerns about algorithmic bias, the potential for these blackbox systems to be running amok without knowing what is going on is what is dominating conversations and policymakers understandably want to address the concerns. Unfortunately, this privatization of oversight is seemingly coming at the expense of the focus of standard development. The activities required to develop standards requires a really robust Scientific Understanding that can serve as a technological underpinning for this oversight. So when we say things like we want a mandate for algorithm transparency was is what some people are calling for, right now algorithmic transparency does not have a definition. We dont know what that means. We dont know how to compare the transparency of one system to another. And rushing to make those rules without actually doing those scientific legwork behind it is going to be shortsighted and any rules will necessarily be arbitrary. So i guess the challenge for us is how do we get nontechnical policymakers to care about this really important technical work. And it is a challenge. Im sure folks know that. But we hope to get out of this conversation today and Going Forward, to educate policymakers about the importance of this kind of scientific legwork in shaping the future of oversight of ai and all the concerns that there are in the public about the potential for the misuse of these systems. We have to make sure that translates into momentum for this kind of scientific investment. So to start off, i am going to tee up with a pretty easy question for the panelists to get the ball rolling. What is your primary focus when it comes to ai Standards Development going on the line . Are you working on developing standards yourself . How are you engaging the community . What did they mean to your business or to your role in the federal government . Good morning and thank you for being here. This is an important activity. My role in this process right now at the white house, i am the assistant director for ai which means i oversee the white houses activities an a i particularly the promote side of ai. One of the important areas i am working on now is all the deliverables that are called for in the executive order and you have heard this morning the main deliverable we are discussing today which is the creation of a plan for how the federal government should prioritize its engagement and teco health Standards Development for ai. This obviously is one of the key actions in the executive order. It is recognized by the administration as an area that we as a nation need to get engaged and there are a lot of good reasons for doing that. We could go into that as we go through the panel. But right now, my role is looking at the great work being done and leading the way here. Certainly the rfi is an important way for stakeholders to provide feedback. We encourage you and your colleagues to provide that feedback. This workshop is an important opportunity to hear from everyone about what the federal government should be emphasizing. As the plan is developed and is issued for public comment, we encourage you to respond to that and provide feedback. I am cheering on all the great work being done here and everything you are doing to contribute to that. Good morning. It is a pleasure to be here as well. Thank you for nist getting this important process underway. At microsoft, i am running a team of Global Practitioners who are involved in standardization at the International Level through primarily, in relation to Artificial Intelligence at jtc one and we will be talking more about that throughout the day. I will point out that as an organization, microsoft is looking at Artificial Intelligence and the role that it will play in a broader spectrum and recognize that standards are one element of the ways that the Technical Community will deal with interoperability. People also looking heavily at Opensource Software and the methods happening there were people are working on exchanges. And in the policy environment, i absolutely agree and support the need for thinking about accountability. And when you go down this path of accountability, and principles or any of the other countries around the world looking at this, we are also recognizing that the relationship between the regulatory approach and the accountability comes down to the criteria by which you measure. That criteria is going to be fundamentally predicated on the good standardization work being done. So i look forward to this panel and thank you again for joining us. Good morning. My name is anthony robbins. For the last 30 years, i have spent my career at the intersection of government and commercial industry. Most of which has been with Silicon Valley companies. In my role at nvidia, i am the person in the field that is trying to predict, guide and coach the federal governments progress as it relates to ai and make sure i do the right work to interpret the need from nvidia and commercial industries. So we were excited in february to see the executive order that was signed on ai. If you look at the executive order, you might convince yourself that it was the first time that an executive order had been signed on one of these Big Technology waves. These big waves being the work that we did with client servers early on to mobility, to cloud to ai. Ai is one thing different than the previous that came before and it is bigger than all of them combined. I will grab on to something. We do spend a lot of time as people and as leaders in this Community Talking about some of the challenges with ai and how secure and robust it is. And where the biased exists and things like that. I think it is really important for us as leaders and especially in this room and in this community, to spend a lot of time talking about the progress that is being made and the things that are being done to help mankind and improve the planet and make a significant contribution. There are children literally and i will say that for High School School kids making a profound impact on the world that we live in and the work that they are doing around ai. As much as we want to go act out of concern and we want to address standards and international and know that the role that they have with the strategies i think it is important for us as leaders to celebrate the amazing progress that is underway because if you look back into the history books i guess the federal government has been touching ai since the 50s and we had a major breakthrough since 2012. I am excited to be a part of this panel and make the contribution. Thank you. The rush to develop the standard is new but the recognition of the importance we have seen this cycle happened before with previous generations of technology. So what kind of precedent exists for how we will be approaching the standards. What other models can we look from where there are fundamentally new challenges that we have to address. I think if you look at historically the way that standards have developed, lets look at the computer era Information Age where we have a lot of technology being developed and we have a lot for the technology. Historically the worldclass ideas were coming from American Companies. Certainly we very much support the voluntary consensus open transparent industry led Standards Development process. They were coming from American Companies for the most part. The folks that were at the table conversing about technical standards were primarily American Companies so the process has worked out well but i think we cannot assume that because it has worked out well for American Companies in the past it will continue to work out while Going Forward. Now the landscape has changed. The economic competitiveness has changed. We have strategic competitors who are also recognizing the importance of technical standards. I think that the process that has worked well in the past, we want to foster that Going Forward but we have to recognize that we are in a new climate now. That Global Competitiveness requires us to be more intentionally proactive in promoting that process so that we can make sure that all of our good ideas are coming out of the United States and have an equal footing. We are not afraid of competition internationally but i think the standards process cant just presume in some sense the federal government broadly speaking is not recognizing the importance of standards because we presume that the process in the past continues to work well Going Forward. I think that the fact that the process has worked well for the computer innovations means we do not need to let our guard down and presume that we do not need to be more proactive promoting the open process of developing technical standards. I concur that if we use history as a guide you look back and say how is it that u. S. Industry in the sector has been so strong. Theres no question that one of the underpinnings has been a dynamic standardization of the environment. That has been created by keeping open all possibilities of the spectrum as it approaches everything from consortia to national processes to the International Processes. U. S. Industry has been adept at making use of that full spectrum depending on what it is that you are trying to get done and what time. I want to take a step back to my opening comments and recognize i agree that things have changed but they changed in a fundamentally different way than people are thinking about. Im going to mention again open source software. You will see a new factor that will play in how technical interoperability will be addressed. Engineers will move to a much more rapid pace of technical interoperability work. That does not then take away from the role that standardization is going to play but it means that things are fundamentally different not in a way that things will be less but it will be function of how engineers and contributors come to the table and played with innovation and ideas that bring about a foundation of understanding that the issues are transparent and you can understand what the technologies are and policies can be built. That is a misuse of standardization if you put up barriers. It will be a function of using standards in a way that they are strong and leading discussion but you do want to preserve that system as much as possible following the principles that have led to the great success. To me i think theres a great deal of merit looking at where the standardization has been but recognizing that there has been a fundamental change but its about who is racing to get the first standard. Those of us in industry will tell you that they can be marketmakers but the products come down to the aggregation of dozens if not hundreds of standards and it is really about the layering of heil value work above standardization that is about market success and so i would put it in a slightly different context. So try not to repeat anything that has been said, what have we learned about standards as they have come across. Most recently would be the work has occurred across where 160 Different Countries have rallied around different aspects of standardization. That is a pretty good model as we think about standard and scale. Where the model may break down a little bit is it is still not at the scale that we are talking about here in the complexity we are talking about with respect to ai. We are talking about nations that want to create immense value for themselves. We are talking about concerns with the application of ai so we have to deal with that. The other thing is as i mentioned in my opening comments that when we think about the society only think about the trust and security an important aspect of the adoption will be the improvement of societys belief and trust in the technology and its ability to be there for. Standards play a really Important Role. I think that nist is really important. There are a couple of things. It is important to the progress that we have made the other thing i would recommend is that it is not just about the standard itself. Its about the use cases for the federal government. On the civilian agency side we think about waste and fraud and abuse there are Different Things we may consider out of the data and citizen privacy then maybe on the department of defense. Or how we think about Cyber Security. So as one of the reports that came out the benchmarks and standards and prototypes i think it is important that we get started and build some prototypes and Lessons Learned that relate to how the federal government might adopt and deploy ai because it may inform some learning and the position on standards. I want to pick up on a team that all of you addressed on this idea about u. S. Leadership and standards of development. Its one of those issues that we talk about folks in industries than in the International StatusBody Community that these organizations exist they are sending huge amounts of delegates that are coordinated. It is a very clear specific goals in mind where the u. S. Approach is different we have industry representatives and we are not trying to champion any particular company but create a competitive and fair open standard. The concern we keep hearing about is that china is much more effective about potentially ship in the scales if they choose to to favor them in an anticompetitive way as they have done in the past so what is the solution here for the United States and how can industry and government Work Together to kind of create a fair and even playing field. I am happy to start but im sure everybody has good opinions on this. It seems to be we are honorable today. I think that there is zero evidence that the chinese have an unfair advantage in the International Standard system. The reason that i say that is because the principles by which most of the Standards Organization that we are talking about follow very explicitly prevent or have rules in place to diminish dominance. I speak about this as a large corporate player. I recognize the dynamics that are playing these bodies. As industry we would far rather have them involved in the International System where they are engaged in one country one vote dynamics or in an environment where they need to bring ideas to the table like the germans and americans and japanese and argue them out in the community rather than put them behind a wall in the country and use things like one belt one road or trade agreements to impose their system on others as part of financial engagements. Those outcomes will be far worse for us than encouraging all countries