Analysis, Artificial Intelligence and threat forecasting. This is part of an intelligence and National Security conference, cohosted by the cia and George Washington university. Thank you. If i could ask everyone to please find their seats. So i get the privilege of actually moderating this next panel. And i can tell you based on the conversations weve had already, youre in for a real treat. These are some really big brains tackling some of the really big issues the Intelligence Community faces. While were looking at, our focus is on forecasting conflict and instability. I think youll see this will expand in many other ways to look at the role that technology plays for the red. The bad guys and for the blue, the good guys, and how we can try to get out in front of some of these issues. Awesome panel. Starting with dr. Stay ecy dixo. Thank you, stacy. Andrew holman, who is Deputy Director for Digital Innovation at cia. So all Things Digital and a very cool reorg within the agency. Rob high, who is Vice President and chief Technology Officer at ibm watson, thank you, rob. And last but certainly not least, fran moore, who is president of fpm consulting and a former cia director of analysis. So weve got an Amazing Group hire. And just to very briefly set the stage. Today is the 60th anniversary of the launch of sputnik, which clearly triggered a space race. Recently, the head of the, head of state of the same country opined that whoever owns Artificial Intelligence owns the world. In essence. This is also on the heels of china launching a Quantum Entanglement satellite capability. So it seems like, to paraphrase samuel clemons, where his, the future may not repeat itself, it does tend to rhyme. I think were seeing some of the rhyming now. But im really optimistic that the good guys are getting their arms around this. So lets sort of peel back the hype from the reality. What should we be thinking now, where is it going, and im not going to ask you to answer whether Vladimir Putin is correct, but clearly, you have a h head of state highlighting this. Stacy, you want to kick us off . Ill start off with a basic definition, Artificial Intelligence is when machines mimic people. We see it every day in our lives in something simple as mapping programs, getting us from point a to point b, personal assistants on our phones. Were thinking of analytic tools that can help the analyst make decisions better, learn lessons from them more clearly and better. I would agree that there is i wont speak specifically on what putin said, yes. So whoever rules a. I. Rules the world . Potentially. It open up the possibility to automate a lot of things that we havent done in the past. Certainly, theres a lot of investment around the world in Artificial Intelligence, and we have to make sure were monitoring that and keeping up with that. We have great researchers and scientists and practitioners in this country that i think will be competing well against those in other parts of the world. So im optimistic that ais really going to have more benefits to society than threats, but we have to make sure that we are watching out for the threats as well. Andrew, i minean, ai is more than simply processing algorithms and technology. What is the agency doing right now to advance the field of antis pa tor eye intelligence and what role does ai play in all that . So as an intelligence organization, the first kind of order of value that we provide is revealing the secrets denied to us by our add vir sarees. So exquisite intelligence collision, signal intelligence, the first sort of business for us. Increasingly, what our policy makers expect and the value that we provide is help us understand how major world events unfold. What are the major muscle movements that move to periods of instability. So we also have an expectation and can provide value in taking what we know from many decades of social sciences and using advances in the yew pick wit and power of computing and sensor data to learn from that, that social science experience and apply that computational ability to enable the higher human cognition of our analysts and our officers, and buy time and space for policy makers by helping them understand where those emerging conditions changing that require a noix,foa shift, a pivot. So the demands on us increasingly are on the more diffuse muscle movements where its a higher volume and fragmentation of data. Oftentimes public sentiment, many types of data sources, diverse sources that we can apply them, computational methods to and Computing Power to enable again that higher human cognition. And andrew, and i wanted you to touch on this and then bring fran in and go to rob. But, you know, isnt expectationsetting a big part of this discussion . I mean, the Intelligence Community, after they miss something, its sort of a witchhunt and what have you, and a lot of fixnger pointing. But the reality is youre estimative, youre not claire voints. Absolutely. People cant call what they see because theyre afraid they might get it wrong. We try to manage expectations by nightinoting, were talking improving our forecasting to reduce the uncertainty, manage the complexity and yes, we can. And we have made advances in improving our ability to forecast unstable events, and that will be a consistent investment we make in a field of forecasting and antis pa tor y e intelligence. We have to invest in the field and develop tool suites for our analysts and officers to manage that complexity, were always going to be in that constant state of investment in the field as it advances. So important to understand the difference between prediction and forecasting in that context. And fran, youve lived, and youve been the leader of so many in this community at the Central Intelligence agency and elsewhere. I mean, looking at it through the eyes of the analyst as well. Where do you see this kind of playing out and how do you enhance, how do you empower their voices, even if theyre occasionally wrong . I think you got to get back to the heart of the analytic mission. The analytic mission is to provide insight to policymakers so that they can make sense of events on the ground and either advance u. S. Interests or protect u. S. Interests. So might a policymaker like it if we were in a position to predict, to give them an accurate prediction of an event . Absolutely. If were in a position with confidence to provide that is correct that, ultimately no surprises is what were looking for. We recognize if you could predict that will, you would be in las vegas. You wouldnt be here work being no the Intelligence Community. But what we need instead is to understand what is happening in ways that give us space to make decisions. Either to do something that advances interest or protects it. So i think for analysts, its actually the understanding that sometimes were not going to be in a position to make that prediction, and the best we can do is to provide understanding of whats taking place on the ground. And i think Technology Offers us an opportunity to extract from information what is latent knowledge now, l latent informan now but could become actionable in that context. When integrated with what experts know. So whether were talking about Artificial Intelligence or forecasting, its the marriage of the power of that computing capability with the subject matter expertise. [ no audio ] ultimately everything we discuss today has to be useful to that end user which is [ no audio ] and ultimately policymakers to make good decisions. Rob, so it, at ibm, youve got some massive processing capabilities. Youve got some Amazing Technology and thinking around these issues. Where do you siee coming dow on some of these matters and how do you make it work . How do you make it work effectively . Because there is a lot of buzz around the technology but theres also [ no audio ] what are your thoughts on all this . And not to put you on the spot, but you see the real animation coming out of uncle sam or out of the private sector . Ill come back to that question. But perhaps to answer your question by [ inaudible ] first of all, i think not withstanding everything else, ai, and certainly, you might hear from some of the celebrities who like to weigh in on this topic. The purpose of ai is not, our current focus on ai is not about replicating the human mind, not in an entirety, not in a general sense. Its purpose primarily is around what i call augment ng the human mind and amplifying the [ no audio ] that allows the information coming in. Were seeing in the field to be distilled. And yes, theres some reasonable similarity for the reasonable processes and nonetheless, we can activate the human mind, to see perspectives that might otherwise be overlooked. To help people see through their buy as. Biases, the trigsers that cause us as humans to have a new idea and to bring that to whatever problem were dealing with, whether thats in intelligence setting. In private sector, of cork turs the similarities are to financial investors, recognizing that most of the decisions are going to be based on predicting and anticipating and forecasting what the markets going to do or in the case of health care doing the same thing for identifying the right treatments for a patient, recognizing that in this case youre predicting and anticipating and forecasting how this patients going to respond to that treatment and likewise, what kinds of side effects are going to experience and so forth and how you deal with that as well. So i think in all these cases what matters is that were concentrating on the parts of the human cognitive process that today, for most of us represents a limitation in our cognition, limitation in our awareness, limb tag limitation in the way that we reason through a problem and bringing ai in at that moment where its ability to distill information will make a difference in the process. Rob touched on a critical point there, and ultimately most tech breakthroughs are at the edge. Looking at it from iarpas perspective, are you starting to sigh more see more in the convergence . Yes, youve got different d dimensions. The adversary can express more pain more cheaply. But id be interested from iarpas perspective how you think about applying some of this, and in particular some of the convergence between human tlerngs technical means and how that all comes together. We started out some of our research in forecasting looking at data itself. What was available from all the publicly available information out there that you can derive intelligence from. We coupled that with human judgment. How do you lit people make decisions about events in the future and grade them in their success success or failure. Machine learning that can parse through all of it and that comes faster than humans can process it oftentimes, how do you put that together, the best of machines together with the best of the human forecasters to figure out if you can make more powerful forecasts. To give Decision Makers more time and more space. We see a lot of promise in bringing it together and allowing machines to help analysts think through the processes that theyre using both in coming up with assessments and looking at the Lessons Learned. Walk them through the processes in a way that sometimes is difficult. In theory, the machines can be developed to be unbiassed in their approach. But they are, they are created by humans. So youd have to watch the human bias that gets put into there. But in theory, you can help bring an analyst through making designatures decisions. For us, the power is in the combination. Seeing it in the workforce at the agency, any early Lessons Learned and observations that i think are important . Because the, a good collector may not even, got is a big part of what they do every day and they cannot necessarily convey that. They just know it when they see it. How does that sort of play out . Your world . Couple points. One is benefits of this field and of this technology and the marriage of the technology to the forecasters. Theres utilities simply in including for our operators in enabling their Situational Awareness. And, as you look to the way that the sensory environment globally is evolving and the digital dust that our operateeaors leave beh, theres greater opportunity to have a greater Situational Awareness of the environment, the targets, the tracking. Were trying to find who we dont yet know, but know from analogous behavior, maybe theres a pattern in there or something to detect to find evolution of a plot. But the other point i would make is, and to the point that you made to rob was wheres the greatest innovation happening, in the federal sphere, the government sphere or commercial sphere. I noticed rob didnt answer. You about i want to hit this, because for us, its really an important point that one, its not an either or but deemands a close partnership. We simply cant we do this internally behind our firewalls, were just not going to be the advancers in this field. Weve got to have that very close partnership. And you are leaning forward, not only with the traditional Big Industries that are doing phenomenal work here and the Defense Industrial space and the integrators but also silicon valley, right . Yeah, theyre partnered very closely with silicon valley. We just have to have that close partnership. Awesome. And fran, i mean, sort of having now been out of the community for a little bit, anything youre seeing that you would have, im not suggesting done differently, but just sort of having that private sector, because i think that really is a critical point. Its become a little trite, Public Private partnerships, but its also genuinely important, and i think that whats important is not the partnership, but its the people that are actually partnering, and they understand, they step in one anothers shoes and know what they can, what piece of the puzzle they can bring to the fight. So any things from your end . One of the most incredible insights once you leave the organization is the isolation, as much as we try to reach out in the organization. The isolation of working in a secure environment like cia creates and the intensity of the mission and the need to on a daytoday basis be completely on juyour mettle to not miss Th Development or event that you are responsible for warning for. I think it tells scopes your ability to focus in an i have in insular way. Its very difficult, no matter how much the agency tries to help officers reach out. So, when i listen to this conversation, whether were talking specifically about working with the technology, working with data scientists to be explicit about the process through which subject Matter Experts arrive at their conclusions or even in some of the more traditional, what i call lowtech ways that weve developed over the last 30 years for anless alysts. There is always an ahha in those exercises. The debate always has been do you provide the output of those processes to the customer . Or is it use the simply to make sure that the analysts and tour has widened and that its open to the possibility of change. So when i look at what im seeing in the private sector is very often those processes are incontinueded to improve the interm quality of the work being done. Its not considered an enduser product. Its a qualitycontrol product. So one of the things i would say is that debate in our own organization on the analytics side, if we make the investment, does it have to produce a prediction thats going to lantd on the president s plate . Or is the process itself and the ability to challenge othur assumptions the actual end goal that ultimately our more it traditional means are informed as possible. And i think thats such an important, its more than just warning. Its context, setting the stage. So you can make sense. Right. Of environment. And its Situational Awareness. The freedom has to come first inside the organization to think expansively without being constrained by the need to produce for the enduser, whos the customer. One thing i might note that the agency did that i felt was quite innovative and i understand quite effective is getting their analysts in the field and getting those who are operators a better sense of what the analyst can really provide, and that sort of, in a tech environment, how many times have we invented tools and the operators werent part of it . And you sort of have the operators coming up with macgyver Like Solutions to everything. So we really do have to move if that direction. Rob, id also be curious, director pompeo in his opening remarks referenced the fact that collection continues to outstrip our ability to process data. And h and this is an ageold signal to noise. We got a lot of data, but what data really matters . Data is the new oil. It is gold, but not necessarily if you dont know how to mine it. So id be curious what youre seeing there and howatson can be brought to this fight. And actually, i need to bring out that this is the exact same thing occurring in the private sector as well. That it is motivating. Not only ibm, buff realt really entire industry to invest as much as we can in ai. It is about how do we get insight from all the information produced out there in every aspect of our lives. Just think about the changes that have occurred in our lives over the last ten years with the introduction of smartphones on a daily, its not even just a daily interaction. Its an every moment of our life is spent, away from it is really spent somehow engaged with these devices. And that is both an opportunity meaning thats a place in which we can really get advantage by having a system thats able to ingest and make sense of information, allow us to make daily decisions, small and large, but its also the source of a lot of information that were pro