Electronics association and the intelligence and National Security alliance. Thank you. I was going to try and get you fired up here because the initial plenary session, we were a little bit subdued. So i want you to feel free to get enthusiastic here. We have a great panel coming. Lots of cheer for them. We are going to have a great discussion. So welcome back. Before we jump into this next session, let me remind everyone that for the plenaries here in the maryland ballroom you can email questions to the panel. And pope flee we will have a logo up that says questions intel summit. Org. By the miracle of technology they are going to appear on an ipad with vince. Vince, if you didnt get the word at some point larry is going to tell you last question. Okay . So it is now my pleasure to introduce the moderator of our Defense Intelligence plenary, Lieutenant General vincent stewart, u. S. Marine corps come on, come on [ applause ] retired. Thats the part i was waiting for. [ laughter ] and vince is the founder and ceo of stewart global solutions, an International Consulting firm. General stewart retired from the u. S. Marine corps in april of this year after more than 38 years of service to the nation. Join me in thanking vince for his service. [ applause ] a career intelligence officer, vince served as the 20th director of the Defense Intelligence agency where he was the first africanamerican, the first jamaicanamerican, and the first marine to hold the position of director dia. [ applause ] general stewart earned masters degrees in National Security and strategic studies from the naval war college. And in National Resource strategy from the National Defense university. Please join me in welcoming former director, good friend of industry, and my friend, Vince Stewart to the stage. Over to you, vince. All right. [ applause ] i dont usually let folks waffle on about my career like that but since this is chucks last time i let him get away with that a little bit. Thank you for joining us this afternoon. We have a distinguished panel here this afternoon. If you dont know them, you are in the wrong place. I am not going to ask them to do an introduction to tell you would they are and where they are from. We have susan white, the deputy at dia. Jeff kruse, who is now at drake. And trey with it worth is on the joint staff. I want to go through a quick round of questions. I want you to have your questions ready for followup. What i will do with the questions is give them the chance the set the stage, talk about what their highest priorities are, what their concerns are, how industry could help and on the back end we will ask the question i used to get asked all the time, what keeps you awake at night. We will do the table setting, asking the questions and we will do this kind of quick pace, fast break basketball. If a question is answered and we dont like the answer, we want to follow up, we will jump right in. I am going to start with ms. Bingen. Lots of conversation about human machine teaming, a. I. , Machine Learning and its implications for the Intelligence Community. You have described project may have been as the pat finder for a. I. Machine learning. Can you give the audience a sense of where we are with project may have been, jake, some of the challenges you have seen and some of the ways that we can be helpful as partners. Sure vince. Thank you. Thank. It is great to be here. Love this even every year. I am happy be the here participating. May have be maven. I like to previoface it by sayi we have a ways to go. Maven is a path finder. When we were here two years ago we were on track, within six months of authority to proceed to get an initial minimum viable product capability out to the field. Since that time, they have scaled pretty significantly in terms of increasing algorithm performance. Scaling geographic locations. My boss was just out in theater a couple weeks ago and got to see it in action. Scaling across different isr platforms, we are in dozens of legacy platforms now. And then scaling across different missionaries, not just full motion video but other areas of intel. We proud of that but have a tremendous amount of work to go. Just a couple of points i would like on what we are learning, i think the key about bag path finder is you learn and you apply that to broader activities that the jake in particular is undertaking. Is one, i would say get your users involved early. This is for us it has been all about fielding. Get that minimum viable project, Agile Development in practice. Get that out to the field, work with the users and build from there, scale it from there. Second thing i would say is where we have learned the most has been the amount of i. Pipeline. Righting the algorithm is probably the easiest part. The up front data access, data labeling, incredibly manual. Back end of weapons systems, accreditation of the confirm the o. D. Systems. That is where we are learning. Now we have got almost two years under our belt now. It is out there in enough see change. We need to see work flows changes we need to see efficiencies we need to see the culture change as a result of bringing a. I. Into the field. Thats what we are looking for in the next year. Okay, thanks. Suzanne . Much has been said about building the analyst of the future. While the analytic trade craft will largely remain the same the analysts of the future will be inundated by a vast amount and growing amount of data. With all that data that the analysts will have to swim their way through, what is dia doing to think about Big Data Analytics . Where does mars a program we all hear about, fit into this analytic framework . There is no question that analyst of the future is something that dia is trying to really hone in on and define more specifically. Clearly Mission Number one for us is providing analytic assessments to all of our Customer Base timely, accurate, insightful, comprehensive understanding of the information out there providing insight and ideally understanding the operational environments. But you are right, the data thats available to the analysts to craft those assessments only increases by the minute. Not only that but the environments that we have to assess are evolving. Space is a great example of that. So the types of data and exquisiteness of date are challenges for the analysts. You named mars. Certainly top of the list if you will, of the initiatives that dia is taking to try to High Pressure our analysts make sense of all that information and fundamentally deliver more insights. Mars is step one. For those in the room when may not be familiarities our effort to develop and expand the data environment in which our analysts it analyzes many more data bases in one fell swoop adjusts for Additional Data sources we dont have and hopely will leverage the a. I. Data as it matures and allow more rapid and creative insights on the part of the analysts. Mars is not it. Open source has been around for quite some time. What dia is doing new is structuring our effort against open source. Even just earlier this year general ashley made some decisions to establish more officially a career path if you will for open source collectors in dia. We are establishing an open source Integration Center we are putting structure around many of the efforts that existed in pockets around our enterprise. The intent there is to bring increased discipline around the activities that many of our analysts and collectors are taking now to drive maturity into the process and ultimately to allow the analysts to leverage open source in a much more official trade craft compliant and useful way. Thats number two that i would highlight. Finally i would highlight j wicks. None of this happens, none of this is effective without a strong j wiks foundation. It has been around for 25 years and it has to continue to evolve to customer demands custom again are only increasing even outside the i. C. User base. The types of things that people are using j wic for are evolving and j wiks needs to be modernized to keep up with that. We are untaking some investments and some approaches to do exactly that. The time is now, given need to support mars, to support owe sent, and over all annist of the future things. I am probable flee going to come back to j wiks a little bit to dig through what the future of j wiks might look like. I will come back to that. I am going to skip jeff for just a second because admiral with it worth and i talked a little bit about the challenges of producing intelligence in a timely manner in this environment. And one of the thing, as suzanne was talking that i was thinking about, when was the last time we used a National Intelligence estimate . Thats a pretty static document thats almost obsolete at point of publication. Can you talk a little bit about the challenges of generating realtime intelligence in an estimated sort of environment n a spiral sort of talk about some of the challenges of developing intelligence in this crazy world, randle. Yes, thank you for the question. The chairman and the joint staff have made it clear that we need to focus on global integration which means we are not going to become fixated on regional conflict just as a regional conflict. We are going to look at going to look at opportunity as it play applies to other adversaries taking advantage. That provides so many permutations of estimates. To your point, when we draw a National Intelligence estimate or something that stays on the several as the 2019 edition, when you are trying to get that integrated and that global, it presents a problem. How do you go back and fill in variables that contributed to that particular estimate with some real time urgency and speed . Enter what we are classifying as our joint concept for intelligence operations. And to your point, it is a spiral estimate. It is basically trying to get those monte carlos going with multiple variables so we can rest assured that at least at some juncture all of the data is being looked that but we are being cued to at least possibly reassess for a new estimate. In kasey our Industry Partners didnt pick up on that, static assessments, the static summaries that drive Decision Making doesnt fit in the modern very dynamic world. How does industry help us to build that very dynamic spiral estimate as the world changes, as the variables changes . How do we see those different permutations so that that drives Decision Making at the policy and strategic level . So thats an area that just having almost every publication that we put out there as an intel estimate begins to be obsolete at the time of printing. How do we make in a more dynamic . Thats the requirement. I am going to leverage jeff a little bit here. I heard lots a questions today and comments about the great fire competition. Jeff spend two years at a j 2 at end all fade com. He got to see one of the great powers carry out its strategy. I am going to ask you to to describe what you saw specifically as china walked through its strategy. I will give you an extra 30 seconds. 30 seconds. Easy for any intel officer. Let me add my thanks to everyone who is joining us today. Great conversations. I think i would preface my observation of china with two quick stories. 2014 to 2015 i happened to be running the keg for general breed love out at uconn and working through what was a similar problem set. Russia was going through a revolution, had conducted a handful of operations that they wanted to dodds under a certain amount of cover. And the question for us is how do you illuminate that and then how do you convince the rest of your allies and partners or your own department of defense the threat that in this case russia presented . The second piece is way back when, going back ten years when i was a student at the National War College we have this simple piece of china. Two column, by two rise, china continuing to rise, china unable to main tin the moment up thin china being friendly to western powers or china being an adversary. Really, a decade ago nobody was picking the quadrant that was china being able to sustain its rise economically and internationally and domestically and militarily while maintaining a cha toning the international order. 2016, showed up at m,anted me t the story. The first piece i came to grips with, with all the assessments we had made a decade previously, china had far surpassed all of the estimations we had made. We used to rely on a handful of thing about china, that we had a quantitative and qualitative and experiencial ability to do whatever they needed to do and go beyond. We were far more advanced in all the areas that mattered. Turns out china had five year plans methodically over 25 or 30 years systemthatically addressing the areas where we had the advantage over the years that they could observe. My first piece, the first 12 or 18 months at pacom was illuminating and really understanding where china is in that journey they were on and what kind of a threat it presented. Great partnerships with the national community. And it is remarkable what china was able toll do in that periof time. The period at the tail end of that first 12 to 18 months was the crafting of the new National Defense strategy. That paralleled back to how do we convince the rest of the department of what china looks like and what are their goals and objectives Going Forward . This piece that i would offer to you is that at the macro level i considered china an open book. Their academics write considerably. They plan as a nation. And they move out in a very large way, in a very public fashion. So we know their National Objectives are to restore or achieve regional hedge emmitt, to displace the u. S. P uu as a gl plower, and to change some of the International Organizations out there to be more advantageous to chinas authoritarian model. So if we know that, it becomesen in what we worked on for the second 18 months which is the hard part, what do you do about it . So the piece that i would offer to the crowd here is that while we used to rely on china being able to build a lot of stuff but not being able to train with it, not being joint enough, they have worked through all of those issues and they are presenting as we have seen in a most recent exercise which concluded not that long ago a very dynamic capability to employ highend equipment against a scenario that should cause a variety of their regional partners to take note of what they are attempting to do. I would say china has tried to keep their intentions below the radar scope so it doesnt elicit a kind of response that we would normally take, and they were certainly counting on an ability to use ties in the academic, business, and other communities to make it challenging for the United States to make some decisions about what we are going to do to address that. Where we sit today in my view and the challenge for all of us is what is it that we can do today we are in a competition space today. China would certainly like us to take a little to no action to focus on the urgent versus the long term issue. I think what i would offer is the important piece for us is to understand great power and competition and do well in that conflict today, posture ourselves so, you know, two, four, five 20 years from now we dont fine ourselves in a position that we dont want. Tee it up that way. I would love to jump on that. We do have a new secretary, deputy secretary, the good thing here is they have maintained our departmentwide focus on the National Defense strategy. We are not seeing any changes there. Prioritizing china and russia and modernization. I think jeff hit it on the head is we are in this strategic competition phase right now. The department we do order of battle well, we do force on force well, we buy aircraft and tanks and ships well. It is the gray zone space that we dont do as well. It is looking at the neck us between economic and National Security nexus between kmi and National Security. I look at chinas one belt one road priority. Roadw roadways, train tracks, telecommunication capabilities. All of this is happening now. If we cant be competitive in that competitive space you are setting yourself up for not being successful if and when the balloon goes up. Just another thing i would mention there is Just Technology theft as well as jeff hit on this, too. We are seeing them in a pretty concerted effort going after specific technologies. In the past we have been fortunate to be able to maintain the technology and thus military advantage. When you are stealing it now we are going to see the same technology that we are doing r and d on now we will face it five or ten years from now in the battlefield. That Technology Advantage is eroding. We are doing more in the Security Firm to do that. Is there something i that industry can help us in securing the technology and managing the supply chain so china doesnt take advantage of the great r and d that we do . What are the things we would like industry to do . I put that up as a jump ball . Supply chain insights, transparency management is increasingly a concern for us. Obviously we have been focused on it particularly in the i. T. Space for years but when we are finding is thats not enough, it is too narrow of a viewpoint. We are opening up our aperture and trying to work most closely with our partners and understand the entirety of the supply chain coming through the dia door in our case. And understanding the tangent that each of the supply chains go into. It is daunting no question, but we have to do it. We are increasingly looking to our vendors to come to us with transparent display, you will, of our supply chain so we can understand it so we can have confidence in it and really make sure that everything that comes through the front door is uncompromised throughout. Whether it is equipment it could be camera, chairs, furniture, i. T. Equipment. I could go on. We have to think more seriously and Work Together on the supply chain. Understanding it makes multiple views of multiple phases of supply chain both in industry and then on the government side, too. It is knowing you are a target. There are multiple methods that the chinese are using right now to go after our technology and intellectual property. So cyber means. Clear defense contractors, we know you are a target. We know it is your unclassified information, your subs that are the target. Basic cyber hygiene. Encrypt data at rest. Dont conduct thumb trifs to your Mission Systems and then the internet. You would be surprised at those who to do that. Basic cyber hygiene. Knowing having strong Insider Threat programs. Better controlling your sensitive but unclassified material. So there is a whole host of things that you can do. We are in the department treating security as a missionary, no longer an admin or a back office function that can be traded away. Those are just a couple of the areas to focus on. Jeff . Sure. If i could add just a couple of pieces. One is for the first 25 years of my career i had the luxury of not paying a lot to economic intelligence. For a large part we relied on various partners or exquisite niche capabilities within the ic to do those pieces for us. In the last three years i have spend more time on economic intelligence and understanding the connects between economic and security issues. Industrys ability to help us work through those issues is certainly welcome. To echo a little bit on the cyber hygiene, defending and protecting your networks, but also having the same approach to counterintelligence that we are doing. We are scaling up our approach to counterintelligence due to what is being related to chinas approach and other peoples approach to the direct access vektdor. The direct access vector to what they want is not necessarily what they are going to take. Chinas Global Presence gives them lots of opportunities. For he moo it is asking industry to be cognizant of your network, your people, your connections, your vendors across the globe. The last piece is our National Strategy really depends on allies and partners. Anything industry can do to help protect and secure the networks of friends, partners, and allies, because of our desire to connect to them at speed and scale with data, means we are going to have to rely on the security of their networks. There is a direct application to u. S. National security. Let me echo Industry Partners a little bit and get the reaction. You guys make it so hard for us to bring you the best technology that solves your problems today. You talk about acquisition reform, you talk about acquisition agility. You build all kinds of structure. Joint universal needs, all sorts of things that fix the symptoms but doesnt allow industry to bring the best ideas, the best to you today. Where is the department . Where are we in really allowing industry to help us and be good partners . Anybody want to jump on that grenade . Because thats what i hear from industry, you make it really really hard to help. The first person that told me that was jake jacoby when i took over at dia. He said, i have got stuff. I cant share it with you. Where is the department on acquisition reform, acquisition agility . What did we do better . Phone a friend and have ellen lorde sitting up here. I know they are doing a lot there. I cant i sorry. There is a lot of truth to that. Sorry. I told them no gotcha questions. That might have been one. There is a truth to that. And me working with joe kernan who was at the edge of the spear and spent time in industry. We know there is plenty of talent out there. We are seeing services castel run or other vehicles to bring in ideas. With maven we brought in over 40 Industry Partners from the bigs to smaller dot com startups. I will say we need to do a better job practicing what we preach. But when we say Agile Development and there is a b. A. Or something out there forgive me something in six months. Dont give me a proposal in six months for how you do it. Maven has been a great example. We put six months as a boeing out there and we sprinted and had phenomenal capabilities from across the spectrum of the bigs to smalls actually deliver algorithms and capability that we are able to integrate into command isr capabilities. Dont bring us a proposal, bring us capabilities. Dont bring me a proposal. Dont bring me 18 power point slides, bring me something thats usable today. Anybody else want to jump on that . I can speak a little bit even to the congressional side of this because even in the context of mars, what we are finding is still in some cases the level of discomfort on the hill with all things Agile Development because it is not well defined by definition. Particularly as they are looking for us to justify resources and investment, they are saying, well, when is it going to be delivered . When are you going to be able to use it . When can i see something concrete . How much is it going to cost . We are saying we are not sure. We are going to Agile Development we are going like some, not like some and we have to adjust. Some are having a hard time embracing and supporting that. Yes the department has work to do on it, for sure. We all have work to do on leveraging some of the flexibilities that already exist. But we have to work on congress on the receiving enbecause they are fnly critical to us actually proceeding. Trey, did you want to you didnt. Happy with the u. S. Dizer spots. You have one of the Biggest Challenges in terms of warning in the j two, trying to get incites. I remember many times the u. S. Forces career commander wanted 72 hours of unambiguous warning. Have you solved the warning problem yet . If you on the have, what are the needs, what are the challenges and how could we get after solving the warning problem for you. There is a lot being written about the warning problem. They are citing crimea, mosul. At the same time, the question is did we have a warning problem . Do we have an omniscience problem . Do we have a listening problem . Do we have a data problem . It gets to a natural segue, what keeps you awake at night . Well it used to be is admiral jacoby here . There he is. I dont remember if you remember, sir, 25 years ago you said i just want it all. I want it all. I want all the data. That was at that time it seemed like a huge quest. But it was at least a goal we could set. Now i am more worried about what we are not going to at least even assess. What we are what hits the cutting room floor. So that really keeps me awake. On the warning problem, what hits the cutting room floor . So we need an application, we need a series of applications, we need some additional help so that we have a certainty that at least diet has gone somewhere and has been processed through i will just offer, perhaps Machine Learning, a. I. , to at least cue us to what i was talking about before, new variables. You need to know this, you need to at least consider this. Thats where i would say thats the kind of answer one on what keep us awake. And it is directly tied to warning. The second would be also related and that is assessing ourselves. And how do we know that we are assess ourselves . This is actually tied into the joint concept for intelligence operations. This is kind the second part, which is a spiral estimate of our own readiness. So as we get into this globely integrated world and so many thing could go wrong, so many warning problems could be out there, opportunity, cost, et cetera, how do we know that we have all systems on go . How do we know in a the fed krags we have set up is working . How do we know that we have got the right number of analysts allocated to different warning problems that might be related . It is a complex answer to what probably seems to people like a very easy question. Jeff . Can i add one thing to that sue . My vignette for warning that was instructed for me was again, at indopacom we had a subordinate, of all the warning problems that are written with precision and tracked by the most number of people to provide the most collaborative results, that is the warning problem. But frankly there is three major aspects to it. One of them is warning of just internal stability for north korea. When we just swapped out and i say just. It has been a little bit. U. S. Fk commanders, the commander came through headquarters and did a series of office calls. My first conversation was sir what do you need warning of . We cant provide you warning of anything. He rattled off three things which required a rewrite of the warning problem. To me thats step one, what do you want warning of. Then designing the warning problem. Being comfortable if two weeks later what i want warning of changes. Offany questions yet, so i dont have any questions yet owe either we are having a great conversation and you dont have any conversations or we put you to sleep. But thats okay. Have you got questions coming . Writing questions . We cant we have got to talk a little bit about Cyberspace Operations. U. S. Cyber command is coming up on nearly ten years now from the early days. How are we doing in terms of intel support to Cyberspace Operations . We thought our way through the right policies . Are we meeting the requirement from the joint staff from the combatant command . Or is it another intel thing thats plugged in this multidomain world . Jump ball. I will take a stab. Having seen it at least from nsas perspective. I was not assigned to cyber combut i was a partner. I think we are postured better than people think because cyber is intelligence driven. People are in the domain, intelligence analysts are in the domain when they do that collect and when they actually help an operation to occur. So to think that its broken and that there is not sufficient intel support to cyber, perhaps reflects some people who arent entirely familiar with what actually happens inside of an ops room. That might seem a bit harsh, but its true. It is very hard to describe just how much intelligence is going into a Cyber Operation without being inside the ops room. I am more optimistic than most people on this issue. Anybody else . Suzanne, do you want to talk about j wiks a little bit . What does it look like in the fit. If j wicks transforms and it is j wiks of the future what does it look like . I wish i had an answer from that. What we are doing is engaging from the Broader Community exactly when they think needs to evolve from it. It started as the core ic system. It was that for many years. What we have seen over the last five years plus is really an expansion of interest in j wiks as the daily system i will say, the c 2 system, the Communication System regardless of class pick. There is multiple reasons for that, its financely trusted in the reliable and security of the system. Understanding all the information you could need would reside on that system instead of going back and forth between super nix and j wiks. We are trying to create that conversation, if we were to create j wikz wiks today what would it look like for the Broader Community, the dod, jc, and even federal. This is an opportunity to have that conversation because there is such a demand signal. We are trying to understand that so that as we head down our path of modernization we are addressing all of those concerns and scenarios of where j wiks will go. More to come on that conversation. Really, the no longer an intel collaboration environment. Its command and control. It is an decision support. Its an intel i video. It is all of the above to all customers at all times. The real question, is that the right model . If it is the right model, what does it mean for us in designing the next version of j wiks, 4. 0, whatever the heck it is. But it has certainly evolved, from an Intel Platform at the highest level to every combat and command. Every j 5, every the j wiks box. The followup is you touched about it is on partnering. What the is partnering environment that allows us to collaborate in that space . There are lots of versions to out there, but it is something we need to think our way through. All right. Jeff, you have been you did you want to oh, no. Jeff, you have been in jacque now for how long where . Where are you now . I work for mr. Kernan and ms. Bingen at the jai. I thought you were going into the jake. Oops, maybe i shouldnt have announced that. Thats another air force general. Another air force general. One of the great challenges we have today is an awful lot of data being collected on all of us. And what do we do to protect our data internal to your organizations, data thats collected on us . Are we doing anything to make sure that that environment is clean for our our intelligence professionals . What can industry help to us do to protect our data, if anything . Jump ball. I guess that, you know, there is at one level there is the you know, we clearly have laws, policies,ing arelations, there is a lot of d. O. D. Directive and instruction on how we protect information. But then there is the actual, you know, technician of it and the technology of i mean, heck, weve got the back on investigation, mission coming to d. O. D. In less than a monday now. And as we look at rebuilding that i. T. Capability, it is insuring that it is, all of our everyone that has a clearance or a credential that your information is protected. So it is building all of those Cyber Security elements in up werent into the system. I think there is a technology piece to it as well that we will continue the need help on. Heres the thing that i want Industry Partners to think about. All of our data is available out there to any of our adversaries. And even the data that we tried to protect is available to your adversaries. We have closed circuit tchbst and cameras in every city thats capturing biometric data. What does that mean for us as a society and as intelligence professionals . How do we protect that data . How do we make sure our adversaries arent able to use that data . Ow industry can help us think our way through biometrics, closedcircuit tv all the data thats out there on the dark web that exposes all of us, but more importantly some of our very sensitive collectors. How can industry help us to solve that problem . Thats where i want to go with that question. You think about the super squirrely stuff we do, counterintelligence, the last 15 years of ct has been focused on vetting taliban and whatnot in afghanistan. When you have a sophisticated adversary a russia or china, those skills atrophied quite a bit over the last decade and a half. We are working on rebuilding it. We are not rebuilding to what we did in the 70s or the 80s. We are rebuilding this the modern Technology Environment where there is constant surveillance, smart cities, everything out there on the internet that our agencies are having to think through how they do that human ci, cover all those aspects that were core intelligence missions. All right. Does d. O. D. Seek commercial satellites as an alternative to National Technical needs . I will say as a compliment to National Technical means, you know, you are still going to have your need out there for some pretty exquisite High Resolution and other features that those satellites have maybe some additional protective measures given where the threat has gone in space that you are willing to pay a premium on. At the same point in time god is it an exciting time to be in commercial space right now. I visited one place a couple months ago where they are finetuning their production line to pump out 6, 7,000 satellites a day. S that unfathomable for someone when it was six or seven years. Exciting things there. We absolutely, from nro to others have to take advantage of that. The other piece of it where industry can be can help us quite a bit is not just on the sensing side buttiped side. You go into a ground station today, it is manual labor intensive. You have people moving around bars on computer screens. When you get into the consolations of the sizes we are talking about, you cant have a person in the loop doing that manually. So all of the automated tasking, automated collection optimization, the processing side of it, moving some of those algorithms and processing upstream further, freeing up the analysts to do other things, there is a whole manual labor intensive linear set of processes out there that we must do better and if we are going to be dynamic, as you were talking about earlier, and much more responsive to the threat environment we are seeing visavis china and russia. Yes . Go ahead. I would offer, as a former j2 for u. S. africa, a place we have some partners that would really appreciate some of that commercial image are you, it has a perfect application there as well. For partnering, as we were talking about. It provides them at least with an avenue. If i could add two vignettes to that. As a former j2, that the commercial imagery gains that we have seen over the last handful of years have been absolutely crucial and helps us gets the most out of the national architecture. For example, indo pay come was 52 of the earths surface. How do you track open ocean areas . How do i track north korea and shiptoship transfers of illicit oil . A lot of times that was only at a certain period doing a highresolution with National Technical means and using commercial imagery at every phase and stitching it together cohesively in way that gave me the best picture and could provide my boss was a combination of the various pieces together. So i think thats the future. How do we stitch that together. And then back to the point on the ability to extract information out of commercial imagery and making that readily available. I just happened to be having a conversation earlier today about synthesis. There is a lot of data. Dhou how do we take that to those two or three take aaways to make decisions in a highspeed priority environment. The ability to extract data out of commercial imagery i think is an important business line to continue to pursue. So this one came from the audience. I was wondering about this earlier. We talk about Machine Learning. We talk about artificial intelligence. We talk about the technology coming into the work force. Is there a Work Force Ready for this . What are we doing to make our work force culturally ready for this influx of technology and sore it the same work force . It might be the same work force, theyll adapt over time. Any thoughts on what are we doing to prepare our work force for technology . I think its a little bit of both. Some of our existing work force surely are adapted already, are hungry for it, are eager to continue to evolve their own capabilities and habits, frankly, on how they leverage technology. That said, there is no question that some of it is going to have to be trained. Some tv is going to have to be hired to some extent. We are trying to hone in on what does that look like. What skills do we have to teach . What skills do we have to seek out and recruit and defining that is still part of the puzzle, if you will, that we are trying to piece together on it. But the work force in many respects, you know, we have such a great work force that they are very adaptable. They are interested in trying new things. They like experimenting. They are innate liqueurious. So you put a piece of new technology in front of them and they will figure it out and figure out what it can do for them and take full advantage of it. So its just a question of how do we leverage that and encourage that and develop that while rounding it out where we still have gaps with the hiring and training. I agree with that. We have analysts, obviously, who are sourced by dia. We are dia. Especially the younger crowd are looking for advantages all the time. They are always looking for an advantage. So a new application is going to be welcomed. I dont think we are going to run into a situation where analysts are going to feel in competition with a Machine Learning environment. I think they are going to find it completely complimentary and they are not going to find it a substitute for their they are going to find it just an addendum, something thqueues th to a better product. How do we train them . Because the training that we do today is probably not the same sort of training and preparation we need for the future analysts or the future work force in this environment. So there something wed like industry to think their way through in terms of how we train, prepare our work force to be successful in this next era . I want you all thinking about virtual training. I want you all thinking about immersive increasingly complex training. I want to think about how painful it would be to have an instructor standing on a podium lecturing as we prepare our work force. What is the different training model . Is there a different training model . I can envision a situation where we would put an emphasis on certification to use the tool so that the analyst knows the left right limits of the product and the queueing coming out of the tool. We probably need to ensure that they are witing. This is based on what could be a bad variable. I need to regroup and try again. We went through this actually a long time ago, you know, with maven and everything thats happening. We actually looked at the automation of point dropping in the targeting community, and there are some tools that made things very quick, but they also cut a few corners in the interest of speed. So as we thought about it we said we want this tool, but at the same time we want to know where it potentially is dangerous to precision and accuracy, and that was the basis of certification for using the tool. The tool is actually used. Its not a substitute for the good Old Fashioned way of dropping points. It happens to compliment the process. So i could envision analytically a very similar environment. Just know what your left right limits are in terms of tolerances so you can queue to yourself and chain of command this was based on Machine Learning, we might need to take another round. It also goes back to we need to preserve the hard Critical Thinking skills of our analysts. Especially with all of these different data sources that are more readily available at their fingertips. Whether it be the Machine Learning tools, the traditional imagery or signals intelligence, even publicly available information. We are using that so much more, but you have to be able to discern how reliable, how credible is this and what kiechbd decisions are being made off of it. I think there is just an evolution of the trade craft and those Critical Thinking skills that have to evolve with the technology and the different data that we are getting our hands on. Lots of the services are talking about multioperations. When i started this business, the army and air force did their thing. The navy and marine corps did their thing. Then joint grenada and air land battle. Now we are talk being multidough nane operations. What does that mean for the Intelligence Community, one . And for the j2, the joint staff, multidomain operations in a globally integrated world. We talked about china as if its a regional problem. Its a Global Challenge set. What are the challenges to multidomain operations in a globally integrated environment . I think i go back to how to assess the opportunity risk as we are getting a bit too fixated on a regional problem. That is not an inherent part of our trade craft at the moment. Thats something we are going to have to work on. Lets imagine we have we have decision tools for something as simple as moving from point a to point b in our smartphones we use either google maps, waze, Something Like that to help us assess how quickly, whats the best way to move from point a to point b. You are going to use different chokepoints. Imagine they are different variables. Everybody has done it in here. You are kind of testing, testing to make sure its correct. Now imagine you have an armada of cars trying to do an objective as well but others have an armada of cars and they actually want to confound your solution. Imagine all the permutations of that. That starts to get very complex when you are thinking about the monte carlos for opportunity costs and how an adversary might take advantage of your dilemma in a particular place. How does an analyst efficiently succinctly address that . That is not something that we necessary train to. I would add that there is a data side on the back end of this. Multidomain operations would require one service to send, other servers to shoot. On perhaps of a Third Service sink a ship. How do we all have that same picture that we can have common understanding and that requires an inneroperability and the data standard that we have yet to achieve, but continue to work on . And then there is the data integrity of the data that flows across there to get after treys point. How do you ensure nobody is putting false data in there. If you are doing long range over the horizon fighters with a shooter that cant see the target in the center that they are not causing fires to go astray. So there is a tech side of multidomain operations that i think underpins it. Once we work through that piece, then it becomes the ttps and the trust and confidence and how that would work in a Global Integrated environment. I dont want to speak for admiral kohler and the navy. I think these things are going on. Now let me just talk as a naval officer. And i think the jury is still out as to if were talking about the domain of space, we are talking about the domain of cyberspace and how specialized these domains are, we have to ask ourselves whether generalization as were rearing specialty officers is the right path or whether we actually might need it think about specialization. Us havi i was having this conversation not too long ago. Now we tend to reward and seek generally itss and we might need to adjust that in cyberspace, going back to your question. Very specialized craft might need to keep people at keyboard for a long time as opposed to back and forth between some of the domains within our services. I think the air force is already moving in this direction. Suzanne, you were going to comment . So the vision for mars is to enable multidomain operations as well. Fundamentally, one of the key efforts is to understand and try to pull together in some form or fashion various capabilities of the individual services we are going to be using and leveraging and how they can bring them together with common access to that data and feeding it back and forth in and out of mars as needed during an operation, too. So thats, you know, a fundamental precept of mars looking forward, is how exactly to enable that future. Yeah, there is a bit amount of plumbing we need to do in our community. Ill use dcgs a s an example. Yes, you can share it was supposed to be the Common Ground system and weve seen each service has developed its own kind of bespoke capability. Although you can share products, the raw data isnt exactly inner operable. There is a lot of work we have to do on inneroperability, data standards and formats and architecture, and a trust piece as well. You have to be able to trust that someone is going to provide data and you can do something with it or act on somebody elses data. The Missile Defense community has had to work through that in the last ten years or so. I think we need to apply some of that learning to other areas as we think through multidomain. Talk little bit about creating an open Source Center. Some thoughts about how that might work. Implications of, if you do this, you are going to bump into u. S. Persons data. And we thought through the policy. Can you spend some time expanding on open Source Center . Sure. And come october is when we are kind of opening the doors, so to speak, on a more formal center than we have ever in the past. And its going to grow over time and evolve over time. So part of it is a structural change for us and organizing the activities a little bit more specifically to include dedicating people into respective centers or commands and who really will be kind of the hub of the activity in that location. But then as you indicate part of it is what is the trade craft behind it . What are the rules behind it . And again i think because so many of our analysts and collectors have been kinda sorta doing it on their own for many years, you know, the imperative is clear that we have to bring some discipline to this, particularly when it comes to u. S. Persons to make sure that everybody is doing it appropriately and accurately and from an oversight perspective that we can stand by it. Its a significant change for a number of reasons for us, but its going to be, you know, an evolving effort because we are not going to wait until we have all the answers figured out before we embark on this. We are going to jump right in and kind of fashion it as we go. But its been years in the making, and the good news is we finally figured out, you know, that this is fundamentally something that is in fact structural, there is a dedicated career for it for many of our folks. There is enough of a career there and enough of a discipline there that we can call as such and manage it as such. So i think what were trying to figure out though, too, is what is the enterprise do. So were focused first and foremost on dia, but clearly the enterprise as a whole, whether its the services, the broader commands, what have you, they are going to have a role as well and thats going to be kind of the next step that we are going to have to define, is how do we leverage the enterprise when it comes to o scent and who can bring what and figure out how to knit it together. Its going to be a continued effort after we open the center. Could i add to that . I think one of the great applications of Machine Learning and a. I. Is in this rel. Its not about necessarily finding meaning in large volumes of data. Its doing attribution of that data and can we use a. I. Machine learning to help discern is the person behind that particular bot or post or whatever tied to u. S. Persons. I think there is great opportunity there for a. I. Machine learning. Yeah, i would agree. The vetting premium is huge here. Cant say enough about o scent as a form of an Early Warning, especially in theaters where we dont have sufficient collection. The speed is unmatched. And so you really need that vetting behind that particular one liner to find out if this is an earn warning that we need to pursue or an Early Warning that we need to ignore. In the Intelligence Communitys ai a. I. Ses. What extent will china dominate 5g communications . How prepared is the u. S. Government prepared to protect from foreign threats . Can you jump on that one . I dont know that i can speak for the broader Intelligence Community. I would say they its one of their national priorities. So they have put a lot of effort and resource behind wanting to dominate in 5g arena. I would say technologically i dont think that they are there yet, so its not a foregone conclusion and we will continue to have to stay after it. But this is an area we in the department are spending a good amount of time on and working with the Intelligence Community as well. First off, just to make sure that folks understand the security risks of it, and our foreign partners, whether they are european, middle east, asian partners, is there is a clear it may be a great package deal today, financing tied up with a bow, but you need to look long term at the security risks. The onus i think is on us in the department to articulate what the military and security implications are of relying on a chinese or huaweibuilt 5g telecommunications architecture. For starters, you have a china National Intelligence law that mandates that companies and individuals compel them to cooperate with the chinese government. But on the military impact side, whether its intelligence sharing or mobility, there are tremendous amount of military areas that would be affected when we do not have trusted telecommunications capabilities with our close partners. And its been a challenge working this with our allies and partners for them to see beyond some of the nearterm economic benefits, but to look at the longerterm security risks and getting the intel and defense, National Security folks to talk to the Commerce Business community and hop those stove pipes. We are working in the department within our research and engineering groups. So dr. Griffin and dr. Porter are leading an effort to create an integration test bed. So whether you are u. S. Or other companies, and i would note there are other companies out there that are probably leading beyond the the u. S. Does not have an end solution. They are working on developing an integrated test bed so companies can come and test out and evolve their technology and work the integration pieces of it so that there are viable alternatives to a chinese 5g system. Okay. We will start on your end, trey. You touched on this a little bit already. What keeps you up at night . Anything else . I am looking for the black swans. Looking for the things that we go russia, china, iran, north korea. Oops, it was what keeps you awake at night . Very few enemies or adversaries keep me away. Its the knowledge of ourselves and readiness of our own systems. Ensuring we are not missing something. Missing the big story. Putting the pieces together. A lot of the things we talked about today, thats what keeps me awake. I would absolutely say the same thing. There is no adversary that keeps me up at night. It is about our ability to sustain over time a longterm disciplined approach for the various problem sets that are out there. How do we address the urgent needs and the important needs concurrently and can we maintain that. Thats probably what keeps me up at night. Your black swan question, for me that is about an area where there is potential for escalation that we have not been through that escalation cycle previously and not know how to scandal. India pack stoon India Pakistan border, there is historical in some of those areas. Things continue to change. Those are the ones i would say are worth watch to go ensure we understand where were going and are providing best advice and assistance to friends and allies around the globe to work through those issues. So i think for me its the risk that we assume every time we have to make a tradeoff decision. So clearly we, you know, we have many more demands than we can ever meet by way of requirements and our resources just wont keep up regardless. So we are almost on a daily basis having to make some choices. Obviously, there is risk in every choice that we make. And we can communicate that risk all we want, but just worried that something is going to result as a result of our decisions. That risk is either not going to be appreciated or we are not going to be able to respond. We have a Tremendous Community out there across defense and National Intelligence. Folks that are working 24 7, 365 days a year. So i dont know that there is anything specifically that keeps me up at night. Actually, i would echo what trey was saying. He and i, just thinking over the last couple of days, he and i were on a phone call last friday and asking ourselves, hey, have we made do we have the policy in place so that the folks that need to take action, you know, over the weekend or on a holiday, that they have the authorities, they have the policy, they have the guidance to do what they need to do. And its making sure weve thought through those and that we, you know, im a policy wean any up in osd, but providing that frame and guidance to them so they can go off and do what they do best. Coming down the line now. One last chance to talk to industry. Top one, two, priorities, what do you think industry could do to help . Start with carrie. Helping us modernize our very manual labor intensive linear processes. And the more i see the more i uncover in this community, there are a lot of them that weve just finetuned and are bespoke that we need help on. So that. Id say helping us evolve in areas like a. I. , maven, mars, getting those scaled, getting them out to the field. And then i would be remiss if i didnt talk the security piece. With the chooirn and russia who are intent on getting after our technology and our sbek lektual property and people, we have to protect that. We have to make sure that we have a trusted work force and we need industrys help in all those areas. So ill emphasize supply chain one more time. But then obviously with mars, and mars is just the best example to date we have of working with industry and asking industry to challenge our assumptions on what we think we need, to help us really think through and think nor creatively about what we need. That obviously takes some understanding and constant engagement to kind of live in our shoes a little bit so that you can, in fact, do that. But i do worry that we all kind of fall into that trap of presuming we know what we need and asking you to deliver it. We have to flip that and you have to challenge us on that a little bit more. Data inneroperability and standards. The older models of proprietary data standards served us well for a period of time. I think we are moving past that. The Business Model of the future is about integrating and understanding data, and the more we partner with industry. And then collaborative development. If we ever do solve the Agile Acquisition kinds of things, it means partnering with industry from the beginning of what are we trying to do, how do we do it, how do we fail together, and deliver a subset of what we started. There is a Business Model in there. There is a path for the government in there and partnering with us to help design that and making congress comfortable with that sort of strategy. We are the functional managers for three disciplines. Warning, collection management, and targeting. We havent talked too much about targeting. I am going to offer three things in that order. In warning, just helping us not sacrifice accuracy and thoroughness, completeness for speed. In collection management, we need some help automating the process. Our automation for collection management is dated. And then in targeting we always need help, and this will be the case from now until eternity, just helping distinguish enemy from nonenemy. Its one of the hardest things we do in the business. Okay. We are going to stop there. I ask you to give the panel a round of applause for spending some time with us. [ applause ] hopefully, we didnt give too many jump balls. Thank you all very much. Into primetime on cspan 3 a discussion with Robert Wilkie on improvements at the department of veterans affairs. You can watch the Heritage Foundation conversation tonight at 8 00 eastern. Following that at 9 00 a u. S. House hearing in chicago on gun violence. The energy and commerce subcommittee went to economchic learn more about the cause of gun violence there and the impact on the community. Lawmakers heard from doctors, a funeral homeowner and a woman who lost her son and brother to gun violence. 9 00 tonight eastern on cspan3. Tonight on the communicators when it comes to facebook, the ftc recently fined that company. How did you come up with 5 billion, and where does that money go . So to take your second question first, the money goes to the u. S. Treasury. In terms of the monetary fine, remember, of course, obviously the monetary fine is only one aspect of the relief that we obtained from facebook, yes, a 5 billion civil penalty, but also broad injunctive relief that con trains the way in way facebook can handle consumer data Going Forward. Watch our interview with ftc commissioner Christine Wilson tonight at 8 00 p. M. Eastern on the communicators on cspan2. Tuesday the Senate Commerce science and Transportation Committee hearing examining the boeing 737 max following two international accidents last year. Testifying before the committee is boeings president and ceo Dennis Muilenburg and Vice President and chief engineer john hamilton. Watch live tuesday at 10 00 a. M. Eastern on cspan3. Online at cspan. Org or listen live wherever you are with the free cspan radio app. Search all of cspans coverage for video on demand of all the congressional briefings and hearings as well as the administrations response during the impeachment inquiry process. Log on to our impeachment inquiry web page at cspan. Org impeachment. Your fast and easy way to watch cspans unfiltered coverage anytime. Up next, a House Appropriations panel looks at funding for a new nasa program to send humans back to the moon. Nasa administrator Jim Bridenstine and the agencys acting associate administrator for human exploration of space took questions about the potential costs of a moon mission