Scene from beloved drones must seem like an angels of death. The gods can see you. All the time. A toy has turned into a deadly weapon. Has this become our god. Tell of the five the white city so far away from and yet so near to war at the beginning of our journey i want to visit a country that is mentally and often physically at war. And he saw his name on the news about my sure that i thought he had somewhere like we are in Israel Aerospace industry was one of the leading manufacturers of drones the day the army and his two colleagues invented something here four decades ago they call themselves the three musketeers without it. And the three of you this is your first three of it im sure this is was the first that will be getting in seventy seven. And nineteen seventy seven where we began to develop it from scratch so mean nothing was happening and we were thinking we knew with that we have to do a flying camera so we will begin to think about it and then this is it three years after we put it out this one this configuration everybody else will have to give us it was really a joke because we were we were playing they called the toys because they always money a mole in the bed. And we were really. Out of the time we were really. Today then youll see that there was. A first increase to the platform because you know. When youre eating the appetite just coming with it. We are allowed into the Ground Control station. Ok so im now im so frightened see. What do you feel when you sit down. This is not the fly do you would be ok i think you are very thank you both was a key moment for c. B. S. We did mouth. To mouth so now i would i could really even see people exactly and i would i can go down with my camera you know. You can switch between. I think it is better if youre used to that i dont play video games but i think if youre used to said you can handle that thing better so basically you need. To play here we used to look in florida. Do you get like somehow close to someone. You. Know for me to objects are given the code names sometimes its in the names of. The names i have no idea. What his. Family nothing. And thats why i can keep it professionally and the check. Of late to perform my mission can you show us a bit how it looks like if you if you track in time get someone how does it look like this so where you can see in this short video its. It was taken during a game cast lead operation in gaza in january two thousand and nine as you can see here this is a must still was dressed as an israeli soldier as you can see carries a firearm and have a backpack a radio device looks like one of our soldiers hes also wearing gear uniform olive drab. Uniform but after a short investigation we saw that. As i mentioned we can look at investigated target he doesnt wear a helmet on his head and hes missing did something with his hands so we find out that hes preparing an i. E. D. Improvised explosive device. Looks like an injured person ok it is out as you can see two legs and a body and its actually a dummy. But. Is hiding improvised explosive explosive device beneath a dummy this is not the behavior of an i. D. F. Soldier ok this is a behavior of a terrorist was it this is this is was an attack helicopter that we called a helicopter in order to attack the terrorist that is hiding here but since we didnt want to hit the un involved civilians the bomb here made almost not damage at all so we told the helicopter helicopter pilot to shoot again this time he aimed a little bit closer so the wall collapsed but the tail was syrian man alive so we called for a Ground Forces in other places there were. The house explode everything but here we make sure that none harm will be done so on involved. I prefer that my missions be to save lives. And not the other way around but. Their reality is really is not that simple unfortunately. At the border to gaza we hear shots. We are stunned by the surveillance devices balloons with cameras attached. Watching everything predicting everything a logic that does not prevent war. In july two thousand and fourteen a thirteen heavily armed attackers from the military arm of hamas crawled through a tunnel into israeli territory. Total surveillance and the deployment of high tech seemed to provoke arcane countermeasures. Wherever this war game and. This is how it began in america since nine eleven the American Government has been deploying drones and other instruments more and more for surveillance and for targeted killing. The n. S. A. Of a home. About home both n. S. A. Surveillance and Surveillance Drones are to a certain degree a socalled response of challenge. Challenge in this case. This means the attackers no longer wear uniforms or openly carry a weapon or launch attacks or move in a unit across real or imaginary borders instead they come from deep within the social space in the cities or towns or wherever. Thats where they operate and that means you cannot deter them. Like when you. So in consequence you have to act in a much more preventative manner if you want to stop them. But that means you have to know what theyre going to do. And you can only do that through dragnet surveillance and even if course you know there are political alternatives if our society were heroic one way you could say we dont want this sort of surveillance the probability that we might die terrorist attack is no higher than that of us dying in a household accident you know we accept that it was. Like a century but our society isnt that heroic finds were willing to take our chances with accidents and we accept them with sullen indifference and what we can accept this sort of terrorist threat. And because we are opposed to roic societies we demand surveillance and Security Systems and including preventative ones. And kind know if i know if i can buy fish by then you can only call drones cowardly if you are more willing to accept the risk of death and steady to name. This. If thats the case we have to accept that a post Heroic Society brings forth the post heroic weapon. But and the drone and all its trappings post is simply opposed to relock weapon one has said that the weapon is the essence of the fighter in light of that the drone and its deployment however its used is the essence of the post Heroic Society. Fighting by means of a monitor. Killing in the same way down. We play or is it the other way around like that. But it was. Better that it was. All over the world any one second you know you could fill dozens and even hundreds of Madison Square gardens you know with the amount of people that are playing this game its incredible i mean its becoming like sports its people are earning a living out of it its very competitive and i think its great and its not something we ever imagined when we first started making this gain. Even the advertising trailer for call of duty shows possible reasons for the games popularity that combines violence heroic poses and futuristic more technology. They used to like you know yeah. Its funny because i remember just watching it remember so many of the stories that you know that happened when there we were making the game and many fun memories trigger in my mind when i watch that. Its really fun game to work on. You know you see a lot of a lot of future type stuff where you get a lot of these Energy Weapons like laser guns and all these kind of thing and we wanted to stay away from that because we feel thats sort of too little too Science Fiction for us. But it actually opened our eyes as to you know that restriction aside how how much is actually being worked on right now this is just some crazy stuff going on drones in particular. Its fascinating to see where this will go because drones now have these things called Mission Packages where you can actually autonomous lee program a drone to go on a mission completely on its own no Human Interaction needed and this technology exists today. You know the day when you will have a drone that can have a Mission Package to go somewhere target somebody take them out and then return is an incredibly scary world. The Paris Air Show is the Biggest Aerospace trade fair in the world and a marketplace for military machines. Wheres this development heading were hoping to get some answers. Please welcome to the podium the secretary of the United States air force Debra Lee James mann. Well good morning to you new. Say you are looking sharp good morning everybody as i said yesterday we in the air force like to talk about Global Vigilance global reach global power but you cant get any of that without Global Industry backing us up and also without our Global Partnership so were very very grateful for all of that because you see collectively you and i all of us together we are all air power and today air power is air and its space and cyber we are all collectively air power and keep in mind and spread the word air power without us you lose thank you very much thank you. After months of trying an american film crew is allowed to enter the plant and general atomics as a german crew we constitute a security risk. This is the predator aircraft. You can carry three thousand pounds of payload beneath its wings another seven hundred. Payload is our point. And let me say without reservation the future of remotely piloted aircraft is right and this company is represented above that in the last twenty five years we have logged over three million flight hours with our remotely piloted aircraft and its interesting to note that that curve is exponential the more we produce the more its flown in fact in that two thousand and fourteen we flew nearly five over five hundred thousand flight hours in one year alone. I think its interesting to note that every second of every day there are fifty seven of our aircraft airborne worldwide. Making soldiers into heroes has long been a tradition not just in america his willingness to sacrifice himself is what makes a soldier deployable. What value do socalled soldierly virtues have in a future where on manned machines dominate the battlefield. Whether technicians we already have these kinds of people theyre in the air force and they scare me they frighten me these are people who have no ethics they have no morals they are amoral they are a political they have no real concerns about anything but the technology in front of them. So it can valley in california has produced thousands of these people i meet them i talk with them i see them and i understand them i think and it scares me they have no ethical compas at all theyre question is how do i perfect the technology how do i make the technology do what my customer wants it to do or what i wanted to do and how do i refine the technology as i go along there is no ethical component to their thought we are producing these people today in our universities in our corporations in our society and they are frightening people. If people no longer half ethics could ethics perhaps be built into the machines we fly to atlanta to the Georgia Institute of technology i have read that professor vonage i can is doing research on an Ethics Software for robots. The most depressing work that i had to do was studying Human Behavior in battlefield environments the battlefield environment brings often in many human beings in some cases the best i mean you can see noble examples these unfortunately are highlights or exceptions in many cases but often we see atrocious behavior literally atrocious behavior where noncombatants are slaughtered and justifiably in the battlefield reading and studying Human Behavior in the battlefield as part of the work that i did is what caused me to think that we can create robotic systems that can outperform human beings in this case they can act in a more rational i would contend that might sound strange but a more rational manner under these sets of circumstances without having fear of selfdestruction or anger or frustration imbedded within them to be able to come up the tension with better decisions for this. The real question is how can we take what human beings have already decided is ethical behavior and make sure that the robot complies with that so what we do is extract from those laws the principles that say youre allowed to do this and youre not allowed to do that we do that with our war fighters as well too we tell our soldiers you are allowed to do this and youre not allowed to do that they dont reason from First Principles in terms of deriving International Humanitarian law where in the battlefield theyve been told what is right and what is wrong and in essence were doing the same thing with the robot were telling it whats right and wrong to code in this particular case. Using a software called ethical governor the votes of the future should control themselves and develop a guilt sensor. In the scenario depicted here an unmanned probe craft is back on a search and destroy mission. While performing its Mission Young man wrote a crappy encounters an enemy convoy within the designated kill zone. The robot searches through its choice available weapons and weapon release positions for a combination that can simultaneously satisfy the allegation and the laws of war all striving to minimize civilian collateral damage. After Battle Damage assessment has occurred for about compares the actual damage yes make it prior to the engagement the resulting Battle Damage exceeds that expected by the robot ethical adaptor computed guilt value corresponding to the actual excessive damage incurred. Theres got value in the rod macqueen dibley plausible model is computed as a function of a range of articles including the military necessity associated with the target and mission the number of casualties including combat and non combatant friendly and the amount of civilian structural day. Image sustained. As a consequence when gold levels increased various weapon systems are deactivated by the ethical adaptor relation to their destructive potential. The as always there are different perspectives. That if the map that of the camera from above and that of the phone camera from milan. Was. My car cant be ethical or not my of it cant be ethical or not my robot cant be ethical or not it can be designed with the designer trying to be ethical or the user trying to be ethical but the machine itself is that it is a machine second it assumes that you are not facing a thinking fo who will take advantage of that programming you could build a awesome. Machine gun armed ground robotic system theres versions like this right now and one of the best counters to it will be a six year old with a can of spray paint. Because it will put an incredible dilemma on the other side you will either have to shoot. Technically on armed six year old or watches that six year old walks out. Phrase paint right over the sensors and defeats your super advanced robot. Thats the point is where where were in this space where i dont think you can just simply engineer your way out of an ethical dilemma. How would i decide. Like the machine. The machine can process all the available information at top speed. Im not that fast. What does it want. Nothing at all. It is sufficient. Is that what the inventor once. Who knows. These machines will make mistakes they will kill civilians there is no doubt about that all Weapons Systems end up killing civilians the real question from my point of view and this is whats called a consequentialist argument or utilitarian argument is well they kill less civilians than Current Technologies and war fighters do from my point of view if we end up with fewer noncombatant casualties thats a good thing. To my surprise ron arkin sends me to the aquarium in atlanta he tells me that he sometimes goes there. His grandson. Oh and. Hes particularly fond of the beluga whales their lightness their beauty and their playfulness. We project many of our longings on to animals something inside us would love to be just as free as we magine they are. Awake to what cues do we put these longings thanks what are we doing to ourselves. Thank you our thanks why are so many machines named after animals drones hawks predators we take their forms and abilities and transform them into Something Else thank the lord. Thank. You thank you and he has. The bully turn ourselves into Something Else in the end the bank. If as i firmly believe it is our imperfection that makes us human what happens if we try to eliminate this imperfection. We project a little bit further out here but youre going to see things like think if you want to assassinate somebody you could send a swarm of insect sized drones that have cameras on them facial Recognition Software and you just put all this stuff into the into the swarm and say go for it this guy and you know stinging him with your cyanide tail or whatever it would be and then you dont even need boots on the ground. To game shows as jones like flying insects anticipating a possible use of the technology for military purposes. In the way you are like lets say test fear that it was the first the military itself it must be fascinating for them to see so its kind of. Close to reality well its incredible and you know im sat at home one day and i see i see an email pop up this is like a year or so after black ops two was being released and its from a former pentagon official who works for a think tank now the Atlantic Council and you know he said he wanted to talk to me about he said hed seen his son playing call of duty black ops two and he was amazed at how intricate and realistic the future storyline was for twenty twenty five and he wanted to talk to me to see how we would come up with that so he actually invited me to washington d. C. So be to go there and speak to him and so you know other people from the military the pentagon. How i came up with this this relationship