A ride. Hop in. Whats the main reaction you get from people riding in the car with you . Raj the typical reaction is one of anxiety and angst. Here, and occasionally fear and occasionally even panic attacks but then they basically watch the vehicle is able to drive, exactly stopping when it should stop. Ts actually riding fairly comfortably with a degree of comfort. We learn more about the move to selfdriving cars and professor raj rajkumar at the lab where his experimental car is kept and worked on. Dr. Raj rajkumar, whats your job here at Carnegie Mellon . Raj im a professor of electric engineering and robotics at Carnegie Mellon. Host howd you get into that . Raj i did my post graduate studies and got my masters at Carnegie Mellon. Then i left c. M. U. Joined i. B. M. Research for three years or so and i came back to c. M. U. Because i liked c. M. U. And pittsburgh that much. Host what kind of things do you work on . Raj i work on what are called embedded systems, the technical term. These are things that embed computers inside them but the research basically uses different devices. For example, your smart phone is a sophisticated device but you dont worry about the components inside there. Televisions, for example, et has components in inside it with you it is meant to be a elevision. That toaster that we use on a daily basis that cost 10 bucks its an embedded computer embedded inside a toaster. These are embedded systems that act smart but its a dedicated device of some other kind. Ive been working on embedded systems since my doctrine and it turns out vehicles that drive themselves are a classical example of embedded systems. So a car is a car that transports people and goods from point a to point b. With components embedded inside them. Host how did you get into the business of Autonomous Vehicles . Raj great question. Ive been working with General Motors, the carmaker since 2004. General motors has been working with the researchers in our Department Since the year 2000. I started applying my experts ease into embedded systems in the automobile. In 2006, the Research Advanced by the agency, the research arm of the u. S. Military announced a competition. The competition was the urban challenge. For vehicles that drive themselves, they need to drive for about 60 miles in fewer han six hours. With other selfdriving vehicles as well as human driven vehicles and following the same rules of the road that you and i have to follow on a daily basements g. M. Became our biggest sponsor. We had about 20 other sponsors but g. M. Was the biggest of them. Because i already had a strong working relationship with g. M. , i became became an integral part of the team that worked on the vehicle, which ended up winning the 1 million prize. When our team from Carnegie Mellon won the competition, g. M. , who was the biggest sponsor of the team said hey, our team actually sponsored the winning team and they said because its related to driving, it clearly has implications to the Consumer Market segment and they opened a second lab on campus focused exclusively on automated driving and ive been running that plan as well since itself launch. So our relationship with g. M. Continues to be extremely strong and very loyal. Host does g. M. Own the technology that you develop . Raj fmpt technology that we sponsor. Were grateful for their support. Is actually owned by caron darn university and we have Carnegie Mellon university and we have some Licensing Agreements with g. M. Host and we learned more about the experimental cadillac as we got ready to take it for a drive. What is this monitor thats up here in the top . Sensors. Six major thats one in the front of the car. Theres one in the bumper here. Right there. Radar, s the second thats the third on the other side of the bumper as well. Theres one behind the side back window and one on the other side. Host what are they read something raj theyre sending out multiple players of layers of beams and when the beams basically hit the on the, they bounce back and come back to we ransmitter and because know the speed of light, we can estimate how far that is. Because there are multiple beams scanning as well, you can get a profile of the object. And because we have lidar allover the car, it shows whats happening around the vehicle in realtime. Host is this car communicating with anybody but itself . Raj its capable of communicating with classic cars and other vehicles with similar roos. Its a automated vehicle. T. A. V. In short. Host i also see some cameras inside this car up here. What are these . Raj we had six laser sensors. Radars eras and three as well. One is inside the cabin. One is pointed downwards looking for landmarkers on the road. The other looks at traffic lights. The third camera at the back of the vehicle is for backing purposes, you can see whats going on. And there are also six radars. Theres one behind this cadillac emblem. It is the metal we replaced with the metal logo with a plastic logo so the radar can see behind the plastic. Theres one also thats behind the bumper. The bumper is made of plastic so the radar can see through the plastic as well. Theres radar on the other side of the bumper and there are two on the side but you cannot see from the outside. Internally theyre enclosed. You dont see in the inside either. Theres radar at the back in a plastic bumper also. Host is this car seeing 360 degrees . Raj it is seeing 34 3 0 degrees all the time, as , who have to ns turn their heads around. Host when you get in this car, whats different in the look than a different cadillac . Raj we tried to make this car look normal on the inside and outside. On the inside it pretty much operates and looks like a normal car. Just like you would rent a car at the airport and pick up the keys and the layout looks slightly different but were still able to drive. Its basically the same thing. Ou bring in your keys, get the vehicle running and start driving. Looking at the dark board there are two things that have changed. A button on the dashboard, an emergency stop and theres a button behind the stick shift. Think of this as the autonomy equivalent of Cruise Control engage button. So you would actually engage this button to go into autonomous driving mold. What you need to do is rotate this switch then pull it upwards to get into autonomous driving mode and that is a very conscious action on the drivers part. Host is this car licensed to drive the streets of pennsylvania . Raj in pennsylvania, the laws allow a vehicle to drive itself as long as two conditions are satisfied. Number one, a licensed human driver in the drivers seat and number two, that human can take over control at any point in time. On those two conditions, vehicles can drive themselves on pennsylvania public roads. Host but this could drive as a normal car too, correct . Raj of course. This autonomous board, you push this down. But you would not have the time do that, you can turn it and press the brake pedals or the gas pedal and the vehicle would still respond. Something could be happening that allows the human to take over. And then this button here is only for strictly emergency purposes because we added a bunch of sensors, computers and motors to the vehicle and ge, for example, totally something totally unexpected happens. But the reek is already driving. Ou push that button in, it mechanically, electronically stops the car at that point in time. Knock on wood, we havent had to engage that when driving yet. Host whats the cost of all the different systems youre added . Raj balls many of the sensors, units, we use are one they tend to be expensive. The real idea to be thinking about is that when the volumes go up, the costs will go down significantly. Our thinking is that when these vehicles are mass produced, it would add about 5,000 extra on top of the cost. Host youre going to give us a driverless drive. Is it truly driverless . Raj it is an automated vehicle. It can drive itself under many conditions but not all conditions yet. Not yet. So lets start driving. Ill explain a few more things. Host and youre driving manually. Raj im driving manually. It says manual on the screen. Ill point out a few things on he screen. Lls us that the screen interface. So i guess if i flip back and forth. This is a screen that we added. By flipping a switch you can go back and forth. Host what are these images were seeing here . Ramona this is the display which shows what the vehicle is doing at any point in time so we as humans can be comfortable that the vehicle is indeed doing the right thing. O what we see here is some i conls here and the icons here and the icons for example say i can launch, stop. Tell the vehicle to go to the airport, go home or go to work. And this basically lets you zoom in or zoom out. And what you see on the screen lines. See two blue they are the lane the vehicle can drive in host there are no lines on this parking lot were in. Raj we have a map of the parking lot that allows to us drive along these lanes and we go to the main public reads. That represents the map that your g. P. S. Has. Then you see a green line there. All the people who basically ride around this car and thats the route that your g. P. S. Device calculates. Map p. S. Has a bitin database. You also see a very short red line up there. That is basically the car knowing the green line route, knowing the blue line maps. T basically uses its sensory data from the radar cameras and basically says for the next 15,20 meters, this is how, where, im going to drive. Host dr. Raj rajkumar did you have to Program Every route ahead of time . Raj it needs to have a map of the roads. Host a g. P. S. Map. Would that count . Raj you can think of this as a g. P. S. Map, g. P. S. Navigation device and then you tell the system where it you want to go. It uses your navigation and your calculation for the route. And the red line is what the vehicle is deciding in real time. The red line. Now youll see if i zoom back a little bit. You see a bunch of dots on the screen. Those dots are the laser point from the laser sensors updating in real time. So what you see here is basically a bunch of yellow dots. Host is that that white car . Raj thats the white car. And this dot dumpster there. And you basically see a host and all these white lines are the trees. Raj basically its able to sense in this case the rampedse and cameras and lasers act as the eyes and ears of the vehicle. Host how far can it see . Raj 775 meters. It has builtin Wireless Communications roos that can go as far as 600 meters. Host all right. Raj so lets engage the vehicle in autonomous mode. The vehicle is in parking mold. Im going to engage autonomous mode. Host while its in park . Raj while its in park. Autonomous driving. Ost it started driving. Does it ever make you nervous . Raj i guess the normal reaction for anybody new is at this is that you feel sedate anxiety and angst about not being in control. Its a very normal reaction but lets see host ok. Dallas, you doing ok back there . Dallas i think so. Host its turn signals. Because you told it where you wanted to go already, correct . Raj yes. Host all right. Coming in host so you hit the brakes . Raj i did not. Even though the legal speed limit is 25, pretty much everybody drives at 35 or 40. But this is a stickler to the rules. So basically right now we have a vehicle behind us and of course the driver doesnt like driving so slow. Host uhhuh. It seems to do a little meandering in the lane. Is that a Fair Assessment . Raj it could be better, yes. Host but hes reading constantly. Raj yes. This is a winding road. Host uhhuh. Raj so im not controlling the steering, brake pedal or the gas pedal and it was able to shift transmission by itself. Jorge i see that. A biker . Raj yes. Host all right. How far have you come in 30 years . Raj we have come a very long way but still some ways to go sically completely host is this vehicle constantly learn something raj this vehicle is not constantly learning but its collecting data and using the data to teach the software about new features and functions. Its not learning actually driving. Its learning after the fact. Host how did it know theres a stop sign there . Raj the map basically has indications about where the stop line is. Host that was the car that did that . Raj the car did it all by itself, yes. Host it wasnt sure of its speed . Raj basically saw the parked cars. There are two ways to get back. Ill take over manually. Ill just push this down. The vehicle has gone back to manual mold. Autonomous ready. Raj it says autonomous ready. Jorge and can you do that on the fly . Raj yes. You can switch back and forth seam lesley. Host and its seeing all of these things . Raj yes. This crosswalk is not on the map. Host oh, ok. Raj these lanes have been changed recently. The maps are have not been completed. Host ok. So its not quite ready to be sent out on a road its never been on before. Raj we have done that on highways. Highways weve never been on before. In the city, bicyclinging and more things. We do that on the highway, not in open corridors. Host ok. A bike. Raj ill zoom and you can see it better. We are back on the curvy, winding road. Host kit read signs . Raj it can read some signs, yes, but not all signs. There are thousands of signs. It does not understand all of hem. So now we see that red light well. The green is the stop line. The red line is exactly on top of that. Host how far have you driven in this car autonomously . Raj a total of about 20,000 iles awe awe tom mousely. Host whats the longest stretch youve ever taken . Raj weve done a couple hundred miles of highways. A system thats been used to basically drive from San Francisco to new york city, about a 3,500mile journey and the vehicle drove itself on highways about 98. 6 of the time. Used ology had been so highways are not a problem. Autonomous ready. Raj so you have taken your first ride in an Autonomous Car. Host when will we do this regularly as consumers . Raj simple question, basic question and ill give you a long answer. You can already buy vehicles, for example, tesla with an auto pilot feature. The vehicle can drive itself but the human must be paying attention. General motors next year will introduce a similar feature they call super cruise where the vehicle can steer itself and apply the brakes and that will be in a cadillac sometime next year. And many highend vehicles can already drive themselves today. Some of these features are already available on the market and then three to five years from now, we expect that the vehicles will be able to drive themselves but in wellspecified, welldefined geographically constrained regions. Where ly, for example, bicyclists are not allowed and there is no heavy rain or heavy snow. So some of these technologies were deployed earlier but when you ask when can the human not drive at all . That basically implies that the technology should be able to drive the vehicle itself from any point a to any point b that any experienced drivers can drive in the u. S. That capability is going to take at least weve come long way over the past couple of decades or so but still a quays to go before the human can go to the back seat and take a nap. Host have you allowed your kids and your wife to ride with you in the Autonomous Car . Raj sure, weve allowed family members to go along on in of the projects, yes. Host i was a little surprised that we didnt have to sign a lease before we got in. Raj i guess if you were with a corporation, but just because we are [indiscernible] raj host why are we talking to you about Autonomous Cars in pittsburgh rather than detroit or Silicon Valley . Raj thats a great question. Carnegie mellon is globally well known, as a has a strong reputation for computer science, engineering as well as robotics. We have a Robotics Institute on campus. International and has more than 100 researchers in it, excluding students, if you will, and theyre all extremely knowledgeable about the robotics and in the field have been built at c. M. U. Since the early 1980s. In fact, we at Carnegie Mellon believe that we are the birthplace of Autonomous Technology dating back to about 1983 or so a couple of years back in 2014, we literally celebrated the 309 birthday of this technology on campus. Host you said before we starred this interview that computers are simultaneously very intelligent and very stupid. Raj yes, exumpletse are simultaneously very intelligent. They can do things that amaze us, right . They can react very quickly and they can make decision that is finds extremely smart. How does it know theyre driving at this speed and so on. They are really intelligent because they process a 360degree view of the vehicle with the multiple sensory data streams from radars and cameras. Very intelligent but at the same time they are stupid, if you will, because they dont really have common sense. For example, we know that when we fall down or when we basically touch fire it hurts and the next time you wont do it but computers cannot make that dink. Hey, i crashed into somebody last time around, next time dont do that. It will do the exact same thing unless this program through something else, specifically by a human being. Host whats the difference between the vehicles here in the lab. The jeep and the cadillac . Raj we see two vehicles. One is a red jeep. The other is the cadillac that we were able to demonstrate the vehicle driving itself. The vehicle on the left, the ed jeep, is called nav lab 11, meaning this was created by a Research Laboratory at c. M. U. Called nav lab, short for Navigation Lab and the 1212 indicates this is the 11th generation of autonomous ehicle the lab has bill bit built. So there were 11 variations of this vehicle. The vehicle on the right is the cadillac that we were able to drive today. That vehicle has been created by the project that i lead with support from General Motors through the u. S. Department of transportation as well as the National Science foundation. So because of our close working relationship with g. M. , we are extremely sensitive to the a sthet aesthetics to the vehicle, the interior and the exterior as well. So that it looks very normal, that something g. M. Would be proud to both design, manufacture and sell.