Brought to you by your television provider. Welcome to todays virtual program, from the Commonwealth Club of california. Im the clubs Vice President of media and your moderator for today. Wed like to thank our members, donors and supporters were making this and all of our programs possible. Were of course grateful and we welcome others joining us during these uncertain times. Today im pleased to be joined by brian christian, programmer and author of the new book the alignment problem, Machine Learning. Which was released last week. He discusses theoretical contributions documented in his bestselling book the most human humans and an outer of them to live by. Christians posing questions of if we continue to rely on Artificial Intelligence to solve our problems, what happens when ai itself is the problem. Hes a professor of philosophy and poetry, tackling the ethical and technological implications of a advanced society becomes more relianton technology. His work intersects elon musk in his book the most human human which is my cable reading. In his new book the alignment problem he lays out our dependence on ai and questions whether or not whether systems are a replication of inevitable humans, how can we get a handle on it before we lose control. Although we train the substance to make decisions for us, essentially humans will be humans. Will be discussing a lot in our next hour and i want to ask your questions. Youre watching live with us, but your question in the chat on youtube. So thank you brian and welcome. Thanks for joining us. Is my pleasure. This is not your first book. I want to ask in the opening question, the obvious question you ask an author it really gets to the job. That is why do you decide to tackle this problem now. Requested area and so the initial idea for this book came after my first look at come out. And as you had mentioned, vanity fair reported that elon musk was bedside reading and i found myself in 2014, attending a Silicon Valley book group that was a bunch of investors and entrepreneurs and they had seen this thing about elon must reading a book and invited him to join and to my surprise, he came. There was this really fascinating moment at the end of the dinner when the organizers thanked me and everyone was getting up to go home for the night. And elon musk forced everyone to sit back down and he said no, but seriously what are we going to do about ai. Im not letting anyone leave this room until you either give me a convincing counterargument white we should be worried about this or you give me anidea. And it was quite a memorable vignette. And i found myself drawing a blank. I didnt have a convincing argument for why we shouldnt be worried. I was aware of the conversation around the risks of ai or for some people its like a human expansion level risk. Other people are focused on the present day ethical problems. I didnt have a reason why we shouldnt be worried but i did have competent suggestion for what to do and that was the general consensus in the room at that time. So the question of, so seriously whats the plan. Kind of pointed me while i was pushing my previous book and i really began to be starting around 2016 seeing within the field to actually put some ideas together both on the ethical questions and the sort of further into the future space questions. And both of those movements have grown i would say explosively between 2016 and now. And these questions of ethics and safety and whats in the book i described as the alignment problem, how do we makeure that the objective that thesystem is carrying out is in fact what were intending for it to do. Brian the movement a try to figure o where in a way of answering this question. What is the plan. What are we doing. Michael really interesting things that i thought about as i wasetting into this. So very complex type of technology and things are going to this. I dont think a lot of people are aware tt this is already being applied and even lifeanddeath situations in our society today. We are talking before we started the program that theres number of examples of ai tools to acmplish with their hoping to perform. But one of the examples is the algorithms that are bng used t just here in california but where instead of cash fail, the judge use algorithms to determine whether or not subject is replaced while hes in jail while hes in trial. And this fall californians are voting on this. It would reaffirm law and replace it with an algorithm. It is a very complicated issue and what really surprised me was in the naacp, because of what they are saying the builtin inequities of the algorithm. But given to that example income go from there. When the problems with the algorithms and how did we get there. Brian absolutely. So there is a nearly 100 year long history of statistical things from the 1920s called actuarial methods in parole. Inattentive to create a probation and parole. That is to say, it started in the 20s or 30s and it really took off with the personal computers in the 80s and 90s. And today its implemented in almost every jurisdiction in the u. S. Municipal county state and federal. There has been an increasing scrutiny does come along with that is been very interesting watching the public discourse, pivot some of these tools. So for example, the New York Times editorial was writing through about 2015, the board was from these letters these open letters saying it is time for new york state to join the 31st century. We need something that is objective, that is evidencebased, we cant just be relying on the whims of folks and ropes behind the bench. We need to bring some kind of something more to it. Sharply that deposition changes. And months later, the end of 2015 in the beginning of 2016, New York Times was running articles saying algorithms are putting people in jail. Algorithms have racial bias. We need to put the brakes prayed and calling out my name the particular tool which is called compass. Its one of the most widely used tools. Threat the united states. In the question, it really ignited an entire subfield within statistics around a portion of what is it mean to say the new tool is fair. The system is designed to make predictions about whether somebody will reoffend. If there on probation or pending trial or pretrial. What does it mean to take these that exist into law things like treatment or equal opportunity etc. The 14th amendment protections etc. What it actually means determining and how that we look at a tool like this and say, whether we feel comfortable actually deplong it. Michael it is interesting when you gave example of how a black suspect in whiteuspect with similar crimesnd backgrods and how much more likely why suspect was to go free including the white suspect was actually advocates there you go. So what goes into if you are the baking of that cake, that built thesento here. Because o unintentionally baked into it. But there still hard baked in some aspects. Brian so this a very big conversation. I think up one pla to start is to look a the data goes into thes so one of the things that they are trying to do is. One of three things. Lets just think about pretrial case for now. Typically is predicting three different things. One is your likelihood to not take your part appointment. The second is to commit a nonViolent Crime by pending trial in the 30s to commit a Violent Crime during trial. The question is what is the data come from on which these models are trained. And if you look at Something Like failure to appear in court. If you failed appear in court, the court knows about it by definition. So that is going to be in an unbiased sense. If you dont show up into court. Just dont. It is nothing like nonViolent Crime, it is the case for example if you pull a young white man and young black men in manhattan. Selfreported are on the usage. They self report that they use marijuana the same right if you come you look at the arrest data and the black person who is 15 times more likely to be arrested for using marijuana in the white person. In manhattan. Other jurisdictions might be eight times more. I think like in iowa and it varies from place. So that is the case where it is really important to remember that the model claims to be able to. Crime. Of what is actually predicting his rearrest for the rearrest is kind of this imperfect and systematically proxy for what we really care about which is crimes. And its ironic to me because as part of this project of researching the system, went back into hysterical literature when i 31 started getting used which was in illinois in 1930s. At the time, a lot of the objections were coming from conservatives, from the political right. And ironically, its almost the same arguments are being made now on the inside. And in the 30s, they were sing women, if a bad guy is able to evade arrest, and the system does not know he committed a crime. It treats him like hes innocent. I will recognizes release and of their people like him. Now we hear it being made from the left which is to say somebodys wrongfully arrested and convicted and they going to the data as a bad person is a criminal and will recommend and detention of other people like them. Through the same argument described in different ways. This a very real problem and we are starting to see groups like the partnerships on ai which is nonprofit Industry CoalitionFacebook Google in a number of groups intact. Hundred different stakeholders. They recommending that we dont take these predictions of nonviolence for the arrests as seriously as we take the predictions of failure to recur. Missing aquatic that i want to highlight here. Its very that, but the second thing is worth highlighting is a question of what you do the prediction once you have it. So lets say you have a hher than average chance that youre going to fail to make your court, schedule a cour court appointment. As a prediction, the separe question which is when we do that information. One thing we do is when the present gel. While they wait for the trial. That is one awer. Turns out theres an emerging vibrant research that shows things like you send them a text message reminder, their much more likel to show up for the part appointment. In their people proposing Solutions Like providing Day Care Services for the kids providing them transportation to the court. Theres this whole separate question wishes as much is going on as much scrutiny as rightfully being directed at the actual algorithm and predictionc question is what we do with those predictions. If youre a judge, and the productions and this person is going to fail to reappear, ideally you would want to recommend some kind of text message alert for them as opposed to jail. That may or may not be available to you in that jurisdiction. So you have to work with what you have. That is not necessarily algorithm per se. But the algorithm is sort of caught in the middle if you will. Michael this ticket out of the crime and punishment area going to different area. Talk later the book about highlights. Amazon, and coming up with this ai system. I would hope that job applicants and what they were finding was that they were, a lot of them were men. And the reasons for this also was baked in the womb the way the system is being trained and the way it was being used. I think also need you to this, also has a question at the end of like why were you trying to find people, is you have people. Tell us about that. And how did they get in. Brian so this is a story that involved amazon around the year 2017. By no means are they a unique example. It just happens to be the example of amazon. Any companies were trying to design and take a little bit of workload off of the human part of it. And if you have an open position and you start getting x number of residents coming in. Ideally you would like some kind of algorithmic system to do some triage. Tommy okay, these are the recommended work thats going on. And were looking at more closely at these things or are they lower priority. And somewhat ironic twist, amazon decided he wanted to rate the applicant on a scale one five stars. The writing of prospective employees, same way that they cut product. But to do that, they were using type of computational language model called word vectors. And without getting too technical, for people are familiar with Neural Networks. These Neural Network models there varies successfully computer succession on 2012 also moved into computational things around 2013. In particular there was this very remarkable family model that were able to sort of imagine words as points in space and so if you had a document, he could. A missing word based on the other words there were nearby in this abstract space. With these models had a bunch of really cool other properties. And actually kind of arithmetic with words. You could do paying minus man plus woman. Answer sure the point in space was nearest to have and you would get clean. And you could do tokyo slightest japan plus england and get london. He sorts of things. So a numerical representation of the word, so that the Neural Network. Is been useful for the surprisingly vast array of tasks. In one of these was trying to figure out the relevance of a given job. And so what we could do it is just to say there are all of the resumes the people that weve hired of the years, those into the sport model final this point in space and friday new resumes close to see which of the words and that cv have the kind of positive attributes in which they would make it. Sounds good enough. But when the team started look at this, i found all sorts of biased. For example, the word women was cited with penalty. The Womens College or a Womens Society something. The word women that cv was getting a negative deduction. Michael so because is located farther away from the more successful words that is been trained to watch wds right. Brian this right. It doesnt appear on theypical resume they would get selected in the past in similar to other words in a fragment. So the red flag goes off. And we can delete this attribute for model. Start noticing those also applying deductions to like womens at sports like field hockey for example brightest in the get rid of that. Or Womens Colleges. So they get rid of that. And they start noticing that its picking up on all of these like very subtle fantastical choices there were more typical of male engineers resumes than females. So like executed and captured. Like i executed a to capture market value or whatever. Certain phrases that were more typical of men. At that point, the basically gave up. It scrapped the project entirely. In the book by comparing to something that happened in the boston Symphony Orchestra in the 50s when they were trying to make the orchestra which had been very male dominated for equitable so they decided to pull the addition behind the wooden screen. But somebody can only later was that is the audition walked out into the wooden parquet floor, and of course, identify whether it was a flat shoe for a highheeled shoe. So is not until i think the 70s and the additionally instructed people to remove their shoes before entering the room. And finally started to see the gender balance in the orchestra to start balance out. And the problem with these language models if they basically always hear the shoes. There always detect think the word executed or captured. The teams in this case just gave up and i said we dont feel comfortable using this Technology Read whatever is going to do, is when you identify and some very subtle pattern in the engineering that we had before in the gender balance is off their. Is just going to sort of replicate that into the future which is not what we want. So this particular case, they just walked away. It is very active area of research. How do you devise a language model. You got all these points in space and try to identify sub spaces within this giant triggered a residual thing which represented stereotypes and can you delete those dimensions. Its an active area is ongoing to the state. Michael published did amazon spend developing that. Brian is a great question. There. Tightlipped about it. People were giving their names and not doing a lot of followup. But as i understand, they actually not only disbanded products but also the team and reissued them to other areas. So the really wash their hands of it. Michael i am asking because i assume millions were put into that in the couldve hired other people. Another example is that self driving car. You talk a lot about this in the book. Well actually the fatality did happen because the way the car was recognizing this person. Again explained that. Brian yes, so this was the death of a gal in 2018. The 31 pedestrian killed by itself inc. The r d vehicle. The full a national highway Transportation Safety work review came out at the end of last year. So is able to get some of that into looking forward to press. Was very illuminating. To read the official break out of everything that went on because it was one of these things where probably six or seven separate things went wrong. And only then, an entire suite of things going on. It mightve ended differently. When the things that was happening was suzie and network to object detection. But it never been given an example of jay walker. So that all of the Training Data this model have been trained on, people walking across the street perfectly correlated with stripes. And they were perfectly correlated with intersections and so forth. As of the model just it really know what it was saying. And when it saw this woman crossing the street. And most object Recognition Systems are taught to classify things into exactly what of a discrete number of categories. So they know how to classify stuff it seems gone in more than one category. Or that seems like assign any categories of this is been again, active research problem. Making headway on it only recently. This particular case, the woman was walking a bicycle. So set the object recognition system kinda into the slumbering state where at 31 blush of the cyclist was she wasnt moving like a cyclist than if she was a pedestrian but in the shape of the bicycle in the navy is just some object this been sort of rolling into the road. Nothing it is a person know i think it is a biker. And due to a quirk in the way that the system was built