The Commonwealth Club of california. I am john zipperer, Vice President of media and editorial and a moderator for this program. We would like to thank our members, donors and supporters for making this and all of our programs possible. We are grateful for their support and we welcome others joined them and supporting the club drink these uncertain times. I am please be joined by brian christian, government author of the new book the alignment problem Machine Learning and human values. His best offer to first get a contribution to the Tech Industry and document in his best bestselling book the most human human and algorithms to live by. In his new book he poses questions such as every content neutral on Artificial Intelligence to solve our problems, what happens when ai itself becomes a problem . With degrees in philosophy, Computer Science and poetry, christian has spent his career tackling the ethical advancing society and become more reliant on technology. His work is got the attention of Technology Leaders such as elon musk once said of his 2011 book, the human human was his night table reading for its new book the alignment problem Christian Lazar depends on ai systems and questioned whether not it can help us. How can we get a handle on technology before we lose control . He argues although we train the system to make decisions for us eventual human when the humans we will be discussing about in the next hour i want to your questions. If youre watching live please put your questions in the text chat a few tips i can work them into our conversation today. Thank you brian. Welcome. Thank you for joining us. Its my pleasure. Thank you for having me. This is not your first book of course. I want to ask in the opening question the obvious question but i think is good job of setting the place for conversation that is why did you decide to tackle this topic now with this book . Great question. The initial feed for this book came after my first book and come out and as you would mention vanity fair reported that elon musk was his bedside reading and i found myself in 2014 attending a Silicon Valley book group that was a bunch of investors and entrepreneurs, and they had seen this thing about elon musk reading the book and invited him to join. To my surprise he came. There was this really fascinating moment at the end of the dinner when the organizers thanked me and everyone is getting up to go home for the night, and elon musk force everyone to sit back down and he said, no, no, no. Seriously, what ever going to do about ai . Im not let anyone leave this rheumatoid you either give me a convincing counter argument but we shouldnt we developed this or give me an idea for something we can do about it. It was quite a memorable vignette. I found myself drawing a blank. I didnt have a convincing argument why we shouldnt be worried. I was aware of the conversation around risk of ai. For some people its like a Human Extinction level risk. Of the people are focused more on the present day ethical problem. I didnt have a reason why we shouldnt be worried but it didnt have concrete suggestion for what to do. That was the general consensus at that time. His question of so seriously, whats the plan . Kind of haunted me while i was finishing my previous book. I begin to see starting i would say around 2016 a Dramatic Movement within the field to actually put some kind of plan together, both on the ethical questions and the sort of further into the future safety questions. Both of those movements have grown explosively between 2015 and now. These questions of ethics and safety and what in the book i describe as the outline a problem, how did make sure the objective the system isarried out is, in fact, what we are intending for it to do. These things have gone from marginal and somewhat philosophical questions at t margins of ai to really today making up the central question in the field. I wanted to tell the story of that movement and figureut in a way answering his question, whats the plan, what are we doing. One interesting things as i was getting io this was theres lot o very compl technology and program the goes to this b i dont think a lot of people are aware of how this is being applied in some even lifeanddeath situation in our society today. We were talking bore the start of the program that you psent a number of examples of ai tools to a conference with their hoping to perform one of her examples connecting more time and thats the algorithms being used by judges not just in california but, instead of cash bail a judge will use this algorithm to determine whether not a suspect is released or remains in jail while awaiting trial. This fall california has ballot proposition 25 that would reaffirm a law that would do away with cash bail and replace with this algorithmbased system. Its a very, get issue but what surprised when its looking at it was the naacp and Human Rights Watch proposed this proposition because of what theyre saying building inequities of the algorithm. Why dont we get into that example and build from there. Whats the problem with the algorithm and how did we get there . There is a nearly 100 year long history of statistical what were in 1920s called actuarial methods in her role. This attempt to create a science of probation and parole. That as i say started in the 20s and 30s but really took off with the rise of personal computers in the 80s and 90s. Today its implemented in almost every jurisdiction in the u. S. Municipal, county, state, federal. There has been an increasing scrutiny that is come along with that and its been interesting watching the Public Discourse have it on some of these tools. For example, the New York Times was writing the New York TimesEditorial Board was writing up through about 2015 these letters saying its time for new york state to join the 21st century. We need something that is subjective, that is evidencebased. We cant just be relying on the whims of folks in robes behind the bench. We need to bring some kind of actual science to this. Sharply that position changes and by just months later the end of 2016, the New York Times was running a series of articles saying algorithms are putting people in jail, algorithms have seeming racial bias. We need to throw on the brakes. Calling out by name this particular tool which is called compass which is one of the most widely used tools throughout the united states. The question, this really has ignited an entire subfield within statistics around this question of what does it mean to say that a tool is fair . This system is designed to make predictions about whether someone will reoffend if they are released, on probation or pending trial, pretrial. What does it mean to take these concepts that exist in the law, things like disparate treatment or equal opportunity, et cetera, 14th amendment protections, et cetera. What does it mean to turn them into the language of code and how to look at a tool like this and say whether we feel comfortable deploying this . You were giving examps of how a black suspect and a white suspect with similar crimes, similar backgrounds andow much more likel the white suspect was to go free, including surprising one of the white suspects it was [inaudible] what goes intohe baking of that cake that build devices in your question advices were intentionally baked into it but they are still hard baked in. This is a very good conversation. One placeo start is to look at e data that goes into the systs. One of the things that the syems are trying to do is predict one of three things. Lets think about a pretrial case for now. Typically a tool like compass is predicting three different things. One is your likelihood to not make your court appointments. The second is to commit a nonViolent Crime while your pending trial here if there is to commit a Violent Crime awaiting trial. The question is where does the data come from which these models are trained . If you look at Something Like failure to appear in court, if you fail to appear in court the court knows about it by definition, right . Thats going to be fairly unbiased innocent regardless of who you are. If you look at Something Like nonViolent Crime, its the case that, for an example if you hold young white men and young black men in manhattan about their rate of selfreported marijuana usage, they selfreport the use marijuana at the same rate and yet if you look at the arrest data, the black person is 15 times more likely to be arrested for using marijuana in the white person is in manhattan. In other jurisdictions it might be eight times as high in iowa. It varies from place to place. Thats a case where its really important to remember that the model claims people to predict crime but what is predicting is we arrest your so we arrest is this imperfect and systematically so proxy for what we really care about which is crime. Its ironic to me because as part of this project researching the systems i went back into the historical literature when they first started getting used, which was in illinois in the 1930s. At the time a lot of the objections were coming from the conservatives come from the political right. Ironically making almost the same argument progresses are making abut from the other side. Conservatives in the late 30s were saying wait a minute, if aa bad guy is able to evade arrest and the system doesnt know he committed a crime and the system treats him like he is innocent and will recommend his release and recommend the release of other people like him. Now we hear it being made from the left which is to say, someone is wrongfully arrested and wrongfully convicted they go into the Training Data as a bad person, as a criminal and it will recommend the detention of the people like them. This is the same argument just frank in different ways. Thats a very real problem and we are starting to see groups like, for example, the partnership on ai which is a nonprofit Industry Coalition of facebook, google and a number of groups, almost 100 different stakeholders, recommending that we dont take these predictions of nonviolent rearrest as seriously as we take for example, addiction of failure to repent. The second come over i want to highlight, its a very fast question but the second think thats with high sliding is this question of what you do with the prediction once you have a prediction. Lets say youve got a higher than average chance that youre ing to feel to make your court, scheded court appointments. Thats a prediction. Theres a separate question which is what do we do with that infoation. One thing you could do is put the person in jail while they wait for the trial. Ats one answer. It turns o theresn emerging body of research that sws things like if you send him a text message reminder, they are much more likely to show up for their court appoiment. There are people proposing Solutions Like providing day care servis for the kids or providing them with subsidized transportation to the court if thats an issue. Theres this who separate question which i as much is going on, as muc scrutiny is rightly directed at the actual algorithmic prediction, theres a much more systemic question which is what do we do with those predictions . If you are a judge and the predictions that this pern is going to fail to reappear, ideally you would want to recommend some kind of text message alert for thes opposed to jail. That may or may not be available to you in that jurisdiction. You have to that systemic problem. Thats not an algorithm per se and algorithm is sort of caught in the middle if you will. Lets take it out of the crime and punishment area into the business area. You talk later in the book about hiring. Amazon has come up with this ai system that would help it cull job applicants in with your fine is they were disproportionate a lot of men. The reason for this also worth baked into way the system was being trained in the way the system was being used and also when you get to the questions at the end of like why were you trying to find people who are just like the people you had . Tell us about that and how did they get so yeah, this is a story that evolved amazon around the year 2017. But by no means either unique example. Just happens to be the example of amazon. They like many coming for trying to design a system that could take a little work load off of human recruiters. If you have an open position you start getting x number of resumes coming in. Ideally you would like some kind of algorithmic system to triage and tell you these are the resumes that are worth forwarding on looking at more closely, and these are lower priority. In a somewhat cute or ironic twist, amazon decided they wanted to write applicants on a scale of one to five stars. Rating perspective points the same way amazon customers rate amazon products. But to do that they were using a type of computational language model called word and beddings and beddings. Without getting too technical for people are familiar with the rights of Neural Networks, these Neural Network models were very successful at Computer Vision around 2012 also start to move into computational linguistics around 2013. In particular there was this very remarkable family of models that were able to imagine words as these points in space. If you had a document you predict a missing word basin of the words nearby and abstract 300 dimensional space if you can imagine that. These models had a bunch of cool other properties. You could you arithmetic with words. You could do king minus man plus women and search for the point in space nearest to that and you would get queen. You could do tokyo minus japan plus england and get london. These sorts of things. These numerical representations of words that fell out of this Neural Network into that and useful for this surprisingly vast array of tasks. One of these was trying to figure out the quoteunquote relevance of a given cd to a different job. When we could do is just say there are all the resumes of people we heard over the years, throw those into this work model and find all those points in space and for any new resume lets just see which of the words in that cv have the kind of positive attributes in which have negative attributes. Sounds good enough. When the team at amazon started looking at this they found all sorts of bias. So, for example, the word women was assigned a penalty. If you went to a Womens College or on the Women Society of something, the word women on that cv was getting a negative deduction. Getting a negative rating or whatever becausets located farther away from the more successful word that it has been trained to watch for, right . Thats right. It doesnt appear on the typical sume that did get selected in the past and it i similar to other words that often appear. Of crse 15, the red flag goes off and they say we can delete this attribute from our model. They start noticing thatts also applying deductiontill like womens sport like field hockey. They get rid of that. Or Womens Colleges, smith college, so they get rid of that. And then they start noticing its thinking about all of these very subtle syntactical choices that were more typical of male engineering resumes than female. For example, the use of the words like executed and captured, like i executed a strategy to capture market value or whatever. Certain phrases that were more typical of men. And at that point they basically gave up and scrapped the project entirely. In the book i compare it to something that happened with the boston Symphony Orchestra in the 1950s where they were trying to make the orchestra of which a been very male dominated a little more equitable so they decided to hold the auditions behind a wooden screen. But when someone found out later was that as the addition or walked out onto the wooden parquet floor, of course they could identify whether it was a flat you or a high future. It was not until i think the 70s when he instructed the people to remove their shoes before entering the room. That finally start to see the gender balance in the orchestra started balance out. The problem with these language models is a basically always hear their shoes. Theyre detecting the were executed or captured. The team in this case just gave up and said we dont feel comfortable using this technology. Whatever its going to do its going to identify some very subtle pattern in the engineering resumes weve had before, the gender balance was off, so its just going to sort of replicate that in to the future which is not what we want. In this particular case they just walked away. Its a very active area of research. How do you d bias a language model . You have all these points in space and you try to identify subspaces within this giant threedimensional thing that represents gender stereotypes. Can you delete those dimensions while preserving the rest . This is an active area and it is kind ongoing. How much did amazon spend developing that . Its a great question. They are pretty tightlipped about it. Most of what we know comes from a reuters article where people were giving their names i wasnt able to do a lot of followup but as i i understand that actually not only disbanded the products but the disbanded the team that made an redistributed the engineers. So they really washed their hands. Im asking because however many i assume millions were put into that, couldve hired an extra h. R. Person or two. Thats right. Another example i want to get into from a different angle is a selfdriving car. You talk in the book about the photography that happened because of the way the car was recognizing this person. Again, explained that. This was the death of Elaine Hertzberg in tempe, arizona, in 2018, the first pedestrian killed by a selfdriving car with uber vehicle. The full kind of national highway Transportation Safety board review cam