That it is the present moment that we survived the shock at all is only possible because the past shoulders us on one side and the future on another. As fleeting a as if maybe it mas us human. Time travel a history. [applause] thank you very much. Its wonderful to have you here. The book will be available at the back of the hall. Thank you for being here today. Enjoy the rest of the festival. The effort of the equation, what about that, that is another balancing act. Said the effort he could h. And i think was my language that i came up with. Please let me not to be plagiarizinbeplagiarizing anyon. But its just an idea that weve come to understand not just environmentally that in our terms of psychic and emotional energy. So the more stuff we hav have, e more we have to organize and repair it and get new stuff when i think people would stuff is out of fashion and its just an endless cycle. So the identity is changing faster and faster in the way that leads to buy stuff to sort of symbolizing the word is changing and all of us are thinking about stuff in a different way in terms of wanting to get rid of it and not consume so much in the first place and highprofile this interesting story trying to make it so you give away your own stuff so its sort of this cycle and its part of telling people dont buy new stuff, which is a kind of deep shift in capitalism but i think they are genuinely interested and what does it mean to be a company that doesnt make new stuff that makes new systems and how does that human eyes and think all of our lives better and im very interested in that. The word enough, can you reflect on that . It is important to me in the sense that i felt like growing up this is typical of a lot of people in our generation. I was born the last day of 79 so i was kind of the millennial elder. [laughter] i felt like i was raised with this sense of bigness like what was i going to do in the world and its all about being privileged. The vocation was to save the world. So i think ive done a lot of spiritual work in my life rescaling my expectations not only for my own impact on the world with about where i want to spend my energy and time, but im much more interested in having in an incredible local network of community and friends than being famous or recalibrating all of that in a different way with enough attention and stuff. Just admitting that it is all finite and i think once you have kids, its hard not to realize because it is suddenly so in your face i feel like having children for any wake up and be like its all going to end. So this is all icons and ive got to do the best i can with every moment. But ithat is the especially haro wrap my head around. It isnt a word about the cultivate a relationship with terry it is lower expectations. Its like its sad which is why i wanted to write about the book. Book. Its a first generation that isnt thought to be better off than their appearance and there is a sadness like you want to be able to afford a big house you are not going to be able to have a fancy job and i want to say you know what, i am not sure that all of that is making people happy. You can be sad or you can look at ways in which maybe they are realizing the big stuff and fancy chopstick and create a lot of wellbeing anyway. And maybe it isnt so sad however we did find small. Maybe it isnt so sad to spend a lot of time investing in a Smaller Community of people maybe its the root of all began and we have to return to the to some degree. Weapons of mass destruction is about the misuse of state data and mathematical algorithms to make decisions that she argues should be made by thoughtful human beings. She talked about her book at washington, d. C. Its one hour and ten minutes. [inaudible conversations] thank you for coming up tonight. Out tonight. My name is david. I had to have couple of housekeeping notes. You could take a couple moments to silence her cell phone for any devices. I also just want to mention we will have a conversation. Raise your hand and i can bring this around so you can have all of the questions recorded and everyone can hear the conversation. If you would like to purchase a book [inaudible] if you look at the calendar you can find where these events are happening. [inaudible] its gone way beyond and we live in an age of the algorithm and increasingly whether you go to school, where we get a car loan, they are made by mathematical models and its done according to the same rule. But the opposite is true. Oneill is an academic [inaudible] turn to be a scientist at the Saint Augustine company and is one of the strongest voices speaking out how to influence our life and because it is implemented it as an unusual book that governs so many aspects of our lives. She has a phd from harvard and switch over to the private sector she left in 2011 and started working in 2013 on the data journalism. Shes a weekly guest and you may know her from her book [inaudible] if you could please join me in welcoming. [applause] [inaudible] maybe we can jump right in and get some ideas out there and talk about the examples of your book. We were chatting in the back as you all came in and i was saying one of the questions weve done is to take all of your data from trigger or facebook and use that. So how many of you have done this sort of Personality Test . It turns out a lot of businesses want you to take those that you dont have to take them anymore because you can find them automatically and then use that in ways that you may like or may not. We are going to talk about with a weapon of mass destruction is and then we thought we might start with an example and we can draw out the characteristics. There are some algorithms i care about and some i dont care about. The ones i care about our constructivareconstructive and e mass destruction characterized by three characteristics that we can start with an example to make it real. The example if you dont mind me starting their, there is this guy that was a College Student near atlanta and he wanted to get a parttime job at clovers grocery store. His friends were leaving so he said you just have to fill out the paperwork online. He starte started to fill out pk and like 60 of job applicants in the country, he was required to take a Personality Test online before he got the interview. So heated. It is a standard procedure for lots of minimum wage jobs. He failed it and he was lucky because most people dont find out they just never get a callback. He found out he got a red light and unusual in a second way, his father is a lawyer. When people apply for minimum wage jobs they dont have a big powerful lawyer said he talked to his father and he asked him what kind of questions were on this test because you are very qualified to get a job as a grocery bagger at the store. Hes going to a competitive college, got straight as in school, whats the problem. He said the questions were a lot like the ones i got at the hospital when i was being treated for bipolar disorder. It looks of things which are you an extrovert or introvert are you anxious or do you get angry easily or super mellow and nothing bothers you. Its easy to find that out on a test but it turns out it shows up in other data that we leave behind. That is so frightening. So what happened, he talked to his father to talk about the fact that it was his father was a wonderful man and said that is illegal. You cant make them take a health exam including a Mental Health exam. But the americans with disabilities act, to prevent this sort of creation of an underclass of people with disabilities and Mental Health problems. So his father is pursuing a lawsuit on behalf of anyone that ever took this test and i should add by the way that he tried to get a job at six other Large Companies in the atlanta area and got the same red light for all of them. He was systematically prevented from getting a job in his area. So thats the first example i want to talk about. That takes us towards this definitional collection. A lot of these insights about people come from these models built on algorithms and a whole bunch of data and the fact that they can harm people on the surface it looks like the nature of the weapons of mass destruction concepts. First it is widespread. So its like the one ones i buin my basement because thats what i do, i go about this all the time. They start caring and should start caring where it affects a lot of people in that this is an example because it was widely used. Criminals to do with this algorithm are located in boston and sold it to all sorts of companies. So it is widespread. Second, it is secret so he didnt understand how he was being scored and most people that took this test didnt understand that they were being scored. Finally, it is destructive so it destroyed his chances of getting a job but it is destructive in a larger sense in that they actually sort of create and reinforce and hear the larger feedback is systematically refusing to give to people with certain disorders. One of the examples you open with i just a secretive the seck box nature of the algorithms including some really crappy ways of validating them and intd about is getting schoolteachers fired so maybe we should talk about that. Second example. So, she was the School Counselor for some time and instituted this policy whereby some people would get fired if they had been a Teacher Assessments and some would get a bonus if they got good Teacher Assessments. The assessments are complicated but the short version is most of the way that a teacher is assessed as they dont have a lot of spread so most people get acceptable or good. People that want to disconnect between good and bad are discriminated to become frustrated. They want more. Like people that are terrible, good, fair, better, very good. So very few of the ways we now assess teachers. They instituted this assessment called the growth score or the value added a score of a teacher. I wont go into it technically, but the very broad way of thinking about this is the idea that a teacher is on the hook for the difference whether the students should have gotten versus what they actually go so igot soit is this underlining ml that estimates what each student in the class should get. So if im teaching you, you are all fifth graders right now can at the end of fourth grade you got a score on your standardized tests. Lets say you got a 75 out of 100. You would be expected to get Something Like 75. Lets say youve got 80. You got more than you were expected. Does that make sense. So now help is expected t this e is determined per student and its actually really complicated and another source of uncertainty you got at the end of fifth grade because after all kids get different scores whether they had breakfast into that sort of thing. So there is uncertainty and the teacher, again to remind you to teacher is on the look for the difference between those two things. Now, actually if you think about the original score, the difference between the expected score and the actual scores called the error turned. Its also called noise. The students to be good teachers are being assessed based on this which is if youre not a statistician i dont want you to be confused. All im saying is we would call this stuff pretty meaningless and in fact the way they came out were almost meaningless and i say that not because i have hard data on that because it is a secret score. The only way i got my hands on any of the data o besides the stories i talked to one teacher got a six out of 12010 and 96 out of 12011, he was from new york city. In general, the secret system is another source. So anyway, going back to washington, d. C. , i interviewed a woman who actually got fired because her overall Teacher Assessment score was too low. It wasnt entirely due to her valueadded score. 50 of it but the other 50 worthies things so most of the information was this terrible valueadded score. One more thing to tell you about this score is a lot of folks, a lot of fourthgraders got good scores at the end of fourth grade. Kids coming into the class of fifth grade had excellent scores but they couldnt read and they couldnt write. So she was suspicious of their scores and has every reason to believe the teachers cheated on the end of the test in order to get better value added models for themselves because they got better than accepted so they set her up to get worse than expected. Does that make sense . So every reason to believe. Its never really got systematically investigated. But in any case, she got fired. Lets go back over the three characteristics. Its widespread. They are being used in more than half the states mostly in urban school districts. When she appealed it she said its mathematical. Its where they dont use the schooling system and then the final thing you will be surprised if you know about this sort of nationwide war on teachers going on. We can talk about whether that even make sense but in any case it was to get rid of the bad teachers and the larger feedback loop is that its just getting rid of good teachers because good teachers retire early and they dont have this scoring system and right now we have a shortage of teachers so i would argue that this regime this valueadded model is very bad and i think something that is especially insidious about this, we talk about the algorithms and they can sound very foreign and things we dont interact with a lot of the algorithms we are talking about here have a lot of similarities and are identical like on netflix or amazon recommends shows and books for you. We totally know how to handle those. Amazon is like you bought a stephen king book. Its right but not helpful. [laughter] sometimes it is totally wrong and youre just where did this come from. Sometimes you get these beautiful insights like this wonderful guide it recommended i buy this knife set. I never would have thought about that. You dont just buy everything amazon recommends or watch everything netflix says to watch but the algorithms youre talking about, people dont treat them as one point in the decision process. They treat them as the oracle of truth even when they are poorly validated and that make them a lot more destructive because it overrides all of the tuition with this little bit of mess. I would argue it is worse than what you just said. Let me give you two examples. You know how the face but trending story, guess what, we notice and give feedback to facebook saying that isnt a real story. Thats something that doesnt happen for the teacher valueadded model. There is no ground truth for the teachers. There is no secondary assessment to compare the scores to. They were just told that this s your score. When i got six out of 100 was ashamed. Next year he got 96 out of 100 he was like what does that mean i went from being terrible to a great teacher . Theres nothing to compar compat to, no feedback. The teacher cant say im actually a really goo good teacr so please update your model. Thats the equivalent of a saying i dont want to see that movie because it is awful. Please dont show that timmy. I dont recommend anything like that to me again. We have ways to tell it but the teachers dont have access to that. Or parents or administrators. They couldnt go in and override. Exactly. So the other thing i want to agree with is that people really do trust the scores because they were mathematical. The way that i came across this algorithm is she started complaining to me about the teachers getting scored and i said can you tell me how they are being scored, like what is the algorithm and she said i asked the department of education and they told me you wouldnt understand. He said that isnt good enough. Mathematics are supposed to clarify, not confuse. So that isnt something you do with math. That is recognizing math. So i keep pushing through each layer. Then she finally got this white paper that wasnt readable to me and i cannot understand it so to get the source code for the algorithm i should say the reason i thought about it is the New York Post had the information act request it had successfully got all the teachers scores so they publish it. But if they can get their scores, it makes sense. I was denied and then i contacted somebody at the institute that built the score and they said i would never again because they had a contract with the city of new york. This was to say secret from officials in the department of education so they cannot explain how they are being scored. Part of the book is the argument i thought maybe you could give us an overview on that because you are building these for a while then he became the enemy of the finance. Joining up by paul street is probably not something youre hedge fund would have smiled upon. I joined by hedge fund in early 2007 and headed straight into the crisis. I was really disillusioned very quickly because the people i thought were experts really didnt seem to know much about what was going on. I understood perfectly that they didnt see it that much better than i did and in the heart of it, by the way thats why i fell in love with math because like how beautiful is the rubiks cube. I had this idea of math being this pure, beautiful, almost artistic and better and then a part of the financial crisis was this mathematical why do the opposite of the beauty of the rubiks cube, the aaa ratings of the mortgagebacked securities which, you know, they are mathematics. They are promises that people that were good at math with a phd o were backed by crunching e numbers and promising mortgages were not going to go bad. They were not doing that. They were creating a lie and selling it for money. The aaa rating people believe that so much the scale of the market was very large and one of the reasons it was such a big deal. Back to one of the things i realized again it was the weaponization of people trusting mathematics and the math itself wasnt the problem. It was that people were basically corrupt and shielding themselves by calling it math and a saying dont look here. And its right, you have to trust it so thats the thing i realized. Then i was like im doing the same thing. Thats what the Data Scientists did, they use Historical Data to predict people. Some of you are predicting personality traits and that didnt seem so bad until i came across this sort of teacher valueaddeor that teachervaluet mentioned. Then i think the moment i decided to quit my job and write this book is the one interaction i had in the startup where the venture capitalists came to visit and he was thinking of investing in the company so we all sat and listened to him talk to i was working on advertising which was relatively benign you get opportunities for some people its a way of segregating society. But i still dont think it was particularly evil. He gave us this idea of what his future dream for the world of the internet was and he was like i have this dream that someday it didn