Historical associations congressional briefing on historical perspectives of artificial. Im alexandra levy, Communications Manager at the American Store association, and i coordinate the briefings for the they thank you all for taking the time to be today. Im going to provide a little context for what were doing here this morning. The chase congressional briefings program, which dates back to 2005, provides congressional staff, journalists and others in the policy community with the historical background necessary to understand the context. Current legislative concerns. The program rests on the premise that everything has a history and that anyone involved in policies and decisions of any kind, whether in the corporate government or nonprofit worlds, ought to understand how positive hints and policies have shaped the current landscape. Our second premise is it is possible to provide that Historical Context in a manner that is nonpartisan and unencumbered with a particular legislative agenda. And we ask all of our speakers to adhere to that standard. The fha is grateful to the Mellon Foundation for providing the Financial Resources that make this work possible. We hope to see you at future briefings and encourage you to email us any ideas for future briefings. We have over 11,000 members and can find a historian to speak on any topic. Todays briefing will be moderated by matthew jones. Dr. Jones is the smith family professor of history at Princeton University in 2023. Norton published his. How did a happened. A history from the age of reason to the age of algorithms written with with chris wiggins. He is completing a book great exploitations on state surveillance of communication and information warfare. His previous books include reckoning with matter, calculating machines, innovation and thinking about, thinking from pascal to babbage and the good life in the scientific revolution. Descartes. Pascal, leibniz and the cultivation of virtue. I will now turn it over to him to introduce the panelists. Thank you, alex. And thank you for of you for coming today. Were enormously lucky in in todays congressional briefing, ai to have two real world experts on the history of ai and its broader place in the development of computing within society. And one expert on the use of ai style tools for historians confronting the vast amount of documentation that we have in in more contemporary forms of forms of history. There are lots of things that historians can bring. Indeed, everything has a history. But in particularly in questions of history, of technology, historians offer an opportunity to get leverage that the history is one that is continued and that is not simply driven by technology in which human choice is crucial, even if it often doesnt seem that and are our our commentators today can illuminate fundamental technical questions of the development of this technology, the privacy and ethical challenges, the threats to expertise how the threat of automation which long come in many other domains, is now threatening many of the sort of most domains we can imagine from Clinical Knowledge to legal knowledge to the practice of history, questions of Environmental Impact and the challenge regulation, good and bad. Theres a lively debate, as most of you will know currently around ai, that questions whether we ought to be focusing on harms in the here now to existing communities versus speculative harms that often take an almost apocalyptic character. Likewise, there is the possibilities that i might provide for people doing historical them for near and for the societies in which we in which we live, which produced documentation that such a scale that the traditional tools of the of the historian often are inadequate or appear inadequate to deal with. So we we have a moment here to think with these wonderful historians, to think about how technology can best comport with the values we have. And ill introduce our three speakers. So first, were going to hear from Geoffrey Jeffrey yost, who is a Research Professor of the history of science and technology and is the director of the Charles Babbage institute for computing information and culture at the university of that is the premier site in the United States, if not the for documentation about the development of Information Technologies computers. He is published seven books, most recently a coauthored wonderful textbook called computer a history of the Information Machine and a crucial book in the labor history of computing called making it work. He coedited a book series called computing culture for John Hopkins University press, and its a journal called interfaces, which i highly recommend too, and founded the Blockchain Society blog. Secondly, our second historian and again, were enormously lucky to have both of these historians of computing is janet abertay, a professor science, technology and society at virginia tech. She wrote to transformative volumes one inventing the internet, which really was the first scholarly attempt to understand how it is that the internet came to be, how it emerged differently than. Many of its progenitors imagined. For good and for bad. She also published a transformer of a volume called recoding gender about womens changing participation in computing, which i can testify is often transformative for students in reading. Most recently, she has an edited volume with stephanie of simon fraser called abstractions and embodiment, which collects many of the sort of most lively historians of computing today. And shes writing a history of Computer Science as an intellectual discipline. So. Yost and and are historians of computing very and the best Matthew Connelly who is a professor of International Global history at columbia and is director of the center for the study of existential risk at cambridge in the uk is is a historian who has pioneered the use of a large number of tools that we now would associate with ai for the study of history. He is the principal of the history lab at columbia, funded by nsf, an nih that data science to analyze data. He has numerous books, including a diplomatic revolution, fatal misconception and most recently and most relevant to discussion here the declassified declassification engine. What history about americas top secret . With that, i will allow the three speakers to talk, after which we will have what i hope is a lively question and answer session. Unfortunately, we have to be done by 1015 at the latest. So if youre asking too question, i will ask you to be concise, which i know for the former academics in the room is a challenge. Okay, thank you, matt. Ive titled these remarks exploring ais dual contextualization and why metaphors matter. Artificial intelligence or a. I. , paradoxically, is overhyped and underappreciated. Is overhyped in a misplaced Science Fiction sense of the nexus of an existential threat. Machines will not take over job losses will occur, but forecasts are exaggerated. If history is a guide, it will be far less than the 300 billion or 400 billion lost or drastically transformed jobs that Goldman Sachs and mckinsey have predicted. Automation far better descriptive term than intelligence thats been hyped three quarters of a century. A quick shift or a cashless idea has been predicted by Management Consultant for 60 years. Electronic transactions have grown, but we still have cash. Air is under appreciated both in terms of its long history and with recent generative ai. The risks of amplifying social biases. Generative pretrained transform. Murmurs their gpt is are new, but they extend from Neural Networks and Natural Language Processing Research areas commencing seven plus decades ago. In 1943, warren mcauliffe and walter pitts conceptualized their first algorithm. Algorithm like neuron natural language. Language processing took off in the 1950s with noam chomsky, his work. My comments today will focus on what i term the dual contextualization of ai, namely taking history and data of context. So in a name. How did we get the term Artificial Intelligence . And why is this important . Prior to World War Two and the advent of Digital Computers, we had mathematicians speculating on future computing machines. In retrospect, this was the start of theoretical Computer Science. By the mid 1930s, alan, was this Research Areas intellectual leader. His 1950 paper computing machinery and intelligence introduced the imitation game as a close analog to thinking can a human correctly guess if a human or machine is responding to them . Thinking or intelligence, however, is more than imitation. This was emphasized by philosopher john searle in 1980 and his critique of turings 1950 paper in late 1940s, 1950s. Journalists and scientist as use terms electronic brains. Giant brains refer to early Digital Computers for. Example scientist edmond berkeley framed. Simon edmund berkeleys famed 1949 book giant brains machines that think so in 1956 and organizing a summer workshop of top scientists exploring ideas of Information Processing and automation at dartmouth. John mccarthy coined the term intelligence. He did so with intentionality. You correctly thought it would help raise military funding for this area of research. It matters little that a. I. Then and a. I. Now was not and is not intelligent and then and ai now was and is not artificial is mathematics abstraction and automation depends heavily on labor and guidance. I was and is a misnomer and distracting metaphor. Responding in 1950s. Responding to 1957. Sputnik. The dod in 1958 founded the advanced Research Projects agency or arpa. In 1962, jcr licklider became the founding director of arpa Information Processing techniques, office or dto in. The 1960s arpa zapdos showered funds and for Computer Science areas. A. I. Graphics, timesharing and networking of the arpanet in the 1960s, a a. I. Labs at carnegie mellon, mit and stanford. Despite generous of funding efforts, floundered relative to grandiose rhetoric and promises by a. I. Researchers, there have been successes in limited domains such as computer chess. But the dod cared about transformative military and economic applications. Hence the 1970s winter or scarcity of research funds. This winter ended with of japans fifth generation a. I. In 1982, catalyzing a flood of u. S. Government funding for a. I. A new summer in the late 1980s and other winter began, and a late 1990 short summer with the dot. Com bubble. A longer winter when it burst and the current summer of the past decade, plus financial greed as well as real and a proper rhetorical fear of or military adversaries drive, hype filled air summers fear of japan in the early 1980s and of china today. Conversely, under delivery of research and recognize an overblown hype leads to winters riding. The seasonal coaster. I expert systems were invented by stanford Computer Scientist edward feigenbaum. He partnered top scientists Joshua Lederberg and edward short to pioneer such systems in the 1960s and 1970s. Short list 1970s era expert system meissen for medical diagnostics drew on an inference engine and database. It could be a tool for advancing evidence based medicine help avoid physician case bias had perfect memory and was never fatigued. It was also overly hierarchical and physician were mixed on its usefulness. My sense for antibiotic prescriptions could reduce overprescribing to combat community antibiotic resistance short less follow an anxious then aided with prescribing complex chemotherapy regimens in medical expert systems, limited domains, transparency, User Feedback and context were critical and they remain a fundamental lesson from history. Jumping to Large Language Models of the past decade, they surveil and they scrape the web publications, other data sources indiscriminately and out of context. The systems are opaque and complex and quality metaphors can help to better understand. Georgetowns helen toner has argued for an acting improv metaphor for a. I. Chat bots. I respectfully disagree. That metaphor highlights machines, humanlike intelligence and creativity. They do not. I much prefer a computational linguist emily bender at all. Stochastic parrots metaphor for generative a. I. Parrots are not human, but they put together sentences, ones devoid of larger context. This is spot on to characterizing how these systems work. Rules and statistics. Many of the responses reasonable hallucinations are a metaphor for a. I. Bots, irrational outputs. But this metaphor is misleading. Conjuring infrequent software bugs. It is not as efficacious in all machines where slip the brown. The very model of is scraping data without proper conduct context. Hence hallucinations are inherent to the system. Hallucinate just imitate non hallucinations. Gpt four uses far more Energy Search as an information seeking tool. Some estimates place it at ten times search regarding cybersecurity, a. I. Seemed in balance, helping white hats protect as much as giving black. Black had hackers new tools. With recent generative a. I. Surveillance, concentration of data and deepfakes, security and privacy risks shifting dangerously in black hats favor openai founders chose its name to associate with the half century old open Source Software movement at islamia to largely models promoted as open source. The open Source Software movement and is a force for sharing and democratizing computing, as signaled president Meredith Whittaker has pointed out. Open and generative a. I. Is quite different and presents significant risks, especially concentration of power with the largest a. I. Enterprises. Those with the most surveillance and data and their models. To sum up, automation is a more accurate descriptor than Artificial Intelligence. Decade long hype cycles followed by winters are the norm. A. I. Is more useful and safer with domain specific and transparent datasets. Chart bots are Energy Intensive blunt instruments for information seeking a. I. Outside of Large Language Models has much to offer, especially in medicine. Generative a. I. May add to productivity in some areas, but questions of quality and remain largely language model based. A. I. Takes data out of context and as such carries substantial risks. Accelerating surveillance, misinformation, proliferation cycles and deep fakes. Along with amplifying racial gender ableist and other biases. Thank you. Thank you, matt. Thank you all for coming here. And its a for organizing this. Ill be using historical examples to unpack a seemingly simple question how do we define success for a. I. . History shows theres been no one definitive answer. Instead, the success or failure of a. I. Depends on what we think it means. Be intelligent to be human and to adequately solve a problem. The notion of intelligence has many different facets that can range from logical reasoning to creative intuition. Emotional intelligence, and the wisdom of experience. We expect human beings to possess all these kinds of intelligence, but computer intelligence, a. I. Has been defined in much more circumscribed bad ways. A few examples will illustrate it, as dr. You just mentioned, one formative idea was the turing test or imitation game proposed by alan turing in 1950, in which a computer is considered intelligent. If it can fool a person into thinking is a human being. The rationale is that if people are intelligent and a computer can act like a person, then it must be intelligent to notice that this is an objective definition of intelligent behavior, but essentially says that intelligence is in the eye the beholder. This idea that i should attempt to imitate how people express themselves in language has resurfaced more recently with generative a. I. A remarkable thing about turing test that is often overlooked is that it essentially equates with the ability to tell convincing lie. The goal is for the machine to deceive and this again anticipates current ais tendency to produce made up results that merely look plausible. Socalled hallucinations. But this is only to be expected if the standard for intelligence is imitation, rather than authenticity or accuracy or accountability. In the 1960s, the intelligence of asis atoms was often measured by their ability to play and win the game of chess and more recently, the game of go been added to that that benchmark. The choice of chess as a measure of intelligence, cultural stereotypes held by largely white male researchers about what types of mental tasks are challenging and worthy of effort. One could argue, for example, it takes more intelligence to raise a child than to win a game of chess. Chess is a very abstract bound game. And of course, its also zero sum game of war. It has no practical utility. Its not a universal human experience. Its its an elite version of what it means to be smart. This shows how benchmarks of intelligence always encode social values that we should be aware of in the seventies and eighties, socalled expert systems gained popularity these systems and attempted to incorporate the Knowledge Base of human experts in order to produce results in specialize domains such as diagnosing disease. Here, intelligence refers to deep knowledge in a limited area that has an identifiable body of data rules for applying it and clear benchmarks for success. This type of expertise practical value and is assumed to be scarce which motivated use of expert systems in a variety of application areas. For example, the oil industry used it to analyze samples and many businesses used it for economic forecasting. So an important point here is are trying to replace intelligence thats actually scarce and valuable or are you trying to replace something that anybody can do . These types of systems are still very much part of current a. I. Expert systems are sometimes seen as something from the past, but actually still alive and well, tend to be playing more of a supporting role and not really advertised as much. And theyre often combine with language systems as a user interface. So another thing to keep in mind is that many a. I. Systems are hybrid as they combine the advantages and pitfalls of multiple techniques. The point of these examples is that a. I. Systems dont have general purpose intelligence. Theyre designed to do specific things that mimic fairly narrow portions of human intelligence. So it might be beneficial to stop using the Artificial Intelligence and instead be more specific about what aspect of Intelligence Systems to display and how it attempts to do that. Its also important to recognize what is left out of a. I. Models, intelligence. Human intelligence is not just about getting facts right or winning games. An important aspect of human intelligence is social intelligence and moral reasoning. These kinds of intelligence stemmed from living as an embodied person in a society with other people whose actions have consequences for one another. A computer has no social ties or obligations and it has nothing at stake. You cant put it in jail or hurt its feelings. We have no way to automate moral responsibility or an ethic of care. Historically, a. I. Research has prioritized disembodied, socially disconnected forms of intelligence. While this work well for playing chess or operating factory robot, its problematic for current uses of a. I. That are meant to occur within or take the place interpersonal social interaction. This is one reason why it would be beneficial for the people who create a. I. To be more representative of the u. S. Population, which is currently not the case, so that we can a wider range of social experience to the design process. A briefly touch on the question of what it means for a. I. To replace a human being, which dr. Yosts already referenced. Human intelligence and human labor have always and continue to be part of the systems that make up a. I. So humans are never really totally replaced. Human experts provided the knowledge for early expert systems, human generated data transfer, a. I. And human workers tag images, flag prohibited speech and edit. A. I. Generated. But these human inputs are often hidden. Sometimes so. Sometimes not. Which supports the fiction of a. I. Being autonomous and that it could actually completely replace the human being. The other point i want to make is that a. I. Works best at producing, expected or stereotyped version of a human being for example, weve probably all encountered service chat bots that follow scripts and these work because were expecting a particular kind of interaction. So are following our half of the script and were not trying to have a general conversation in the chat, but given familiar cues, human users will project intelligence the software. An early example is a 1960s program called eliza that imitated a therapist. Eliza convinced many users that it was actually selfaware and, was responding thoughtfully when they confided troubles to it through the keyboard. And this was not even intended by the designers. There was somewhat horrified by. But people just kind of fell into that interaction. So part of the success of ai in imitating or replacing human is that we see what we want or expect to see and we respond to conversational with our social instincts. We fill the blanks. So this is another way that people are doing some of the work that makes a. I. Appear to be intelligent. Finally if the purpose of ai is to answer questions, solve problems, we need to ask, what does it mean for i to solve a problem . How do we know if the answer were given should count as a solution . History shows a wide range of definitions for solution in the paradigm by paradigm example of chess. The goal is to win win the game and the rules for winning are clear. But most ai solutions are not as black and white as chess. For an expert system, rules are encoded that ideally lead the system through a series of decisions to arrive at a correct answer, a finite set of possibilities. For example, a medical diagnosis. In this case, the correctness is not. But the computers answer could be compared with an experts diagnosis. Or you could wait. See how the disease progresses. And retrospective judge the answer because is often done in expert systems tend to be used in real world cases. Eventually you will know the answer, the quality of the system of the solution depends on both the quality. The expert information that was built into it and on how variable and unpredictable the domain is. If things are mutating or changing rapidly. The system might not be able to keep up for pattern, which underlies many current. An ordinary person often decide if the computer is and all of this. Do this every time you take one of those online tests to youre not a robot. You know, clicking on all the pictures of a motorcycle youre actually training an ai recognize images. But this also means that there is no objectively correct answer. If the people looking at the picture say its been correctly labeled and thats considered the solution, if these people biased, those biases will become part of the system and shape future answers, which has already happened with labeling of images for for systems like chad gpt. The answers are even more subjective since its completely up to the user to decide if they like the result. Notably, there is nothing into the system to guarantee that answers are factually correct or even make. So its important to be clear about our criteria for success. When we talk about a. I. Are there objectively right answers, or is it an imitation game where correctness is in the eye of the beholder. Less obviously we need criteria for failure. Failure is not just a lack of success. For example, many generative a. I. Programs are blocked from creating hate speech, even if the user requests it. In this case, the users criterion for success conflicts with a public interest. Criterion for failure. As another example spread of disinformation on social media is a failure for society. But its a technical and economic success for the platforms. The algorithm is doing what it was designed to do by promoting speech that gets a reaction. So failure or failure is not just a matter. Not meeting the users or designers criteria. Failure could also be defined by a set of principles that we as a society dont want to be violated. We could decide that a system has failed if it produces gender bias. Identity theft, deepfake, pornography or other harms. But experience that. What we cannot do is. Expect an ai to be able to make that. Thank you. All right. So thanks, everyone. Thanks for coming. Thank you also for the organizers and and the hra. So i opened my laptop here because i asked chachi to compose my remarks. Im just going im just going to read them out now. But i do have some remarks so. Im going to mainly focus on uses of ai and how i think they actually are and will increasingly be indispensable for doing historical research. But let me i just want to quibble a little bit with my colleagues. So i completely agree with jeffrey yost how one of the best ways to evaluate predictions that people make about ai and Everything Else is to look at the track record in that particular domain, to see how well those predictions have have been accurate in the past. But one thing ive been struck by is how in recent years, the people working on ai, if anything, theyve been exceeding expectations and think thats one reason why a lot of the people working in that field have revised their own expectation as to as to what we might see going. And another, you know, thought that occurred to me in listening to to janet is how if were going to move the goalposts and decide that what we really expect from Artificial Intelligence is not that it can imitate humans a convincing way, but that its going to be authentic, accurate and reliable. Then our expectations are going to exceed the capabilities of many humans. So. So anyway, i theres a reason why were here right . I mean, whats happening in generative ai right now is really remarkable. I mean it really is extraordinary. And it really has changed the way ive been thinking about this. Most of us working even adjacent to the field. If you wanted to show you knew what talking about, you didnt even use the term ai. You talked about Machine Learning because i agree that a lot of what were talking about is, i think, more accurately described in terms of algorithms. Right. And rather than something resembling human intelligence. But i think one reason why even some of us who think we know what were talking about are starting to use ai again, is actually some of systems really are in terms of their and the things that they can do. They really are beginning to resemble the things that humans do in some ways are all too human and and reproducing our failings. But the only reason we can talk about these things as historians is because we have records to work with. And so most of what going to talk about is how it is that ai is going to be essential to do history. Now, one thing we havent really talked about is the other ways in which historians be contributing to the conversation. We havent talked much about, though, jeff did. He alluded to the idea that Large Language Models, the idea that theyre going to take the world is Science Fiction. And if you describe it that way, i to agree. But one thing that, you know, weve been talking about even before we started this discussion is how there are billions and billions of dollars being invested in thousands and, thousands of very talented and capable people, all working to make that nightmare a reality. So and many of those people themselves are about the way theyre working and the direction heading and the things that people talk about that might, you know, mitigate these risks. Theyre not doing any of those things. And so i think the real there really are reasons to be worried. And theres actually a lot of new and interesting history. History Research Studies thats been done on what we could learn from the past. You know, in terms of trying to control a Dangerous New Technology and some of it is reassuring. You know, if you look, for instance, at the history of different kinds of risky technology, you dont find in every instance, in fact in most instances, these technologies are not developed. You know, people have crazy ideas and typically they dont pursue them, or at least to the point of of actual products. Right. That could disrupt the world. Similarly, theres a certain idea that theres a logic to arms races that we have to obey that logic. But actually history doesnt reflect that. And typically, you know, we find that, you know, the logic of political scientists doesnt describe the world very well. And then finally know theres a history of a Regulatory Agency so we could benefit from i mean, theres a lot of history thats super relevant, you know, to the problems were facing now and how we try to control a. I. But again, you know, where do we get those insights from . You know, they depend on our collective capacity to preserve a record of the past, and that depends on the work of records managers, archivists and librarians. Now, if you talk to archivist yes, they will tell you that world is changing. And it has been for decades. Now, historians are just starting to realize this because most of us work on stuff from 30 and 40 years ago. But when i started work on, you know, the history of the 1970s, its the beginning of electronic Record Keeping that is at least textual records. And the u. S. Federal government was an early adopter. The state department developed a system because during the nixon administration, they thought it would help them to find leakers. They that if they could track that people had access to different records, theyd be able to find out who was leaking them. Its ironic, right . Because the same, you know, database that they began to create the central Foreign Policy files is the same that wikileaks eventually shared with the world. So now you can leak at scale because these systems are originally at least the very beginning, created to try to stop leaking. And i think its particularly this world you this dark world, you know, the dark state i sometimes call it it produces these dark archives as jason barren and others have described them that are filled with classified information. And i think that this is some of the most Important Information of all, because this is the information that determines whether you and i live or die. Right. So i think its particularly important that we understand whats happening with that history, because that history, most of it you know, as jason would say, and he should know, he was with the Office General counsel for the National Archives. He would tell you that, you know, 99. 75 , for instance, president ial records, electronic records had never been reviewed. Were getting a tiny sliver of history and the rest of it is invisible to us. And that whole world that whole world is in crisis. And there are three main causes for that. One of them is all the pathologies, the dark state, what i call the dark side. I call it the dark state, not the deep state, because in fact, nobody loves secrecy more than president s and ultimately, president s really are sovereign over what secret what the rest of us are allowed to know. And theyve kept it this way because serves their interests and what theyve kept in place is the system and where all the incentives are for keeping secrets. And there are no incentives to release information to the public. But who benefits from that . Well, you know the approximately 5 Million People who have security clearances and especially the people who at the very top of that system, the ones who have not the need to know. Right. Who are able to access information types, levels. But they have access to all the special Access Programs and the rest of it. And the system such as it is reviewing those records, deciding what the rest of us are allowed to know is the same system weve had for the last 80 years. And in fact, you know, by an act of, you know, ever since the cairo amendment, theres a legal required that every page that might possibly have information related to Nuclear Weapons has to be reviewed page by page. You. So what does that mean . Well, it means that, for example, when theyre reviewing the records of the clinton administration, theyre already reviewing terabytes of data. So theyre not textual records. Like most of this data is not textual in form. But if it were textual. Were talking about 86 million pages of documents for each. The Trump Administration was only one administration compared to clinton that produced hundred 50 terabytes of data. So i just think what would mean right to to review all of that page by page or even a significant portion it. To take one other example, ten years ago, the state was producing 2 billion email a year. How do you think were going to get to see any of that without using a. I. Now, the third problem is underfunding the mission. The National Archives has grown, you know, tremendously with historians of mathematics. I hesitate to use word exponential, but i think its appropriate in this case. But i think to draw a historical analogy, its a little bit like the veterans, just like the the National Archives has to with all the legacies of all our wars and all of our fiascos, Foreign Policy and over the last, you know, many decades, its very technology, more and more so its skilled. Now, imagine if the Va Administration had the same budget it had 30 years ago. Just think what va hospitals would look like. Thats what the National Archives are like now. All right. So why does this matter . Well, if you think that at least some of this information could get people killed, then you should care about the fact that the systems out of control. Right. And i think we all know examples. You know, how airmen, second class tactics, era somehow managed to release information again, i think actually was dangerous war plans and whatnot. Some former president s have been in their custody of classified information, but this is going to keep happening. I mean, think of the office of Personnel Management hack millions of people who underwent the process of being reviewed for security clearances shared the most intimate information about their drug taking, their affairs, their Mental Health histories and all the rest of it. All that information is now in the hands we think of the chinese government. None of it was classified. Its just one example of how our government is failing, you know, to prioritize the records that really are dangerous that really could hurt people. But also matters to us, all of us, not just historians. But, you know, if historians cant do our work, right, reviewing the records of the past and trying to hold Public Officials accountable, then citizens cant do. And if policy makers working the cloak of secrecy, if they cant be held to account even in the court of history, they really are accountable to one. Now, this might seem like an exaggeration. Let me just give you two examples by act of congress, by law. The state department is responsible for producing an accurate record. American Foreign Relations called the Foreign Relations the United States. And theyve been doing this since the civil war more than 150 years. You know, year by year they produce, you know, typically thousands and thousands of declassified records. Now how much have they produced in this year, in 2023 . How many records been declassified that are part of the official record of american Foreign Relations . Does anybody. Zero zero. All right. The information security, oversight office. A tiny, underfunded and relatively powerless part of the national. It was put there so they couldnt mess with anybody else. They put them there so that they could produce reports. Again, this is a legal requirement. Theyre supposed to track how much the government spends on secrecy how many records they review and release each year. Since 2017, they havent able to produce a single estimate. And the last time they had that office produced a report, he said that they could no longer keep their heads above the tsunami. Okay. So this is history, right . This history, even if you back, you know, i think a lot of us would think maybe the history of the 1973 war is relevant. Maybe we could learn something from that. Maybe we could learn something from the history of the end of the cold war. Vladimir thinks that history is relevant. Hes constantly using it as a weapon against. The United States and our allies. Now, i cant comment on pending. Im not allowed. You could ask about it. Id be delighted to tell you what i think. But one, you know, remarkable thing and i think its wonderful. And if you havent about it, its because its so incredibly bipartisan. There are actually senators and congressmen whove roused themselves and are finally fighting back against the overweening power of the executive branch. And theyve actually begun to do something about this. And we have to get behind them. Well, im not telling you anybody do that. Some of you might think its a good idea, but real problem all along has been that theres no. Its an unfunded mandate. So there are recommendations, but theres no money to back it up. Even so, im finding in recent months that im hearing from all kinds of firms, you know, some of them have longstanding histories of contracting for the federal government, several of them building llc based systems to try to accelerate declassification. But unfortunately, when they talk to me about their customers, they say what their really want to know is what is it that these systems show they should not declassified . So theyre building declassification engines. Luckily for all of us, perhaps, but theyre building them perhaps with the purpose or at least the outcome where they might hold back more information rather than less. So thats why we have to get in the game. You know, historians have to be working with Data Scientists and Building Systems are going to allow us to hold these contractors account and evaluate the claims that they make about the performance of these systems. You know, the last thing id say is, like the cia, state department, theyve already been conducting for years now Research Using ai to try to assist declassification. And theyve made virtually none of that public. Right. So we can only speculate for the most part as to what theyre doing and what going to do with it. But at least part of what theyre trying to do is to figure out ways they can automatically records that they decide are worth preserving. Wouldnt we like to know how those systems work . Theyre doing this without any consultation with the National Archives. Even though the federal records act know that the federal the National Archives be responsible for this. All right. So i appreciate my colleagues concern about i share many of them, you know, but my argument would be that, if we want to preserve history, if we want to analyze it history, try to understand it, we have to use all the intelligence we have. Some of that intelligence is going to be artificial. And thats a good thing. All right. So thank looking forward to your questions. Great. Thank all three of. So on your seat, you ought to you probably have this this web minus one technology. If you have questions for any or all of our please write them and we will grab them. I want to say i want to thank my colleagues for, their sort of brevity and succinct this and clarity in all of these issues. And one thing that really comes across is that there are a lot different forms of history that there exhibiting for you. On the one hand, a sort of stripping away from some of the dangerous metaphorical language at work to underlie the various sort of concrete, technical and labor processes that theyve uncovered. And on the other hand, the the salience of metaphors, intelligence or Artificial Intelligence, which at once do not describe those phenomena, but are enormously important in history, in the advocacy and development of these systems, whether its funding from whether funding from state actors or indeed from sort of corporate actors. So while youre writing down your questions, i just give i wanted to give professor yost an appetite and a moment to reply to some of the things that professor said, since he directed them. He raised them at the beginning of his remarks. Ill just say, to be clear, wasnt saying we shouldnt use Artificial Intelligence, but that we should be clear about what we think were doing when we use it and. Whats realistic to expect and know the whole point is that its not going to be accountable unless we make it accountable. Youre here. I totally agree. And i agree. Probably 90 of what you said certainly about the a dark state and and the need for transparency and access to records for democracy in terms of the the scientists and the Corporate Leaders that have spoken out and largely on the existential risk of of ai. Im a little bit more cynical, especially with the ceos. I that you can emphasize the existential threat and its conveying the power the technology. You see stock prices go up in there. I talking about. A moratorium. Its disingenuous. I dont think they think of that as i dont think they ever thought of that as a possibility. I think it was is was just some maneuvering and and, you know, the full you emphasize its full steam ahead with regard to investment in a. I. So. So thank im now getting some of the questions. One one iteration of some of what youre talking is youve talked a lot, all three of you, about the history of overhyping. But i think often there have been moments where theres been as it were under hyping of actual technical capacities. And so i think here often we see in debates about privacy, whether its privacy relation to, say, large scale corporate Automated Systems or privacy, say in wake of the snowden revelations in, which there was a sort of downplaying capacity to say, oh, in fact the machines are far less. Capable than one anticipate and one saw this for example, in the wake of the snowden revelations, when the judges in the the the fisc court came to appreciate that collecting metadata may not be as innocuous as doj and nsa suggested. So the first question ive received received asks, how is it that you think about the the ways we will maintain an accurate record of the past, given that many of these new generative technologies are powerful tools for distributing inaccurate and falsified records . And that increasingly includes visual auditory as well as printed records. Reflections on this challenging issue. Well, i would say that makes it all the more important that the official records that havent been generated, chat bots actually get put into the historical record. And i remember when i was writing the internet book, i had to file oia requests and some retired from arpa was going through page by page of these, you know, old Research Records from the sixties that didnt have anything remotely to with national security, but it was just Network Research but had to be, you know, like a year to get, get those in. So i am all for having the you know, the official actual accurate records more available so that maybe the truth can drown out the misinformation. So im director of the babbage institute, which is the largest and i broadest for the history of information technology. And we, i think four large digital collections. We have 320 collections of paper records and if you start them all up, it would be over a mile and a half and and theres something to be said for paper records and and and i think have to be very careful with to digital records to ensure theres mechanisms for authentication and and that there are there is an altering of of of digital records because that obviously can occur in a different way than paper records in an archive. Yeah, my Computer Science friends have told me that the that were keeping, you know, that we know to be authentic before generative a. I. Going to become increasingly valuable over time relative to the accumulate mass of misinformation that were likely to see going forward. So, yes, i agree. That makes this information the more precious. Yeah, id say one of the key in the development of history, a profession is in the 17th century in europe, there was a massive what we call epistemological crisis, a doubt that any of the records were true. And the response to this was the development of a very well as what seems an antiquarian field of diplomatic but diplomatic which was a way of verifying whether charters were real because was really about who owned what and had private rights. I bring this up only to suggest that historians long been central ways of thinking about how to authentic date and think through evidence and in the famous stochastic paper about the dangers of well, they call upon sociologists and other. Well, historians have a key role in thinking through all these sorts of documents and their processing and that that role is going to be technologie legal and otherwise. Okay, i have some more questions and i shouldnt opine too much. One question asks that it seems that it seems fair to say that ai has been driven on the one hand by, large Corporate Investment and the other hand by large, in principle, federal investment. And so the question is how do you view the relative of these two forces in shaping and driving the future of ai . You asking matt about corporate government . Yeah. Yeah. So, you know, im just struck by how it is as a society. We seem to letting, you know, private entities know, drive the frontier in terms of whats possible when, you know, i do think like this technology is likely to be very and dangerous at least for some people. So yeah, i think, you know, we ought to have the public equivalent right, of the most ambitious kinds of frontier model projects. I dont know why we wouldnt want that. Right. And i would assume at some point, you know, the pentagon, theyve got billions sloshing around. So theyll probably get to it eventually. But we wouldnt necessarily know anything about it, would we so . Or maybe the nsa. Theyll have their system too. Right. Because theyve got billions sloshing around over there as well. But yes, i think it be fantastic if the nsf, they could be empowered and funded, you know, to the equivalent, because i think we, you know, to understand like what these systems are capable of. And i think a society we should know what that is, at least as soon as that our corporate lords, lords and masters know, you know, and some of these private entities. I see it as a huge that. That we have a handful of of multitrillion dollar or trillion dollar it corporations that the leaders in this and and are largely determining where where its going in terms of of government the power seems to rest largely with the military and dod and their contractors like volunteer so i definitely agree with matt that there has to be better mechanisms that have a fuller fuller representation and and to to the technology and its development and use. I so another question from the audience involves the something that several of you mentioned was that in the 1980s the revival of certain kinds of ai funding emerged in just strategic concerns about the position of japan today that. Conversation very much focuses rather the peoples republic of china. And so an audience asks what is lost in thinking the race for strategic . What harms may come from a no no holds based race to get to the best inverted commas . I first. Thats a great question. I think, you know, people have commented it on the film, oppenheimer, you know, which is impressive in many ways. But one of the things i think if you see the film and you didnt know much about the history, youd come away thinking, well, they had no choice. You know, it was race against the nazis. But, you know, subsequently learned that, in fact, know, it wasnt much of a race, you know, and in fact, you know, the germans knew that the United States trying to develop atomic weapons, so did japan, you know, and obviously so did the soviet union. But theres a long history, you know, of how countries in some cases choose not to engage in races. Right. So theres this idea this kind of determinism. You know as to whether, you know, you think something is going to be a game changer or Silver Bullet or what have you and then we have to rush headlong whatever risks may be. Weve been wrong before and caused real harm. Right. So i absolutely agree. We need to pause and consider this history before we plunge ahead. I think its also to think about what we could learn in a positive direction from looking at other countries. I have a grad student whos looking at the use of ai in the chinese legal system example, which is not anything do with competing with us or, destroying us. Its simply to get more efficiencies in their legal system. And, you know, some of that is things that other countries could learn from. So i think instead of looking at everything as a threat, we can also at what might be a positive example example. So i mentioned earlier that i thought it was largely disingenuous of of some large ceos to be signing a letters like the 22 word letter from from late may about a. I. Being an existential along the same lines as pandemic and Nuclear Weapons. I. Meanwhile theyre running trillion Dollar Companies and and as theyre as theyre saying this they are trumpeting oftentimes the loudest about the risk of china and i so i think we need to be very careful with looking at history and looking at the rhetoric of whats being used to move full steam ahead with with ai and with Large Language Models. They they scrape data from social networking, from publications, from from many different and china doesnt have as open a society. And so they their government probably would not be able to exploit that if that is such a powerful technology anyway or they would not want to do that. So. I think the risk is greatly overblown. So just to extend this a little bit, so its a you know, i dont discount that many of the people raising the concerns are they are very heartfelt but its unquestionably its also the case that it is a powerful it is a powerful persuasive tool. I wonder if you might just briefly talk about earlier technological moments in which the changing of the vocabulary of description helped to transform the place of those kinds of technologies in everyday lives. Now the questions are coming fast and so if you dont, you wouldnt want to answer that. Well, this isnt really about ai, but think about the history of the internet. Yeah, its been positioned when al gore was promoting federal investment and kind of opening up the internet use that metaphor of the information superhighway, which not only is trying to express information can flow, but of course, u. S. Government funded the interstate system. And so that metaphor implies that its the governments role and that this is a fundamental infrastructure, not just a bunch of computers. And so maybe we need to kind of counteract the sort of apocalyptic speech. Maybe we need some kind of positive role models and metaphors for what would be a positive government use this and i want to get back to this idea of data sources because. You know, things like Church Liberty and other similar models. You know, they scraped the open internet and thats kind of a in garbage out situation. But you can create systems on, you know, limited curated data sets that going to be much more accurate. So, for example, the chinese example they basically use all of their legal rulings and their legal dont know their laws and textbooks and things as the corpus for that. So it is not just taking random stuff from the internet, its looking at past case. Theres pros and cons for that. But the point is you can have you know you could take Just State Department records or just legal records and have things that are robust and much less likely to give you kind of random hallucinations. So we should be thinking about what government. The government has a ton of data that needs to act on. And so, you know, can be thinking about doing positive kind of more controlled things with that data and whats the positive rather than just being so reactive, which i think is how were facing it now. Yeah. Think its useful. You know, i agree completely, you know, the those who would want to this Large Language Models by pointing to the hallucination problems. You know i think that they need to keep in mind how there are a lot of people working that problem right now. And there are many approaches to trying to address the hallucination problem. Im working with a at columbia. You know, he has good idea. Weve talked to others about it turns out other people had similar ideas. So, you know, there are a lot of us working on that because yes, the potential is really like extraordinary and i agree like another is going to be limited to declassified documents. You know, millions of them because we think that that if you can address hallucination problem and you can limit the source base, you know, to authentic, you know, records and you also provide in which the user can access those records and thereby evaluate answers theyre getting. Then this is going to be a terrific tool for research. Its going to be a hell of a lot better than just, you know, searching, i mean, keyword is really quite flawed, you know, many of us, like, whether we admit it or not we become dependent on it, the research that we have about how well keyword searching works in terms of giving you the records that youre actually interested in, you get about 20 of the records that. You actually, if you read all them, would find to be relevant. You think you have them all, but you really dont. And so keyword searching is really its like having a flashlight in a large warehouse and so, you know, absolutely think limbs are going to be much more powerful tools for for research. And so another project im involved in is trying to create a public facing, you know, system to give people more trustworthy information on, you know, about the history of american Foreign Relations and other things. So so i like, you know, long before we get to artificial generalized intelligence you know, maybe never i think were going to find lots of uses for for our lives. Yeah so i just the other day uploaded 1959 document which nsa gave to me under freedom of information in which they articulated at a first conference an information precisely the set of issues, authentication of records, need for retrieval and called for the creation of certain. So im reminded that not just historians but also our intelligence colleagues are interested in the authentication of records. One audience member asks given that records may be produced or indeed may be primarily produced now from automated, what sort of ways do we need to track and characterize provenance and the that are important for capturing that as matter of historical practice and role should historians be playing and in thinking through those matters both for the historical record and more broadly in thinking about government policy . Well, dont know how others felt, but when we learned, you know, how they had trained to accept using what seemed like a pretty, you know, random assortment, its like a dogs breakfast, you know, sources, you know, and understanding to a bit about like how it is they would wait like different corpora, you know, i thought didnt they have anybody on team who could have talked them about, you know, what kinds of corpora would likely be most. I mean thats to be even more important when we build know these more customized systems, particular applications and yes, you know i think are going to have a very important role. You know, when i work with Data Scientists, you know, the things that theyre incentivized to do are to, you know, build technology that can outperform current technology. And they, you know, in many cases are relatively indifferent as to the kind of data theyre thats obviously different, you know, when were talking about systems that are all about, you know, the language that youre using to train these systems. But, you know, my hope is, you know, that when you know as historians we can share with our Data Scientist friends like data, thats actually really kind of important, you know, like, like historical records. Theyre going to find that as another place in which they can develop tools and do research that, you know, i would argue like matter not just in terms of technology development, also the things that we could learn about our history. Right. And not just history. Right. But but other fields as well. Well, that raises a good question about the criteria for and it seems like maybe an agency like this have a role in trying to set benjamin for, you know, does it mean to have a high performing, you know, what accuracy or like good metadata to show where that information came from. And then if we had standards like that, then that might help to create incentives that could potentially move the research in that direction. Yeah, i like that. The, the point a lot. I mean, actually part of history of how it is that Machine Learning was successful in ways other parts of ai were not, is that earnest set down a set of it down a single metric criteria by which things were to be judged and many people internally in Machine Learning have said it is precisely that that provided the secret sauce. It wasnt that they were aiming for vague notions of intelligence. They were trying optimize a metric, but that that of one metric was fantastic, except that it did not include all of these sorts of things. Why should that model of the success of Government Intervention in both corporate and Academic Research not serve us more broadly . Our time is short, but i do want to theres one last question that points in different direction. I think it captures something of the excitement of Large Language Models, which is that we are all able to use them on one hand, they are products of very corporations, indeed only large corporations, as Meredith Whittaker has underscored, have the capital to undertake these. And yet they seem in some ways as a kind of democratization of technology and audience members, how should we think of ai for everyday and citizens. You have 5 minutes down. Yeah, a lot a lot of us are going back to blue book exams and even oral exams. You know, one one of the things that i was very, you know, unhappy about a lot professors were is how they launched output in november, you know, right before finals. So, you know, it would have been nice, you know if they thought about like the impact that would in academia in our ability to teach our students. So i mean, thats just one example, right . Yeah. Weve actually gone the other way and, had exercises where students will and answer a question using lgbt versus, you know team uses Just Research and not it sort of compares results because i think the important thing again is that not that we use it or dont use it but that understand what were getting when we use ai and make some kind of informed judgments over, you know, what we do want to use it for. I agree with dr. About that. It is important to teach about it and. Use it in instruction and and and also to design. The assignments and tasks that students will be evaluated on in ways that. Are difficult for chapter four and other chapter to excel. What. Well, unfortunately our time come to an end and i to thank all three speakers for not us hallucinations but actual words grounded in knowledge and expertise, particularly the sets of expertise that are distinctive. The competencies that are important for providing historical perspective. If you didnt get your answer, your question answered, the panelists will be around for briefly afterwards and want to encourage all of you to take a look at some of the literature outside, the entrance here about the important work of the american historical association. We hope. See you at the next briefing, which will be on the history of housing on november 29th on the house side of the capitol. And please join me in thinking our panel