Panel because were pressed for time. Im pleased to have ted mccann, from the house of representatives. Bob greenstein. And bob doar from ai next door. Now were going to talk about moving away from the academics and talking more about the practical issues associated with implementation. And i want to start again sort of in a big picture. We spent the morning talking about the Academic Studies, and making sure studies are done well and the effects of policies. You know, this would have been a reasonable question, i think a year ago, and two years ago. I think we feel like maybe its even a more important question now. Douse evidence matter to the political process . Do these studies have an effect . If so, like, how . Im going to take all of your i think it matters a lot. I think that in my experience in new york, and in new york state, new york city, we paid attention to evidence, we paid attention to outcomes. It has an effect on how you do your debate. I do think you have to be careful about overstating the evidence. You cant oversell. And i think you also have to know your audience. I think that, when looking at this data about investments in Human Capital, i think the best finding concerns medicaid, food stamps behind that. Im not so sure the finding is as strong in cash. When you go in and talk to people who have particular viewpoint, and here im going to talk about my friends, republicans, theyre going to want to hear about employment effects. And theyre not going to want it to be dismissed as unimportant. And it really matters. Theyll want to know the truth about that. And theyre also going to want to hear the extent to which these necessity of investing in Human Capital is driven by an absence of another parent, for instance, or the issue of single parenthood. You cant just write a series of papers and make a statement about investing in Human Capital for young children, and only talk about poverty and race and not also talk about single parents. I do think it matters a lot. I think the speaker in his better way proposal that was referenced mentioned evidence. Certainly the hottest republican election to some people in town was todd young in indiana. He was an evidence guy, pay for performance guy. I think people want to start with the premise that, in the new world we live in now, evidence doesnt matter, certainly arent helping their case. Whats your take . If i didnt think evidence mattered at all, none of us up here would be doing what we do. But having said that, i think my take is a lot less sanguine than roberts. I think evidence can matter a lot when decisions are being made in a nonpolitically polarized atmosphere. But when an issue has a lot of Politics Around it, you can find members simultaneously talking about evidence basing, and then in the same statement, or the same document, making statements that are actually contrary to the evidence. A couple of examples, you know, of things that bother me now. We continue to have members of congress and others say as though this was based on evidence, that any poverty programs are a failure because the poverty rate is the same today as it was in the 60s. When every reputable analyst knows that that comparison is based on the official poverty measure that doesnt count hardly any of the programs that have expanded because theyre generally noncash, and broad agreement among analysts that poverty measures, particularly when youre doing stark comparisons, should count the noncash benefits, and when you do, theres been a big reduction in poverty. Weve had hearings on the hill. I was in one where there were two republican and one democratic witness. The witnesses echoed each other if youre doing historical measures, you cant use the official measure or youll get misleading results. Members in the room and heard this agreement within days were repeating the line, everything failed, the poverty rate didnt go down. One more example. If you look at a lot of the discussions on the hill, there was an earlier panel, i think it was greg and kristen were talking about how much of policy discussions were in the context of perceived labor supply, labor market. And theres a particular line you hear over and over again in policy discussions, that lowincome people in large numbers face 80 marginal tax rates and often worse off if they take a job. We have a cbo study that came out a year and a half ago that said the median marginal tax rate is 14 on people below the poverty line. We did an extensive analysis. How do you get to 80 . There are people who face 80 rates. But you have to be in a narrow income range and get an atypical combination of benefits. It turns out that about 3 of singleparent families with kids can be subject to something in the 80 range. It doesnt matter. The data is out there. It keeps being repeated as though this is the norm. So i think where we run into challenges, or where the data are the evidence, dont support an ideological held position, or arent helpful in a polarized political fight. And i think our big challenge is how do we bring evidence basing, even into those kinds of debates. A response . Well, i guess ill talk a bit about evidence more broadly in the political process. I think actually we do use evidence fairly regularly in the political process. I think youve seen speaker and then senator murray move forward with the Commission Based commission on policy. You mentioned senator young, talking about pay for success. Not only that, but whenever ive been in the room, trying to figure out how to move policies forward, even when its republican, democrats negotiating things out, typically theres a conversation about what the correct policy is. Its not a conversation devoid of what we think the best policy is. Its a conversation based on the political reality, and then combined with sort of what the best policy Going Forward we think should be jointly. A lot of that is getting agreement on what the evidence shows. On a lot of issues, the evidence is mixed. You cant come down on one side or the other side. And folks end up arguing from probably more political point of view. But again, thats just part of living in the messy world. And i would just say, just on the first example that bob gave, most of the time when im in the room, and that statement is made, its always with the proviso that we have Material Hardship when properly measured with poverty, but we havent yet achieved a goal where people are earning their own ability to rise above poverty. That isnt just a political statement, i think there are americans who care about having people get out of poverty so that they at the end of the day dont need the itc or food stamps or value of Public Health insurance to provide the difference between what they earn, or make on their own, and the poverty line. So i acknowledge that that, you know, political statement we both have on both sides egregious behavior by politicians who sometimes say things that arent quite right. But that one is a little more complicated, because people do aspire to having the poverty measure be achieved without transfer payments. I have to disagree. Robert, what you just said is fine. But in most public debates, you look at various members of congress and others on sunday talk shows and the like, they do not express it more often than not in the way you just mentioned. It is used as an emblem and as evidence that any poverty programs have failed, and we should radically change them. Thats the most common rhetoric. The war on poverty failed. It didnt move the needle. A very common statement. It shows were doing things wrong and current things dont work. The same people that say welfare reform was a success. Im not getting into the welfare well, the welfare i think we can agree i think we can probably argue different poverty measures show different things. Yes. We can agree with that. The more accurate is the poverty based poverty measure. Which we dont have at the moment. I also heard you say, though, what people say in public and in the back rooms are kind of different. We worry about the disservice done when we know the facts, but were playing politics all the time. Is there anything that academics, or think tanks could do better to try to get, you know, more agreement, where the evidence where everybody sort of agrees on the basic factsing . Theres another problem here. Im not sure exactly what think tanks should do about it. I think everyone on the panel would agree, that on a lot of complicated issues, getting away from poverty measurement, and research on impacts, on a lot of issues there are multiple studies, and they sometimes go in different directions. But there are five studies and four go in one direction and the other goes in the other. It is not uncommon in this town for people for whom the one that goes in their direction, they cite only that one and they dont tell you the other four exist. And then, hey, Evidence Based. Heres the study. And so i dont know how one achieves the following goal. But there ought to be a norm. Increasingly common term these days. There ought to be a norm that when were discussing an issue on which theres research and evidence, that if various pieces of research go in various directions, its perfectly fine for someone to say why they think a given study or studies are the best ones. But you need to inform your audience of the studies that go the other way. And if you think theyre wrong or not as solid, why you think that. But what should be viewed with suspicion is anytime there are multiple studies, if someone cherrypicks one and doesnt tell you about the others, quality studies going in the other direction, to me thats not evidence basing. I dont think you can argue with that. Thats a good aspect to look at a full array of studies. I also think its important when citing the studies to be really clear about the findings. And the margins between finding that shows Significant Impact and not. I think sometimes in town people have a tendency to say, the evidence suggests, or the evidence shows, or theres a lot of evidence that proves, and you look beneath that and find out the evidence in the outcome wasnt that great. Or studies that show something different. I think when that happens, at least in my limited experience here in town, when when that credibility problem comes into effect, it really undermines the whole finding. And you dont want to do that. You want to be clear about the caveats and the extent to which whatever intervention, especially in poverty policy, had an impact, maybe wasnt quite as big as you want to pretend it was. I also would say one other thing. And that is that we cant underestimate the prevailing economic forces at work in the country that are sometimes more important, or appear to be more important. Especially in poverty. And sometimes theres you had a question, you said to us, well, why doesnt everybody talk about this when all they want to talk about is dynamic scoring. And i said, because people are worried about the overall economy as much as they are worried about a particular intervention in poverty. I just think that has to be considered. So lets move on from that a little bit. The thing that we try to talk about today, was to sort of bring bunches of different kinds of policies together, because we think really these are policies that are actually have the potential to be investments. We all want to raise future growth. Were worried about productivity, about supporting medicare and social security, things that sort of raises future growth. So were like really focused on whats consumption and whats investment. Is that something that politicians actually care about, what happens if 10, 20 years down the road from something or really that youve got to make the case that its jobs, or is it all about labor market stuff . Why dont you start. I think politicians care a lot about that. We just passed 21st century cures which is about seeing down the roads about benefits from producing diseases. With the aca, theres a lot of talk about prevention. I think, you know, i think thats something politicians consider fairly closely. Now, you do have the rules that dont really allow you to capture those savings. So it makes it more difficult to include them. So going back to the evidence a little bit. Can we stick on the growth a little bit . Yeah. Look, if youre a politician, you can be in either party, and youre favoring a policy, it could be a tax si, it could be a spending policy, more often than not you will promote that your idea will promote growth. I would also argue that the standards that are used for claiming growth effects are not equal on the spending on the tax side. Almost any tax cut that anybody favors, it can be in either party, is often claimed as producing growth. In most cases, claims well beyond the evidence. Whats interesting on the social program side, your paper i think does a great job of summarizing and synthesizing this, i find even after the last five to ten years of research which youve captured so well, if im on capitol hill, and im not talking about republicans versus democrats here, its pretty much the case with all of them. There is a real lack of knowledge of the Research Showing some of the longterm positive effects on kids, poor kids of certain kinds of Program Interventions as you discussed in your paper. Whereas among both parties, there are kind of assumptions often fairly evidencefree, or the evidence may come from a k street lobbying firm thats producing data to serve its clients, whatever it is, the tax cut or tax break du jour will have all these terrific effects on growth. Id love to see somewhat more attention on spending side. But for both of them, equally rigorous standards for what we know about growth effects, i think were a long way from that right now. So is there a role for a cbo here, for example, to be evaluating policies and sort of standard part of its tool kit, sort of saying, we think this is likely to be growth enhancing, or not, or have an impact on distribution . For policies both on the tax side and spending side . My view is, it would be useful for cbo to do this on both sides as analysis. And that cbo should do it on neither side for scoring. I dont favor dynamic scoring for taxes. Depending on the model that you choose, you get wildly differing results. I suspect 10 or 20 years from now based on whatever data we have then, if one is doing dynamic scoring on taxes we might be in different ballparks than we are today. My view is we should get the best analysis we can of those effects on both taxes and spending. But i dont actually favor doing dynamic scoring on either side of the budget. I think it undercuts the solidity of the fiscal numbers. Would it make a big difference, do you think, in terms of what kinds of policies were enacted if there was some kind of score to it, or some kind of standard analysis they would do, and people would look to . If you could find savings for a policy, that makes a big difference. Looking in the window, thats generally how with escore. 20 years down the road, you know, theres no numbers way of really making it if you can show its going to save money and show it will be overall net positive, thats going to be in a markets favor. In terms of the actual political moving it, its much very, very helpful to have a negative policy. Because then it can pay for some of the other things wed like to pay for. Lets face it, thats the main reason we have dynamic scoring on tax policies, it makes it easier for tax cuts to pass. Im not saying it is or it isnt. The main political motivation to beginning to use it was to make tax cuts easier to pass. One of the things that you talked about was sort of like, we shouldnt oversell the research results, but as kristen mentioned, its really, really difficult to get, you know, solid, broad research, you know, you have to wait 30, 40 years, 50 years to know that these things really, really matter. But we get snippets, by understanding that young, you know, fetal situation matters a lot. There are a lot of things we piece together. Thats not the same as saying we have incredible evidence. At the same time, not doing policies, while they may be incredibly helpful because we havent waited the 50 years to see, what should the bar be . How should evidence play in and how good does the evidence have to be for policy to be based on it . One of the big difficulties if you create a government program, its very difficult to shut it down. I think thats one of the big issues that conservatives especially have. If we were to look at something that shows promise, that we would know if it didnt pan out, that we could turn it off, or no longer continue the program, i think youd probably see a lot more willingness to take risks in this area. But i think generally, the thought is, well, if we do something, it doesnt work, its never going to end. The spigot will continue going on forever, and well just add additional programs in the future. Are you weighing that it will probably work against yeah. Theres sort of that general are there ways to evaluate the level of rigorousness of an evaluati evaluation . And there are technical ways of determining when you have a good study that does a very, you know, randomly controlled experiment that shows stronger studies. And we know what those are. And when you have a good one, you want to go hard with it. Because you do want to appeal to the congress desire to make changes now that make life better for the future. What im worried about is when you have a close call, or when the caveats inside the study are actually pretty serious. And you dont explain those, as you present the results. Then i think you undermine the entire exercise, because there is a gotcha game here and its not helpful if people say, you didnt actually explain the detail there, or you made it sound like the impact was really significant when it really wasnt. Theres definitely a bar. I cant describe what it is. But what i would say is if you have it, dont say you do have it if you dont. Roberts point about, i certainly agree that particularly where findings are not statistically not significant, people shouldnt be presenting them as a real finding. By the same token, we all know in the lowincome area, if you have an intervention and lets say theres an improvement for kids or employment, whatever, im making up, 10 percentage points, the average person they say that sounds very small. We know in the world of policy, thats actually a big effect. So we also have to help people with that. I think one of the really tough things is what happens when we have social science research, and policymakers draw policy conclusions from it, and then as the years go by, we get more research that sometimes actually goes in the opposite direction. Two examples. For years the Research Indicated a lack of lasting effects from headstart. Now theres Newer Research that suggests longterm positive effects. The same is true on moving