Panel because were pressed for time. Im pleased to have ted mccann, from the house of representatives. Bob greenstein. And bob doar from ai next door. Now were going to talk about moving away from the academics and talking more about the practical issues associated with implementation. And i want to start again sort of in a big picture. We spent the morning talking about the Academic Studies, and making sure studies are done well and the effects of policies. You know, this would have been a reasonable question, i think a year ago, and two years ago. I think we feel like maybe its even a more important question now. Douse evidence matter to the political process . Do these studies have an effect . If so, like, how . Im going to take all of your i think it matters a lot. I think that in my experience in new york, and in new york state, new york city, we paid attention to evidence, we paid attention to outcomes. It has an effect on how you do your debate. I do think you have to be careful about overstating the evidence. You cant oversell. And i think you also have to know your audience. I think that, when looking at this data about investments in Human Capital, i think the best finding concerns medicaid, food stamps behind that. Im not so sure the finding is as strong in cash. When you go in and talk to people who have particular viewpoint, and here im going to talk about my friends, republicans, theyre going to want to hear about employment effects. And theyre not going to want it to be dismissed as unimportant. And it really matters. Theyll want to know the truth about that. And theyre also going to want to hear the extent to which these necessity of investing in Human Capital is driven by an absence of another parent, for instance, or the issue of single parenthood. You cant just write a series of papers and make a statement about investing in Human Capital for young children, and only talk about poverty and race and not also talk about single parents. I do think it matters a lot. I think the speaker in his better way proposal that was referenced mentioned evidence. Certainly the hottest republican election to some people in town was todd young in indiana. He was an evidence guy, pay for performance guy. I think people want to start with the premise that, in the new world we live in now, evidence doesnt matter, certainly arent helping their case. Whats your take . If i didnt think evidence mattered at all, none of us up here would be doing what we do. But having said that, i think my take is a lot less sanguine than roberts. I think evidence can matter a lot when decisions are being made in a nonpolitically polarized atmosphere. But when an issue has a lot of Politics Around it, you can find members simultaneously talking about evidence basing, and then in the same statement, or the same document, making statements that are actually contrary to the evidence. A couple of examples, you know, of things that bother me now. We continue to have members of congress and others say as though this was based on evidence, that any poverty programs are a failure because the poverty rate is the same today as it was in the 60s. When every reputable analyst knows that that comparison is based on the official poverty measure that doesnt count hardly any of the programs that have expanded because theyre generally noncash, and broad agreement among analysts that poverty measures, particularly when youre doing stark comparisons, should count the noncash benefits, and when you do, theres been a big reduction in poverty. Weve had hearings on the hill. I was in one where there were two republican and one democratic witness. The witnesses echoed each other if youre doing historical measures, you cant use the official measure or youll get misleading results. Members in the room and heard this agreement within days were repeating the line, everything failed, the poverty rate didnt go down. One more example. If you look at a lot of the discussions on the hill, there was an earlier panel, i think it was greg and kristen were talking about how much of policy discussions were in the context of perceived labor supply, labor market. And theres a particular line you hear over and over again in policy discussions, that lowincome people in large numbers face 80 marginal tax rates and often worse off if they take a job. We have a cbo study that came out a year and a half ago that said the median marginal tax rate is 14 on people below the poverty line. We did an extensive analysis. How do you get to 80 . There are people who face 80 rates. But you have to be in a narrow income range and get an atypical combination of benefits. It turns out that about 3 of singleparent families with kids can be subject to something in the 80 range. It doesnt matter. The data is out there. It keeps being repeated as though this is the norm. So i think where we run into challenges, or where the data are the evidence, dont support an ideological held position, or arent helpful in a polarized political fight. And i think our big challenge is how do we bring evidence basing, even into those kinds of debates. A response . Well, i guess ill talk a bit about evidence more broadly in the political process. I think actually we do use evidence fairly regularly in the political process. I think youve seen speaker and then senator murray move forward with the Commission Based commission on policy. You mentioned senator young, talking about pay for success. Not only that, but whenever ive been in the room, trying to figure out how to move policies forward, even when its republican, democrats negotiating things out, typically theres a conversation about what the correct policy is. Its not a conversation devoid of what we think the best policy is. Its a conversation based on the political reality, and then combined with sort of what the best policy Going Forward we think should be jointly. A lot of that is getting agreement on what the evidence shows. On a lot of issues, the evidence is mixed. You cant come down on one side or the other side. And folks end up arguing from probably more political point of view. But again, thats just part of living in the messy world. And i would just say, just on the first example that bob gave, most of the time when im in the room, and that statement is made, its always with the proviso that we have Material Hardship when properly measured with poverty, but we havent yet achieved a goal where people are earning their own ability to rise above poverty. That isnt just a political statement, i think there are americans who care about having people get out of poverty so that they at the end of the day dont need the itc or food stamps or value of Public Health insurance to provide the difference between what they earn, or make on their own, and the poverty line. So i acknowledge that that, you know, political statement we both have on both sides egregious behavior by politicians who sometimes say things that arent quite right. But that one is a little more complicated, because people do aspire to having the poverty measure be achieved without transfer payments. I have to disagree. Robert, what you just said is fine. But in most public debates, you look at various members of congress and others on sunday talk shows and the like, they do not express it more often than not in the way you just mentioned. It is used as an emblem and as evidence that any poverty programs have failed, and we should radically change them. Thats the most common rhetoric. The war on poverty failed. It didnt move the needle. A very common statement. It shows were doing things wrong and current things dont work. The same people that say welfare reform was a success. Im not getting into the welfare well, the welfare i think we can agree i think we can probably argue different poverty measures show different things. Yes. We can agree with that. The more accurate is the poverty based poverty measure. Which we dont have at the moment. I also heard you say, though, what people say in public and in the back rooms are kind of different. We worry about the disservice done when we know the facts, but were playing politics all the time. Is there anything that academics, or think tanks could do better to try to get, you know, more agreement, where the evidence where everybody sort of agrees on the basic factsing . Theres another problem here. Im not sure exactly what think tanks should do about it. I think everyone on the panel would agree, that on a lot of complicated issues, getting away from poverty measurement, and research on impacts, on a lot of issues there are multiple studies, and they sometimes go in different directions. But there are five studies and four go in one direction and the other goes in the other. It is not uncommon in this town for people for whom the one that goes in their direction, they cite only that one and they dont tell you the other four exist. And then, hey, Evidence Based. Heres the study. And so i dont know how one achieves the following goal. But there ought to be a norm. Increasingly common term these days. There ought to be a norm that when were discussing an issue on which theres research and evidence, that if various pieces of research go in various directions, its perfectly fine for someone to say why they think a given study or studies are the best ones. But you need to inform your audience of the studies that go the other way. And if you think theyre wrong or not as solid, why you think that. But what should be viewed with suspicion is anytime there are multiple studies, if someone cherrypicks one and doesnt tell you about the others, quality studies going in the other direction, to me thats not evidence basing. I dont think you can argue with that. Thats a good aspect to look at a full array of studies. I also think its important when citing the studies to be really clear about the findings. And the margins between finding that shows Significant Impact and not. I think sometimes in town people have a tendency to say, the evidence suggests, or the evidence shows, or theres a lot of evidence that proves, and you look beneath that and find out the evidence in the outcome wasnt that great. Or studies that show something different. I think when that happens, at least in my limited experience here in town, when when that credibility problem comes into effect, it really undermines the whole finding. And you dont want to do that. You want to be clear about the caveats and the extent to which whatever intervention, especially in poverty policy, had an impact, maybe wasnt quite as big as you want to pretend it was. I also would say one other thing. And that is that we cant underestimate the prevailing economic forces at work in the country that are sometimes more important, or appear to be more important. Especially in poverty. And sometimes theres you had a question, you said to us, well, why doesnt everybody talk about this when all they want to talk about is dynamic scoring. And i said, because people are worried about the overall economy as much as they are worried about a particular intervention in poverty. I just think that has to be considered. So lets move on from that a little bit. The thing that we try to talk about today, was to sort of bring bunches of different kinds of policies together, because we think really these are policies that are actually have the potential to be investments. We all want to raise future growth. Were worried about productivity, about supporting medicare and social security, things that sort of raises future growth. So were like really focused on whats consumption and whats investment. Is that something that politicians actually care about, what happens if 10, 20 years down the road from something or really that youve got to make the case that its jobs, or is it all about labor market stuff . Why dont you start. I think politicians care a lot about that. We just passed 21st century cures which is about seeing down the roads about benefits from producing diseases. With the aca, theres a lot of talk about prevention. I think, you know, i think thats something politicians consider fairly closely. Now, you do have the rules that dont really allow you to capture those savings. So it makes it more difficult to include them. So going back to the evidence a little bit. Can we stick on the growth a little bit . Yeah. Look, if youre a politician, you can be in either party, and youre favoring a policy, it could be a tax si, it could be a spending policy, more often than not you will promote that your idea will promote growth. I would also argue that the standards that are used for claiming growth effects are not equal on the spending on the tax side. Almost any tax cut that anybody favors, it can be in either party, is often claimed as producing growth. In most cases, claims well beyond the evidence. Whats interesting on the social program side, your paper i think does a great job of summarizing and synthesizing this, i find even after the last five to ten years of research which youve captured so well, if im on capitol hill, and im not talking about republicans versus democrats here, its pretty much the case with all of them. There is a real lack of knowledge of the Research Showing some of the longterm positive effects on kids, poor kids of certain kinds of Program Interventions as you discussed in your paper. Whereas among both parties, there are kind of assumptions often fairly evidencefree, or the evidence may come from a k street lobbying firm thats producing data to serve its clients, whatever it is, the tax cut or tax break du jour will have all these terrific effects on growth. Id love to see somewhat more attention on spending side. But for both of them, equally rigorous standards for what we know about growth effects, i think were a long way from that right now. So is there a role for a cbo here, for example, to be evaluating policies and sort of standard part of its tool kit, sort of saying, we think this is likely to be growth enhancing, or not, or have an impact on distribution . For policies both on the tax side and spending side . My view is, it would be useful for cbo to do this on both sides as analysis. And that cbo should do it on neither side for scoring. I dont favor dynamic scoring for taxes. Depending on the model that you choose, you get wildly differing results. I suspect 10 or 20 years from now based on whatever data we have then, if one is doing dynamic scoring on taxes we might be in different ballparks than we are today. My view is we should get the best analysis we can of those effects on both taxes and spending. But i dont actually favor doing dynamic scoring on either side of the budget. I think it undercuts the solidity of the fiscal numbers. Would it make a big difference, do you think, in terms of what kinds of policies were enacted if there was some kind of score to it, or some kind of standard analysis they would do, and people would look to . If you could find savings for a policy, that makes a big difference. Looking in the window, thats generally how with escore. 20 years down the road, you know, theres no numbers way of really making it if you can show its going to save money and show it will be overall net positive, thats going to be in a markets favor. In terms of the actual political moving it, its much very, very helpful to have a negative policy. Because then it can pay for some of the other things wed like to pay for. Lets face it, thats the main reason we have dynamic scoring on tax policies, it makes it easier for tax cuts to pass. Im not saying it is or it isnt. The main political motivation to beginning to use it was to make tax cuts easier to pass. One of the things that you talked about was sort of like, we shouldnt oversell the research results, but as kristen mentioned, its really, really difficult to get, you know, solid, broad research, you know, you have to wait 30, 40 years, 50 years to know that these things really, really matter. But we get snippets, by understanding that young, you know, fetal situation matters a lot. There are a lot of things we piece together. Thats not the same as saying we have incredible evidence. At the same time, not doing policies, while they may be incredibly helpful because we havent waited the 50 years to see, what should the bar be . How should evidence play in and how good does the evidence have to be for policy to be based on it . One of the big difficulties if you create a government program, its very difficult to shut it down. I think thats one of the big issues that conservatives especially have. If we were to look at something that shows promise, that we would know if it didnt pan out, that we could turn it off, or no longer continue the program, i think youd probably see a lot more willingness to take risks in this area. But i think generally, the thought is, well, if we do something, it doesnt work, its never going to end. The spigot will continue going on forever, and well just add additional programs in the future. Are you weighing that it will probably work against yeah. Theres sort of that general are there ways to evaluate the level of rigorousness of an evaluati evaluation . And there are technical ways of determining when you have a good study that does a very, you know, randomly controlled experiment that shows stronger studies. And we know what those are. And when you have a good one, you want to go hard with it. Because you do want to appeal to the congress desire to make changes now that make life better for the future. What im worried about is when you have a close call, or when the caveats inside the study are actually pretty serious. And you dont explain those, as you present the results. Then i think you undermine the entire exercise, because there is a gotcha game here and its not helpful if people say, you didnt actually explain the detail there, or you made it sound like the impact was really significant when it really wasnt. Theres definitely a bar. I cant describe what it is. But what i would say is if you have it, dont say you do have it if you dont. Roberts point about, i certainly agree that particularly where findings are not statistically not significant, people shouldnt be presenting them as a real finding. By the same token, we all know in the lowincome area, if you have an intervention and lets say theres an improvement for kids or employment, whatever, im making up, 10 percentage points, the average person they say that sounds very small. We know in the world of policy, thats actually a big effect. So we also have to help people with that. I think one of the really tough things is what happens when we have social science research, and policymakers draw policy conclusions from it, and then as the years go by, we get more research that sometimes actually goes in the opposite direction. Two examples. For years the Research Indicated a lack of lasting effects from headstart. Now theres Newer Research that suggests longterm positive effects. The same is true on moving the opportunity, the original results were disappointing in most respects. It has striking and important positive effects for families of kids under 13 who use the vouchers to move to low poverty neighborhoods. In the area of welfare reform, the Academic Studies through about 2004 and 2005 were a lot more positive than the more recent Academic Studies. And i think the political system has some difficulty, its enough to get people to use studies in the first place. And when they start using them and get used to it, and a new set of studies come along that go in a different direction, it can take years to get people to get their minds out of the earlier set of studies, and they assume that thats still where the Evidence Base is. On the other hand, you can have a series of studies that showed limited impact of social interventions, or positive effect of welfare reform, and then when you get the one that goes against that, all of a sudden that becomes the most important study ever done. So you have to be careful both ways. I agree. There has to be more than one study. Let me ask a question about sort of the what sarah documented and the basic agreement. On both sides of the aisle, about the finding that where someone is born and the family circumstances in which theyre born in has enormous impact on their eventual yut come, their earnings, health, all these things. Is that a fact that everybody agrees on or is there a basic disagreement on whether thats actually true, or how true that is . Yeah, i think thats yeah, i think everybody agrees with that. Yeah. Absolutely. I think that yeah. The disagreement is about, is the disagreement about the remedies or what the federal rules should be in the remedies . What ive experienced in observing the debate is that its both about what are the characteristics of the household that lead to those disparities, and whether we should talk about all of them, which includes the absence of another parent, or marriage, as well as race, income, neighborhood, community, or just a few of them. As i mentioned earlier, i do think its an important point for people that want to persuade, especially conservatives, about intervening in the lives of poor kids, its important for them to start by including in why we need to do this, the family issue. Because when you dont mention family issue, youve immediately got a representative who is waiting for that. Where is that . Why havent you mentioned that . I dont think anyone disagrees it isnt part of it. Everyone agrees that. So just include it. So i think theres an agreement that and i think speaker ryan speaks to that. He says in some of his remarks, its not right that the Household Income determines your future. And we have to talk about how and why we can persuade people on both sides to come together in interventions that reduce those disparities. I think this can get complicated and challenging. So again, lets take the evidence synthesized in the paper on income effects on young kids. And the research indicating that particularly for certain kinds of interventions, s. N. A. P. , others, refundable tax credits, a variety of things, evidence that the added income early in childhood appears to be linked to increased Educational Attainment, and some studies like dianes in terms of girls who got food stamps early on increased selfsufficiency in adulthood, we have an array of strong studies showing that better Educational Attainment in school is going to increase employment and earnings in adulthood. Okay. So we have a challenge. Were going to have a debate this year among other things on importing tanif style work requirements in programs like medicaid, s. N. A. P. And housing programs. The evidence on tanif style work requirements is presented in another forum in the same auditorium a few months ago, one of the leading authorities on this in the country. The biggest effect, not the only effect, were lots of sanctions. Bob said they did larger effect in sanctioning, in taking removing benefits than in increasing work. Okay. If you put these two pieces together, would this mean that largescale sanctioning by reducing income potentially increasing tax distress among young kids could reduce their Employment Earnings in adulthood, and would the net effect other time from these work requirements, unemployment and earnings be positive or negative . The answer is, i think, we dont know. We dont have the kind of data to draw the conclusion of which side would be higher. But the question isnt even being asked. I really do think that the findings on the longterm effects on kids has barely penetrated in policy debates on capitol hill at this point. And a challenge for the years ahead is, can it become, not the sole determining factor, but a factor amidst a bunch of other factors and other pieces of evidence as we have policy debates. I agree that its new, and its it is so new, the extent to which its penetrated into congress, and understandable in my view. But i also think that its not part of the discussion. I think it will be part of the discussion. And im not sure that just the way bob summarized the finding on sanctions and tanif was i think the general perception is the introduction of work, expectations in afdc through the tanif reform led to more work, less receipt of cash welfare, and less poverty. Lets open it up to the audience for questions. Our mics are coming around. George . Hi. I dont know whether one would consider the Tax Foundation a member a provider of research or merely a reporter, but there what theyre doing is of extraordinary importance right now. Tax reform. They did a study on the republican plan back in december, and all they did was report the numbers with the dynamic scoring, and report the numbers without dynamic scoring. I know you were on joint tax. How does one get beyond that kind of nonsense . Or is it just something we have to live with . The Tax Foundation is an example of my skepticism of dynamic scoring. The Tax Foundation modeled a pa. The Tax Foundation model rests on a series of assumptions, some of which are shall we say outside mainstream economics. And the dynamic scores they get are very different than those cbo gets and those in the Tax Policy Center analysis this fall. The range of results under dynamic scoring are so enormous and i think thats because, you know, while in concept who could disagree, you want to know the economic effects and if you could actually get the scores on it, you would want to i donuse scores. In reality its hard in my view to trust any particular dynamic scoring when so much depends on the assumptions underlying the particular model you elect to use to do the dynamic scoring. Question . Anybody . Over there. Tell us who you are, please. Im mara weinstein. My question is about using evidence and what all of your takes are on the fact that there is a ton of evidence in the Research Community that goes unpublished, a lot with negative findings, so when you have Research Presented to you in a world that really thrives on the cliffs notes or the twitter version of the results, whose onus is it to dig deeper into that information, the presenter or the Decision Maker, and how do you advise people to go about that . Anybody . I would say honestly its probably on the Decision Maker more than anybody. When i was at ways and means we spent a lot of time talking with folks in the Academic Community but also folks on the ground to get a sense of does this academic finding make sense with what other folks are saying. So you weigh all of that and try to come to the best possible conclusion. I think the academic is trying to show a change and show something nice and its our job to make sure that it makes sense. You could also attest to the importance of institutions like the congressional budget office, Congressional Research service, a number of crs reports, im thinking of some in the tax area but they are in the spending area as well, where they evaluate, review various pieces of research. Its also, its not a government agency, but this is another reason i find the urban brookings Tax Policy Center is a critically important institution. We need institutions that do high quality work that arent politically motivated and that people regardless of their political viewpoint hopefully can look to for help in sorting through a lot of complex data and numbers, and helping them to evaluate an array of, a blizzard, often, of information and sometimes numerous studies that go in different directions. Terrific. I think we will end it now. Thank you so very much for participating. [ applause ] so i was about to say we saved the best for last but i was afraid wed be assaulted by the excellent presentations weve had before and i was also