Polls Haven't Been This Accurate Since At Least 1998 01:02:08

You're a podcast listener, and this is a podcast ad. Reach great listeners like yourself with podcast advertising from Lipson Ads. Choose from hundreds of top podcasts offering host endorsements, or run a reproduced ad like this one across thousands of shows to reach your target audience with Lipson Ads. Go to LipsonAds.com now. That's L-I-B-S-Y-N-Ads.com.

Did everybody watch the Oscars last night? Of course. I'm a good ABC company employee. I went home early, but what was the joke involving Malala and the cocaine bear?

It was, it was, they were adjacent jokes. So he was interviewing people in the audience, including Malala. And then for some reason, the cocaine bear was crawling behind him and like got up and tried to talk to Malala or something. Did Malala get to interact with cocaine bear? We couldn't see because the bear was obstructing her. But she did. She did very diplomatically answer a question about whether Harry Styles spat on Chris Pine. And what did she say?

She said she's only interested in peace, which is like a very politician answer. Hello and welcome to the FiveThirtyEight Politics Podcast. I'm Galen Druk.

What if I told you that 2022 was polling's most accurate election cycle since at least 1998? That after all the hand-wringing about polling generally and the sniping about a Republican or Democratic bias specifically, if you actually crunched the numbers, polling had its smallest average error of the past 25 years in 2022 tied with the 2004 cycle.

Well, that's what FiveThirtyEight's updated pollster rankings bear out. So today we're going to talk about how we came to that conclusion and what it says about the polling industry. There are certainly still meaningful questions to ask about the state of polling, and we'll ask some of those as well. Here with me to do it is Editor-in-Chief Nate Silver. Hello, Nate. How's it going? Good. How are you? I'm doing pretty well. You know, hanging in there Monday morning after a... I didn't know you were supposed to answer that question literally. I didn't mean that literally. Okay.

Okay. I'm glad you're doing well. Thank you. Thank you. Senior elections analyst Nathaniel Rakich is also here with us. How are you doing? Good morning, Galen. I'm doing well. How are you?

I'm still pretty good. You say it hasn't changed in the last five seconds? No, it hasn't. But thank you for asking. Okay, I'll keep on checking throughout the show. Okay, when you ask someone how they're doing, you have to say good. You can elaborate a little, but you can't say bad. Well, the choices are okay and good. And okay means bad. Yeah, in which case I think you can't even say okay. I think you said okay, actually.

Did I say okay? Oh, yeah. Maybe it was before we started recording. Well, I said I was doing okay because they're currently pruning all of the trees on the street outside of my apartment. So listeners, if you hear the chomping of trees or honking horns during this podcast, my sincere apologies. Please blame the city of New York.

I should also say that later in the show, senior reporter Amelia Thompson-DeVoe is going to join me to discuss the state of the debate over abortion in 2023, how public opinion has continued to evolve, in which states the debate over legislation is still live, and what it all means on the ground. So we will get to that later.

But right now, at long last, we can speak conclusively about how the polls did in 2022. So Nathaniel, you did the write-up of this year's pollster rankings. And when we say the polls were more accurate in 2022 than in any cycle since at least 1998, with almost no bias toward either party, those are your words. What does that mean in practice?

In practice, it means the polls were really freaking good in 2022 and 2021, which is included in our 2022 cycle calculations. And how did you come to that conclusion, really?

So basically, you know, we have three different ways of measuring pollster accuracy here at FiveThirtyEight, but our preferred is pollster error, which is literally just the difference between the polls margin and the actual margin. So, for example, if you saw a poll that had the Democrat leading by six and the Democrat ended up winning by four, that would be a two point error. And we literally we collect polls as folks.

certainly know if you're listening to this podcast. And for the pollster ratings and for this analysis, we look at virtually all polls that are conducted within the three weeks before an election. And when you do that, we literally looked at the margin of every poll versus the margin of the election it was measuring. And we took the average error for those polls weighted by pollsters. So if a pollster was particularly prolific, we didn't want to give it

too much weight in the average, so we kind of down-weighted those polls. But once you calculate that weighted average error, you come to the conclusion that 2022 had, on average, a 4.8 point error in its polls. And that is, as you mentioned, Galen, the lowest since at least 1998, which is when, as far back as our pollster ratings database goes, it is tied with 2004, but actually it just narrowly edges it out. So 2004 was a 4.83 point

And 2022 was 4.80. And a more typical error, if you do an average of every cycle, it's around six points. Although it should be noted that polls of larger geographic areas tend to be better and of higher up the ballot. So presidential polls, national polls tend to be

the most accurate, and then like Senate and governor polls, and then like House district polls tend to be less accurate. And then also, primary polls tend to be even less accurate than that. Overall polling is still very accurate, despite, or, you know, as accurate as it can be, despite kind of everybody wanting to throw up their hands about it after the 2020 election. So we got some looking under the hood to do here still, but you said something that might tip people off a little bit, which is that

you down-weighted pollsters that were particularly prolific. And some of the most prolific pollsters in the 2022 cycle included folks like Trafalgar, who were publishing a lot of polls, particularly in competitive races.

So like, why wouldn't we just keep that pollster weighted as it is and say, well, they put out a lot of polls and their polls weren't all that accurate. And so that shaped perceptions of the race. And we shouldn't be down weighting them to like make them less influential because they were influential during the 2022 cycle. So what we actually do, I don't know if down weight is the right term. You have to figure out a way to weight different polls or pollsters. And let's say that polling firm A is

does 50 polls and polling firm B does one poll, right? Should you give polling firm A 50 times the influence in evaluating how the polling industry as a whole did? We think no, but it should be more than one to one either, right? So we use a square root. So basically if a polling firm does 50 polls and they get a weight of about seven and a polling firm does one poll, they get a weight of about one. So it still puts a lot more weight on the more prolific pollsters, but it does...

curb it a little bit because, again, we're looking at how the industry as a whole looks at its polls and in sophisticated

models like 538s, we also find ways to kind of give less influence just because you're prolific, right? We use things like the House effects adjustment where it says, yeah, we know, Trafalgar, you want to survey every race three times. We also know that you have a Republican lean or bias, if you prefer, in your polls. So we adjust for that and you don't get that much influence really. So it represents a compromise that we think is pretty fair and

Because people spent, I think there was a lot of energy spent on looking at the Trafalgar's and the Rasmussen's and methods like 538s that are robust, weren't that influenced by them necessarily and had pretty good years. So it's kind of reflecting the way that the industry did in practice. So this is the top line.

But here's my question before we get any further. Are your concerns about polling allayed after looking at this data? Because we talked pretty bluntly about how this was a very important cycle for the polling industry because of what had happened in 2020.

2020. It was a larger polling error in 2020 than 2016. And it seemed like the industry was struggling to get a representative sample and then figure out how to weight that sample appropriately. So are those concerns nullified at this point? Look, it's been weird because we have this back and forth, this kind of ping pong, right? You had

pretty bad years in 2016 and a really bad year in 2020 and then good years in 2018 and 2022. I mean, if your basic thesis is that polling error is unpredictable or random, these past four cycles seem like they're a good demonstration of that. I do think still in the long run, response rates are down in the industry. The classic way of just surveying lots of people by random digit dial or voter lists is not as foolproof as it once was. So I think

In the long run, you might expect some kind of secular decline in the industry. But empirically, there isn't much of a pattern. And it may be – I mean my kind of story is like I think this is the year that pollsters realize they have to put some –

secret sauce into things, right? The classic assumptions don't work as well. And so you are fighting a battle where you have to like make what you can out of maybe not the best response rates and not the best raw data is one way to put it. Also, you know, you can tell a different story about Donald Trump himself maybe having somehow been a problematic candidate to ask about in polls, in elections without Donald Trump. The polls have been perfectly fine lately in elections with Trump somewhere in the

2020, you did have this, I think, fairly convincing excuse about COVID and that made more people, more Democrats stay at home and answer phone calls out of self-imposed quarantine or whatever. But so, no, I don't think it allays the concerns long term, but like people are also maybe more concerned than they should be, quote unquote. Or like their perceptions of polls zoom around from like, oh, they're perfectly accurate to, oh, they're total bulls**t.

or total crap, right? When like, in fact, it's a sliding scale, and it's somewhere in between. And this year is some evidence that we're within historical norms still. But you know, the polls will have will have bad years again, I'm sure. Exactly. I think, you know, moderation in all things, I think after 2020, people were too worried about the future of polls. And maybe after 2022, you know, people will be too confident in the future of polls, although

Yeah, I don't know about that, Nathaniel. Not based on anything I've heard or read. But yeah, but I think my perception of things is that polling is still pretty good. They're especially considering the challenges that the industry faces, but those challenges do exist and they are going to have to continue to contend with them.

So Nathaniel, you said there are three ways that we can calculate accuracy in the polling error. And the one that we've been talking about is what we think is the most significant, which is average error. You know, how close pollsters got to the result, regardless of which side of the result it was on. But the second way that we assess these polls is by looking at like, did they call the race correctly? Which for your maybe average consumer of these things, that's the most important thing. And basically,

Get ready, folks, because according to that metric, this was the worst cycle since 1998.

Meaning that the polls did worse in terms of calling the winner than in any other cycle since 1998, even though they got the closest to the actual answer. How should we explain that conflict? I know that you may be like, well, there is no conflict. But I don't know. It seems like a weird coincidence that both things would be true in the same year.

Right. So only 72% of polls, quote unquote, called the winner of the election correctly this year. The kind of average overall since 1998 is 78%. So you're right, Galen, technically, that is the lowest kind of correct call percentage. But I don't really care about this. As folks should be aware of if they're listening to this podcast, right, polls are not meant to be

predictions calling the winner correctly. If you have a poll showing a one point quote unquote lead that is well within the margin of error, that's essentially a toss up. And so I also had a chart in my article that shows that basically correct call percentage is basically just a function of how close the polls are. So polls that are within

a three point margin only pick the winner 55% of the time polls that are worth it between a three and six point margin pick the winner 69% of the time between six and 10 points, it's 86% of the time, et cetera, et cetera, until you get up to polls that are like double digits, which pick the winner like 99% of the time. And so like this makes intuitive sense, right? And you can actually look at the

the polls by year and it's basically like it is a function of like whatever the you know a given cycle like 2022 or 2000 or you know like 2004 like whatever percentage of the of races they quote unquote call correctly is basically like it's it's

It's highly correlated with how close the polls were that year. So this year, for example, 55% of the polls we analyzed were within a six-point margin, and that is higher than in any other cycle since 1998. So it makes perfect sense that the correct call percentage would be the lowest.

What should we take away from that stat then? Does that mean that these were the most competitive midterms basically in the past 25 years? Or does it mean that pollsters are hurting to some extent?

I think it's a combination of it was definitely one of the closer elections in recent years. It wasn't the closest, so that's not the entire answer. And closest in terms of looking at like the average margin of races that were polled. But so it was, I want to distinguish there. It did have the closest polls, right?

of any cycle we looked at. It didn't have the closest actual margins, right? But it was, I think it was like third closest or something like that. Another possibility that Nate brought to my attention while we were writing this article was the idea that maybe in this day and age when pollsters are kind of strapped for cash, a lot of their, you know,

polling budgets are lower than they've been. Maybe they're focusing in on those closest races, the most suspenseful races. And that means that a higher share of the polls are of these races in which there's genuine suspense. They just don't have the money to poll a race like the Connecticut Senate race that everybody knows about. And as a consequence, you know, they're not racking up those kind of easy wins, quote unquote. No, that seems pretty, I mean, I do think that like, because there are fewer polls now than there were at least a couple of cycles ago.

So where did those lack of polls come from? I don't think it's like the Georgia Senate races or Pennsylvania's, which did get polled a lot. Right. I think it is like just from having like worked on this problem for many cycles, it seems like you used to definitely get more polls of noncompetitive races than you do these days.

There's a third and final way that we judge accuracy, which is polling bias. Basically, does the bias lean towards Republicans or Democrats? Of course, we've had a few high profile cycles in which the polls were biased in favor of Democrats. And there was a lot of what's the word that I should use?

Hand-wringing? Consternation. I think at some points it got past the point of consternation into cable news hand-wringing over whether there was a Republican bias in this cycle. Sounds like you came to the conclusion that there was not. Tell us a little bit about that.

Yeah, so overall, the polls this year, or last year, I should say, were biased toward Democrats by 0.8 points, which, of course, is not a very significant bias. And I think the overall takeaway is that polls were not very biased this year, certainly in contrast to years like 2020, when it was a 4.7 point bias toward Democrats.

and 2016 when it was a three-point bias toward Democrats. But yeah, it was pretty interesting because I think there was this perception that the polls, if anything, that they overestimated Republicans. And that was the case in a few kind of key races. So you can kind of see where someone who's like cherry picking or just kind of like

eyeballing it, would come away with that conclusion. So like Arizona governor, Pennsylvania Senate, those races, the polls in those races were too good for Republicans in the end. But a lot of races that people weren't paying as close attention to, those polls had Democratic biases. And also the generic ballot polls overall, which were the most common type of poll in 2022, they had a Democratic bias of about two points.

although that may be because of uncontested races. But overall, the lion's share of the polls, when kind of weighted for all the considerations we had talked about, were slightly too good for Democrats. Yeah, I mean, I think those caveats are important. If you look at the generic ballot, then we treat the generic ballot as a forecast of the national popular vote. However, in some districts, you don't have a candidate from both parties, and that probably gave Republicans about

point or so of extra margin in the actual popular vote that we don't try to correct for You know technically by the way this pollster rating update covers every poll in the past two years including things in like special elections I don't know how much that affects things but like but the narrative of like oh a huge systematic and

Pro our bias is just not really true. You certainly had some individual polling firms who did quite poorly and they are sort of prolific. But like people haven't really like I don't know, man, you would think people would actually do the work to kind of prove that their thesis is true. But increasingly, people don't.

I think one of the issues here is that in terms of perception versus this rigorous conclusion that we've come to is that so few of our elections are actually competitive, that there's a lot of focus on states like Georgia or Pennsylvania or Arizona.

or, you know, Wisconsin. And some of the states that have gotten the most focus, most attention, have seen some of the largest polling errors. So Pennsylvania is a prime example in 2022. Wisconsin is a prime example in 2020. Obviously, they were in opposite directions. So the polls were biased in sort of Republicans' favor in Pennsylvania in 2022, biased in Democrats' favor in Wisconsin in 2020. And so I'm curious if, like,

Is it worth doing another round of this and looking at the most competitive races only? Or if you were to only look at the most competitive races or the races that we considered in our forecast to be competitive, how we would see this cycle as any different? Like if there was a larger error in the most competitive races, for example. I mean, in some ways, I think the whole point of doing this exercise is to give you more arm's length distance advantage.

from the narratives and from the race that everyone focused on. Right. Cause it's all data that's predictive in some sense about what's going to happen going forward in the sea of the industry. And, you know, pollsters deserve credit if they're a bunch of non-controversial races where they're really close or, or,

or blame if actually the races that are not covered as much, they were big polling errors. Because sometimes you actually, sometimes a race that's not on the radar screen winds up surprising, right? I think it was the Maryland Governor's race some years ago where Larry Hogan won and the race like wasn't polled in the final three weeks by anyone pretty much. So, you know, I think taking the bigger picture view is a feature and not a bug. Yeah.

Yeah, I think, you know, what we're trying to do here is look at the performance of the polling industry holistically. And I think that just looking at the close races would only give you a small part of the picture, right? And like, let's say that things were a little...

So.

If you just looked at the close races, you would think everything's fine with polling. We don't need to make any adjustments. But like those polls of non-competitive races count to methodologically. They're they're the same. And so like in terms of like looking at, you know, is there this long term systematic democratic bias in the polls?

you know, that is valuable information. Got to look at the whole picture. Okay, so another sort of specific question here, which is we came to the conclusion that house polling was the most accurate of any of the polls conducted in 2022, which I think will come as another surprise to listeners because it was the results in the House that perhaps led

the furthest off from what the perception was. Like, I think people going into the election felt like Republicans are probably going to win the House. And of course they did. But it took us a while to figure out what the final number was going to be. And it was, I think, I mean, it was still in the fat part of the bell curve, as we always say, but it was closer than I think a lot of people expected. So how should we think about those two facts that polling was actually the most accurate in the House, but that also it was the most surprising result?

So a lot of the reason why the polling was so accurate in the House, and yeah, you're right, Galen, this was actually probably the single most surprising thing for me when I crunched the numbers. Because as I mentioned at the top, normally state-level polls like Senate and Governor are more accurate than House polls. But this year, for the first time, I think since June,

2000, the House polls were the most accurate type of polls. But a big part of that, I dug into the numbers, a big part of that is because of these generic ballot polls, which of course are national, and it's just easier to get a representative national sample than it is of a district level poll. About half of our House polls this year were generic ballot polls, which is far and away the highest proportion that we have had. Usually it's like somewhere like 20% of polls are generic ballot polls, and the rest 80% are district polls. So

So that kind of methodologically explains, right? So like a higher proportion of the more accurate type of poll were House polls this year. And that's why you saw the average error of House polls in our averages be so low. But also kind of narrative wise, when so much of your perception of the competitiveness of the House is based on generic ballot polls, you're going to get a certain perception. And then we had kind of a weird year where the national popular vote didn't translate super well to

um, to like the seat distribution, like Democrats like did surprisingly well because they just had, like, I think they did a good job kind of like focusing in on these competitive races and they maybe didn't run up the margin in some of these deep blue seats. Um, and that cost them in the house popular vote. And they'd already mentioned the, you know, Democrats left more seats uncontested as well. Um,

But like in these competitive House seats, the ones that obviously make the difference for control of the House, they did surprisingly well. Republicans nominated some weak candidates. And so this led to this imbalance. So I think it was mostly about people, including us. And, you know, I wouldn't have guessed this before the election. People kind of miss misreading how the national popular vote would map on to the seat distribution.

One more thing about the house is that I suspect that pollsters who choose to pull individual districts are like a higher quality group of pollsters on average You know, you have to have first of all some list of voters in a particular district That's not trivial right if you're some fly-by-night polling firm that just says like a list of phone numbers or all registered voters in the state right if you can't pin them to a district then

you can't really do a credible district poll. So people who are willing to venture out and poll a particular house district are often among the better firms. And I think maybe we're specifically talking here about the New York Times-Siena College polling, which has...

to their guns and continued to do House-level polling. And they were quite accurate in 2022, maybe despite some of the things that we talked about on this podcast, including that they wrote up those polls suggesting that they were good for Republicans, even though they were good for Democrats. But look, you know, we don't consider that. We don't consider a narrative building in the pollster ratings. We just consider the polls. So let's get to that point, actually. We did all of them. We were able to come to a conclusion about the best poll

in America and the worst pollsters in America. I've sort of jumped the gun on this, but the best pollster in America, folks, is Nathaniel.

It's, uh, now the New York times, Siena college by a hair over Seltzer and company. Although really, you know, I think you probably would have to consider the two tied. Um, they have a kind of biometric predictive plus minus. Um, they have a fairly significant lead over, um, kind of the third place finisher quote unquote, but really I think, uh, you know, New York times, Siena college and, uh, Seltzer and company are clearly the two best pollsters in America. Um,

So specifically for 2022, Suffolk University and the New York Times-Siena College were the most accurate pollsters. They had average errors of only 1.9 points. And New York Times-Siena College even had no bias whatsoever, like a zero point bias. So they really had an excellent election cycle and it paid off in terms of kind of nudging them barely into the top spot in our pollster ratings.

All right, so congratulations to New York Times, Santa College, and Suffolk University. We did, however, see some notable pollsters get downgraded in this cycle of our pollster ratings. Who were those pollsters?

Yeah, so we have this table in the article that I published on Friday of the most and least accurate pollsters of 2022. And it really is quite striking when you kind of scroll down to the bottom in terms of the least accurate pollsters in 2022. You see they have a few things in common. They generally had a Republican biases, so they overestimated Republicans in the polls. And the pollsters themselves tend to be affiliated with Republicans.

kind of conservative causes and republican campaigns so these are places like trafalgar group uh insider advantage rrh elections the more information group um

So, you know, I think probably listeners are familiar with Trafalgar. They kind of are infamous for having this, you know, the secret sauce where they tend to produce results that are sometimes head-scratchingly good for Republicans. And in 2020, that ended up being correct, obviously, because it was pretty much every other pollster that was wrong about being too good for Democrats. But this year when, you know, polling was

was quite accurate overall. Trafalgar's Republican-leaning House effects really ended up biting them in the ass. And they had an average error of 5.3 points, and they were biased toward Republicans by about five points as well. And as we mentioned up near the top, they were one of the most prolific pollsters of cycle. They were really kind of contributing to the red wave narrative that didn't end up happening.

But they are by far not alone in terms of being a kind of conservative affiliated pollster that kind of misjudged the red wave that wasn't. And so they've fallen from where to where?

So they used to have a pollster rating of A-, mostly based on their strong performance in 2020, but now they have fallen to a B. And that, I think, is the largest downgrade of any pollster in our rating. So, Nate, what does that mean in practice for the influence they have in the forecast going forward? It means they have meaningfully less influence, but a B is an average grade. Mm-hmm.

And not a poor one. I mean, you do have to remember, like, these sample sizes aren't that large, right? We only have 113 polls in our database from Trafalgar that qualify to be rated, which, you know, is not that many. I think SurveyUSA has, like, almost 900 now or something. So, and they did have good 2020. I mean, people are like, oh, these guys are always wrong, right? They're not always wrong. They always have Republican lean or house effect or bias, if you want to use that term, right? Right.

But they're kind of playing this game where if the polls have a GOP bias or a democratic bias, rather as they did in 2016 and 2020, they get to look really smart. And you can be – how to put this? You can be like cynical. You can have a subjective cynicism about kind of what their business model is and how good their polling is. But still, you have to come up with rules and not just be totally –

subject to, right? When you had people in like, I mean, real clear politics this year, like, ah, let's just subjectively kind of chuck out without, maybe there's some standard, but they're not explaining it very well at the very least, right? Let's just kind of chuck out these liberal leading polls we don't like, right? And their averages wound up being quite a bit less accurate than ours because they were just using subjective or unexplained premises for which to evaluate the pollsters. And so, you know, I mean, if I had to

put my own money on the line? Would I discount Trafalgar a little bit more than we do? I mean, probably, but they've also gotten feedback from the market. There'll be a few people who actually sign up to work with them as a client. They probably ought to examine their own methodology and see what went wrong here, right? And so there is some self-correction taking place in a free market.

Yeah.

Or recently, we banned Upholster for engaging in betting markets, which of course is unethical if you are pulling those markets.

But yeah, but that's the reason basically that we haven't banned Trafalgar. You know, we have had issues with their methodology in the past, in 2020, for example, when they were sneakily non-transparent about who was funding their polls. But, you know, the kind of the pollster ratings and, you know, what goes into our models are just based on the bloodless facts. And as Nate said, you know, Trafalgar is no longer...

is going to be significantly dinged by the fact that they had a bad 22-inch hue cycle.

Those are our 2022 pollster ratings. Nate, you mentioned the markets, which are in a little bit of turmoil, I think, at the minute, and gives me an opportunity to pivot into a question that I wanted to ask before I let you guys go, which is I think people are focusing a little bit on Silicon Valley Bank over the weekend and today, and also the political response to that bank's failure last Friday.

If this becomes a sort of contagion scenario or if we see political fallout from this, we will talk about this again. But since I have you here, Nate, what are you thinking about the situation three days in? Yeah, look, I mean, it put the White House in a difficult spot because bailouts and there's like a very online debate about is this a bailout or not? And I'll explain why I kind of

think it is if you want in a moment, right? But like bailouts are going to be unpopular, particularly for industries that are concentrating wealth in the hands of a few very rich people and businesses. On the other hand, to have some type of

financial crisis would also be bad for President Biden trying to get reelected as the kind of campaign is slowly shifting into gear. So they're in a spot where they could probably say, well, look, the politics of this is probably a wash. What's best for the economy? And I guess they decided having greater stability is worth the increased risk of moral hazard. By moral hazard, I mean that if people don't bear the consequences of their own actions, they can

take riskier actions that aren't necessarily good for the economy as a whole. If you can make some bad bets on interest rates, for example, and have, and to be fair, Silicon Valley Bank, the shareholders will not be made whole, they'll lose, but still, you know, their customers can say, okay, you guys are going to like make some more aggressive gambles. You can take like a small equity stake in these businesses. And as a result, you know, you basically kind of get to, um,

kind of free ride a little bit. If you make really big mistakes, then at the very least, your customers are made whole. And so therefore, your customers can feel safe and they deposit with you, then you kind of gamble with that money. And so I don't know, but I can't really speak to whether it was the right decision. Of course. Financially, but I think it is going to lead to some further tension between kind of Washington, D.C. and Silicon Valley.

At a time when that tension is already kind of increasing. You have a Democratic White House. You have – Silicon Valley has traditionally been a Democratic constituency. But you have more and more outspoken conservatives in Silicon Valley about stuff like wokeness, quote-unquote. Also, there's a big regulatory agenda. How will AI be regulated? How is social media regulated?

you know, and now financial regulation, these are all pretty high stakes issues. I do think that like Silicon Valley would say, oh, people in DC, they don't really understand our culture, right? But it's kind of like mutual to some extent, right? I mean, there's a lot of people who have talked about how we want free markets and free enterprise, right? And then they're kind of like asking for help from the government, right? Despite a libertarian ethos. I mean, they look a little bit hypocritical here. And I think that tension could lead toward more efforts at regulation. Yeah.

Well, as they say, everything that's old is new again. It's 2023 and we're back to talking about moral hazard. But of course, hopefully, fingers crossed, this is nothing compared to all of that. We will see how it goes and how politics reacts. But thank you, Nate and Nathaniel, for explaining polling accuracy and a little bit of what is going on in the banking industry and politics this morning. Thanks, Galen. Thank you.

You're a podcast listener, and this is a podcast ad. Reach great listeners like yourself with podcast advertising from Lipson Ads. Choose from hundreds of top podcasts offering host endorsements, or run a reproduced ad like this one across thousands of shows to reach your target audience with Lipson Ads. Go to LipsonAds.com now. That's L-I-B-S-Y-N-Ads.com.

You're a podcast listener, and this is a podcast ad. Reach great listeners like yourself with podcast advertising from Lipson Ads. Choose from hundreds of top podcasts offering host endorsements, or run a reproduced ad like this one across thousands of shows to reach your target audience with Lipson Ads. Go to LipsonAds.com now. That's L-I-B-S-Y-N-Ads.com.

It's been a little over four months since the midterms, when Democrats overperformed historical expectations in part because of the issue of abortion.

It looks pretty clear from the data that the Supreme Court's overturning of Roe v. Wade pushed voters away from Republicans, particularly in states where legal abortion was a live issue. Today, we're going to take a look at what has happened in the debate over abortion since then. Has public opinion changed at all? What approaches have states taken to outlawing or expanding access to abortion after seeing the results of the midterms?

and how are legal challenges playing out in courts? Here with me to discuss is senior reporter Amelia Thompson DeVoe. Welcome to the podcast. Hey, Galen. There was a flurry of legislation in the aftermath of the Dobbs decision, and abortion was a present and, according to voters, as we saw in the polls, salient issue in 2022. How are voters thinking about abortion with the midterms now in the rearview mirror?

Well, it looks like they're thinking about the issue pretty much in the same way that they were around the midterms. For example, an NPR-Ipsos poll conducted around the anniversary of Roe v. Wade in late January found that 60% of Americans think the Roe decision, which of course was overturned last summer, was the correct decision. Only 41% think that the Dobbs decision, which was the decision that came out in late June, was the correct decision.

And Americans are generally not happy with the way that state lawmakers have handled this issue, which notably is consistent across party lines. So Democrats, as well as Republicans, were about as likely to say that they were unhappy and that overall was about half of Americans who said they disapproved of how their state lawmakers were handling abortion over the past year.

And really interestingly, the part of the Dobbs decision that seems to have most resonated with voters is the idea of handing decisions about abortion back to the people.

That was a big theme in the Supreme Court's decision. And according to the poll, 69% of Americans say they want abortions legality to be decided by ballot measure or referendum rather than by state legislators, which I thought was pretty interesting.

If that were to happen, a majority of Americans say they 54% say they'd vote in favor of abortions legality at the state level, while only 27% say they would vote against abortions legality. And again, continuing a theme we saw in the midterms.

Democrats are a lot more united on this than Republicans are. So the vast majority of Democrats say they'd vote for abortions legality. And more than half of Republicans did say they'd vote against abortions legality, but it was a much smaller share. So only 56%, which is certainly not overwhelming.

Have state level or even national politicians changed their approaches to the issue, you know, as a result of the midterms and what appeared to be a backlash to the Dobbs decision? So anti-abortion advocates sure haven't. So the people who are really hardcore on this issue are just saying, you know,

Yes, the response in the midterms was unexpected. I've heard a few advocates say to me that they should have been more prepared. They should have had better messaging to explain why it was that states were banning abortion and what the impact would be on women and that they were caught a little bit flat footed there.

But they've largely said, you know, we need to stay the course. We need to continue to ban abortion everywhere that Republicans are in control. And we need to convince Americans that this is the right path to be on. However, as we're discovering and as anti-abortion advocates have also been grappling with,

Not every lawmaker who calls themselves pro-life is on the same page about what should be done on abortion. And it was a lot easier to jump on board with more extreme pro-life priorities while abortion was protected by Roe than it is now, because politicians are being held politically accountable for this in a way that they're not or they weren't.

before Roe because they would pass a law and immediately would get held up by a court. And it was really a symbolic thing that let them kind of brush up their pro-life bona fides, but was not going to hurt them at the ballot box. Now, if they vote to ban abortion, abortion could be banned. And so I think that's why we're seeing, I,

I hesitate to call it moderation because the proposals that we're seeing are not moderate. But we are seeing a little bit of pulling back. For example, in Nebraska, which is a state where I would have expected to see a full ban, the

legislation that's being pushed forward this year is a six week ban, which will still affect a lot of people, but obviously does not completely outlaw abortion. Similarly, Florida lawmakers put forward a six week ban. On the other hand, state lawmakers in Utah and Wyoming went ahead with full bans. So I think it's a it's a bit of a mixed bag.

You mentioned four states that are currently grappling with the debate over abortion rights. Are there other states where the debate is still live or has it pretty much been settled given the current legislative makeup elsewhere? Well, it has been settled.

It hasn't been fully settled in even some of the states that banned abortion in the wake of Dobbs, which is interesting. So Tennessee is an example of where there is a live and extremely contentious debate happening in the legislature over the past few weeks.

about whether the state's ban should include some very limited exceptions. Right now, Tennessee has one of the most restrictive abortion bans in the country, and some Republican lawmakers have said, hey, this is too restrictive. We need to put in more exceptions. And so that's been a fight that's been happening. A Republican lawmaker in Kentucky said,

introduced a similar bill that would introduce some exceptions into the state's ban. It's not clear right now if that is going to have legs. And then there's been an ongoing debate

over abortion in South Carolina, where lawmakers are in a bit of a weird position because, first of all, there's been this continuing standoff between the state House and the state Senate, which have been divided since last summer about how far they want to go on abortion. And then the fact that earlier this year, the South Carolina Supreme Court overturned the state's pre-existing six-week abortion ban. And so now there are questions about whether

another ban hypothetically passed by the legislature would actually hold up in court. So lawmakers are having to grapple with that as well.

I understand that this is not really how politics works, but you mentioned hypothetical six-week bans, having some exceptions versus no exceptions, et cetera. If you were to try to take some sort of average of what was the most popular law on abortion, I think we've talked about in the past, but just to contextualize where these laws stand, what would fly with Americans?

So I think the abortion ban that would be likeliest to fly with Americans would be something that basically replicated what Roe put into existence, which is a ban around fetal viability that would be...

22 to 24 weeks of pregnancy, roughly, or even in some parts of the country, a 15-week ban or even a 12-week ban. There are certainly this would not be something that would be popular in blue states or even in many purple states where voters want abortion to be legal into the second trimester of pregnancy.

But Americans are more divided and have been more divided historically about whether abortion should be legal in that second trimester. And so, you know, for a state like Florida, I think what Republicans are dealing with is that a full ban would be quite unpopular. I think it's pretty obvious why they're not going after that this year. Yeah.

And even a six-week ban is likely to be less well received than something like a 12-week ban because what we see in the polling is that voters and Americans generally are overwhelmingly pretty uninformed about abortion. But one of the things they do tend to know is that most abortions happen in the first trimester of pregnancy. And Americans are also overwhelmingly likely to support legal abortion in the first trimester of pregnancy.

So any of these laws that we're talking about that are being proposed that would restrict abortion in the first trimester, and especially something like a six-week ban, which really doesn't give people who are thinking about having an abortion much time to get one, and it can be tricky to get an appointment in this world where there are fewer abortion clinics than there were a year ago. I think all of these are

politically dicey. And that's important to remember because they do seem more moderate than a full ban. But they're still out of step with where most Americans are on this issue. Yeah. Speaking of the question of, you know, actually accessing an appointment, you and our colleague Maggie Kurth recently published a map showing how access to abortion has changed across the country since Dobbs and how it could continue to change pending the debates that we've discussed a bit.

And this is not just from the perspective of sort of state by state, where are things legal and up until what point. You actually calculated sort of driving time and appointment availability depending on the county that you live in in America. And so could you describe for us a little bit? And I should say this is an interactive map that folks can go on our website and use and take a look at. And this is audio. So we won't be able to do it justice here. But can you just describe a little bit how things change?

have changed from that metric as opposed to just state by state since the Dobbs decision.

Yeah. So just to give a little bit more credit for this interactive, which I hope folks do check out, um, it's based on data that was provided to us by Caitlin Myers, who's an economics professor at Middlebury college and has really been at the forefront of trying to do the research and provide the data that will help us understand the impact of the Dobbs decision in real time. Um, and my colleagues on the interactives team, Elena Mejia and Aaron Bycoff did a fantastic job, um,

making this just one of the most beautiful interactives I've ever seen. So folks should definitely go check it out. But to talk about the meat of what we uncovered in the interactive,

I mean, travel times have changed enormously since Dobbs. So the average driving time to an abortion clinic was above 100 miles in only three states pre-Dobbs. That was Wyoming, North Dakota, and South Dakota. And those were states where there weren't many abortion clinics, and they're big states. So, you know, they're not states where I think anyone would be particularly surprised to see longer driving distances. Right.

In the wake of Dobbs, it's now above 100 miles of driving distance for 14 states, and that's including Texas and Louisiana, where the average driving time increased by more than 400 miles. So that is hours and hours and hours of driving to get to an abortion clinic. And that's significant because researchers have found that when people have to travel more than around 100 or 150 miles to get an abortion,

That really affects whether they will actually end up getting the procedure. So a 400 mile driving distance obviously is something that could be a real deterrent for people. And for legislators, that's the point.

Right. That's exactly the point. Yeah. So this is, you know, this is a huge win for legislators in states like Texas and Louisiana, where geography is on their side because those states happen to be bordered by a bunch of other states who also banned abortion. And that's one of the big takeaways from the interactive as well, is that it's not about what any one state does on abortion at this point. It's about what the neighboring states did. So the fact that Kansas

still has legal abortion into the second trimester is kind of a thorn in the side of red state lawmakers in the South and Midwest because people can still travel to Kansas. And those clinics are serving a huge population of potential patients. And that was the other dimension we looked at in the interactive. This is another metric that Caitlin Myers provided to us.

And it looks at the average service population of clinics in a particular region. So it's basically not, it's not looking directly at appointment availability, which is something that Caitlin has gotten at in a different project, but we didn't put it in that specific metric into this interactive area.

We looked at average service population, which is a general sense of like how many people are likely to go to this clinic if they need an abortion. And so that unsurprisingly changed too in the aftermath of Dobbs when hundreds of clinics closed.

Now clinics that serve women in Texas, for example, which remember are also really far away, are serving hundreds of thousands or even more than a million reproductive age women. And congestion is particularly bad in Kansas, where some clinics are serving a population of 1.8 million women, and Southern Illinois, where some clinics are serving a population of more than 700,000 women. So those clinics...

Again, we don't have direct data in the interactive about how long you might have to wait for an appointment, but those clinics are under tremendous, tremendous strain, and it is more difficult to get an appointment in those areas of the country than it was a year ago.

And I should say, if folks go to the website and look at this interactive, they can also see how these dynamics and these numbers would change if other states that are currently sort of places where people are traveling to to get abortions, like Kansas or, say, South Carolina or North Carolina were to ban abortion, you know, what would happen and how would these numbers change? So folks can check that out. I'm curious in all of this, are any blue states

making access to abortion easier or expanding the limit for when a woman can get an abortion in the aftermath of DOPS.

So blue states are definitely trying to make it easier to get an abortion in a bunch of different ways. One of the things that they're also doing is trying to protect people who come into their states for abortions, and in particular trying to protect providers from potential legal liability. The reality, though, is that there are some states that sort of have considered blue

making it easier to get an abortion later in pregnancy. But as we were discussing earlier, you know, that's something that really affects a very small number of people. It's an incredibly important thing to abortion rights advocates, but it's not something that has a really broad impact. So the more people

impactful move for blue states who want to make sure that abortion access is still protected and that people can safely travel into their states, which is a stated goal of many Democratic governors, is to try to make sure that providers are still comfortable offering abortions to people who are coming from out of state. And, you know, the broadest

range of clinicians possible can provide abortions. So one of the classic ways that red state politicians would restrict access to abortion was to make it so that only physicians could provide abortions. Blue states, on the other hand, have tried to expand access to abortion, particularly in rural areas by broadening the scope of

of clinicians who can provide some abortions. So making it possible for nurses, for example, to prescribe abortion medication or to perform simple aspiration abortions, that kind of thing. So there are definitely efforts by blue state lawmakers that we already saw right at the beginning of the session. I mean, Minnesota and

Illinois had abortion protection bills right out of the gate. There's a constitutional amendment to protect abortion rights moving in Maryland. So this is something that blue state lawmakers are also definitely working on.

One other piece of state-level legislation that you have looked at is what's happening in some red states in terms of Republican legislators wanting to expand the social safety net for new mothers. Actually, you wrote an article about this last week titled, Some GOP Legislators Are Trying to Show Their Pro-Life, Not Just Anti-Abortion. What's that dynamic? To what extent have we seen actual legislation being passed? What legislation is being debated and why?

So this is one of the most fascinating post-Dobbs dynamics to me. And it's something that has been bubbling in certain corners of the anti-abortion movement for a while. It's not something that emerged wholesale after Dobbs.

But the idea is basically that there's both a moral and political imperative on the lawmakers who are banning abortion to make sure that if more babies are going to be born as the result of their policies, then parents and children have the support they need.

And so there's been an argument in some corners of the anti-abortion movement for a while that this needs to go with abortion bans. The focus was on banning abortions first. So I think people in that corner of the movement are feeling like they're more empowered and

have more of a platform to say, okay, you know, this is this is the next step, guys, like we got what we wanted. So let's follow through. And I think there's also a political piece of it, which is that it's been very easy for Democrats to point to Republicans and say, look at look at these guys, they only care about banning abortion. They don't care about things like health care for pregnant women or new moms. The states that haven't expanded Medicaid are

likely to be in the column of states that have banned abortion. So you can look at them. They also tend to have higher rates of maternal mortality, higher rates of infant mortality. So it's easier to look at them and say, these are states that are not doing as much as other states to protect the health of mothers and babies. And yet they're focusing in this laser focused extreme way on banning abortion.

So what we're seeing is a, I would say, limited beginning to try to address some of this among Republican lawmakers. The one that has been really low-hanging fruit in terms of policy is expanding Medicaid coverage for women who have had babies from 60 days postpartum to a full year.

And this is something that a lot of states adopted when it was offered as a new option in the Democrats' 2021 pandemic relief bill. But some states like Mississippi held out because they're just very opposed to expanding Medicaid in any form. The governor and the speaker of the House, who had both been pretty publicly opposed to it, came around on that this year. And now it has passed both houses of the legislature. And I don't know if the governor signed it, but it seems imminent if he hasn't.

So that's an example, but there have been other states where requirement income eligibility for benefits like food stamps have been proposed to be raised, that there's been discussion of giving tax credits for unborn fetuses or trying to have better child support enforcement mechanisms for unborn fetuses. And, you know, so...

And then there's also been talk of more support tax credits and government money for crisis pregnancy centers. So there's more room for bipartisan consensus on some of these than others. Some of them are pretty longstanding anti-abortion priorities like the crisis pregnancy center funding. But it is interesting to see Republican lawmakers in some states like Mississippi getting behind, you know,

you know, an expanded eligibility period for Medicaid for some people, really, it seems, because of Dobbs.

All right. So there is a final piece here, and it could be a key one in the discussion of the debate over abortion post-Dobbs. And it's that there is an ongoing lawsuit over the legality of a drug used for medication abortions. In fact, whether or not the FDA approval process was above board. Can you tell us a little bit about where that lawsuit stands and how likely it is that this could be removed from the market?

So the challenge is, as you were mentioning, about the nature of the FDA approval process for mifepristone, which is one of the two drugs that is commonly used for medication abortion in the U.S. It's been on the market for 20 years. Anti-abortion advocates have been trying to get it off the market unsuccessfully for 20 years.

And they filed this lawsuit in the courtroom of a judge named Matthew Kazmarek, who is a Trump appointee who's been on the bench for a few years and before that worked in the Christian conservative legal advocacy space.

So he's clearly someone who is likely to be friendly to this kind of argument. He is having a hearing on the lawsuit midweek. So we're, you know, this is still very much in process. We don't know how he's going to rule, but the expectation is that there's a real possibility that

And he could rule that the FDA's approval of mifepristone over 20 years ago was invalid. And he could tell the FDA that he needs to take the pill, that they need to take the pill off the market, which would be a very big deal for the FDA and also for abortion providers. I mean, at this point, do we have a sense of how likely it is that

that ruling comes down? And if it does, like, would this change things overnight? Or is this something that we expect to make it to the Supreme Court eventually, no matter what? Yeah, I mean, kind of all of the above. So it's a little bit hard to predict exactly how it would play out if he does rule that the FDA unlawfully approved Mifepristone, which

It's hard to predict how this will turn out because in the courtroom of pretty much any other judge in the country, this lawsuit would not go anywhere. So the fact that he's taking it as seriously as he is is signaling to a lot of people that this could mean that he's willing to rule that Mifepristone was unlawfully approved. Let's say he did that. Some of this would depend on what the FDA does next.

Some of it would also depend on what abortion providers do, because there is a possibility for them to offer medication abortions just using the other drug in the two-drug sequence that's used for medication abortion now. And so they might still just...

continue to offer medication abortions even if mifepristone wasn't available and it would just be using this other protocol. So on the other hand, the other protocol is slightly less effective than the protocol that uses mifepristone and this is likely to cause quite a bit of chaos.

And the abortion provider infrastructure, which is already under a lot of pressure, as we were discussing earlier. So it's a ruling that has the potential to unlock a lot of chaos, but it's still too soon to know what would happen. There are a lot of moving pieces here.

All right. Well, Amelia, this has been a lot of really good information for understanding how this debate is playing out, you know, as we approach a year since the Dobbs decision came down. So thank you so much for chatting with me today. Thanks for having me, Galen.

My name is Galen Druk, and Tony Chow is in the control room. You can get in touch by emailing us at podcast at 538.com. You can also, of course, tweet at us with any questions or comments. If you're a fan of the show, leave us a rating or review in the Apple Podcast Store or wherever you listen to your podcasts, or tell someone about us. Thanks for listening, and we will see you soon.