\n<\/p>\n
When I was Politics editor of The Times in 2018, polls indicated that Democrats were likely to ride a wave of anti-Trump energy to win big in the midterm elections. Democrats went on to capture control of the House of Representatives, but Republicans did better than expected in a couple of red states \u2014 Missouri and Indiana \u2014 and increased their Senate majority.<\/p>\n
Now we face another midterm election with President Trump in office, and polls again favor the Democrats, including in the House. Recent New York Times\/Siena polls also show the party with a slight edge to flip the Senate. But that may come down once more to Senate races in red states like Texas, Ohio and Kansas.<\/p>\n
Are polls overestimating the Democrats\u2019 chances? What are we doing to accurately capture support for candidates? I\u2019ve been thinking about these questions in my current role focusing on reader trust in The Times. For instance, we published a poll this month showing a six-point lead for James Talarico, the Democratic nominee for Senate in Texas. Many other polls also show Talarico leading. Given the base-line partisanship of Texas and voters\u2019 track record of electing Republicans there, regardless of headwinds, I wondered whether readers would trust our polls or question their credibility.<\/p>\n
I recently spoke about polling and the midterms with Nate Cohn, our chief political analyst, and William P. Davis, who leads our Election Analytics department. They work on our Times\/Siena Poll, which is widely considered to be one of the most accurate polls in U.S. politics, and Nate recently explored the question of polls and Democrats\u2019 chances in his newsletter, The Tilt. Our interview has been edited and condensed.<\/p>\n
Let\u2019s start with a point-blank question. Should people trust these polls showing Democrats running ahead or neck-and-neck in red states?<\/strong><\/p>\n NATE COHN:<\/strong> You can trust them to a degree. You can trust the polls to usually be in the ballpark, but you can\u2019t trust them to be exactly \u201cright.\u201d Polls are inherently imprecise and subject to many sources of error.<\/p>\n So if by \u201ctrust\u201d you mean \u201cassume that the leading candidate will win,\u201d then the answer is \u201cno.\u201d The polls have been wrong by this standard before and they will be wrong again. But if trust means \u201crely on the polls as the best source of information on the electorate, even if an inexact one,\u201d then the answer is yes.<\/p>\n WILLIAM P. DAVIS:<\/strong> There are a lot of indicators pointing in the same direction this year \u2014 polls of specific races, President Trump\u2019s approval rating, the fact that Republican candidates in deep-red states are distancing themselves from the president, the expectation that Democrats have a turnout advantage in midterm elections, etc.<\/p>\n The question becomes: How good of a year will it be for Democrats? Polls suggest a pretty good one. But whether the national results swing eight or 10 or 12 points to the left from 2024 will make a material difference in whether or not Democrats are able to take the Senate, and polls can\u2019t guarantee that level of precision.<\/p>\n But some people read a Times poll headline saying so-and-so is ahead and take that to the bank. What do you say to people who see our Senate polls and regard them as predictions for Election Day?<\/strong><\/p>\n WILL:<\/strong> Polls measure public opinion at a specific point in time. If people are still making up their minds, or late-breaking information upends a race, any prior poll will miss it. Polls can help us understand which races are close, and why; we won\u2019t know until the results are in how they will shake out.<\/p>\n As rigorous as our polls are, I\u2019m a little skeptical that a Democratic Senate candidate will win in Texas, given historical candidate performance there. What\u2019s your pushback on that?<\/strong><\/p>\n NATE:<\/strong> I\u2019d say that you shouldn\u2019t be quite so skeptical. Over the Trump era, Texas Republicans have become dependent on elevated levels of support from Hispanic voters. Without them, Texas is highly competitive, as Republicans have lost enormous support among white voters in the state\u2019s highly educated suburbs. Every available piece of evidence \u2014 from our polls to on-the-ground reporting \u2014 suggests that the G.O.P.\u2019s gains among this group have evaporated.<\/p>\n For good measure, the G.O.P. has nominated a candidate \u2014 Ken Paxton \u2014 who has been investigated, indicted and impeached. Add in a hostile national political environment for Republicans, and I don\u2019t think there would be anything surprising about a Democratic win in Texas. Nearly every nonpartisan poll since at least Labor Day has shown Talarico ahead.<\/p>\n WILL:<\/strong> It\u2019s reasonable, even good, to be skeptical. But people also have a false memory of Texas polls in the past. Most polls did not show Beto O\u2019Rourke ahead in the 2018 Senate race against Ted Cruz. Our Times\/Siena poll of that race had O\u2019Rourke down eight points, and he actually only lost by 2.5 points, so we underestimated him a bit in that race. This year\u2019s race is still pretty close, but it\u2019s undeniable that Talarico is doing better there in polls than any Democrat in recent history.<\/p>\n I raised the question earlier of reader trust in polls. Readers have asked me if we\u2019re \u201csure\u201d that our Texas polling is right and whether we would kill the poll if we weren\u2019t sure. How do you see that? And do we ever not publish polls that we\u2019ve conducted?<\/strong><\/p>\n NATE:<\/strong> It might seem strange, but it\u2019s pretty important that we publish every survey result. We\u2019ve never withheld a Times\/Siena poll result. Many smart election analysts would have never imagined Trump winning in 2016, would have never imagined that Democrats could have avoided a \u201cred wave\u201d in 2022, or would have never believed Trump could surge among young and nonwhite voters in 2024 \u2014 if it weren\u2019t for polls raising it as a real possibility.<\/p>\n And if The Times had withheld positive polls for Trump in 2016, we would have been accused of bias.<\/strong><\/p>\n NATE:<\/strong> Yes. We would absolutely be accused of cherry-picking polls that fit a partisan bias or, even worse, in order to fit or construct a partisan narrative.<\/p>\n Of course, there are sometimes good reasons to wonder whether our poll is in the ballpark. That\u2019s especially true when there\u2019s a lot of other polling or our reporting shows something different. When we get an outlier like that, we can be cautious about the result in our write-up.<\/p>\n Even when the poll does appear to be a true outlier, it doesn\u2019t mean it\u2019s wrong. In 2022, I downplayed the result of a poll in Kansas\u2019s Third District, which showed the Democrat Sharice Davids ahead by 14 points. This seemed totally bonkers. The race was considered a true tossup, with both parties spending millions of dollars. One Democratic pollster called the result \u201ccray\u201d and sent the laughing-while-crying emoji. In a very rare move, the Democratic campaign even disputed the result: \u201cI don\u2019t know what model The Times is using but this is going to be a very tight race. Our polls show it\u2019s a true tossup at this point.\u201d<\/p>\n In the end, Davids won by 12 points. I still get flack for downplaying the result.<\/p>\n Even modest polling errors in the past have underestimated Republican strength. Why is that, and what have we done to account for it?<\/strong><\/p>\n WILL:<\/strong> Poll error from years when Donald Trump has been on the ballot are top of mind for most people, and with good reason. The chief theory for why polls have missed relates to who polls have a hard time reaching: low-education voters who don\u2019t vote as regularly. If those voters overwhelmingly favor one candidate over another, and if polls don\u2019t reach enough of these voters, that can add up to the sorts of issues many pollsters saw in 2016.<\/p>\n At the Times\/Siena Poll, we put a lot of effort into reaching these voters, in part by placing more calls to voters who are less likely to respond. That ends up costing us a lot more money, but it pays off: In 2024, we saw early on that Trump was building a lead with these voters.<\/p>\n And we are continually reviewing how our poll performs and making improvements. In 2024, we made a change to keep in our poll results respondents who hang up mid-survey. This year, we made significant changes to how we weight our polls, with the goal being to better account for the types of people we have a hard time reaching.<\/p>\n Do we know how to poll Democrats better than we know how to poll Republicans?<\/strong><\/p>\n NATE:<\/strong> Not necessarily. When polls underestimate Republicans, it\u2019s entirely possible that Democrats contribute to the problem. Imagine, for instance, if the \u201cresistance\u201d Democrats are much likelier to take a poll than a working class registered Democrat who sometimes votes for Trump.<\/p>\n What else have we learned from past polls, and what are we doing differently this year?<\/strong><\/p>\n NATE:<\/strong> The thing that we\u2019ve learned over the last decade \u2014 and that the industry as a whole has learned \u2014 is that even a perfectly designed survey will probably underrepresent Trump\u2019s supporters.<\/p>\n As Will mentioned, we\u2019ve adopted a new method for \u201cweighting,\u201d the technique pollsters use to give more weight to underrepresented groups. The new approach lets us leverage the rich data we have on our survey respondents while weighting the poll to more thoroughly represent the population. This approach improved our past polls when we took the data from our old surveys and applied the new weighting approach. It would have especially helped in the white working-class states where polls have erred in the past, but that\u2019s no guarantee of better results in this cycle or the future.<\/p>\n WILL:<\/strong> There are serious challenges to making sure you have the right methodology, that you\u2019re reaching a representative group, that your weighting is correcting for the things that matter most. We sweat every one of those decisions. <\/p>\n And there are other ways to mess up beyond the methodology of the poll. The specific wording of questions is difficult to get right. Our colleague Ruth Igielnik leads our questionnaire development process, and we often spend weeks refining the specific language we use. But we sometimes realize after scrutinizing the results, reviewing recordings of calls, calling back voters and comparing our results to other polls that we didn\u2019t quite get it right.<\/p>\n One example was in 2023, when we found that more than 20 percent of voters in the presidential battlegrounds said they would support Robert F. Kennedy Jr. for president. We had a lot of reasons to believe that number was way too high: We compared our numbers to what others were finding, we looked at what respondents supporting Kennedy said in other questions, and we had a theory about what caused the issue.<\/p>\n We still released the results, but we worked hard to place it in the proper context when we wrote our story about the finding. It was uncomfortable for us, to somewhat disavow our own finding. But we thought it revealed something about how unpopular Trump and Biden were, and we tried to explain that. And a few months later, we conducted an experiment to try to prove out our theory of why we got such a flawed result. Our theory was proved correct, and we wrote that up, too, to try to help readers better understand how polling works.<\/p>\n We <\/strong>reported<\/strong> the other day that a fake polling firm had deceived The Times and other polling aggregators with several phony poll results. They said they were trying to \u201cdemonstrate the vulnerabilities\u201d in the political polling field. Size this up \u2014 what did they prove and what did they not?<\/strong><\/p>\n WILL:<\/strong> It\u2019s worth noting that the \u201cpolls\u201d put out by this group, which went by Wedgewood Polls, had no bearing on our own Times\/Siena polls. They appeared in our poll trackers but did not receive prominence in our coverage.<\/p>\n Wedgewood certainly proved that we\u2019re not perfect. But they claimed that The Times and others had included their polls without vetting them, and that\u2019s not true. The Times vets all new pollsters, asking a series of questions designed to determine whether a group is attempting to field representative polls, learn where their funding comes from.<\/p>\n This process has weeded out untrustworthy pollsters. But Wedgewood deceived us \u2014 as they appear to have deceived others who asked similar questions or even gave them money. It\u2019s not clear to me that their true goal was to demonstrate vulnerabilities. We are putting a lot of thought into how we could have found these issues out earlier.<\/p>\n One thing the episode highlighted is the way that newer pollsters can attract attention. The group was able to quickly gain traction, in part by claiming to be associated with another polling group and in part by polling some races that had little polling.<\/p>\n It\u2019s gotten evermore expensive to field a high-quality poll and ever-cheaper to field a low-quality one. And A.I. is making it easier than ever to fake things. The Times\/Siena Poll is one of a few pollsters left with the resources to both put a focus on quality and poll many of the key races each election. Increasingly, pollsters are having to choose between one or the other.<\/p>\n Last question. If Election Day results are materially different than our polling suggests, is the polling worth the risk to our credibility?<\/strong><\/p>\n WILL:<\/strong> In the end, I will regard how well we did as journalists by how well our coverage reflected what ended up happening.<\/p>\n If Democrats do really poorly, we should do some soul searching about both our methods and the questions we asked, as we would for any other story we got wrong.<\/p>\n If Democrats do well but fail to take the Senate, I would ask myself how well our coverage prepared readers for that possibility. I hope the answer is that it prepared them well, because we\u2019re constantly talking about how to ensure our coverage prepares people for this scenario. I would also ask how well our poll questions helped us understand and explain why these races were so close.<\/p>\n Similarly, if Democrats do much better than expected, I would ask myself how well we understood the circumstances that caused so many Trump voters to either stay home or switch to Democrats.<\/p>\n NATE:<\/strong> Polling isn\u2019t like a lot of the other journalism at The Times. Most of the report is about facts. If Peter Baker says that Trump\u2019s going to Illinois tomorrow, that\u2019s a fact. A poll is not a fact. It\u2019s a best measurement. There\u2019s no guarantee that a \u201cbest\u201d measurement is good enough \u2014 whether in terms of The Times\u2019s credibility or whether it improves our understanding of politics. Historically, there have absolutely been elections when it didn\u2019t feel like polling passed that test. The 2016 election is the easiest example; there will be another election like it again.<\/p>\n That said, readers come to The Times to understand American politics, which includes an understanding of public opinion during an election. For better or worse, polling is really the only quantitative measure of the attitudes of the electorate. There just isn\u2019t an alternative. The polls will still drive the political conversation, even if The Times doesn\u2019t conduct any of its own polling. Campaigns run on polls. Our coverage would inevitably be affected by the polls that others are conducting. If that\u2019s going to be the case, it\u2019s probably better that we\u2019re conducting our own.<\/p>\n","protected":false},"excerpt":{"rendered":" When I was Politics editor of The Times in 2018, polls indicated that Democrats were likely to ride a wave of anti-Trump energy to win big in the midterm elections. Democrats went on to capture control of the House of Representatives, but Republicans did better than expected in a couple of red states \u2014 Missouri<\/p>\n","protected":false},"author":1,"featured_media":52522,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[56],"tags":[3580,8797,881,132,17979,4427,6718,951,81,127],"class_list":{"0":"post-52521","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-crime-justice","8":"tag-elections","9":"tag-georgia","10":"tag-harris","11":"tag-live","12":"tag-midterm","13":"tag-obama","14":"tag-ohio","15":"tag-returns","16":"tag-trump","17":"tag-updates"},"_links":{"self":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts\/52521","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=52521"}],"version-history":[{"count":0,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/posts\/52521\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=\/wp\/v2\/media\/52522"}],"wp:attachment":[{"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=52521"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=52521"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/naijaglobalnews.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=52521"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}