Good, bad and ugly: Making sense of the presidential polls

Trump is behind — and not by a little. Even though Trump has trailed Joe Biden in the polls for over 2 years, he had been lurking near or just outside the margin of error. With strong underlying numbers on the economy and a notable lack of popular enthusiasm for Biden, it was more than plausible that Trump could end up winning (not to mention that a plurality of voters consistently thought Trump would win). 

The past two weeks, however, have been terrible for Trump, with numbers sinking well beyond the margin of error. Joe Biden’s strategy of letting Trump lose is being vindicated.

But how accurate is the polling and how should it be considered?

First off, polls don’t predict. Each poll is a snapshot of what the electorate is thinking at the time of the poll. Only when a series of polls agree over a period of time can we make short-term predictions — and only if current conditions hold or change only modestly. The ever-changing conditions in elections are what makes prediction difficult — anyone picking a presidential winner more than a few months out is really making an educated guess and hoping their luck holds.

The RealClearPolitics and FiveThirtyEight averages are a good approximation of where the race stands. The deficiency is that they include some bad polls and polls that are up to two weeks old, which makes them lag current conditions. But looking at the most recent polls can be a problem if the poll is poorly done — and the problem is compounded by the media’s preference for shock over substance. Media often will give time to polls with shocking numbers when the reality is those “shocking” numbers are very often an indication of bad polling.

Consider CNN’s polling. Its Oct. 4 poll — giving Biden a 57 percent to 41 percent lead — was not just a bigger margin than any other poll for months, it was outside the RCP average by well over the margin of error. Bad polling is not new for CNN. Of its last four polls, all have been above the RCP average for Biden and two have been well outside the margin of error. When a result stands far apart from others, it’s called an “outlier,” and it should be viewed with skepticism. CNN recently has specialized in “outlier” polls. What’s more, the CNN “battleground states” poll lumped several states together instead of looking at them separately, rendering the numbers worthless. In this election, in my opinion, CNN has conducted the worst polling of any major news organization.

At the other end of the spectrum, Investors Business Daily has been overly favorable to Trump, with its Oct. 1 poll putting the president within 3 point of Biden. IBD’s problem is that its sampling included almost the same number of Republicans as Democrats. This is an extraordinarily generous turnout assumption, as Democrats far outnumber Republicans currently and have generally led the GOP in party identification for decades. There would have to be an enormous turnout imbalance for the IBD numbers to be true.

And this is the most difficult issue for polling: guessing turnout composition. Who votes obviously matters and — while turnout composition rarely changes dramatically — changes of just a few percentage points can mean a lot. Unfortunately, very few pollsters release full cross-tabulations that identify the number of people in each demographic category. YouGov does, which is why I write about their polls. They provide transparency, so a reader can analyze their method as well as their result. All polls should — the ones that don’t invite suspicion.

Rassmussen — long considered a pro-Trump and GOP polling firm — has consistently shown better numbers for Trump over the past four years. But Rassmussen reports only “likely voters” not just registered voters, which tends to favor Trump. Rassmussen likely has a model that thinks more Republican voters will turn out. However, without the crosstabs, we cannot be certain.

Recent Rassmussen polling has turned sharply against Trump, going from a 48-47 percent Biden advantage on Sept. 22 to a 52-40 percent advantage on Oct. 6. Although Trump has been losing ground, that’s a big move, and my bet is that Rassmussen adjusted its turnout model. If so, credit Rassmussen for changing its methodology to reflect how it sees the race unfolding. But it would help if it would make public its sampling composition. 

Sampling composition is not the only problem for surveys. You should always be very cautious trusting polls with low samples. Any poll with fewer than 800 respondents — whether national or state level — should be dismissed out-of-hand. The sample is simply too small and thus has a large margin of error. Even 800-respondent polls are a problem as it is difficult to analyze component demographics.

The Oct. 1 NBC/Wall Street Journal poll only had a sample size of 800. Of that number, 19 percent were over 65 (152 respondents). That is an exceedingly small sample and subject to large error. MSNBC reported a stunning decline of support for Trump among seniors to 62-35 percent favoring Biden. But seniors are one of Trump’s best demographics, and the most recent YouGov poll puts Trump ahead 49 percent to 43 percent in the 65+ demographic. When you see an outlier like the NBC/WSJ result, it usually means the poll is a mess. But NBC would rather get the momentary shock headline.

A final word on the mysteries of the margin of error. Margin of error is simple: Since a survey is a subset of the larger population, it cannot be absolutely 100 percent accurate. When a poll reports a margin of error of 4 percent, it means that 95 percent of the time the true percentage (if you polled the entire population) could be up to 4 percentage points off.

However, if a survey gives a candidate 50 percent with a 4-point margin of error it does not mean the true result is equally likely to be 46 to 54 percent. Fifty percent is the best estimate with a one-point difference more likely than a 2-point difference, etc. Being off by the full extent of the margin of error is very unlikely. The recent YouGov has Biden ahead 50-41 percent with a 4.5 percent margin of error. At the extreme, the ballot could be tied at 45.5 to 45.5, but that is highly unlikely.

One last note, when Candidate A leads Candidate B within the margin of error, they are not “statistically tied.” Candidate A is statistically ahead, there is just some uncertainty. Knowing this fact alone makes you smarter than almost every pundit on TV.

Keith Naughton, Ph.D. is co-founder of Silent Majority Strategies, a public and regulatory affairs consulting firm. Dr. Naughton is a former Pennsylvania political campaign consultant. Follow him on Twitter @KNaughton711.