Accountability matters. There is too little follow-up in this world.
When pollsters or presidents, reporters or editors make mistakes, they
should be accountable — which in the first instance means acknowledging
the mistakes. (Of course, it’s always easier to hold others
accountable, which is what I’m doing here.)
A few weeks ago I argued that “independent” should not be confused with “accurate,” taking to task the Forum of Fargo-Moorhead for a poll of the North Dakota Senate race, which was “independent” but fatally flawed.
{mosads}With facts now in hand, reassessment is appropriate. At the core of my critique was the need to sample the right people. By restricting the sample to primary voters, the Forum could not accurately gauge sentiment in the general election, which they attempted to do.
Forum editor Matt Von Pinnon defended his newspaper, asserting that this year primary turnout in North Dakota will likely be “similar … to a general election … So is it not reasonable to think their answers might mirror those of the general electorate?” I responded to this question simply. “No. It is not only unreasonable, it’s foolish. Start with the premise — that turnout in the primary will be similar to the general election. Just how likely is this? About as likely as it is for Von Pinnon to praise this column in his.”
Well, I received no paean from Von Pinnon, but the primary returns are now in — 175,206 ballots were cast. In fairness to Von Pinnon, that’s more than in other North Dakota primaries but still far fewer than the 321,000 who voted in the 2008 general election or the 316,000 who cast ballots in the general election of 2004.
Are the 150,000 people who will cast ballots in the general election but didn’t vote in the primary likely to be different from those who participated in the primary? Absolutely. For one, they were interested enough to turn out in June. And generally, the kinds of voters who don’t participate in low-turnout affairs are a bit more likely to vote Democratic when they do cast ballots.
The second shoe dropped on the Forum poll when two other surveys were released with results identical to each other, but different from the Forum’s. One was independent, conducted for KVLY/KFYR TV by Mason-Dixon, the other done for the state party, both of which gave Democrat Heidi Heitkamp the same 1-point lead over Rep. Rick Berg (R). By contrast, the Forum poll was alone in finding a 7-point Berg advantage. I say alone because the Berg campaign itself paid $55,000 for polling in April and you’d bet that if Berg had numbers like the Forum’s, he would have released them. He didn’t.
The Forum poll could have done well in projecting results for the June election. It was very much on target on state Measure 2 and on Berg’s primary. The results for Measure 4 were, however, much more lopsided than the Forum poll would have led us to believe — the poll gave “yes” a 12-point margin, but in the end voters approved it by 35 points.
But nothing they got right or wrong in the primary is related in any way to their attempt to extrapolate from the primary electorate to the much larger November electorate. That was a serious error.
Despite this obvious mistake, many in the press treated the Forum’s Senate poll as meaningful. The normally reliable RealClearPolitics entered the poll into its catalog, noting that it was conducted among “likely voters” but failing to mention they were likely primary voters. Politico emphasized the independent nature of the poll, dismissing the critique as a “gripe.”
Give credit where it’s due. Mark Blumenthal’s HuffPost Pollster drew attention to the controversy, focusing on the poll’s problems.
Let’s hope for better next time.
Mellman is president of The Mellman Group and has worked for Democratic candidates and causes since 1982. Current clients include the majority leader of the Senate and the Democratic whip in the House.