Take debate research with grain of salt

As a pollster, I am obviously a numbers junkie, so Monday night was a cornucopia. Moment-to-moment dial-testing by multiple focus groups, instant polls, prolonged parsing of the numbers. It was all good entertainment, but not very good research. The worst offender, qualitatively and quantitatively, was CNN. They were running a live dial-test focus group composed of about 25 allegedly undecided voters. The legitimacy of this exercise was immediately suspect, given that at least five of the panelists appeared to be black females. Give me a break. If there was ever an outlier sampling, or should we say “outliar,” this was it.

Consider the probabilities. In 2008, 95 percent of black women voted for Barack Obama. In polls this year, black women have continued to be the president’s stoutest defenders. In one Wall Street Journal/NBC News poll taken in August, Romney didn’t attract a single black vote, zero percent of the poll, nada, not even one accidental vote. Ninety-four percent were voting to reelect their president. Every poll I have seen since that breaks out results by race shows black support for Obama in the mid-90s. So we arrive in Florida on Oct. 22 and CNN wants us to believe that blacks are suddenly in play, with black women accounting for about one-fifth of the undecided vote. This is what happens when SMU communications professors become election research experts. I was once a professor, too (though only a Texas Aggie poly-sci professor, not a North Dallas expert). I also thought I knew a lot back then, but didn’t. Only in the real world did I learn that some people will say they are undecided when, alas, they are not, just to get into a focus group.

{mosads}On another channel, Frank Luntz continued his quest for a late-night comedy show host gig, jibbing and jiving with his “undecided” focus-group participants, adopting a interview style closer to Charlie Sheen than Leno or Letterman, so it will be interesting to see where he lands. But the science was similarly suspect. The post-debate comments from his Fox News groups suggested that many of these participants were faking their undecided status. I began to wonder whether some of these people are even registered to vote. Same with the CNN group. Did anyone even check on that? But it was all very entertaining, theatrically speaking. And some of the comments made by a few panelists in each group seemed sincere and insightful.

 The “instant” polling was equally questionable. CBS piously reported that its poll was “a scientifically representative poll of uncommitted voters’ reaction to the presidential debate.” This is a stretch if the statement is meant to imply that we can impute the nationwide reaction of all uncommitted voters from the responses of the CBS online panel. Pre-recruited panels completely violate spontaneity. Once research subjects know they will be involved with such an exercise, they behave differently, studying up for the debates, reading news that otherwise they might never read, checking polls to see what others think, etc. In short, they are decidedly not “au naturel” undecided voters watching the debate. CNN wisely dispensed with the whole undecided nonsense and just polled viewers, whom they told us skewed Republican. And forget that all this was happening after many normal people on the East Coast might have gone to bed, leaving the night owls to have the last word on who won.

 The sad purpose of all this seemed to be to declare a “scientifically determined” winner, so that any remaining genuinely undecided voters might be given a cue on how to vote, courtesy of their waffling, debate-watching compatriots. I hope and trust that few undecided voters will actually take their cues from such dubious inquiries.

David Hill is a pollster that has worked for Republican candidates and causes since 1984.