About those SurveyUSA polls...

We get a good laugh every time Democrats throw a parade over a SurveyUSA poll that shows their candidates ahead.  It's typical for parties and campaigns to spin poll results to their advantage, but what usually gets brushed under the table is the actual credibility of the polls themselves.

Take SurveyUSA, for example.  The company's polling methodology is frequently criticized by pollsters, academics, and political experts as one of the most questionable - if not unreliable - in the survey industry. 

In fact, consider this comment from Jim Jordan, the former executive director of the Democratic Senatorial Campaign Committee and campaign manager for John Kerry:

"Survey USA numbers are the polling equivalent of a college professor's quote.  That is, for lazy reporters they offer the ring of authenticity while in reality being cheap, uninformed, unreliable and meaningless." ("Dialing up a controversy," Chris Cillizza, Roll Call, 8/1/02)

SurveyUSA's polling methodology is often referred to as IVT (Interactive Voice Technology), a much-debated tactic in the survey industry.  The polls are conducted through automated phone surveys rather than by a live operator, which polling experts say jeopardizes the ability to screen respondents.

Here are a couple of anecdotal examples.  The first is from Brad Coker, president of Mason-Dixon polling:

Coker, who lives in Baltimore, received a call several years back from Survey USA, which was at that time polling for WBAL, a local NBC affiliate. He said his 9-year-old daughter picked up the phone and participated in the survey.

ChuckTodd, editor of the Hotline, a daily political tipsheet, had a similar negative experience while taking a SurveyUSA poll.

"I was able to identify myself as a 19-year-old Republican Latina," said Todd. "Live callers could be deceived by respondents, but not like that."  ("Dialing up a controversy," Chris Cillizza, Roll Call, 8/1/02)

Despite SurveyUSA's repeated efforts to spin the validity of its results, even the left-leaning Salon.com published an analysis of polling firms during the 2004 election and found the company's IVT methodology to be "unreliable."  They cited CNN polling chief Keating Holland:

Holland does not let CNN report results from IVT polls. "I find [IVT] polls unreliable," he said. "I've actually been polled, and it was far too easy to screw around with it, which I did." He added, "People feel a bigger obligation to tell the truth to a real person." (Salon.com)

We're not suggesting SurveyUSA's results are always off or that they can never be trusted, just that their methodology is believed to be less reliable.  That's why most major media outlets don't publicize SurveyUSA polls, and it's why you won't find us dancing about them here - regardless of whether they show our candidates up or down.