By Ira Basen - Four polls, all released on April 20, all conducted by reputable Canadian polling firms. And yet the results, in some cases, are so out of sync it almost appears that the pollsters were interviewing voters in different countries.

How can there be an almost nine-point difference in the Conservative vote between Ipsos Reid and Ekos?

Or more than four points for the Greens between Ekos and Nanos, and more than five for the Liberals between Ipsos and Nanos?

If the pollsters are so far apart, how can we rely on their interpretation of what is happening "out there" in the Canadian electorate?

The question is important because we have reached the stage of the campaign where polls have become the story. The platforms have been released, the promises rolled out, the debates are a fading memory.

This campaign needed a new story line to carry it to election day, and it has found that in the so-called NDP surge.

Campaign coverage is now focused almost exclusively on the horse race and the strategic decisions each party is making to come to grips with the new reality that the pollsters assure us we are now confronting.

But what if they're wrong?

Dirty little secret

It's a question pollsters themselves have been asking lately.

In recent months, several prominent Canadian pollsters have been raising some pretty fundamental questions about their industry.

The most provocative critic has been Allan Gregg, chairman of Harris-Decima, which provides political polling for the Canadian Press. He is also a regular member of The National's At Issue panel on CBC TV.

Gregg has been doing political polling since the 1970s and in an interview with the Canadian Press he said that "there's broad consensus among pollsters that proliferating political polls suffer from a combination of methodological problems, commercial pressures and an unhealthy relationship with the media.

"The dirty little secret of the polling business," he went on, "is that our ability to yield results accurately from samples that reflect the total population has probably never been worse in the 30 to 35 years that the discipline has been active in Canada."

Methodological problems

Amongst the methodological problems that Gregg and others identify is the incredible shrinking response rate for polls conducted by telephone.

Thirty years ago, about 70-80 per cent of people called by pollsters agreed to be surveyed. Today, that rate is under 15 per cent and Gregg believes those people tend to be older, less well-educated and more rural than the general population.

But for the purposes of their polling, researchers are obliged to assume that the 15 per cent of callers who agree to spend 20 minutes talking to them are representative of the 85 per cent who are too busy or whatever to participate or who never pick up at all because they can identify a pollster through Caller ID.

The growing number of households without landlines also poses significant challenges.

There are now more cellphones than wired phones in Canada (25 million vs. 17.5 million), and those cellphone numbers are harder for pollsters to get. That leaves a large number of people, many of them younger, whose views may never be surveyed.

In both these cases, researchers have developed "models" that they hope can compensate for these and other instances where polls are conducted on an unrepresentative sample. But the accuracy of these models remains in question.

The basic methodological assumption of the polling industry has always revolved around random probability sampling, meaning that everyone has an equal chance of being interviewed. That is now clearly no longer the case.

Commercial pressures

Firms like Nanos, Decima, Ipsos-Reid and Ekos have become household names in Canada because of their high-profile political polling.

But political polling is a loss leader for these companies. They offer their services to media outlets at a deeply discounted rate, or sometimes even for free, because the profile they develop at election time helps them in their core business, which is traditional market research.

They make their money asking people what margarine they spread on their toast, not who they are likely to vote for.

In fact, that voter preference question is often buried inside a longer survey about some completely unrelated subjects.

Over the years, researchers have discovered that where the political question is placed in the survey, and what else is being asked of the respondent, can affect how a person answers the question.

But these placement concerns are rarely factored in to the results.

Many industry veterans now think that too much poorly executed and poorly resourced polling is causing significant harm to the industry.

"I believe the quality overall has been driven to unacceptably low levels by the fact that there's this competitive auction to the bottom, with most of this stuff being paid for by insufficient or no resources by the media," Frank Graves of Ekos told the Canadian Press in February.

"You know what? You get what you pay for."

Unhealthy relationship with the media

Still, no one expects the media's love affair with opinion polls to end any time soon. Polls are the best news that money can buy.

They keep the campaign story moving along, even when everything else has been thoroughly talked out.

In this campaign, there were 19 days between the English-language leaders' debate and election day. In 2008, there were only 12.

Also, in 2008, the last platform was dropped just six days before voters went to the polls. This year, it was more than three weeks before. These kinds of gaps require new story lines.

But the problem with using polls here is that, too often, the reporting of them is based on creating drama where none exists.

In the process, non-trivial issues like margin of error, problems with samples and methodologies tend to get pushed aside.

Is that what's happening with the story of the NDP surge? That will be the subject of the next post.