sda2.jpg

June 1, 2009

What Would We Do Without Pollsters?

This is how Strategic Counsel polls "most Canadians" for CTV...

Results are based on a national sample size of 1,000 voting-age Canadians with an over-sample of 500 respondents from Quebec.

That Peter Donolo, he's quite the joker.

h/t Orville H.

Posted by Kate at June 1, 2009 8:35 PM
Comments

Nice....and very predictable.

Hmmm, I wonder how many other polls are sneakily done like this?

Posted by: Soccermom at June 1, 2009 9:19 PM

Move over Decima, you have competition.

Posted by: Speedy at June 1, 2009 9:20 PM

"That said, it's important to note that the gap between the Conservatives and Liberals has narrowed quite a bit."


There's the money shot. Predetermined meme; the poll was just "fill in the numbers".

Pure fantasy.

Posted by: A. Cooper at June 1, 2009 9:22 PM

*sigh*

Do you know why oversampling is done? To eliminate possible bias. Do you know how the overall figures are calculated? By pro-rating the oversampling back (weighting it down) when the percentages from each province/region are obtained and combined.

Posted by: MsMew at June 1, 2009 9:24 PM

I think I'll undersample CTV from now on.

Posted by: David at June 1, 2009 9:28 PM

Hmm... I wonder if polling firms regularly over-sample certain geographic regions so as to get results they want?

If one will do it...

Fancy math... like the IPCC computer program, perhaps???

Arrogant pollsters... they're a dime a dozen.

Posted by: Canadian Sentinel at June 1, 2009 9:31 PM

MsMew, do you actually know anything about statistics? If I pulled a stunt like that in a medical study the results would be useless. Because they don't represent reality, they represent the choices made by the study designer. Aka bias.

If half the sample is from one discrete and -small- regional population, and the other half is taken from five or six other discrete regional populations, you can't generalize the study to the whole country. Its not science, its propaganda.

Yet here's CTV treating this foolishness as if it were serious business. Hence CTV's ever shrinking viewer ship.

*sigh*

Posted by: The Phantom at June 1, 2009 9:37 PM

Oversampling - in this case, stratified sampling, from Quebec assumes that the population there is homogeneous and also will come up with different responses than the entire rest of Canada.

But, you could get different responses from the West, the North, as well.

And Quebec has only 20% of the population, so, how accurate is such a poll (1,000/500) as a measure of the whole population?

Posted by: ET at June 1, 2009 9:40 PM

"Do you know why oversampling is done? To eliminate possible bias"

Ya, that's why it looks like they decided not to poll anywhere west of Mississauga. That's why 50% of those polled were from a province that supports a seperatist party, and where the governing party is polling the lowest.

Good God, do you even think before you write stuff like that?

Posted by: paulsstuff at June 1, 2009 9:43 PM

Good Lord. This is too easy.

Oversampling is done to eliminate bias, not to create it. It allows for greater stratification in the sample, which is the very reverse of what ET claims it is--one stratifies a sample precisely because one recognizes that the population is NOT homogeneous. For a number of different reasons, too small a sample might be skewed. I'm speculating, but Quebec has an anglophone population, a significant Native one, and a francophone one. Suppose a sondage of 20% produced a skew, for whatever reason, so that the sub-sample from the anglophone population was disproportionately small. That's why one oversamples--to make that random skew less likely.

The point is that, after all the results are in from across the country, the Quebec sample is weighted back down to the proper 20% of the national sample. Some here, including the poster, seems to imagine that the oversampling from Quebec is simply lumped into the national results without weighting. That would create serious bias, of course, but that's not what oversampling is.

Posted by: MsMew at June 1, 2009 9:50 PM

What more do we need to know?

Peter Donolo is a political strategist and communications advisor associated with the Liberal Party of Canada. From 1993 to 1999, he was the Director of Communications in the office of Prime Minister Jean Chrétien.

No bias here, folks. Just move along ...

Posted by: batb at June 1, 2009 9:55 PM

It allows for greater stratification in the sample

Too hastily written, sorry. It does that, but more pertinent here, it allows for more accurate results from stratification in this case.

Posted by: MsMew at June 1, 2009 9:55 PM

MsMew, I'm afraid that sounds like, well, gobbleddygook.

Much like the IPCC stuff.

Sorry, not buying.

Posted by: Canadian Sentinel at June 1, 2009 10:03 PM

Yet Saskatchewan and Manitoba, which also have significant Native populations, weren't declared "heterogenous".

I can also assume that the Cree, Innu and Inuit populations of Labrador weren't declared "heterogenous" from Newfoundland either.

Ethnic/racial politics, gotta love it.

Posted by: jwkozak91 at June 1, 2009 10:11 PM

* fart * oops , so why bother with the rest of the country at all . . Just phone 500 more in Quebec and voila !

Posted by: cantuc at June 1, 2009 10:12 PM

Nope, not "gobbleddygook." Survey methodology 101. Time for some of you folks to do a little homework before you make greater fools of yourselves.

Posted by: MsMew at June 1, 2009 10:13 PM

"it allows for more accurate results from stratification in this case."

Not if you over-sample the smallest, least homogenous part of the total population Ma'am.

Quebec opinion is in the tail of the curve on most issues. Furthermore I'll bet you a donut that Montreal is the source of most of the Quebec sample, skewing it even farther. Oversampling what you know is the two sigma tail does not give information about the center of the curve. You can't get anything useful out of a sampling like that just by changing the weighting. You'd have to know what the outcome was supposed to be first.

Oh, wait...

BTW, while I may be an over trained, glorified nurse, ET is the gold plated Real Thing(TM). I'd suggest you go crack a textbook if you want to disagree with her on stats and study design.

Posted by: The Phantom at June 1, 2009 10:19 PM

MsMew, why Quebec vs. any other province? Apparently the GTA alone has a population of about 5.5 million, vs. a Quebec provincial population of under 8 million.
And how do you decide which categories are significant enough to start fiddling with? There's a francophone category and an anglophone category, okay. Is there an age category? Income? Is churchgoing significant? Birth order?
I'm not trying to be sarcastic; I just think that there's inherent bias in these attempts to eliminate bias.

Posted by: Black Mamba at June 1, 2009 10:23 PM

This looks to be much ado about nothing. Oversampling Quebec on this issue seems to be an appropriate and reasonable response.

If 20% of the survey population is from Quebec, that sample is probably too small (200 people) to make any valid claims, however adding the extra 500 to give a sample size of 700, probably does.

The results of the 500 people from the oversample are not used in any of the original survey results.

The bias in this, or any other survey would most likely not come from data manipulation, but from the way the questions are phrased and the answer interpreted.

Posted by: SDH at June 1, 2009 10:27 PM

The full Strategic Counsel report is here, there is more than was quoted by CTV.
http://www.thestrategiccounsel.com/our_news/polls/2009-05-28%20-%20GMCTV%20FINAL.pdf

Posted by: Rich at June 1, 2009 10:31 PM

SDH: Then the same principle applies nationally. The population of Prince Edward Island is 140,000 vs. a national population of about 34,000,000.

Posted by: Black Mamba at June 1, 2009 10:41 PM

The population of PEI is also a darn sight more homogeneous than the population of Quebec.

Posted by: MsMew at June 1, 2009 10:46 PM

I get a kick out of the way polls are presented, ie +/- 3% 19 times out of 20.
19 times out of 20 =5% so why don't they say +/-8%

Posted by: Tony W at June 1, 2009 11:06 PM

ms Mew - the population of Ontario is the most non-homogeneous of the entire country. Check out Canada Stats to see the ratio of immigrants in Ontario. And any other province in the West has more immigrants than Quebec. What about New Brunswick - with its francophone, anglophone and indigeneous?

So, I don't buy your explanation that oversampling was required to rule out Quebec bias. The statistical facts of Canada is the heterogeneity is higher outside of Quebec - and - furthermore, you can't assume that a Quebecois anglophone will vote on these questions differently than a Quebecois francophone.

SDH - I didn't say that 20% of the survey sample population was from Quebec but that 20% of the Canadian population is Quebecois. The sample population of the survey need not match the provincial ratio. After all, if you selected your respondents by provincial ratio, then, your sample from such provinces as PEI, Nunavut, NB, etc..would be equally small. And who says that the sample must reflect provincial opinions?

My question is why the researchers assumed that Quebecers would have biased responses and therefore, they took a larger sample from that area (which is why this is also a stratified sample), to rule out bias. Why wouldn't they assume that, eg, 'red-neck Alberta' would provide biased sample responses?

Posted by: ET at June 1, 2009 11:10 PM

Well, yes. I was responding to SDH, taking his/her point to what I took to be it's logical conclusion, i.e. that 1,000 people out of a population of 34,000,000 isn't really a big enough sample to be reliable.
My original point was that there is an inherent bias in adjusting statistics based on assumptions about which population subdivisions are significant.

People who like this sort of thing should google:
Iowahawk "sampling error"
Nobody's ever regretted googling Iowahawk.

Posted by: Black Mamba at June 1, 2009 11:10 PM

Homogeneous? Who cares if one province has more people that lie about their sexuality than another? Batb's comment about Donolo's background sums up this poll very well,liberal BS,trotted out as facts. Just like that Drummond fellow always giving his opinions on the economy,yet it is never mentioned that before he worked for TD, he worked for the libs for about 25 years.

Posted by: wallyj at June 1, 2009 11:13 PM

Oh for crying out loud. This is getting outrageous.

See, Tony, there are two different measures in that formulation, not one. One is called a confidence interval: that's 95% (19 times out of twenty). That tells you that the results of the sampling will statistically speaking be found to fall within the range(+/- 3%) 95% of the time.

I ought to charge for this.

Posted by: MsMew at June 1, 2009 11:15 PM

In the sciences we just called it 'cooking it in'.

Posted by: Speedy at June 1, 2009 11:18 PM

I actually think I understand this oversample stuff. If you poll everyone in Quebec,you will get a true response. However if you only poll ten people,you may get a skewed sample. Therefore you poll more and then bring it back to reflect the size of the original polling.Eureka,quebec is a skewed province,and the ROC is just screwed.

Posted by: wallyj at June 1, 2009 11:21 PM

There is something fishy about this oversamnple stuff. Why is Quebec opinion so critical with this little pissant poll yet regular polls by this outfit do not oversample Quebec? What is so critical about this poll? Why do we need to know who is the prefered dinner guest out of PMSH and MI and why is it so critical that we get a 'true' feeling of how Quebec feels on this? Its a push poll designed to counteract the MI ads pure and simple.....what a piece of shit these outfits are?

Posted by: Jooge at June 1, 2009 11:49 PM

MsMew is correct in her analysis of the process


and I go with ET in questioning the need to over sample queerbeck, because as she, ET, states , Terana is a large and with greater diversity population, and it was not over sampled


as someone mentioned, the questions asked in the poll, the time of day it was taken, and who did the polling (what persons/s) are all critical in polling

Posted by: GYM at June 2, 2009 12:20 AM

I'd trust a poll that oversampled homosexuals from Qwebec, with an average income over $100k. At least then the underemployed druggies would not be a significant factor!!!

Posted by: eastern paul at June 2, 2009 12:48 AM

for god's sake why does anyone ever concern themselves with what Quebecers allegedly think?

you know they're going to 'think'(vote) whatever complements and compliments their position a la exchequer vis a vis the 'current federal position'...always have always will...

the Quebecers are NOT serious members of MY Canada...i disabused myself of that notion not long after i started wearing long pants and i first met that bald guy ...the one married the nutbar out west here...gee...he was SO intellectual and grooving with Fidel and all....it was cool..cool times period.

Posted by: john begley at June 2, 2009 12:50 AM

I suspect the Leftist Big Media and the Leftist pollsters are colluding to IPCC-ize data to get results they want to propagandize.

They want people to think the Iggy Ads are "backfiring" and so on and so forth.

Of course pollsters can be biased and corrupt, though I'm not going to point any fingers of specificity. Just food for thought. Hey, if scientists can be corrupt...

Posted by: Canadian Sentinel at June 2, 2009 6:33 AM

And, then, this brilliant salvo in the TorStar yesterday under the headline "Tory ads attacking Ignatieff backfiring on Harper, poll finds":

"The Conservatives' biting partisan attack ads aimed at Michael Ignatieff are taking a toll -- on Prime Minister Stephen Harper as well as Liberal leader, according to a new poll."

How's that for talking out of both sides of one's mouth? And how's that for a "partisan" headline? (The poll was conducted by TorStar and Angus Reid whose VP is Mario Canseco. 'Anyone know his background?)

I long ago stopped paying attention to polls except to note that the MSM tends to promote polls whose results show the CPC and PMSH in a bad light -- and which are often proven wrong come election time.

Yawn.

Posted by: batb at June 2, 2009 7:47 AM

Comments at CTV quickly closed...right after an honest non-partisan opinion with first hand experience is posted...
and the title using 'most Canadians' was very misleading.
'nuff said...media bias is rampant in this article.
I'd love to have Prime Minister Harper and family over for coffee or a meal anytime.

Posted by: bluetech at June 2, 2009 8:27 AM

Talk about SKEWED and screwed up polling!
According to Donolo Quebec is one province but warrants 50% of respondents to his polls. We get the game now, it gets the desired results in favour of the Liberals. It's a "national" poll? Yeah, right. How about a "half national" poll?

What a freaking conniving bunch of desperate people.

As to the Conservative truth ads, they're working. We know by the reaction from the Liberals and their toadies.

Statements calling them "biting, partisan attack ads" is really rich when they are the facts about Iggy, some right from his own mouth. It also tells us the Liberals know those facts are not helpful and want to hide the truth from the voters.

If the ads are hurting or biting Iggy who's to blame? We didn't make up the facts, they can't be denied or glossed over.

Posted by: Liz J at June 2, 2009 8:28 AM

Wow. This thread makes me want to bang my head against a wall.

MsMew's overview of Statistics 101 is sound. While I haven't read the actual survey methodology for this poll, over-sampling is common in national (or other large population) research projects and, as others have mentioned, the final results are then weighted back down to the correct representation.

The reason for doing so isn't to bias the overall results, but rather to provide more reliable and representative results for the sub-set.

Let's say that I have a population with 990 boys and 10 girls and I am conducting a survey of 100. Such a sample would normally only include one girl, which would be pretty much useless from an analytical stand-point. I'd be able to talk about the results as a whole, but I would not be able to gain any insight into the girls, as the margin of error would be far too high. To compensate for this, I would over-sample the girls.

To keep things simple, lets say that I'll speak to 25 girls and still speak to 99 boys. When I calculate the final numbers for the total population, I will first adjust (weight) the girls results back down to their proportion of the population, i.e. divide their results 1/25.

The end result is that I can get better insight into the girls' opinions, but they still only represent 1% of the overall results.

The "Why oversample Quebec?" in a second.

Posted by: Jim at June 2, 2009 8:50 AM

Do all the weighting people note they rarely if ever give the questions asked and how they are asked? I used to answer polls to see which way the bias was going. When the pollsters said who they were it was so fast I never caught who it was. Nanos is noted as being the best pollster. Why? It's the questions he asks and he is the only pollster that calls cellphones. The rest talk to your grandma or your kids.

Posted by: Speedy at June 2, 2009 8:58 AM

Who gives a rats-ass what Quebec thinks anyway.
They have had a love-in with all things socialist for decades...just like the MSM.

Posted by: The Glengarrian at June 2, 2009 9:08 AM

Several of you are bothered/confused/suspicious as to why Quebec is being oversampled.

There are a bunch of reasons for this, some that are actually research-related, others that are not.

The first is pure marketing. Quebec has a unique media market that is interested in Quebec. The Quebec media wants to report on what Quebecers think, whereas the (typically Torontocentric) English media are generally happy to report on broader Canadian results. Because of this, there is a greater need / value in having more statistically reliable results for Quebec than Manitoba.

Similarly, the media still enjoy / cling to the old "two solitudes" political narrative. It may be lazy and (perhaps) outdated, but the national media are interested in comparing what Quebecers think compared to the ROC. Again, in order to do so, you need to oversample Quebec.

On the research side, there are several solid reasons to oversample the Quebec population. As you all know, Quebec has a disproportionate (ad significant) influence on national politics both in the number of HOC seats and the focus of national policies. You can hate that all you want but, as long as the situation exists, smart research says that you dig deeper into such a group.

The Quebec populace is also far more politically volatile. Toronto may have a more ethnically diverse population, but when was the last time that the city didn't send an army of Liberals to parliament? 1984? The swings in both popular vote and seat count can be fairly wild in Quebec, hence the need to dig deeper into (and have greater confidence in) the numbers.

Can pollsters be biased? Of course. Can their biases influence the results of their studies? Yes, but this would more commonly occur in the nature of the questions asked or by sampling non-representative populations (and then trying to suggest that their opinions have broader application). Most commonly, the bias occurs with the interpretation and/or presentation of the results.

Hope this helps.

Posted by: Jim at June 2, 2009 9:17 AM

a sample would include quebec at about 20%. the oversample is quebec at 100% giving a quebec bias to the poll of 46% , only appropriate when you are trying with all your lieberal heart to promote iggy.

Posted by: cal2 at June 2, 2009 9:17 AM

Just to wrap-up, Speedy hits on a very important point. Very few pollsters have properly adapted to call screening and the growing percentage of people who do not have landlines.

The concern is, of course, that the people who actually answer the phones and take the poll are not representative of the average person (regardless of province).

I'm not sure that Nanos has it exactly right, but it at least represents an attempt to address this growing problem.

Bias in actual questions (which I mentioned in my second post) is a clear concern, but has nothing to do with weighting or over-sampling.

Posted by: Jim at June 2, 2009 9:24 AM

following MsMew's line of thinking - my question is how come they didn't do 500 respondents from Alberta. After all - Alberta is 100% Conservative (well except for the "strategic voting" blip in Edmonton Strathcona).

Wouldn't doing an over sample in Alberta acheive the same thing?

Just askin.

As I learned in Statistics - you can make "statistics" say whatever you want them to say.

I would suggest that this poll is a prime example of that.

Posted by: Alberta Girl at June 2, 2009 9:32 AM

Jim - I acknowledge your analysis of 'why to oversample' but I don't agree with your analysis of why to oversample Quebec.

Such an act gives an overweight to the Quebec response that I feel it doesn't merit. You ask when Toronto didn't return a Liberal Set to parliament - and, with the introduction of multiculturalism, Toronto The City of Immigrants moved into the grip of the political party, the Liberals, who based their voting power on creating isolate sets of dependent immigrant groups.

Equally, when did Quebec ever return a non-socialist Set? And, with the disastrous enabling of its own political party - when did it move away from the Bloc - its very own socialist party? Quebec isn't volatile; it's totally predictable. Indeed, in all of Canada, Quebec is the most predictable; it will vote for whichever party will give it the most federal money with the least responsibility.

Its 75 seats in the House, heavily held by the Bloc, mean that the government will almost always function as a minority. But, with the new seats in the West, due to their increased population, Quebec's generation of dominance in Canada is fading. At the moment, Quebec has 11 more seats than the West, despite their populations being equal. The change in seat counts will deal with that.

Therefore, my question remains - why oversample Quebec?

Posted by: ET at June 2, 2009 9:38 AM

Alberta Girl - You've completely misunderstood MsMew's explanation and apparently only attended the first day of your stats class.

AGAIN, over-sampling does NOT mean that the sample population becomes over-represented in the national figures. The results are weighted to restore the correct representation in the overall results. The oversampling is simply to increase the reliability of the subset.

Posted by: Jim at June 2, 2009 9:45 AM


Half from Quebec !?
Aren't they the same freaks that think Justin Trudeau represents their interests?

Posted by: richfisher at June 2, 2009 9:45 AM

What would the poll results have been if 500 of the respondents were from the prairies and the other 500 from the rest of Canada?
It is incredible that main street media has the gall and lack of integrity to put the results of such a misleading poll on the national news.
Doesn't the President of CTV take some responsibility for what his network is delivering to Canadians under their license - bring on more cable,satellite and the World Wide Web because the media have proven they can't be trusted and need a watchdog - and I haven't even commented on the CBC.

Posted by: Peter at June 2, 2009 9:57 AM

ET - I'm pinched for time, and I pretty much agree with your feelings if not all of your facts.

Quebec seat totals have varied significantly over the past 25 years. Huge majorities for the Tories in 84 & 88, a Bloc sweep in 93, Liberal rebounds in the subsequent two elections, a Bloc comeback post-adscam, surprise Tory inroads in the first Harper victory, and another Bloc rebound this past year.

I'm not saying that it is right or fair - just a valid / interesting group for researchers to study.

Posted by: Jim at June 2, 2009 10:01 AM

Pollsters are part of the opinion creation industry. I believe this game was described as manufactured consent by Bernays. They give us the opinions we should have to compliment the political agenda of the day.

In any real sense opinion polling can be related to fishing for the right answers.

Posted by: voltaire's bastard at June 2, 2009 10:09 AM

Peter - In theory, the results would be similar, but with a larger margin of error as the sample sizes for Ontario and Quebec would have been a bit small.

Posted by: Jim at June 2, 2009 10:10 AM

Jim
The " margin of error is greater" says it all for me as far as accuracy of polls that are over weighted.

Posted by: Peter at June 2, 2009 10:35 AM

Jim - the Mulroney Progressive Conservatives were a completely different political party than the Harper Conservatives. Mulroney's PC were, as you know, Red Tories, or Liberals.

Essentially, ignoring the superficial party names, Quebec has always returned only one type of federal party. Socialist and one that in particular, has its key figures from Quebec (Mulroney, Chretien, Martin). The Bloc didn't exist in the 80's.

This means that they vote for a party that gives them socialism, that gives them the highest proportion of the federal authorities in Ottawa and the highest proportion of federal money with the least interaction with the ROC.

They are totally predictable.

To others in this thread - some of you misunderstand. The survey wasn't of 1,000 respondents with half from Quebec. It was 1,000 over all of Canada, plus a second survey, that 'oversample' of 500 more respondents confined to Quebec.

This was done because the researchers assumed that the Quebec response would be unique and they wanted to 'test' it (with that extra 500). Then, they 'reduced' that 500, with its unique interpretation, to the proportion of Quebecers in the 1,000 test sample.

My question has been focused only on why the researchers assume that Quebec alone will have a unique response to the survey.

As for bias, that's basic to surveys. The first bias comes from getting your sample population. Getting a valid representation isn't easy. And the next comes from your questions which can be quite heavily biased. And of course, the presentation of the responses can be biased.

Posted by: ET at June 2, 2009 10:39 AM

Peter - I'm not sure that you're following the math.

The problem wouldn't be with the overweighting of the Prairies, but with the sample size of Eastern Canada being too small.

I just reviewed the actual sample distribution for the poll (http://www.thestrategiccounsel.com/our_news/polls/2009-05-28%20-%20GMCTV%20FINAL.pdf):

Canada 1000
Quebec 743
Rest of Canada 757
Ontario 383
West 300

Just to be clear, they did not sample 500 people from Quebec and 500 people from the rest of Canada. A nationally representative sample was drawn proportionate to regional populations. An additional 500 surveys were conducted in Quebec. The Quebec results were then weighted back down to represent 24.3% of the national figures.

Peter - to answer your initial question, if the same thing had occurred for the West, rather than Quebec, the results would theoretically be the same. My earlier comment on the greater margin of error was not correct, as the sampling did ensure proper base samples for each region. I mistakenly assumed that the ROC base was 500, not 757.

Again, all over-sampling does is increase the reliability of the data subset.

Posted by: Jim at June 2, 2009 10:54 AM

ET - Your macro-analysis of Quebec politics is pretty solid, but you're forgetting that modern Canadian politics is all about "brands" and "teams", not real philosophical differences or competing ideologies (see Harper, Stephen).

At the end of the day, Quebec voters switch brands more often than those in the rest of the country, and pollsters want to be able to follow/predict/understand those changes.

Posted by: Jim at June 2, 2009 10:59 AM

ET - As I mentioned earlier, I think part of the focus on Quebec is pure laziness. Its an easy narrative and an easy market to get attention in (and thus get more corporate business, which is the real reason for doing these fairly useless polls).

Personally, I'd be way more interested in better BC polling data (an equally volatile political population) or in the seemingly more conservative (but ethnically diverse) areas around Toronto such as Mississauga and Brampton. Unfortunately, the media have little to no interest in reporting such polls, and want to cling to that old "Quebec / ROC" narrative.

Posted by: Jim at June 2, 2009 11:06 AM

Jim
Are we talking about the same Quebec? The Quebec I know has been returning Bloc MP's in a majority number now for many years in federal elections. Your comment that Quebec voters switch brands is not based on fact - I expect the rational for the polling methods are not based on facts either.

Posted by: Peter at June 2, 2009 11:46 AM

Peter - You're out to lunch.

Here are the popular vote swings for the past eight elections (2008 - 1984)

Bloc - 38% / 42% / 49% / 40% / 38% / 49% / na / na
Liberal - 24% / 21% / 34% / 44% / 37% / 33% / 30% / 36%
Tory* - 22% / 25% / 9% / 12% / 22% / 14% / 53% / 50%

*I've combined Alliance/Reform/Tory votes for 00/97/93 elections.

Posted by: Jim at June 2, 2009 12:22 PM

MsMew

Confidence intervals have nothing at all to say about sample bias and it certainly has nothing to say on the matter of leading questions. It isn't what it measures. You can have a high level of confidence at either 95% or 98% confidence interval and still have an invalid stat. You can replicate the same invalid survey and achieve the same incorrect result with a high level of confidence over an over again. All you have to do is repeat the same errors (deliberate ones or otherwise) every time.

What you are implying is that they Oversample then reduce the weighting back to represent a statistically representative sample size to avoid sample bias. There is no information to suggest that is what they are doing - you are making an assumption. I personally don't trust liberals not to make their manipulations very deliberately but that isn't a statistical argument.

Further, if they reduce the weighting of these 500 people in a total sample of 1000 you reduce your total confidence interval. Further still, if you only have 500 people sampled in the RoC your confidence interval for this data set would be reduced dramatically.

So, Houston, we have a problem.

Posted by: Jason at June 2, 2009 2:37 PM

Regardless of the differences in interpretation of the polls, one conclusion clearly emerges ... There is a window of opportunity for Ignatieff and the Liberals to win a snap June election. Canadians no longer want a Harper government guiding the nation through the global recession.

If Ignatieff hesitates and doesn't grab the opportunity, he will be abandoning Canadians to the Harper mismanagement of the nation ... and that would be unconscionable and unpatriotic too.

The remaining days in June for Ignatieff to push a no confidence motion and a snap election will be a test of his leadership and patriotism.

Iggy must pull the plug in June before the HoCs recesses for the summer. If he doesn't act decisively and defeat the Harper government, he will be facing a new batch of CPC attack ads on TV and radio ... to which he will not be able to counter and thus be defined as incompetent, unsuitable and NOT worthy to be Canada's prime minister.

Posted by: Observant at June 2, 2009 2:46 PM

Observant

lol. Dreams are cheap my friend. They are also worthless.

Posted by: Jason at June 2, 2009 2:51 PM

I was simply explaining the concept of confidence interval to someone earlier in the thread. But oversampling is a different matter: it reduces sampling error. The only assumption I am making here is that the oversampling itself isn't deliberately skewed to obtain some desired result or other.

The confidence interval for the weighted sample as a whole is 95%.

Posted by: MsMew at June 2, 2009 2:53 PM

Jason - Read the survey methodology:

http://www.thestrategiccounsel.com/our_news/polls/2009-05-28%20-%20GMCTV%20FINAL.pdf

You are wrong, and MsMew is correct.

Not everything is a Liberal plot, folks.

Posted by: Jim at June 2, 2009 2:55 PM

Playing just a little bit with the numbers from Jim's post above, Kate, the main poll breaks by region into

Ontario - 383
West - 300
Quebec - 243
Atlantic provinces - 74
I'm assuming here that Northern Canada is included in the PQ, Ont and West counts.
The regional result from 74 respondents is going to be real accurate - less than 10% of the total. Better make sure, though, that the result from Quebec is accurate.

Heh, heh.

This was also an "online" poll. Can someone tell us how you get truly random data sets when the respondents are online?

If we actually all could invite Ignatieff or Harper or both to dinner and have a real chance that they would accept, seeing that 56% of Canadians wouldn't invite either really saddens me. What a nation of political morons we are.

Posted by: BJG at June 2, 2009 3:23 PM

ok ! Ok !! OK !!

We can settle this once and for all !

Someone, anyone, just look up the list of those citizens "chosen" to be polled and their unadulterated, unfudged, responses. The raw data.

Which one of you has access to it ? Anyone ? Has anyone ever seen it ? Heard of it being available ?

If not, then they are probably hiding something. IOW, the bias, the skew, the slant. Push-polls.

I have asked Chantal Hebert this question many times. Crickets.......

Posted by: ron in kelowna at June 2, 2009 3:30 PM

MsMew

The confidence interval of 95% isn't the measurment. It's the level of confidence at which the measurement is made.

You can measure your confidence interval at the 95% level (+/- Two standard deviations) or the 98% confidence level (+/- One standard deviation.)

What is important is the Standard Deviation from the mean. The larger the SD the less accurate the poll.

Jim

OK, the total sample was 1500 people. They oversampled QC and reduced the weighting. That means they took the effort to get really good data from the place where the libs look better and less accurate data from everywhere else.

It still doesn't invalidate any of the other points listed above.

Posted by: Jason at June 2, 2009 3:31 PM

The integrity of the method means diddly squat if the data is pulled out of thin air or the respondents are "chosen" by someone with an agenda.

Is it possible that media orgs have an agenda ? Could they be politically motivated ?? Ya think !?

The first run of the Printing Press in 1462 had not even cooled when Political Organizations of the day started up their own media orgs. Nothing but nothing is more politically bent than media.

Could media polls of today be a tool ? Follow the money.

Posted by: ron in kelowna at June 2, 2009 3:42 PM

Jason - Your conclusion is pretty accurate.

BJG - I assume (but do not have any facts to back it up), that SRC has created an online database that is supposedly representative of the Canadian population, and then drew their sample from that.

I have serious reservations about this sort of polling, as it would seemingly rely on self-selection into the process (to get into the database in the first place). It is still preferable, though, to the typical Internet straw polls that get passed off as research.

Posted by: JIm at June 2, 2009 4:28 PM

Jim,

To be representative, the sample has to be random. So, a master list is out of the question. Random samples are easier said than done at the best of times.

Some issues include no representation from those with unlisted phone numbers and cell phones (or no phone at all) along with those who refuse to talk to you, etc.

Polling when done right is still somewhat akin to voodoo. It's easier to get the result you are looking for than it is to get a representative result. In other words, lies are easy - it's the truth that's hard. That goes double when you add in political considerations.

Posted by: Jason at June 2, 2009 4:42 PM

oh, and internet polls are not meant to be valid. They're meant to drive traffic and make money.

Posted by: Jason at June 2, 2009 4:48 PM

Let me say this, Quebec is not only oversampled in polling, it's over served.

Anyone who thinks over sampling one province gets a legitimate result to present as voter intention or support has to be playing politics of the most desperate order. It's bogus.

Posted by: Liz J at June 2, 2009 5:07 PM

Jason - I share all of your concerns with Internet polling (hence my mention of serious reservations), and noted the same concerns that you've highlighted with telephone polling earlier in the thread. I was just trying to differentiate between the likely approach that the SRC took versus the straw polls that you might see attached to a news article or blog.

There are many, many valid criticisms of the opinion polling industry, and my responses here have not been an attempt to defend the industry at large, but rather to correct some extremely inaccurate assumptions/understandings of basic statistics and research methodology as it relates to sampling.

Not everything is a Liberal conspiracy and some people (not yourself) would be better served by trying to understand what they are ranting about before posting!

Posted by: Jason at June 2, 2009 5:07 PM

And I am not saying polls cannot be quite accurate. They can. Sometimes uncannily so. Predicting elections, ect.

But a six week election campaign gives the pollster ample time to supersede the push-pull polls with ones more close to reality.

No ? Have you ever seen raw poll data ? Is human nature such that integrity rules even when not under the threat of being exposed ?

Posted by: ron in kelowna at June 2, 2009 5:11 PM

Jim
Thanks for posting the numbers so readers can see how you are prepared to skew things. You included in your comparison the Mulroney years when the Bloc didn't exist - neat comparison if to thought you could get away with that nonsense.
Facts are the number of MP's the Bloc got out of the 75 seats available from Quebec was a pretty constant majority of 54, 44, 54, 51 and 49 seats ( 1993- 2008).
Jim you wrote the following drivel "The Quebec populace is also far more politically volatile." and used that argument to justify the polling nonsense.
Don't treat the people on this board like mushrooms - they are a little more sophisticated than yo give them credit for - maybe you should carry through with your threat and go "bang your head against the wall" - from your rational on this board you have overdone that to this point in time.

Posted by: Peter at June 2, 2009 6:17 PM

Quebec voters know they have probably milked the conservatives for all they're going to get so now they are playing a bidding war . It doesn't matter what the poll results say , if they ask Nanos or Donello it looks good for the liberals , for some reason . Theres a question in there somewhere that will give them a positive result for the liberals.

Posted by: cantuc at June 2, 2009 7:46 PM

When the largest response is none of the above, its a true indication that the respondents saw the poll for what it was.

What are the odds that most of the responses where eliminated for some reason?

Posted by: gimbol at June 2, 2009 9:05 PM

So, Canadians are being told how to think like Quebeckers...???

Posted by: RW at June 2, 2009 9:07 PM

Canadian Sentinel at June 1, 2009 9:31 PM

I do not disagree with you. But, remember, there are two reasons to make a poll.

1) To propagandize

2) To measure the effect of the propaganda

Posted by: RW at June 2, 2009 9:33 PM

Dief once said that polls were for dogs.
Here is why I agree with him.

When the federal Liberals and their supportive cast of American haters (CAW et al) were fighting against free trade with the US, Liberal polster guru Angus Reid published a poll stating that 78% of Canadians were against free trade.

They claimed a large sample of over 2000 respondents, accurate to plus or minus 1.5%, as I recall.

But the week after that AR poll was published, another FACT (not a poll) was published in the business section of the newspaper.
It stated over 5 million Canadians had made 1 day trips to the US in the previous month.

Now how could these polling maroones find Canadians were 78% against free trade?
And where did they find them?
Outside the auto plant gates?
Did they also 'over-sample' Quebecers?
Also to eliminate the bias?

I ridiculed Angus Reid polls for years after.

Canadian political polls are still for the dogs.

Posted by: rockyt at June 2, 2009 10:18 PM

to put an end to this polling debate, why does any party care what anyone west of Mississauga thinks? No party has won a majority catering to the rural population of Central and Western Canada. What matters is Quebec, Ontario and Vancouver and the rest can be paid lip service. Sounds harsh, but the math just doesn't work.

People who live in large cities have always made the decisions for their nations. Of course, the Nazi Party was an exception, being borne in the wheat fields of Bavaria.

Posted by: yo yo yo at June 3, 2009 12:26 AM
Post a comment

Before submitting, review the post to ensure your comment is on topic and does not contain words that might get caught in the spam filter (eg: insurance, viagra, online, poker). This is not a forum or a repository for off-topic link dumps. Profanity is discouraged. Take your extended debates and/or flamewars to private email. Thankyou.










Remember personal info?






Site
Meter