What Would We Do Without Pollsters?

80 Replies to “What Would We Do Without Pollsters?”

1. Peter – In theory, the results would be similar, but with a larger margin of error as the sample sizes for Ontario and Quebec would have been a bit small.

2. Jim
   The ” margin of error is greater” says it all for me as far as accuracy of polls that are over weighted.

3. Jim – the Mulroney Progressive Conservatives were a completely different political party than the Harper Conservatives. Mulroney’s PC were, as you know, Red Tories, or Liberals.
   Essentially, ignoring the superficial party names, Quebec has always returned only one type of federal party. Socialist and one that in particular, has its key figures from Quebec (Mulroney, Chretien, Martin). The Bloc didn’t exist in the 80’s.
   This means that they vote for a party that gives them socialism, that gives them the highest proportion of the federal authorities in Ottawa and the highest proportion of federal money with the least interaction with the ROC.
   They are totally predictable.
   To others in this thread – some of you misunderstand. The survey wasn’t of 1,000 respondents with half from Quebec. It was 1,000 over all of Canada, plus a second survey, that ‘oversample’ of 500 more respondents confined to Quebec.
   This was done because the researchers assumed that the Quebec response would be unique and they wanted to ‘test’ it (with that extra 500). Then, they ‘reduced’ that 500, with its unique interpretation, to the proportion of Quebecers in the 1,000 test sample.
   My question has been focused only on why the researchers assume that Quebec alone will have a unique response to the survey.
   As for bias, that’s basic to surveys. The first bias comes from getting your sample population. Getting a valid representation isn’t easy. And the next comes from your questions which can be quite heavily biased. And of course, the presentation of the responses can be biased.

4. Peter – I’m not sure that you’re following the math.
   The problem wouldn’t be with the overweighting of the Prairies, but with the sample size of Eastern Canada being too small.
   I just reviewed the actual sample distribution for the poll (http://www.thestrategiccounsel.com/our_news/polls/2009-05-28%20-%20GMCTV%20FINAL.pdf):
   Canada 1000
   Quebec 743
   Rest of Canada 757
   Ontario 383
   West 300
   Just to be clear, they did not sample 500 people from Quebec and 500 people from the rest of Canada. A nationally representative sample was drawn proportionate to regional populations. An additional 500 surveys were conducted in Quebec. The Quebec results were then weighted back down to represent 24.3% of the national figures.
   Peter – to answer your initial question, if the same thing had occurred for the West, rather than Quebec, the results would theoretically be the same. My earlier comment on the greater margin of error was not correct, as the sampling did ensure proper base samples for each region. I mistakenly assumed that the ROC base was 500, not 757.
   Again, all over-sampling does is increase the reliability of the data subset.

5. ET – Your macro-analysis of Quebec politics is pretty solid, but you’re forgetting that modern Canadian politics is all about “brands” and “teams”, not real philosophical differences or competing ideologies (see Harper, Stephen).
   At the end of the day, Quebec voters switch brands more often than those in the rest of the country, and pollsters want to be able to follow/predict/understand those changes.

6. ET – As I mentioned earlier, I think part of the focus on Quebec is pure laziness. Its an easy narrative and an easy market to get attention in (and thus get more corporate business, which is the real reason for doing these fairly useless polls).
   Personally, I’d be way more interested in better BC polling data (an equally volatile political population) or in the seemingly more conservative (but ethnically diverse) areas around Toronto such as Mississauga and Brampton. Unfortunately, the media have little to no interest in reporting such polls, and want to cling to that old “Quebec / ROC” narrative.

7. Jim
   Are we talking about the same Quebec? The Quebec I know has been returning Bloc MP’s in a majority number now for many years in federal elections. Your comment that Quebec voters switch brands is not based on fact – I expect the rational for the polling methods are not based on facts either.

8. Peter – You’re out to lunch.
   Here are the popular vote swings for the past eight elections (2008 – 1984)
   Bloc – 38% / 42% / 49% / 40% / 38% / 49% / na / na
   Liberal – 24% / 21% / 34% / 44% / 37% / 33% / 30% / 36%
   Tory* – 22% / 25% / 9% / 12% / 22% / 14% / 53% / 50%
   *I’ve combined Alliance/Reform/Tory votes for 00/97/93 elections.

9. MsMew
   Confidence intervals have nothing at all to say about sample bias and it certainly has nothing to say on the matter of leading questions. It isn’t what it measures. You can have a high level of confidence at either 95% or 98% confidence interval and still have an invalid stat. You can replicate the same invalid survey and achieve the same incorrect result with a high level of confidence over an over again. All you have to do is repeat the same errors (deliberate ones or otherwise) every time.
   What you are implying is that they Oversample then reduce the weighting back to represent a statistically representative sample size to avoid sample bias. There is no information to suggest that is what they are doing – you are making an assumption. I personally don’t trust liberals not to make their manipulations very deliberately but that isn’t a statistical argument.
   Further, if they reduce the weighting of these 500 people in a total sample of 1000 you reduce your total confidence interval. Further still, if you only have 500 people sampled in the RoC your confidence interval for this data set would be reduced dramatically.
   So, Houston, we have a problem.

10. Regardless of the differences in interpretation of the polls, one conclusion clearly emerges … There is a window of opportunity for Ignatieff and the Liberals to win a snap June election. Canadians no longer want a Harper government guiding the nation through the global recession.
    If Ignatieff hesitates and doesn’t grab the opportunity, he will be abandoning Canadians to the Harper mismanagement of the nation … and that would be unconscionable and unpatriotic too.
    The remaining days in June for Ignatieff to push a no confidence motion and a snap election will be a test of his leadership and patriotism.
    Iggy must pull the plug in June before the HoCs recesses for the summer. If he doesn’t act decisively and defeat the Harper government, he will be facing a new batch of CPC attack ads on TV and radio … to which he will not be able to counter and thus be defined as incompetent, unsuitable and NOT worthy to be Canada’s prime minister.

11. Observant
    lol. Dreams are cheap my friend. They are also worthless.

12. I was simply explaining the concept of confidence interval to someone earlier in the thread. But oversampling is a different matter: it reduces sampling error. The only assumption I am making here is that the oversampling itself isn’t deliberately skewed to obtain some desired result or other.
    The confidence interval for the weighted sample as a whole is 95%.

13. Jason – Read the survey methodology:
    http://www.thestrategiccounsel.com/our_news/polls/2009-05-28%20-%20GMCTV%20FINAL.pdf
    You are wrong, and MsMew is correct.
    Not everything is a Liberal plot, folks.

14. Playing just a little bit with the numbers from Jim’s post above, Kate, the main poll breaks by region into
    Ontario – 383
    West – 300
    Quebec – 243
    Atlantic provinces – 74
    I’m assuming here that Northern Canada is included in the PQ, Ont and West counts.
    The regional result from 74 respondents is going to be real accurate – less than 10% of the total. Better make sure, though, that the result from Quebec is accurate.
    Heh, heh.
    This was also an “online” poll. Can someone tell us how you get truly random data sets when the respondents are online?
    If we actually all could invite Ignatieff or Harper or both to dinner and have a real chance that they would accept, seeing that 56% of Canadians wouldn’t invite either really saddens me. What a nation of political morons we are.

15. ok ! Ok !! OK !!
    We can settle this once and for all !
    Someone, anyone, just look up the list of those citizens “chosen” to be polled and their unadulterated, unfudged, responses. The raw data.
    Which one of you has access to it ? Anyone ? Has anyone ever seen it ? Heard of it being available ?
    If not, then they are probably hiding something. IOW, the bias, the skew, the slant. Push-polls.
    I have asked Chantal Hebert this question many times. Crickets…….

16. MsMew
    The confidence interval of 95% isn’t the measurment. It’s the level of confidence at which the measurement is made.
    You can measure your confidence interval at the 95% level (+/- Two standard deviations) or the 98% confidence level (+/- One standard deviation.)
    What is important is the Standard Deviation from the mean. The larger the SD the less accurate the poll.
    Jim
    OK, the total sample was 1500 people. They oversampled QC and reduced the weighting. That means they took the effort to get really good data from the place where the libs look better and less accurate data from everywhere else.
    It still doesn’t invalidate any of the other points listed above.

17. The integrity of the method means diddly squat if the data is pulled out of thin air or the respondents are “chosen” by someone with an agenda.
    Is it possible that media orgs have an agenda ? Could they be politically motivated ?? Ya think !?
    The first run of the Printing Press in 1462 had not even cooled when Political Organizations of the day started up their own media orgs. Nothing but nothing is more politically bent than media.
    Could media polls of today be a tool ? Follow the money.

18. Jason – Your conclusion is pretty accurate.
    BJG – I assume (but do not have any facts to back it up), that SRC has created an online database that is supposedly representative of the Canadian population, and then drew their sample from that.
    I have serious reservations about this sort of polling, as it would seemingly rely on self-selection into the process (to get into the database in the first place). It is still preferable, though, to the typical Internet straw polls that get passed off as research.

19. Jim,
    To be representative, the sample has to be random. So, a master list is out of the question. Random samples are easier said than done at the best of times.
    Some issues include no representation from those with unlisted phone numbers and cell phones (or no phone at all) along with those who refuse to talk to you, etc.
    Polling when done right is still somewhat akin to voodoo. It’s easier to get the result you are looking for than it is to get a representative result. In other words, lies are easy – it’s the truth that’s hard. That goes double when you add in political considerations.

20. oh, and internet polls are not meant to be valid. They’re meant to drive traffic and make money.

21. Let me say this, Quebec is not only oversampled in polling, it’s over served.
    Anyone who thinks over sampling one province gets a legitimate result to present as voter intention or support has to be playing politics of the most desperate order. It’s bogus.

22. Jason – I share all of your concerns with Internet polling (hence my mention of serious reservations), and noted the same concerns that you’ve highlighted with telephone polling earlier in the thread. I was just trying to differentiate between the likely approach that the SRC took versus the straw polls that you might see attached to a news article or blog.
    There are many, many valid criticisms of the opinion polling industry, and my responses here have not been an attempt to defend the industry at large, but rather to correct some extremely inaccurate assumptions/understandings of basic statistics and research methodology as it relates to sampling.
    Not everything is a Liberal conspiracy and some people (not yourself) would be better served by trying to understand what they are ranting about before posting!

23. And I am not saying polls cannot be quite accurate. They can. Sometimes uncannily so. Predicting elections, ect.
    But a six week election campaign gives the pollster ample time to supersede the push-pull polls with ones more close to reality.
    No ? Have you ever seen raw poll data ? Is human nature such that integrity rules even when not under the threat of being exposed ?

24. Jim
    Thanks for posting the numbers so readers can see how you are prepared to skew things. You included in your comparison the Mulroney years when the Bloc didn’t exist – neat comparison if to thought you could get away with that nonsense.
    Facts are the number of MP’s the Bloc got out of the 75 seats available from Quebec was a pretty constant majority of 54, 44, 54, 51 and 49 seats ( 1993- 2008).
    Jim you wrote the following drivel “The Quebec populace is also far more politically volatile.” and used that argument to justify the polling nonsense.
    Don’t treat the people on this board like mushrooms – they are a little more sophisticated than yo give them credit for – maybe you should carry through with your threat and go “bang your head against the wall” – from your rational on this board you have overdone that to this point in time.

25. Quebec voters know they have probably milked the conservatives for all they’re going to get so now they are playing a bidding war . It doesn’t matter what the poll results say , if they ask Nanos or Donello it looks good for the liberals , for some reason . Theres a question in there somewhere that will give them a positive result for the liberals.

26. When the largest response is none of the above, its a true indication that the respondents saw the poll for what it was.
    What are the odds that most of the responses where eliminated for some reason?

27. So, Canadians are being told how to think like Quebeckers…???

28. Canadian Sentinel at June 1, 2009 9:31 PM
    I do not disagree with you. But, remember, there are two reasons to make a poll.
    1) To propagandize
    2) To measure the effect of the propaganda

29. Dief once said that polls were for dogs.
    Here is why I agree with him.
    When the federal Liberals and their supportive cast of American haters (CAW et al) were fighting against free trade with the US, Liberal polster guru Angus Reid published a poll stating that 78% of Canadians were against free trade.
    They claimed a large sample of over 2000 respondents, accurate to plus or minus 1.5%, as I recall.
    But the week after that AR poll was published, another FACT (not a poll) was published in the business section of the newspaper.
    It stated over 5 million Canadians had made 1 day trips to the US in the previous month.
    Now how could these polling maroones find Canadians were 78% against free trade?
    And where did they find them?
    Outside the auto plant gates?
    Did they also ‘over-sample’ Quebecers?
    Also to eliminate the bias?
    I ridiculed Angus Reid polls for years after.
    Canadian political polls are still for the dogs.

30. to put an end to this polling debate, why does any party care what anyone west of Mississauga thinks? No party has won a majority catering to the rural population of Central and Western Canada. What matters is Quebec, Ontario and Vancouver and the rest can be paid lip service. Sounds harsh, but the math just doesn’t work.
    People who live in large cities have always made the decisions for their nations. Of course, the Nazi Party was an exception, being borne in the wheat fields of Bavaria.