We get a lot of polls during a federal elections. Most of them, and especially the ones getting media attention, are nationwide polls with provincial breakdowns. However, we also get a fair share of riding polls. They usually have much smaller sample sizes (between 300 and 600 observations) and are often conducted by firms a little bit less know (Mainstreet, Environics, Segma, etc instead of the big names of Ekos, Ipsos, etc).
I decided to do a systematic comparison of the projections I get based on provincial polls and the riding-level numbers from the riding ones. I looked at every riding poll conducted after August 15th (there were a few at the very beginning of August but I decided ignore them as it was the beginning of the campaign). For the projections, I used the average at the provincial levels during the time (most riding polls were either conducted last week or around August 16th anyway).
Here are my findings:
- Riding polls underestimate the support for the Liberals. Indeed, if I average the percentages for the Liberals in my projections in these ridings, I get 31.7% while I only get 28.7% in the riding polls. This is the only party with such a big difference. Conservatives are similar (30.9 vs 31.9), so is the NDP (30.8 vs 30) (The Bloc is highly underestimated in my projections but this is mostly based on the 3 polls in Quebec by Segma). The riding polls aren't selected randomly and evenly across the country, so we shouldn't expect them to give us the same overall percentages as nationwide polls (even though it's quite close despite a lot of these polls being conducted in Alberta ridings). But here, I'm comparing in the same ridings.
In other words, either the Liberals are overestimated by national polls, or underestimated by riding ones. Of course, there is the third solution where my projections are just wrong. This is possible but my projections have the Liberals in average where the polls have them (for each province). So it'd be a little bit weird for my model to be so wrong in only these ridings.
- The riding polls would predict the following seat count (note: two ridings were polled more than once during the last month so I only counted each once. The 0.5 is because in Ajax, the two polls didn't agree with each other):
CPC: 11.5
LPC: 12.5
NDP: 15
While my projections would give:
CPC: 13
LPC: 15
NDP: 11
The 3 net gains for the Liberals in my projections (compared to the polls) are in Vancouver Granville (Environics poll has the NDP ahead), Edmonton Centre (poll has CPC), Toronto Centre (poll shows small NDP lead, I have small LPC lead), Kitchener Centre (poll has NDP), Eglington Lawrence (CPC in the poll), Ajax (Mainstreet poll has CPC at 39% vs 37% for LPC). At the same time, the polls give 2 ridings to the Liberals (Cumberland-Colchester, Peterborough-Kawartha and Calgary Confederation) where my projections have a different winner. Overall, it seem the riding polls especially differ from my projections in Ontario.
Overall, the polls agree with my projections 30 times out of 40. At the same time, let's remember these riding polls have big margins of error and are generally less accurate that other polls. With that said, my projections only have the Liberals within the margins of error a quarter of the time (and in only 1 out of 12 polls on Ontario!). So there is a discrepancy between projections and riding polls as far as the Liberals are concerned.
If you think my overall projections are hard on the Liberals, remember that riding polls are even harder!
