My projections are based on what I believe to be the easiest numbers to be able to predict: the province-wide numbers for each party. I base my model on the province-wide numbers because these will be the most accurate numbers provided by the polls, much more than regional or riding level data. With that said, I do make further adjustments if my regional averages are too far away from the polling ones. And when we have riding polls, even though they are less accurate, I do an average between them and my projections when the latter are 5 points or more off.
The problem right now is that the riding polls, all conducted by Mainstreet as part of their daily tracker (you need to pay $30 to have access) absolutely do not match up with my projections. This could obviously mean my model is wrong and this is indeed possible. But the riding polls also don't match with the current province-wide numbers. And this is problematic.
How different are the two types of polls? The province-wide ones show a close race with the NDP slightly ahead (in the popular vote) while the riding polls are showing a landslide Conservative victory.
Since the riding polls are behind a paywall, I can't reveal the numbers. But I can talk about general trends and differences.
There have been 38 riding polls published so far. We don't have the exact dates (Mainstreet said they'd add this info) but they were all published over the last week (and the discrepancy between polls and projections have been more pronounced in recent days). Ridings are pretty much all over the province, from the North (Kenora-Rainy River, Sault Ste. Marie, etc), the East (Ottawa South and West-Nepean, Glenngarry-Presscott Russel, etc), the SW (Sarnia-Lampton, etc), the golden horseshow (Guelph, Kitchener-Conestoga, etc) or the GTA (Toronto Centre, multiple Brampton ridings, etc). It's a fairly varied "sample".
If I compare my projections to these polls (when they were published of course), I currently seem to be overestimating the Liberals by 2.2 points and the NDP by a whopping 5.2 points while I underestimate the PC by 3.4 points. This is significant, it means the average PC-NDP gap is almost 9 points off. This leads to multiple ridings where I see the NDP ahead but the riding polls have the PC first. In some ridings, my model is projecting the NDP significantly up compared to 2014 while the polls have them barely higher.
So again, at first, it seems more likely that my projections are wrong. And for a while I thought this was the case and I made the necessary adjustments. But the more polls we get and the more I think the issue is bigger than that.
If we average the 38 polls, we get the table below. I also added the average deviation with respect to my projections.
If I only use the riding polls published during the last two days, then the projections are underestimating the PC by 4.6pt and overestimating the NDP by 7.4! So the problem got worse.
By the way, I realize some of you might object that these 38 ridings can't be a good "sample" of the entire province (think for instance if you only polled ridings in the 416, your results wouldn't be representative). I did a simple average of the results of 2014 and found 42%, 30% and 21% for the OLP, PC and NDP respectively. This compares fairly nicely to the official province wide results of 39%, 31% and 24%. So while not a perfect sample, it's also not overly skewed. And anyway, when I compare them to my projections, the bias, if there is one, should be the same.
If riding polls were simply providing extra information, I shouldn't have a systematic over or underestimation with my projections. I should sometimes overestimate the NDP and sometime underestimate it. But it is clearly not happening.
Notice also how the average difference between polls and projections match up with the overall difference. Take the 38.8% and subtract the 3.4 points, you get 35.4%, almost exactly the current polling average. Do the same for the NDP and you get 31.7%+5.2%=36.9%, there as well almost exactly the average.
What this shows is that my projections are doing a good job at transposing the province-wide percentages into riding ones. Therefore the average of my riding projections will give you the NDP ahead by around 2 points, as it should. The riding polls on the other hand are showing a situation where Doug Ford and the Tories would still be largely ahead and en route to a landslide majority.
Landslide majority based on the riding polls isn't an exaggeration. If the Mainstreet numbers are right, the PC is about to win the entire 905, all of Central Ontario and Eastern (minus 1-2 seats in Ottawa). It would also win most of the Southwest including ridings we'd think the NDP would be ahead such as Sarnia-Lampton. The PC could even be competitive in the North. So really, the entire province minus some seats in the core urban environments. We are talking a 75-80 seats majority here possibly (what the projections were showing 2 weeks ago basically).
The province-wide polls (along with regional polling averages) show a very different situation, one where the NDP is crushing it in Toronto, competitive in some of the 905, ahead in the Southwest and the North. This would be a close race with a slight edge for the PC.
I don't know why the riding polls are so different. Maybe they were conducted a while ago? It's not even a Mainstreet bias as their provincial polls are in line with others. I'll ask Mainstreet about this. But for now, I can't use the riding polls. I believe the provincial ones more and therefore my projections more. I'll use the data of some of these polls such as Guelph (showing the Green competitive) or the ridings in the North. For the rest? I'll remove the adjustments for now.
Riding polls cost me two correct calls last year in BC and I think that this article should convince you that they aren't telling the same story. So I have to make a choice as to which polls I trust. And the choice between multiple province-wide ones or some riding ones is easy. After, I may be wrong and will regret it next week, but at least you know where I stand.
