The Progressive Conservatives proved the polls right (well, mostly, see below) and got their majority. I don't have all the results per riding but I have enough to write a quick post-mortem to an election which ultimately was a good one for me and people making projections.
1. The polls.
Polls weren't bad but they also weren't great, at least not in average. The IVR polls (Forum, Ekos and Mainstreet) did really well while online polls (and live callers) missed the PC-NDP gap by quite a lot. And one firm, Abacus, missed completely. Mainstreet had the best overall accuracy but they didn't have the Green within their Margins of error (note: MoE vary with the level of support for a party. A party at 5% does NOT have the same MoE as a party at 50%. Polls usually only report the MoE for a party at 50%). Forum had all 4 parties within the MoE... so hard to say who won. Let's call it a tie between Forum and Mainstreet. Funny enough, these are the two firms often criticized online (I have some annoying people on Twitter for instance who always want me to ignore these firms lol).
See the table below for the detailed performances:
I had mentioned the IVR-online gap before but since the two methods had been equally good at Canada over the last decade, I couldn't pick who was right. With that said, even IVR polls underestimated the PC and overestimated the NDP. Should I have had the PC higher given the late trend? Maybe but the late polls were also mostly all IVR (except Research Co.) and I didn't want to trust one method more than the others. Hindsight is always 20/20 but I'd do the same again. I'm not in the business of trying to guess who is right, I follow the average and the numbers.
People will likely remember this election as one the pollsters got right (they were predicting a PC majority and got one) but going from a 2 points PC lead to a 7 points one isn't close at all. Abacus was the worst one by far and they'll argue that it's because they finished polling 6 days before the election. While it obviously doesn't help, it also shouldn't matter to you. First of all, Abacus could have polled later. They simply chose not to. And I strongly suspect they did so to have a way out in case they were wrong. Second of all, Abacus was showing a bigger NDP lead than most other pollsters anyway.
I think IVR pollsters got it right. Online ones were off.
A note about the Mainstreet riding polls. They ultimately were not bad but I feel vindicated in the point I had been making for a week: they were consistent with a province-wide result where the PC is largely ahead of the NDP. It's interesting that the riding polls were showing this situation way before the provincial polls.
2. The projections.
I'm quite happy with my forecast. The average deviation per party is 3 seats, which is one of the best forecast I saw out there.
I correctly called 110 ridings out of 124, that's a 89% success rate (it pisses me off to miss on the 90% mark!). And 1/3 of the mistakes are from Scarborough! I'm two three mistakes away from Earl Washburn from Ekos, the guy I consider as being the best in this country (and he does have access to riding polls from Ekos that we don't get to see). So I'm quite happy. With the actual vote percentages, I'd have actually made 16 mistakes! But I'd have had the PC at 82 seats and the NDP at 40, so closer than my normal forecast. I'd have missed the Liberals even more though.
Here are the ridings I missed. For each, I also discuss whether I could have done better.
Brampton Center: was projecting a close race with the PC winning by 0.6pt, the NDP won it by 0.26pt! Let's be clear here, getting such a close race right is just luck, nothing else. I'm quite pleased with the forecast.
Brampton-West: bigger miss there but I had the PC with around 20% chances of winning, so not a shocking surprise. The polls underestimating the PC lead didn't help though. By the way, the riding polls also had it going NDP...
Chatham-Kent-Leamington: Pretty big miss percentage wise, but the probabilities were showing the PC had a 36% chance of winning. Interestingly, I had noticed that my regional average in the SW was becoming too biased in favour of the NDP compared to the polling average over the last 2 weeks. I corrected it for it but not fully. That explains part of the miss.
Don Valley West: Wynne won by a small margin and proved me wrong. I don't have much to add here, it's a miss for sure. The riding poll from Mainstreet had it right.
Kenora-Rainy-River: the riding poll was right and I did include it, but ultimately my model still had it going NDP. A clear miss too.
Orléans: one of the seats the Liberals managed to save. Look, I was expecting the OLP to be stronger than expected in some seats, the question was to identify (guess?) which ones. The riding polls were even more off than me so I don't feel too bad.
Ottawa South: similar story here, including the riding poll being wrong. I'll say one thing though: the one trend over the last week for the Liberals was an increase in the East. So I was expecting the Liberals to possibly keep more seats than forecast in the Ottawa region but I didn't include this subjective reading into the model (beyond adjusting the regional numbers of course). So it didn't shock me when the Liberals kept it.
Sarnia-Lampton: similar to Chatham-Kent-Leamington. The riding poll was better.
Four mistakes in Scarborough: Not much to say, this is one region where I got it wrong (or got unlucky). With that said, the Liberals keeping Scarborough-Guildwood is the biggest surprise of the night and I haven't seen anyone projecting it to remain red. The chances of winning were only 9.24%. Still, when your biggest mistake is one where the chances were 9.24%, you know you had a good night! After all, if my probabilistic model is correctly calibrated, you need some people projected with only 10% chances to actually win!
Thunder Bay-Superior North: the North is always more unpredictable, although most were expecting the Liberals to keep the other Thunder Bay riding.
Toronto St. Paul's: I knew the NDP had its chances there but given the advance turnout that wasn't increased compared to 2014, I thought the NDP surge in the core of Toronto wouldn't take this riding. Oh well.
So, any mistake I regret? Maybe the two in the SW (Sarnia and Chatham) as there was some data showing they would go PC. Maybe Don Valley West too (although advance turnout showed a big increase, which isn't usually good for the incumbent). For the rest, I consider correctly predicting almost 90% of the ridings correctly to be a good night. I don't think it was really possible to anticipate 7 OLP wins. Not without doing some crazy subjective adjustments.
The probabilistic model worked again very well.
| Projected chances of winning | Actual win percentage |
| 0% | 0% |
| 1-20% | 2% |
| 20-40% | 29% |
| 40-60% | 52% |
| 60-80% | 64% |
| 80-99% | 99% |
| 100% | 100% |
Adding this election to the others (so we have more observations as some bins/cells are quite low for one election only), the model has now the following accuracy:
| Projected chances of winning | Actual win percentage |
| 0% | 0% |
| 1-20% | 9% |
| 20-40% | 30% |
| 40-60% | 50% |
| 60-80% | 64% |
| 80-99% | 88% |
| 100% | 100% |
This is a well calibrated model. The only one not perfectly matching is for the 60-80% range. But it's still close enough.
That's it for now. I'll not write as much in the coming weeks as I want to take a little break (and there is nothing to write about anyway lol). I hope you enjoyed my coverage during the election and see you next time.
