Ok so I'm back from Mexico (it was really nice, thanks for asking). I looked at the projections and here are some comments.
1) Polls were underestimating the Conservatives.
While the polls were mostly right for the other parties, they clearly underestimated the Conservatives. If you take the last couple of polls published during the week-end, they all put the CPC around 35-37%. Given that these polls had a sample size of 2000-3000 most of the time (Ekos, Angus, Leger), it means the actual Tory score is out of the margin of error. Numbers can be even worse at the province-level. For instance in Ontario, you never had the Tories at 45% like they did on Election Day. Pollsters will (and already have in some cases like Ekos) have to revise how they survey people I think.
2) Atlantic
Using the actual percentages for this regions (which are, respectively for the CPC, LPC and NDP 37.9%, 29.3% and 29.5%), you would get 9 mistakes. Most are coming from the tricky NF-L. However, it seems that my assumption that some of the ABC effect was gone was right. Indeed, the Tories increased more in NF-L that in the rest of the Atlantic: they increased from 16.6% to 28.4%. But the effect wasn't 50% gone, more like 25% then. With that being said, weird things happened anyway. The only riding won by Harper was Labrador and unless I wasn't aware of a new star-candidate, it was definitely not the most likely to switch. In the rest of the Atlantic, the model performed quite well.
3) Quebec
The big orange wave. It's funny, with my final projections, Iw as the harshest on the Bloc (as compared to 308 or DS) and I was a little bit worried to underestimate this party... Boy I actually overestimated them. Even by using the actual percentages, I would still have the Bloc with 10 seats. I would have the Liberals at 12, but winning 6 out of 7 close races. Therefore a more adjusted projections would be 9-10. The same for the Tories with 9 seats but 5 out of 6 clsoe races, leading to adjusted results of 7. By the way, if you actually compare the close races projected to be won by the Liberals over the NDP to the actual results, you see that the model was actually quite close most of the time, instead that the NDP won these ridings slightly. Having the correct "call" is important, but being accurate is as well.
The model would have worked quite well in Quebec everywhere except in Gaspésie where the NDP won almost everything while the model was still giving most of the ridings to the Bloc. It's because historically, the Bloc was more resilient there and the NDP's swing was clearly decreased there. With that being said, th Bloc ending up with only 4 seats despite finishing second in the popular vote with as much as 23% is puzzling. I would actually be worried to have my model correctly predicting that! lol I mean, it seems the Bloc was caught in a wave AND was freaking unlucky on top of that.
4) Ontario
Ok here I need to mention something first. The following comparisons are made using a simplified model that I had. As for Quebec, I had other models ready for Ontario in case all parties would enter extrapolation-territories. Since the final polls were showing the NDP around 25%, Liberals around 27% and, most importantly, the Conservatives slightly down compared to 2008, I decided to use the full model, and not the simplified, more extrapolation-friendly one. BUT, if the polls had shown the CPP at 45%, I would have switched to the other model. People reading this blog regularly know that I have many specifications. After all, I did switch for Quebec after the orange wave started.
So anyway, no matter which model I would have used, my projections would have have been mostly ok in Ontario, except in the GTA. There is no way to reconcile historical trends and what we saw last Monday. In particular, while the provincial swings explain how the Tories won ridings in the suburbs of Toronto, they shouldn't have won seats in Toronto. For instance, the seat of Ignatieff should have remained a Liberal one. Historically, the Tories swing was decreased in Toronto. Look at the following table:
| 04 to 06 | 06 to 08 | ||||||
| CPC | LPC | NDP | CPC | LPC | NDP | ||
| east905 | 3.3% | -4.0% | 1.0% | 6.8% | -10.1% | -0.7% | |
| west905 | 4.1% | -3.9% | 0.2% | 3.8% | -5.5% | -0.8% | |
| Tor. east | 4.0% | -3.1% | 0.5% | 2.5% | -4.7% | -1.3% | |
| Tor. West | 3.3% | -3.8% | 1.0% | 3.9% | -5.4% | -1.0% | |
| prov. swing | 3.6% | -4.8% | 1.3% | 4.1% | -6.1% | -1.2% |
As you can see, traditionally, the Tories were increasing a little bit less in this region while the Liberals were holding up better (except in the east 905 where the Tories already had all but one seats since 2008). So ridings liks the one from Ignatieff where the difference between the Tories and the Liberals was more than 11-points were not likely to switch when the provincial swing for the Tories was +5 and -8.5 for the Liberals. You have a lot of ridings in this situations where the provincial swings were actually amplified for both parties! What it means is that there was a major change occuring in the GTA during this election. Maybe it actually all makes sense. I mean, yes traditionally the LPC was holding up better, but now that the Liberals were already low everywhere but in the GTA, it kind of makes sense that any further drop province-wide would have to come from the GTA! I'll think about it while building the next model. In any way, I was very clear that my model was probably not appropriate for extrapolation purposes. For this, simplistic model assuming an uniform swing (or a proportional one) as better suited.
5) What to do next?
My objectives has always been to turn such a model into a published paper. Now that we have more data (and in particular, I now have variation for the NDP, a party that used to move very little and was tricky to estimate), I will be able to test various specifications more in details. I couldn't do that before because, in order to estimate the coefficients, I need more elections. Indeed, with only 3 (2004 to 2008) and thus 2 swings, I hadn't enough data points. Now, I can use data from 2004 to 2008 to see how I could have explained 2011 better. One thing I'll have to try to make the model more extrapolation-friendly. With a majority government, I have more time to do that! I'll keep you posted with the updates.
Doing seat projections is always tricky. You can go simple and be right sometimes, but sometimes you are wrong. In 2008, for the Quebec election, my model was the only one to predict 7 ADQ MLAs instead of the 2-3 predicted everywhere else. Why? Because I had regional coefficients. But when the regional patterns change, like this time in Quebec (Gaspésie) or in the GTA, a model based in historical patterns would necessarily be wrong. Seat projection is not an exact science. However, I still think there is a place for a scientific, complicated and transparent model like mine.
I'll also be back for the next Quebec election, around 2012 I guess. I don't know yet if I'll have time to build a model for Ontario or BC.
