Taking stock …

Cranking this blog up again. Current writing can be found at Inside Story and the Drum.

Older writing at the Oz here and here.

An Australian Nate Silver?

- First published in the Australian, 23 November 2012.

Since Nate Silver’s fame was well and truly cemented with the US presidential election result, some have suggested that Australia “needs a Nate Silver”.
Silver, who blogs for the New York Times, rose to fame with PECOTA, a system for forecasting baseball players’ performance. (Wikipedia entry here.) I can’t claim to know much about that, but assuming it does what it’s supposed to it sounds like a work of great genius and originality.
What he did for the presidential election was more mundane: synthesise the voting intentions opinion polls and produce state-by-state and overall odds.
Silver wasn’t the only person to do this, just the most famous. Simon Jackman, an Australian academic at Stanford University (who sometimes writes for this newspaper) was another, and he came up with pretty well the same odds.
On election eve they gave Obama about an 80 per cent chance of victory and Romney about 20 per cent. That 80 per cent encompassed everything from a narrow win to a big win. What was unusual was that all the polls were right in all the states. I wonder what the odds of that were?
The fact that Obama won didn’t make Silver “right”, and if Mitt Romney had won it wouldn’t have proved him “wrong”.
If anyone was “right” it was the opinion polls. Silver no doubt crunched them in a very sophisticated way, digging beneath the surface by obtaining details from pollsters and doing some of the weighting and other manipulation that pollsters generally do themselves.
In 2008 Silver apparently got all the states right except one (Indiana) but what that really meant was the polls got all the states right except one.
It was surprising to learn afterwards that the Romney camp actually expected to win the election. One imagines a campaign team operates two mindsets concurrently: the realistic one and the hopeful one. That’s fair enough. On one level you know you’re not likely to win, but on the other you say “I’m feeling pretty good; we might just do it.”
And the polls, while great in number, all showed it being reasonably close. They weren’t pointing to a landslide.
But it seems the Republicans really expected Romney would be comfortably elected. For that belief to take hold they had to have hard-headed number-crunching types telling them it was so.
And people like Karl Rove and Dick Morris did do that. They too had convinced themselves; they thought most of the pollsters’ and Silver’s assumptions and measurements regarding turnout of different groups of people, such as Latinos and Blacks, were wrong. (Read Morris’ mea culpa here.)
There was also something about surveyed Independents (when Americans register to vote they usually have to describe themselves as “Republican” or “Democrat” or “Independent”) slightly favouring Romney. Usually whoever Independents vote for wins. But everyone knew Independents this time consisted of an extra group of people to the right of the Republicans who would favour Romney. People were saying that before the election; there was no excuse for not realising it.
Camp Romney created its reality, from the ground up.
But back to the original proposition of an Australian Nate Silver. For such a creature to exist would require two things. The first is lots and lots more published opinion polls. Americans don’t just have 14 times the poll data we have (14 being the approximate magnitude of their greater electorate size); they have more than that.
The second aspect relates to the electoral architecture and makes the task even harder. Americans particularly poll the “swing states”—those that tend to go to the winner and which have more than a handful of Electoral College votes. The Australian House of Representatives has three times as many “electorates”, 150 of them currently, and they are of equal size. To predict seat-by-seat, pollsters would have to survey individually all that have the slightest chance of being in play.
That would be a massive task. The economies of scale do not remotely exist.
Currently the closest we have are people like Pollytics and Poliquant. Both have a Nate-like understanding of stats and they average and weight and project the latest published polls to give a two-party-preferred number they then apply to the pendulum, or state-by-state bits of the pendulum. (Pollytics has other “secret” data but that would play only a small part.)
To me this is going a bit overboard given the meagre data available. (They both also make other calculations.) We can all see the recent published polls and can get the vibe ourselves. It will be more useful during the campaign, but even then there are so few polls it’s debatable whether a formula that takes into account last week’s polls is more useful than just looking at today’s.
Nate reckons he’s not that much into politics actually. With the election out of the way, sport is creeping into his blog.
Which is here.

The first returning officers

A tiny slice of my PhD thesis. A five minute slide show on Youtube.


Electorate tables

Federal election 2013 and other.


Bolt from the blue

THIS afternoon a strange thing happened on my Twitter stream. I started receiving angry tweets about something I’d written at 1:13pm yesterday (Saturday).

In addition someone found my email address and sent a note that I’m a “low life scumbag … you and your fellow left wing/Green ratbags have absolutely no shame … You should be ashamed of yourself and your employer should sack you.”

“Scumbag” got a mention in the odd tweet too. Most of them were from people I’d never heard from before, who don’t follow me and who I don’t follow. I’ve retweeted most of them, so you can see them on my account if you could be bothered wading through.

Then I worked out what happened. That nice Andrew Bolt had given me a plug on the country’s most widely-read blog.

Now continued here.

A post …

just because ….

Election 2013 census table gallery

Apart from archives there’s nothing new to see at this site since I started blogging at the Oz.

Except for the federal election census and seat table gallery. Check it out here.

Nonsense reporting about direct enrolment

This was published in the Australian on 12 December 2012. It is pasted here un
Here are some simple facts about the electoral roll and direct enrolment.
There are currently around 14.3 million people on the electoral roll. At the last election, in 2010, there were 14.1 million. The AEC estimates that around 1.5 million eligible voters living in Australia are not on the roll, and puts the 2010 number at 1.4 million.
So about one in ten eligible votes are not on the roll.
But—and this bit seems to cause confusion among some journalists and academics—it is not true to say most of those 1.5 million are young people. Most of them are in fact over 30. There are more unenrolled people over 50 than aged 18–19. I spelt this out in a recent post.
And it is not true to say most of the 1.5 million have never been enrolled. Most of them have, but dropped off at some stage when they moved home.
So young never-enrolled are a small subset of the “missing 1.5 million”. This is important.
Another misunderstanding, or assumption behind shock-horror analysis, is that direct enrolment will get all these unenrolled into the roll. It won’t. It won’t even come close, not even by 2020.
As you might expect, the commission is proceeding with care, slowly.
As Milanda Rout in the Australian has written: http://www.theaustralian.com.au/national-affairs/labor-in-campaign-to-mobilise-the-young/story-fn59niix-1226534036231
“The AEC expects to automatically enrol 500,000 to 600,000 over the next two to three electoral cycles as a result of the new laws. A number of the remaining one million “missing” voters may not be picked up under the new system, including the homeless, indigenous people and some non-English speaking communities”
So only about a third of the “missing” will eventually be scooped up—after four to seven years.
So, what about the 2013 election? How many electors will be directly enrolled by then? I reckon 200 thousand is as good a guess as any. It assumes some low-hanging fruit in the early stages of the process.
The unenrolled are disproportionately young, and mobile, and renters, all people who lean left-of-centre more than the rest. (There’s much overlap between those groups of course.)
Earlier this year an ABC 730 segment and Drum article asserted that the unenrolled would favour the ALP, after preferences, by around 10 per cent more than the rest of the country. I think that’s crazy. It may apply to the very youngest age cohort, the 18 to 19 year olds, but not the rest. I reckon it would be less than 3 per cent across all age-groups, but let’s call it 3 per cent for the sake of argument.
Let’s also pretend every one of the extra 200 thousand turns out to vote next year. In reality, for pretty obvious reasons, we can be sure they’ll turn out at a lower rate than everyone else.
(NSW started direct enrolling before its 2011 election and about 80 per cent of them turned out.)
And we’ll also assume all of them will vote formal, when for similar reasons we can be pretty sure they’ll register a higher informal rate than everyone else.
Finally, let’s apply it to the last election, which again is overly generous, as 200 thousand will be a smaller proportion of total voters in 2013 than 2010.
Here are the sums.
In 2010 the two-party-preferred votes for the ALP and Coalition were 6,216,445 and 6,185,918 respectively, 50.12 to 49.88 per cent. If we add 200 thousand more votes, favouring the ALP 53.12 46.88,
Labor on 50.17 and Coalition on 49.83. A difference of 0.05 per cent of the total vote.
It would be vary a little across electorates of course. And 0.04 can make a difference in close seats. But no seat was that close in 2010.
In 2010 several hundred thousand people tried to vote but couldn’t because they weren’t on the roll. If direct enrolment brings that number down, it will be a good thing.
How can making the electoral roll more accurate be a “rort”?
Mr Pyne needs a Bex and a lie down and so do a few others.

*Overseas Aussies is a whole different category. There are around a million of them, but less than 100,000 on the roll and voting.

Mumble has moved to the Oz


Mumble moves to the Oz

Dear Mumble-readers, Mumble moves today to the Australian. It will be a mainly online gig, doing the kind of stuff I’ve done here but more frantically and often.

I love youse all, thanks for indulging me at this place and I invite you to continue at the new one. If only to judge if/how the piper-payer calls the tune.

New site here. Comments here closed; any comments there please. (They will eventually appear, just working that out.)