While everyone spoke of New York Times blogger, Nate Silver’s uncanny, almost witchlike ability to call the election last night, the big winner was the triumph of big data and smart algorithms over gut feel and egos.
Those in tech that have been following Nate Silver’s FiveThirtyEight blog at the New York Times broke out in collective high-fives when FiveThirtyEight finished the evening correctly calling 50 out of 50 of the states (besting his 2008 call of 49 out of 50). A baseball statistics geek, Mr. Silver turned to politics and the aggregation of state and national polls as a playground of data ripe for his insights. Traditional polling agencies such as Gallup accuse Nate Silver of standing on their backs and taking all the glory (1 in 5 visits to nytimes.com stopped by to visit FiveThirtyEight). Their complaint is one we’ve heard before, that without their original polling data, Nate would have nothing to aggregate.
Sounds like the what the newspapers used to say about Google News.
But in reality it’s more than just aggregation. Nate Silver and others like him (Votamatic, Princeton Election Consortium) rigorously analyzed what they pulled together and revealed patterns that let the data speak for itself. The accuracy of this approach is a huge wake up call to any pundit that did not take into consideration a data driven approach.
While Nate Silver has put the Science back into Political Science, the data-driven approach to politics is also transforming the sell side, the people that run the campaigns. Time magazine has a fascinating piece on the team that would use modern data aggregation techniques borrowed from online advertising exchanges and e-commerce funnel analysis to segment and target potential supporters of the Obama campaign.
As one official put it, the time of “guys sitting in a back room smoking cigars, saying ‘We always buy 60 Minutes’” is over. In politics, the era of big data has arrived.
– Inside the Secret World of the Data Crunchers Who Helped Obama Win
With the use of data to predict a winner or run a campaign, it is only natural that news organizations too use data as a way to make a point. Data visualizations are one way to convey information that is now becoming de riguer for any self-respecting newsroom. The Guardian started the Data Blog and the New York Times launched beta620 to experiment with data. Some of the best coverage of the local and state elections (such as the image below) came from the Los Angeles Times’ Data Desk which I think is a great idea for any media organization, anything that raises data literacy.
Data can be the source of data journalism, or it can be the tool with which the story is told — or it can be both. Like any source, it should be treated with scepticism; and like any tool, we should be conscious of how it can shape and restrict the stories that are created with it. – Data Journalism Handbook
Leave a comment