Another data map for you. This one by the New York Times based on the most recent census data shows the percent in each region that have either public or private insurance or are uninsured at all. Be sure to allow your browser to read your location and you’ll shortcut to your local region.
The New York Times Magazine had a cover piece on the Obama data mining team that used modern data-mining techniques to more efficiently target the undecided voters that they needed to bring across the fence to win the election. Check out the last line (emphasis mine) on their clever use of Facebook photo tags as a way to further refine their targeting to determine who your real friends were. If they identified any of your close friends as potential voters that were on their “undecided” list, they would then put them on a list of friends for you to ask to vote for Obama.
They started with a list that grew to a million people who had signed into the campaign Web site through Facebook. When people opted to do so, they were met with a prompt asking to grant the campaign permission to scan their Facebook friends lists, their photos and other personal information. In another prompt, the campaign asked for access to the users’ Facebook news feeds, which 25 percent declined, St. Clair said.
Once permission was granted, the campaign had access to millions of names and faces they could match against their lists of persuadable voters, potential donors, unregistered voters and so on. “It would take us 5 to 10 seconds to get a friends list and match it against the voter list,” St. Clair said. They found matches about 50 percent of the time, he said. But the campaign’s ultimate goal was to deputize the closest Obama-supporting friends of voters who were wavering in their affections for the president. “We would grab the top 50 you were most active with and then crawl their wall” to figure out who were most likely to be their real-life friends, not just casual Facebook acquaintances. St. Clair, a former high-school marching-band member who now wears a leather Diesel jacket, explained: “We asked to see photos but really we were looking for who were tagged in photos with you, which was a really great way to dredge up old college friends — and ex-girlfriends,” he said.
My father (who started his own blog) shares what you need to do to get a gun in Japan.
To get a gun in Japan, first, you have to attend an all-day class and pass a written test, which are held only once per month. You also must take and pass a shooting range class. Then, head over to a hospital for a mental test and drug test (Japan is unusual in that potential gun owners must affirmatively prove their mental fitness), which you’ll file with the police. Finally, pass a rigorous background check for any criminal record or association with criminal or extremist groups, and you will be the proud new owner of your shotgun or air rifle. Just don’t forget to provide police with documentation on the specific location of the gun in your home, as well as the ammo, both of which must be locked and stored separately.
And remember to have the police inspect the gun once per year and to re-take the class and exam every three years.
Oh, pink assault rifle? Yes, it’s real – in California.
While everyone spoke of New York Times blogger, Nate Silver’s uncanny, almost witchlike ability to call the election last night, the big winner was the triumph of big data and smart algorithms over gut feel and egos.
Those in tech that have been following Nate Silver’s FiveThirtyEight blog at the New York Times broke out in collective high-fives when FiveThirtyEight finished the evening correctly calling 50 out of 50 of the states (besting his 2008 call of 49 out of 50). A baseball statistics geek, Mr. Silver turned to politics and the aggregation of state and national polls as a playground of data ripe for his insights. Traditional polling agencies such as Gallup accuse Nate Silver of standing on their backs and taking all the glory (1 in 5 visits to nytimes.com stopped by to visit FiveThirtyEight). Their complaint is one we’ve heard before, that without their original polling data, Nate would have nothing to aggregate.
Sounds like the what the newspapers used to say about Google News.
But in reality it’s more than just aggregation. Nate Silver and others like him (Votamatic, Princeton Election Consortium) rigorously analyzed what they pulled together and revealed patterns that let the data speak for itself. The accuracy of this approach is a huge wake up call to any pundit that did not take into consideration a data driven approach.
While Nate Silver has put the Science back into Political Science, the data-driven approach to politics is also transforming the sell side, the people that run the campaigns. Time magazine has a fascinating piece on the team that would use modern data aggregation techniques borrowed from online advertising exchanges and e-commerce funnel analysis to segment and target potential supporters of the Obama campaign.
As one official put it, the time of “guys sitting in a back room smoking cigars, saying ‘We always buy 60 Minutes’” is over. In politics, the era of big data has arrived.
With the use of data to predict a winner or run a campaign, it is only natural that news organizations too use data as a way to make a point. Data visualizations are one way to convey information that is now becoming de riguer for any self-respecting newsroom. The Guardian started the Data Blog and the New York Times launched beta620 to experiment with data. Some of the best coverage of the local and state elections (such as the image below) came from the Los Angeles Times’ Data Desk which I think is a great idea for any media organization, anything that raises data literacy.
Data can be the source of data journalism, or it can be the tool with which the story is told — or it can be both. Like any source, it should be treated with scepticism; and like any tool, we should be conscious of how it can shape and restrict the stories that are created with it. – Data Journalism Handbook
The New York Times’ wonky FiveThirtyEight blog posted this rather dramatic graph showing the poll numbers of Obama and Romney over the past few weeks. What used to be a 1-2 point difference is now a 5-6 point gulf with the lines getting further, not closer, together.
There’s a chill wind blowing through the our government.
Since 9/11 it’s been known that the NSA has been wiretapping email and phone calls as part of a domestic spying program. Now the evidence is piling up that as of December of this year, a new data center in Utah is getting ready to come on line to store every single bit of data they can capture from banking transactions to your Amazon shopping history.
Now the guy who wrote an important piece of the data-mining software (he originally wrote it to spy on the Soviet Union) is coming out in public protest to his software is being used to spy on US citizens. Documentary filmmaker Laura Poitras profiles William Binney, a 32-year veteran of the National Security Agency, in her upcoming film to be released in 2013.
Wired magazine ran a profile on the NSA datacenter in Utah back in March.
The Electronic Frontier Foundation (eff.org) has filed suit against the NSA and has been pursuing the government in court. Read more at Jewel v. NSA.
I always make it a habit to read the release notes of apps when they push updates to see what’s new. Usually it’s pretty dry stuff, what’s fixed, what’s new. Yelp is always good for a few laughs too and their latest release pokes fun at the latest Rick Perry gaffe.
For those that haven’t been following along, here’s what is being referred to.
Eric Fischer takes large datasets and turns them into art. His flickr stream is a collection of fascinating time-series maps plotting data over time to draw out shapes which take on a greater meaning. Weather it’s a map of taxis in San Francisco or an overlay of flickr metadata on top of NYC, Eric’s creations are at once beautiful and informative.
Last month Eric was able to use an open-sourced version of Chrome’s language detector library to parse a week’s worth of geo-tagged tweets and identify who what tweeting in what language, where. What you see above is is the result. Note how languages such as Portuguese, Spanish, English, Dutch, Italian, Swedish, and German stick to and define the borders of their nations. Within each country, major transportation hubs are lit up like avenues. One can only imagine it is the result of people tweeting while enroute somewhere on a train or bus.
The images coming out of Cairo are very disturbing. Nothing good can come out of a situation where the two sides resort to the throwing of blunt objects at each other. My hope is for calmer minds to prevail tomorrow. Google (and Issac Asimov) tell us, “Violence is the last refuge of the incompetent.”
Ignore the “american as cherry pie” stuff – that’s just the incompetent Americans that are saying that.
. . . when you are trying to instill global confidence in your leadership abilities.
In response to questions from the international press about S&P issuing it’s first downgrade on Japanese credit worthiness in nine years,
Prime Minister Naoto Kan had little reassurance to offer. “I just heard that news,” a flustered-looking Mr. Kan told reporters. “I am a little ignorant on those kind of matters,” he said. “Let me look into it more.”