I wrote the following post on Friday and was going to mull it over for a bit but then this article came out in today’s Washington Post that made the issues raised here all the more timely.
An interesting topic was brought up that was glossed over in coverage of Friday’s Search SIG. John Battelle warned that the search engine industry is eventually going to hit a privacy wall. In pursuit of the perfect search result (which we all know is relative) it’s implicit that a search engine needs to know a bit about the person running the search. The more a search engine knows about you, the more relevant the results. If you identify yourself as a car fanatic and type in "jaguar," an informed search engine can skew the Jaguar car information over the stuff on big black cats.
We are ever more connected to the data cloud called the internet and mobile devices will prompt more and more of us to upload our information such as calendars & contacts into this cloud so we can access it anytime and anywhere we want. How many of you have clicked that little "sync" button in Plaxo and later realized that every contact you’ve every met now joyfully reminds you every year of their birthday? It’s so easy to forget what pieces of information you’re throwing around and how that data is used and shared.
Battelle calls for search engine companies to kick off a debate on guidelines & best practices around privacy before something terrible happens that forces the government to step in with heavy-handed regulations that would bog down development of the social web. I would argue that this debate over privacy is already happening with the cases of Choicepoint and Westlaw earlier this year. Joi Ito has also posted at length about the privacy debate as it related to a program to roll out a National ID database in Japan.
Clearly the elephant in the room as we all dance down the road to one big inter-connected nirvana. Being absolutely clear to the public about the trade off between sharing personal information and greater utility is an important point that should not be glossed over in the name of progress.
UPDATE: John Battelle has an op-ed piece in the Sunday San Jose Mercury News on the topic. Related to privacy, I also point you to Barton Gellman who has written at length about "national security letters" in The Washington Post. These letters, which are authorized under the USA Patriot Act, give US agents broad powers to ask for and receive personal information in the pursuit of national security and obligates the provider of the information to keep the disclosure secret. Over 30,000 of these letters have been served and including the case of an orange alert in Las Vegas at the end of 2003. In order to try and locate a potential terrorist threat, the hotel records of an estimated 1 million guests at Las Vegas hotels were sent to the FBI for a data mining excercise. In this case, what happened in Vegas didn’t necessarily stay in Vegas.