Privacy Wall

I wrote the following post on Friday and was going to mull it over for a bit but then this article came out in today’s Washington Post that made the issues raised here all the more timely.

An interesting topic was brought up that was glossed over in coverage of Friday’s Search SIG. John Battelle warned that the search engine industry is eventually going to hit a privacy wall. In pursuit of the perfect search result (which we all know is relative) it’s implicit that a search engine needs to know a bit about the person running the search. The more a search engine knows about you, the more relevant the results. If you identify yourself as a car fanatic and type in "jaguar," an informed search engine can skew the Jaguar car information over the stuff on big black cats.

As search engines pull in increasing amounts of information to gain context, there is going to be a point where the search engine companies begin to tread into a grey area where the rules and practices of how to handle private information have not been worked out. Every site has it’s privacy policy but who takes the time to read through these anymore? It would  be helpful to have a debate about this now. Who owns your information? What are best practices on how it’s stored? How can your data be used in aggregate? What is the opt-out procedures? Can there a simple way to indicate the level of privacy control, through the use of icons, much in the same way that Creative Commons indicates copyright control?

We are ever more connected to the data cloud called the internet and mobile devices will prompt more and more of us to upload our information such as calendars & contacts into this cloud so we can access it anytime and anywhere we want. How many of you have clicked that little "sync" button in Plaxo and later realized that every contact you’ve every met now joyfully reminds you every year of their birthday? It’s so easy to forget what pieces of information you’re throwing around and how that data is used and shared.

Battelle calls for search engine companies to kick off a debate on guidelines & best practices around privacy before something terrible happens that forces the government to step in with heavy-handed regulations that would bog down development of the social web. I would argue that this debate over privacy is already happening with the cases of Choicepoint and Westlaw earlier this year. Joi Ito has also posted at length about the privacy debate as it related to a program to roll out a National ID database in Japan.

Clearly the elephant in the room as we all dance down the road to one big inter-connected nirvana. Being absolutely clear to the public about the trade off between sharing personal information and greater utility is an important point that should not be glossed over in the name of progress.

UPDATE: John Battelle has an op-ed piece in the Sunday San Jose Mercury News on the topic. Related to privacy, I also point you to Barton Gellman who has written at length about "national security letters" in The Washington Post. These letters, which are authorized under the USA Patriot Act, give US agents broad powers to ask for and receive personal information in the pursuit of national security and obligates the provider of the information to keep the disclosure secret. Over 30,000 of these letters have been served and including the case of an orange alert in Las Vegas at the end of 2003. In order to try and locate a potential terrorist threat, the hotel records of an estimated 1 million guests at Las Vegas hotels were sent to the FBI for a data mining excercise. In this case, what happened in Vegas didn’t necessarily stay in Vegas.

SD Forum Search SIG

I went to the SD Forum Search SIG over at the Microsoft Silicon Valley campus last night. The first session was a talk between John Battelle and his old editor at MacWorld, Dan Farber (now Editor in Chief at ZDNet). Then there was a panel discussion featuring the viewpoints of four companies that are building vertical search engines

I must say I like these smaller gatherings better. When John asked how many in the audience were working on their own search engine, almost 30% of the room of 150-odd folks raised their hands! With such a knowledgeable audience, the questions were good and the speakers could go deep into what interested them.

Some notes:

Battelle: the web 1.0 companies that survived were the ones that created platforms, not web sites.

What is the number one frustration users have with search engines? Bad results.
What is the number one frustration search engine companies have? Bad questions.

Search marketing is not advertising – it’s a sales channel. If done correctly, every dollar you put in should return more than what is invested in revenue. It’s measurable so budgets can be held accountable. This is why, to a certain extent, John Battelle feels that Google’s dependence upon advertising revenue is somewhat immune to any downturn in the traditionally cyclical advertising market. It’s not advertising, it’s sales. More on this on CrunchNotes.

Interesting note – while one of the keys to success is a platform which improves based on user interaction, none of the companies that were demoed yesterday had a real social aspect to them. To that end, they are more nodes for information than hubs of activity. Opportunity?

TimesSelect Annual Revenues Nearly $5M

The New York Times announced that it has over 270,000 subscribers to it’s premium TimesSelect service. PaidContent does some back of the envelope calculations and calculates that they are pulling in $4,954,000 in annual revenue from this venture only 52 days after launch.