I had a great day yesterday at the GigaOM Roadmap conference. The agenda had a number of great speakers including Brian Cheskey of AirBnB and Tony Fadell of Nest, the red hot company that is re-defining what a thermostat should look like.

The thesis the conference explored is one that Om Malik (now my boss) has put forth a number of times. If you think of the steam engine as the PC of our age and the portable version of this technology, the locomotive, as the mobile phone, what does increasing bandwidth and the enabled mobility mean for society and businesses going forward?

Each speaker chipped away at this thesis with their own slant but Jack Dorsey, as he described how Twitter has enabled empathy on a global scale and how Square has removed the barriers of a Point-of-Sale system and the, “massive counter” that sits between a customer and the vendor, more than anyone else opened my eyes to the incredible transformation going on around us.

Yet, in light of all these incredible transformations, Dorsey challenged us to maintain a balance between the “sleek, modern perfection and the rustic, zen-like chaos” and to build products that maintain this “balance in-between”. He referenced the Japanese design aesthetic of wabi-sabi (if you want to read a great book about the topic, I highly recommend Wabi-Sabi: for Artists, Designers, Poets & Philosophers).

In the end, Dorsey advised all product managers to guide themselves with these two principles.

1. Simplification, work real hard to get technology down to its essence (of an interaction). Take away the “conceptual debris”
2. Make things fun, remember to be human, relate, “have some whimsy” in your application and make it human.

The whole interview is worth a listen. I’ve embedded it below.

From a site called @peanutweeter which combines tweets with stills from Peanuts, updating the 1950′s comic strip to a commentary of our time.

“The site arose from the concept that the amusing and sometimes outrageous tweets out there would be even funnier or sometimes darker if they came from someone that everyone could identify with,” site creator T. Jason Agnello told Wired.com by e-mail.

I spent the year end holidays in Tokyo with family and friends. As with every visit I was blown away by the pace and energy of the city and came away re-charged with optimism. I am especially happy when I hear about new companies such as Twitter coming to Japan and finding a fit. Not only is there a television drama centered around characters that tweet to each other, corporate twitter handles are regularly mentioned on advertising and the mass media assumption is that everyone knows what twitter is and how it works.

The latest proof point of twitter’s growing popularity (Japanese make up 16% of all twitter users compared to less than 10% in the US) is a post on the twitter blog. Almost 7,000 tweets per second which is more than double their last record of 3,000 during the World Cup.

It’s hard to wrap up a major conference, especially when you didn’t attend, but viewing things from a distance sometimes helps because only the loudest messages make it all the way over.

Before the conference even started, Fred Wilson threw out a one-liner that got people thinking. He called it the Golden Triangle.

The three current big megatrends in the web/tech sector are mobile, social, and real-time.

To Fred, the vectors between each of these points on his triangle represented the biggest opportunities over the next few years and where he, as a technology VC, was going to focus his attention.

Ross Mayfield, his line from the first Web 2.0 conference is still relevant, added Geo to Fred’s Triangle and posted his virtual napkin up on flickr.

The importance of Geo cannot be ignored as the most obvious (and easiest) way to add context to information which is being harvested and sent our way in increasingly alarming rates. We talk about a world in which there are 1 billion mobile devices. Imagine what happens when each of these gets a camera, gps, and bluetooth sensor and an IP connection to pull in real-time updates. Adds a new dimension to Right Here, Right Now.

So while HTML Page Indexers of yore were failing at finding us the best Chinese in Helsinki or plumber in London, Social Discovery became the new nectar. Facebook leads to FriendFeed leads to Twitter and now our capacity to consume and process has overloaded. Groups, Hashtags, Lists, Folders, call them what you will but this manual organization of streams is beginning to feel like e-mail folder management all over again. The Googles and Microsofts have added the Twitter firehose to their indexes but somehow I don’t see that as solving the problem unless they can filter on your social connections as well (rumor has it Google Profiles are about to play a much more important role Google Social Search is now live).

Which brings us to Social Filters.

Marshall Kirkpatrick has been following this topic for a long time. He bangs the Social Filter drum again in a post about Facebook’s News Feed redesign,

Someday social networking is going to be like the telephone. Today you can’t send messages from Facebook to people on MySpace or LinkedIn but that isn’t going to last forever. Just as you can call someone who uses T-Mobile from your Sprint phone, someday sharing and messaging between online social networks will be a given.

How will social networks retain users then? Why stick with Facebook when some smaller service offers a decentralized social networking service outside of Facebook’s control but still tied into your friends on Facebook and elsewhere?

These services will someday have to compete on user experience, when they no longer have your social connections locked-in. The service that does the best job filtering up the most important information you have coming your way will likely be the service you stick with. That’s going to be a key area of competition between social networks.

Yes, it’s no longer about who “owns” the social graph – it’s who provides the best services on top of a shared graph. Someone mentioned that Tim Berners Lee said at the conference that AOL was to WWW as Facebook is to distributed social networks. Just as we thought it silly that AOL wanted to put it’s famous wall around the internet, we may also look back in amazement thinking that anyone could have the audacity to think they could own the world’s social address book.

Some historical perspective from Tim O’Reilly and John Battelle in Web Squared: Web 2.0 Five Years On

There is a race on right now to own the social graph. But we must ask whether this service is so fundamental that it needs to be open to all.

It’s easy to forget that only 15 years ago, email was as fragmented as social networking is today, with hundreds of incompatible email systems joined by fragile and congested gateways. One of those systems – internet RFC 822 email – became the gold standard for interchange.

We expect to see similar standardization in key internet utilities and subsystems. Vendors who are competing with a winner-takes-all mindset would be advised to join together to enable systems built from the best-of-breed data subsystems of cooperating companies.

Bringing it all together you can almost hear the synapses of the global brain achieve self-awareness. Not only are we moving to a web of sensors feeding real-time data into the grid, we are annotating it by injecting bits of human commentary and behaviors across an increasingly distributed social graph.

A phone in one corner of the world sends off a snapshot which is immediately re-tweeted via the world’s largest telephone tree. More reasoned minds pick up the samples, turn it over and examine it and later conclude that no, the calculated mass of the balloon could in fact not hold a small boy aloft – rumor refuted! Lesson learned and the network becomes a little smarter, more skeptical, less knee-jerk adolescent. Sentient if you will.

The pieces are in place, the machines are warmed up. It was fun while it lasted but it’s time to put Failblog aside and see if we can move on to tackle bigger problems. O’Reilly and Battelle wrap up with their call to arms,

2009 marks a pivot point in the history of the Web. It’s time to leverage the true power of the platform we’ve built. The Web is no longer an industry unto itself – the Web is now the world.

I’m not so sure how the whole Facebook namespace landrush is going to work out for me (cheeky of them to have us all sit around camped out facebook.com on a Friday night!) but for now I’m going with this twitter card as a way to get my social media dialtone for now. To make you’re own, just edit the source at twitter.com and crop to fit. Or, if you’re lazy, you can use the Twitter Meishi Generator.

There’s been good debate around how the source of traffic to sites is changing, shifting from the search engines to social sites such as Facebook and Twitter. I confirmed that I too am seeing a greater percentage of traffic come in via links shared on social sites and shared a colleague’s theory about what this would mean for Google’s advertising revenues. Fred Wilson also posted about this topic here and here.

What about attention? How does the average visitor from a social site compare to someone from a search engine? Niall Kennedy tweeted the following stats from his referral logs:

• Twitter: 7 seconds
• Digg: 20 seconds
• StumbleUpon: 40 seconds
• Facebook: 52 seconds
• Delicious: 82 seconds

Here are my stats for the past year:

• StumbleUpon: 40 seconds
• Digg: 42 seconds
• FriendFeed: 53 seconds
• Facebook: 60 seconds
• Twitter: 86 seconds*
• Delicious: 110 seconds
• Techmeme: 114
• MyBlogLog: 176 seconds

Compared to the attention span of those coming from the major search engines we get:

• Live.com (MSFT): 21 seconds
• Yahoo: 35 seconds
• Google : 40 seconds
• Ask: 46 seconds

It would be interesting to see figures from other sites, especially online shopping sites which are the ones most interested in getting (and therefore likely to pay for) traffic. While it’s clear that visitors from social sites are more engaged with my blog because they tend to hang around a bit longer, that may not be the case with a shopping site where there is less intent to purchase than if they come from a search engine but Mark Essel thinks otherwise.

Increasingly, the flow of web links is being made between individuals via social media sites.  Your good fishing buddy who knows the Bay area, shares a link to his favorite supply store.  As focused communities become populated across geographic barriers, local quality referrals become more likely.  But what if you want to know what store fishermen prefer in San Francisco?  You could simply use twitter search for fishing san francisco.  In real time you could send a message to several individuals who are interested in fishing in that region.

Shared links are compelling but they need to be matched with impulse buying or discoverable when you’re looking for it. I’m thinking of O’Reilly’s flash discount shared via twitter (44% off to celebrate the 44th president) which was effective in bumping registration at the recent Web 2.0 Expo. The other way to generate business via shared links is to make them searchable so you can find what you need when you want it – but then we’re right back at sending traffic via search again.  Yes, you can search twitter but you can find this stuff on Google too.

The jury’s still out as I think this will be a slow shift of behavior that will take a long time to impact existing business models. The real prize is back to social search which would combine the best of the recommendation trusts of social networks with the ability to find what you need when you’re looking for it.

Facebook recommendations married to Google ‘s structure and ranking? That’s the subject of another post.

—o—

* I dug into the Twitter figures because they’re so out of whack with what Niall is seeing and it looks like there are a few visitors that hung out for a long time that are pushing that average higher than it should be.

Jason Calacanis expands upon his offer to pay twitter to get his Mahalo account onto twitter’s suggested user page. It’s the distribution channel and potential click-thru traffic he’s looking for, and click-thrus to his site equals registered users and a lifetime relationship with Mahalo that can be monetized over time.

The top 20 slots on Twitter are actually worth–to some people–2-10x what I offered. If Southwest, Amazon, eBay or Zappos were to get their hands on one of these accounts, they could easily make one uber-compelling tweet a week. 50 tweets from Amazon with things like “Top 100 Science Fiction Movies of All Time–as rated by George Lucas” would garner a two to 10% CTR. A Zappos tweet with “Back to school: Buy two pairs of shoes get one free” would get a huge response on August 20th.

A JetBlue daily notice saying “The first 1,000 folks to respond to this alert get $25 off their next flight” would mean never having another empty seat.

The point is that Twitter has the ability to unleash a direct
marketing business the likes of which the world has NEVER seen. I predict they will, and when they do, they will make the Twitter nay-sayers look like the donkeys they really are. (Note: you ever notice the folks who have the most to say about making money are the ones who’ve never made any? Exactly.)

Direct marketing by mail changed retail forever, as did the Web and email.

Twitter will take that to an entirely new level. Why? Because people *live* inside of Twitter like they have never lived inside of a product before. We have NBA stars twittering about their performance at half-time and a president who leveraged the service to get elected.

It doesn’t take a genius to understand that there is something
disruptive going on here.

Wasn’t this the promise of SMS? A direct channel to which consumers could sign up and get notifications of local deals in their area?

Liz Gannes posted that Perez Hilton is now seeing more traffic coming in via Facebook than Google.

My colleague Udo Szabo at Nokia HQ in Finland has a theory that I call the Unified Theory of Interweb Economics. The theory goes something like this:

1. Advertising is a function of your traffic volume, the more traffic that comes to your site, the higher rates you can charge.
2. Social sites such as Facebook and Twitter have a lot of link sharing going on as friends post links to share them with each other.
3. When Facebook and Twitter send more traffic than Google search referrals, advertising dollars will follow the source of that traffic.
4. Google’s dominance in online advertising will be threatened.

One would think we’re seeing the first hints of that with perezhilton.com and it will take a long time before we see this trend across the board. But in hindsight this is obvious and looking at my stats for the past month I find that 44% of my visitors are coming from search engines and 45% from referring sites with a vast majority coming from StumbleUpon, a social site for sharing links.

The more you think about it, the shift in balance of power has already taken place.  SEO is still a big industry but now there are companies that will help you with SMO (“social media optimization”).  On twitter we’re starting to see scam artists try and insert themselves into the conversation just as they used to do with Splogs.

And so the wheel turns round once again and there appears to be a new king of the hill – AOL Keyword > Yahoo Category Link  > Google Keyword Ranking > Social Site referral.

The jury’s still out as to which social site will send you more links but it’s looking more and more like Facebook. We used to have the Digg effect but no one talks about that anymore. Twitter certainly has the ability to send you a bunch of traffic in short order but the audience is still mostly the early-adopter set and any traffic will most likely be short-lived as the re-tweets scroll into the past.

Facebook, with 175 million users, certainly has the right broad-based distribution to broadcast a link and send back traffic. They have added new features such as FriendFeed’s “like” feature to make quick sharing easy. The Facebook Connect and Comments widgets also will help insert new links into Facebook for redistribution.

But while it’s great to get an influx of new visitors via a social site, it’s no good if it’s not a lasting reference. Has the pendulum swung to far the other way? There are some searches that work great on the real-time web, the latest viral video comes to mind, but others such as the listing for your local plumber just don’t work on something like twitter. Not that it’s worth anything but I still get a regular influx of traffic because I’m the #2 listing for insulting british slang even though the post is over three years old.

I think it’s fair to say that the social sites such as Facebook and Twitter will erode Google’s monopoly on online advertising. Google’s never been really good at building social sites and have shared in the growth of social networking in the past by providing the advertising engine for the most popular social sites. What happens with these sites start to go direct to the advertisers? Will Facebook advertsing revenues come from traditional cost-per-click/auction model or is some other type of model required to succeed in a social networking site?

Is twitter a directory or a utility? This is the question that Charles Hudson raises in his post The Database of Intentions is More Valuable than the Database of Musings. While investigating prospective business models, he raises good questions about the ability of a collection of “accumulated musings” to determine intent which is what is most valuable to advertisers.

But maybe advertising  is not the great revenue driver of the next generation of startups after all, at least not advertising as we know it. Maybe it’s just me but I feel a need to make sense of all the stuff we share with each other. There seems to be value in tapping into the pulse of the “now web” but the methods of pulling meaning out of the noise seem crude. Keyword searches? Is that the best we can do?

Something went wrong with the Intense Debate comments on last night’s post on Keywords and Meaning. It’s unfortunate because there were some really thoughtful responses to the post which I’ll repeat in this post because they are worth reading.

Todd writes:

Keyword extraction from Twitter could be cool, but may kill of serendipitous discovery, my favorite aspect of Twitter. If keywords or meta-categories are predetermined truly unique hawtness, unprecedented new things ( a Twitter specialty ) will just get deleted? That would be FAIL.

I wonder if more of a “people with attributes” are really what’s needed. Example, I do want to know what’s going on with the latest developments for Symbian operating system, particularly activity streams and address book stuff. Rather than rely on keyword extraction, I could just assign an attribute to your tweets…

twitteruser:iankennedy=novi

…I can be fairly assured news filtered by real humans, THEN assigned an attribute of my choosing will bring me some good results. A tag cloud of all tweets containing “symbian, activity stream, address book” would be noisy ( pollute with people asking each other for tech support? ), difficult to pull meaning from while drinking beer at my favorite bar.

Jonathan Strauss writes:

The TechCrunch post you cite was inspired by John Borthwick’s very interesting essay on how Google’s approach to content filtering breaks in the realm of what he calls the ‘Now Web.’ Like you say above: “Google’s PageRank, while valueable in sorting out the reputation and tossing the hucksters, is no good when applied to real-time news which is too fresh to build up a linkmap.”

In the (relatively) static web, the network nodes are pages and the endorsement actions are the links between them which are effectively permanent as well as public, and thus crawlable. In the Now Web, the network nodes are people and the endorsements are ephemeral share actions, the majority of which are not public or crawlable (i.e. email, IM, Facebook — what I call the ‘Deep Now Web’). And so, authority also takes on a different form from the aggregate view that PageRank provides to the personal measure of how much influence an individual has with her social network on a particular topic at a given moment.

I agree that we need to have a means of systematically capturing the newly important metadata of share actions and that it needs to be done at the point of sharing (see Jeff Jonas). But, I believe the more easily adopted (and thus ultimately more useful) taxonomy will be one of contextual metadata (i.e. who/what/when/where/why/how) rather than the more personal folksonomy/tagging approach you suggest.

There was also reactions via twitter from Kevin Marks:

1. 
   kevinmarks
   
   @iankennedy surely the point is that we refine meaning by deciding whom we follow
   
   
   
   16 Feb 2009 – 01:09
   
   from mobile web
   
2. 
   iankennedy
   
   @kevinmarks yes, to a certain extent, you are who you read. Is your OPML and Follow list the digital equivalent of DNA?
   
   
   
   16 Feb 2009 – 01:13
   
   from twhirl
   
3. 
   kevinmarks
   
   @iankennedy thats not what i meant; i meant we filter through others: we are each others’ media
   
   
   
   16 Feb 2009 – 01:17
   
   from mobile web
   

— this quote was brought to you by quoteurl

The act of sharing links, photos, or other metadata on social networks is an action, to a certain extent, that gesture is more interesting than the actual data itself. The fact that my usually dormant cycle racing friends are now extremely active on twitter these past few days as the Tour of California is on is as much an indicator of interest as the actual substance of their conversation.

Keywords are part of the picture – the complete context around who/when/where/why/how are just as important as the tidbit of data itself. The meta-data contains more clues than the data.

The cellphone is a rich source of meta-data which can be captured at the source, the moment of sharing. Feeding contexts captured from the cell phone would be a great way to add context to any act of sharing.  There are privacy concerns and ownership questions. There needs to be a real value demonstrated to the potential user before they give up some of this privacy. But that’s a topic for another post.

TechCrunch asks if twitter search gets us closer to being able to mine the world’s collective thoughts. We may be getting there as millions text their latest thoughts into their cellphones. With a simple text message, the hive mind has the potential for 4 billion nodes out in the real world (for comparison, the human brain has 100 billion neurons)

News junkies of the world turn to twitter as the latest source of raw, unfiltered information. Peering over the shoulder of various members of the House and Senate who twitter is a unique view into our government. What you see is a more intimate, human view of the people that make the news. Yet, how do you harness that noise and turn it’s output into information?

Twitter follows a long line of services which break through editorial filters, get at the source of a story so you can make your own judgements. Blogs occupied this space just a few years ago and real-time indexes such as Technorati rose to prominence as a way to get a jump on the news.

Sidenote: Alacra, admitting important news about companies breaks on the web, is launching Pulse which applies their analytics engine to extract company names from their hand-picked collection of 2,000 RSS feeds.

The need for speed is nothing new. Former Wall Street Journal newsman Craig Forman draws an arc that extends through the real-time newswires used in the financial world back to the pidgeons of Baron Reuter that delivered news of  Napoleon’s defeat at Waterloo. If there’s a way for someone to profit from the knowing something before anyone else, there’s always going to be people looking for a way to get at a scoop and others looking for a way to deliver.

We want to look to twitter for the scoops but we are doomed to learn the same lessons as we have in the past about authenticity. What we gain in speed and convenience, we lose in validation and measured fact-checking. Google’s PageRank, while valueable in sorting out the reputation and tossing the hucksters, is no good when applied to real-time news which is too fresh to build up a linkmap.

Working for Dow Jones in Tokyo, I would work with bankers and reporters who would use digital newswires to deliver them the latest news from around the world. As a systems engineer setting up their workstations, I would often be asked to set up their news filters to narrow the feeds down to something reasonable (the typical newswire delivers hundreds of stories an hour, most subsribe to several newswires). In the late-90′s the tools were crude and after getting frustrated by throwing in a few keywords, I would get called in to refine things using additional tools such as company ticker symbols, or a few undocumented codes from a taxonomy of subjects that varied from newswire to newswire.

Today the problem of information overload has spread to the greater population trying to derive value from the rushing torrent of updates coming out of twitter and facebook. How do I manage all this stuff and figure out what’s important? We use the tools we have but if you think about it, Google Trends and twitter search are just keyword searches with very crude resolution. We have a long way to go before such tools will let us tap into the collective mind.

Perhaps it’s time for a crude taxonomy for social networks to help sort out the types of messages flowing back and forth? Imagine if all your tweets, facebook messages, and friendfeed streams came pre-tagged with the following tags or categories?

• look at me, I’m doing something cool
• check this out, it’s funny
• books, movies, music, food, or sports
• this is touching and will change your life
• gadgets and meta, technology post about using technology
• weather and the natural world
• babies and kittens
• my obscure hobby
• breaking news, OMG!
• make money now!

What other categories would you add? Librarians of the world, what keywords would you put into your search filters to help grep out what goes where? Categorization is the first step towards ranking and with ranking you get useful filters.