This did not occur to me. As an algorithm gets better at recommending content that matches and reinforces what a community is looking for, the negative complaints go down which makes it harder for someone outside (such as platform moderators) the filter bubble from detecting these closed communities in the first place.
The algorithm is doing what it was designed to do but without any moral compass, its overall contribution to society is questionable.
Here’s someone who worked on the YouTube algorithm commenting on this (emphasis mine).
Using recommendation algorithms, YouTube’s AI is designed to increase the time that people spend online. Those algorithms track and measure the previous viewing habits of the user—and users like them—to find and recommend other videos that they will engage with.
In the case of the pedophile scandal, YouTube’s AI was actively recommending suggestive videos of children to users who were most likely to engage with those videos. The stronger the AI becomes—that is, the more data it has—the more efficient it will become at recommending specific user-targeted content.
Here’s where it gets dangerous: As the AI improves, it will be able to more precisely predict who is interested in this content; thus, it’s also less likely to recommend such content to those who aren’t. At that stage, problems with the algorithm become exponentially harder to notice, as content is unlikely to be flagged or reported. In the case of the pedophilia recommendation chain, YouTube should be grateful to the user who found and exposed it. Without him, the cycle could have continued for years.
This is a shameless pitch for a plugin to WordPress my company just published but there are also broader ideas proposed here and I would love your feedback.
SmartNews is a mobile news aggregation app. The backend tech is pretty nifty. It uses machine learning and what we call a “discovery algorithm” to expose users to new points of view that they might not see if they are using a social network or personalized news service to read their news. You can read more about that stuff here, that’s not what this blog post is about.
SmartNews aggregates news from our partners. If we have no news, we have nothing for our users to read. While it’s possible to crawl the web and pull in stories as we find them on the open web, we would prefer a relationship with each of our publishing partners so they send us their articles and feel in control of how their content is used by SmartNews. We want our partners to feel as if the SmartNews app is an extension of their CMS. If our partners are not successful, neither will SmartNews.
While we do send traffic to our publishers (lots of it) that’s not the only benefit we offer. We have architected the product to offer a snappier, native view (think of Safari or Chrome’s reader mode) of the articles. Because this view is hosted on our app, our users can read while offline. The SmartView page in SmartNews was designed to serve the subway commuter in Tokyo where signals were spotty.
But we wanted to make sure publishers had a benefit when readers chose to read their articles via the SmartView page. Thus the SmartFormat feed spec was born. SmartFormat is a simple variant of the RSS standard with a couple new elements to provide greater portability of not only a publisher’s articles but also their advertising and analytics.
The <snf:advertisement> element lets publishers provide an ad tag which we run on the SmartView page. Because this is the publisher’s ad tag, the publisher keeps 100% of the revenue.
The <snf:analytics> element lets the publisher send along the analytics bug so they can include SmartView pageviews in the total pageviews that they see on their dashboard.
Combined, both the <snf:advertisment> and <snf:analytics> allow for portability of not only content but also advertising. Now when a publisher distributes a full text feed to SmartNews they also are distributing the monetization and analtyics footprint as well. While other platforms require you to opt in to revenue shares on the platform’s advertising and analytics, SmartNews lets you use and optimize your own, dynamically, on a feed endpoint you control.
Now to the fun part. If you’re running on WordPress, we have a simple plugin that will open up two text boxes, one for and one for and will build a SmartFormat feed compliant with the SmartFormat feed spec. In order to get distribution on SmartNews, all you have to do is apply to be a publisher on SmartNews, install the plug-in, then you’re ready to go!
As for the broader proposal, I was curious (and I could very well be looking in the wrong places) why no one has attempted to extend RSS in this way before? Feedburner had something where they injected Google Ads into their RSS feeds but it never really took off because those ads only ran in the feed or feedreader, not on the downstream aggregation sites or platforms. As much as I am loath to try and extend a standard, wouldn’t it benefit publishers to have a place where they can add their ad tag, analytics scripts, and even subscription CTAs so that the business travels along with the editorial?
<snf:advertisement> and <snf:analytics> work great for SmartNews but what about extending it for others? The more platforms that accept this extended feed, the more incentive there is for publishers to create these feeds. Seems like the classic win-win all around. Besides the bureaucratic lift of trying to extend a “standard” such as RSS am I missing something?
April Fool’s Day is always a busy day at SmartNews as our news discovery algorithm get overly excited with all the cool, interesting and unique news released that day. Here’s a running log of what we’ve had to gently remind the algorithm is actually just a joke.
After watching the HBO documentary about Elizabeth Holmes and Theranos, I’m all in everything about how she pulled the wool over so many eyes and got so many people (mostly older men points out my wife) to hand over their wallets. It’s a cautionary tale for the fast & loose culture of Silicon Valley that is starting to bump up against more traditional and heavily regulated industries such as medicine, banking, and transportation. Think of CRISPR, cryptocurrency, Space-X and Tesla as examples of projects that need to take extra care as they innovate so as not to lose the public’s trust.
Here’s my reading list:
Bad Blood – by John Carreyrou – the book by the guy that broke the story
It’s been twelve years since I last was at SXSW – time flies – I wroteaboutitthen so for continuity’s sake I might as well share some highlights from this year.
What I learned
Following the success of the New York Times’ The Daily podcast, the Grey Lady will dip its toe into broadcast TV with a weekly show called, you guessed it, The Weekly. Following the same formula, The Weekly will go deep on one story and, in 30 minutes, bring viewers the broader picture of how a news article comes together and convey things via video that cannot be expressed in print. The Weekly will air sometime in June on FX.
I saw Senator Amy Klobuchar expertly deflect a pointed question about her comb/salad incident, brushing it off as, a mom thing and moving on to a broader point about expecting the best from herself and her staff.
I saw the intersection of two writing cultures in the The New Yorker but Make it Internet panel. I learned that while you “post” something online, you “close” it in print. The young writers who post regularly online shared their experiencing getting edited for longer pieces in print where they still use words such as “teen-ager” and “web site” is written with two words.
Lots of talk about Artificial Intelligence and how it will impact journalism and storytelling (film). Sony demonstrated software that can write music. Many media outlets admitted to deploying AI to write summaries of high school sports or financial earnings reports and some are using AI to look for anomalies in large datasets for primary investigation. Several funny stories shared of flat-footed AI recommendation systems, “You’re here for the parenting, stay for Iraq!” and how one-dimensional AI can be (intense interest in how to raise a baby does not evolve into how to raise a child).
This reminds me of what Kathy Sierra said about apps in 2007 at my first SXSW,
All apps have a Asperger’s Syndrome. They cannot pick up on visual queues from their users such as when someone is angry, frustrated, or confused. If a user has these reactions to software, they quickly fall below the suck threshold.
Kathy Sierra, SXSW Keynote, 2007
We must always remember that any Artificial Intelligence was born in the mind of someone, somewhere and has the biases of that person or team of people baked right into it and remains static, not taking in the inexplicit contexts of its environment,
What I experienced
SXSW is much bigger than it was 12 years ago. While I was last in Austin for the annual Online News Association meeting which took over the JW Marriott, SXSW sprawled over not only the JW Marriott but also the Austin Convention Center and Fairmont Hotels with pavilion and bar takeovers all across town. I don’t think Rainy Street was a thing back in 2007. It is not only the Interactive festival but also gathering for music and new film releases which I was somehow oblivious of the last time. I thought I could plan ahead and pick and chose what I wanted to see but soon realized that would be an exercise in frustration and that it was better to just have a rough idea of a few things you wanted to seek out and then let the winds of chance take you from one thing to the next.
I wandered into the Japan pavilion where the Japanese comedian, Yuriyan Retriever (ゆりやんレトリィバァ) narrated a guy who was creating a stop-motion video on his iPad in front of the most-patient audience I had ever seen. I’ve written about her before, she’s got a great bit on the Oscar’s acceptance speech.
SXSW with your son
Tyler had the week off from college so I suggested he come down from Boston to Austin and take advantage of the hotel room and get to know the city and take in the festival. Unfortunately he came down with a nasty cold for the first two days but we did get a chance to walk (and scooter) around a bit on Monday and Tuesday. I loved just walking around and experiencing things through his eyes.
While Twitter and Foursquare famously launched at SXSW – this year was the coming out party for electric scooters. The grab-and-go rentals were everywhere with competing pods from Uber/Jump, Lyft, Bird, Lime, and others. In my unofficial survey, the late-model Lyft scooters had the most “umph” to them.
The Interactive badge got me into all the media sessions that I wanted to attend and they had this new SXXPress system that, like a Disneyland FastPass, let you jump the line and grab a guaranteed seat. Also discovered a hack to get into the more popular talks was to attend the talk in the time slot prior and squat in a seat and make small talk with your neighbors. This hack probably won’t work next year as many were catching on to this as the festival went on.
Be sure to check out a film or band in the evenings. While the Interactive lanyard will not give you first dibs or get you into the big name premiers, it will get you in for free if space is available. I caught the premier of the excellent documentary, Mr. Jimmy and the Welsh band Novo Amor.
I booked my hotel very late but was able to score a room at the Marriott Residence Inn at The Domain which has a Costco-sized Whole Foods which is a wonder to behold. The Domain is way out of town and costs $20+ in a Lyft but discovered a tram station 10-minute’s walk away that only costs $3 and takes you right into town, next to the Convention Center. Riding the tram I learned:
West Austin is where rich housewives gather for yoga sessions an Kombucha exchange parties,
People in Austin get discounted passes to SXSW, many also volunteer which gives them access to events when they’re not working,
The Domain is a soulless retail development and signaled to many locals the end of Austin
Yeah the BBQ is awesome. Iron Works, Coopers and all the other places – there are tons of places to eat. If you’re looking for something else, Gus’s Chicken (it’s world famous ya know) is good for lunch and the Texas Chili Parlor is a must do when finishing out a night on the town.
Kevin Kelly is the master of optimistic projection. Drawing a line Just as VR and AR start their decent into the trough of disillusionment, Kevin Kelly and Wired comes out with a cover story that looks beyond the immediate applications and imagines the opportunities in the great beyond.
The first big technology platform was the web, which digitized information, subjecting knowledge to the power of algorithms; it came to be dominated by Google. The second great platform was social media, running primarily on mobile phones. It digitized people and subjected human behavior and relationships to the power of algorithms, and it is ruled by Facebook and WeChat.
We are now at the dawn of the third platform, which will digitize the rest of the world. On this platform, all things and places will be machine-readable, subject to the power of algorithms. Whoever dominates this grand third platform will become among the wealthiest and most powerful people and companies in history, just as those who now dominate the first two platforms have. Also, like its predecessors, this new platform will unleash the prosperity of thousands more companies in its ecosystem, and a million new ideas—and problems—that weren’t possible before machines could read the world.
Information – People – Places & Objects. Who is going to figure out search on the mirrorworld? The social graph was the innovation that organized the social web. What will be the innovation to organize the world of places and objects? Voice UI?
When asked what we’re trying to build at SmartNews, I sometimes explain it with a department store metaphor. When algorithms are applied to online shopping, they are optimized to show you exactly what you are looking for. Amazon and Netflix are famous for perfecting the “others-that-bought-what-you-bought-also-bought-this” algorithm to great effect.
If you’re looking for a red sweater, they will show you the best red sweater. But that only works if you’re looking for a red sweater.
What if you are just browsing around? What if you are wandering around, looking for inspiration, not sure what exactly you want? Japan has a some of the world’s best department stores. They are wonderful at curating interesting things from around the world and introducing them to the sophisticated urban consumer. They have a long tradition of doing this.
Ginza has a new department store called Ginza Six that opened a couple years ago. Besides these wonderfully packaged $20 tubes of toothpaste from Italy (above) you can also check out the completely impractical spiked boots (below). It’s a curious browser’s delight.
What I’m getting at is that SmartNews is trying to re-create the Japanese Department Store experience online with news. I keep talking about “hidden gems” and this is what I mean. You never know what you’re going to find but we’ll do our best to make an algorithm which is optimized for finding something serendipitously, interesting.
Anyway, more on this in this interview with my boss, Rich Jaroslovsky, below as he describes the concept of “personalized discovery.”
The key is personalized discovery. Of course that also means sometimes users will see stories they don’t like. I’m generalizing, but a conservative might see a Mother Jones story, or a liberal might see something from Fox News.
That’s probably the biggest complaint we get: That the stories are all “left wing,” or “right wing.” But when we look at studies about user engagement in news apps, our audience is far more engaged than that of any other news app.
My argument is that those are two sides of the same coin. As a 40-year journalist, when all you see is stuff you already know or already think you’re interested in… news gets boring. There’s no serendipity. You don’t get to learn anything new. You don’t get to discover.
That’s the textbook definition of a filter bubble.
As a commuter, I have two blocks of time going to and from the office when I am not able to read. During these times, I listen to podcasts as a way to get a deeper perspective on the news of the day or learn something new.
Most podcast apps ask you to subscribe or follow podcasts you like and push new episodes to you each time you open the app. As you add new shows to your “feed” you inevitably get overwhelmed with a backlog of unlistened episodes which discourages you from finding new episodes.
Breaker was the first app I found that solved this problem by adding a social network to the platform. Breaker pays attention to their users’ behavior of listening, liking, and commenting to determine how to rank the best episodes. Instead of subscribing to shows, you listen to episodes. The selection is more granular and has more variety. As with other social networks, because you get a daily feed which you graze through, the pressure to clear your queue is removed.
I believe podcasts are an important component of the media diet of anyone that is interested in current events. The intimacy of hearing a primary source tell their story is unique to audio and I have been eager to find a way to add podcasts in a meaningful way to the SmartNews app. My initial attempt was to leverage SmartNews’ algorithm to locate and feature podcasts in a single channel but the results were never quite right.
The SmartNews algorithm is optimized for text so the selection and ranking of stories didn’t work well.
Sometimes the audio would stop when you put the app into the background or your phone went into standby mode.
Each episode’s page featured a play button in a different location on the page so the user had to hunt around to find it.
The Breaker app solved all these problems and I was eager to find a way to introduce elements of what Breaker had to offer to SmartNews users.
I met with Erik Berlin, co-founder of Breaker, and we talked about how SmartNews might be able to integrate Breaker’s social ranking of stories to create a channel that would feature the best episodes to SmartNews users in a way they could sample what they had to offer easily, within the SmartNews app.
The Breaker team worked on a feed of popular stories on their network that we could integrate into SmartNews. They built an episode page with a built in player that continued to play in the background and featured a prominent play button that was placed in a consistent location for quick and easy access. The idea was to give SmartNews iOS and Android users a place to go to discover the best audio stories of the day to introduce them to this new aspect of story-telling.
I hope you enjoy the new Podcast channel on SmartNews (if you’re on mobile and don’t have SmartNews, you can download it here). If you find yourself embracing podcasts as I do, download the Breaker app to unlock extra features where you can share and discuss your favorite episodes on their social network.
Oh, and if you’re curious about what I’m listening to and enjoy, follow me on Breaker at @iankennedy
The Citizen app is available in NYC and the SF Bay Area. The app is designed to alert you when crime happens nearby with location-based push notifications. But, because the platform is crowdsourced, it exposes all the idiosyncratic definitions of “crime” that you would expect from its voyeuristic users.
Here are some of my favorites. If you have a good one, send it my way and I’ll add it.
We used to have Florida Man stories. Now we have Citizen.