everwas

a blog by Ian Kennedy

Month: December 2017

  • Reuters Tracer combs Twitter for news

    According to internal research, Reuters determined that 10-20% of news broke first on Twitter.

    Reuters, the news agency that first scooped its rivals with the use of carrier pigeons, is seeing good results from an algorithm to sift through Twitter (over 12 million tweets/day,  2% of total volume) to search for signal in the noise. Reuters Tracer is the system summarized in MIT’s Technology Review, How Reuters’s Revolutionary AI System Gathers Global News

    The first step in the process is to siphon the Twitter data stream. Tracer examines about 12 million tweets a day, 2 percent of the total. Half of these are sampled at random; the other half come from a list of Twitter accounts curated by Reuters’s human journalists. They include the accounts of other news organizations, significant companies, influential individuals, and so on.

    The next stage is to determine when a news event has occurred. Tracer does this by assuming that an event has occurred if several people start talking about it at once. So it uses a clustering algorithm to find these conversations.

    Of course, these clusters include spam, advertisements, ordinary chat, and so on. Only some of them refer to newsworthy events.

    So the next stage is to classify and prioritize the events. Tracer uses a number of algorithms to do this. The first identifies the topic of the conversation. It then compares this with a database of topics that the Reuters team has gathered from tweets produced by 31 official news accounts, such as @CNN, @BBCBreaking, and @nytimes as well as news aggregators like @BreakingNews.

    At this stage, the algorithm also determines the location of the event using a database of cities and location-based keywords.

    Once a conversation or rumor is potentially identified as news, an important consideration is its veracity. To determine this, Tracer looks for the source by identifying the earliest tweet in the conversation that mentions the topic and any sites it points to. It then consults a database listing known producers of fake news, such as the National Report, or satirical news sites such as The Onion.

    Finally, the system writes a headline and summary and distributes the news throughout the Reuters organization.

    Three recent events and their corresponding Tracer’s and Reuters alerts.

    More details (and attached screenshots) sourced from the paper, Reuters Tracer: Toward Automated News ProductionUsing Large Scale Social Media Data

  • The week that was

    Trying something new. A weekly wrap-up of sorts.

    The Austin headquarters for the Texas Health and Human Services Commission was overrun by “several hundred” rats.

    A Russian rocket launch went awry when the wrong directions were loaded into it’s flight plan and sent it to the bottom of the ocean.

    Meanwhile, the Voyager team dusted off the thruster controls to spacecraft now 21 billion kilometers away. After waiting 20 hours for the instructions to get there and waiting another 20 hours for the response, the damn thing worked!

    Finally, Mike Hughes’ rocket launch into the “atmosflat” where he planned to take photographs to prove the earth is flat and “expose the conspiracy” has been delayed until next week.

    Los Angeles’ free street paper, the LA Weekly, has a post asking if anyone can tell them who owns them. The next day, the paper answered it’s own question.

    And for another year, Kate Dries has still not made it into Hendrik Pöhler’s Erotic Carp Fishing Calendar. The Amazon reviewers are not pleased.