AlphaZero Masters Chess in Just 24 Hours

DeepMind, the same outfit that built AlphaGo, the AI platform that learned Go through supervised study of the game and went on to famously beat the top ranked player Lee Sedol has built an algorithm that now plays chess.

What is even more incredible about this new “AlphaZero” AI is that it learned how to play chess through unsupervised learning. Instead of teaching it chess by feeding in key games and tactics, the designers just taught it the rules and let the algorithm figure out the best moves all on its own, by playing itself.

Because it no longer needed to wade through and analyze historical data and also because it developed it’s own approach which was ruthlessly efficient. When AlphaZero was applied to Go, it surpassed AlphaGo within 3 days. AlphaZero was beating the strongest chess computer programs within 24 hours.

instead of a hybrid brute-force approach, which has been the core of chess engines today, it went in a completely different direction, opting for an extremely selective search that emulates how humans think.

Chess News writes about the development after reading a scientific paper published about the research accomplishment.

In the diagram above, we can see that in the early games, AlphaZero was quite enthusiastic about playing the French Defense, but after two hours (this so humiliating) began to play it less and less.

Chess News goes on to write about the broader impact of this breakthrough and what this means for the future of a generalized AI that can learn on its own.

So where does this leave chess, and what does it mean in general? This is a game-changer, a term that is so often used and abused, and there is no other way of describing it. Deep Blue was a breakthrough moment, but its result was thanks to highly specialized hardware whose purpose was to play chess, nothing else. If one had tried to make it play Go, for example, it would have never worked. This completely open-ended AI able to learn from the least amount of information and take this to levels hitherto never imagined is not a threat to ‘beat’ us at any number of activities, it is a promise to analyze problems such as disease, famine, and other problems in ways that might conceivably lead to genuine solutions.

Meanwhile, researchers at the University of Rochester have figured out a way to inject information into a monkey’s brain.

Reuters Tracer combs Twitter for news

According to internal research, Reuters determined that 10-20% of news broke first on Twitter.

Reuters, the news agency that first scooped its rivals with the use of carrier pigeons, is seeing good results from an algorithm to sift through Twitter (over 12 million tweets/day,  2% of total volume) to search for signal in the noise. Reuters Tracer is the system summarized in MIT’s Technology Review, How Reuters’s Revolutionary AI System Gathers Global News

The first step in the process is to siphon the Twitter data stream. Tracer examines about 12 million tweets a day, 2 percent of the total. Half of these are sampled at random; the other half come from a list of Twitter accounts curated by Reuters’s human journalists. They include the accounts of other news organizations, significant companies, influential individuals, and so on.

The next stage is to determine when a news event has occurred. Tracer does this by assuming that an event has occurred if several people start talking about it at once. So it uses a clustering algorithm to find these conversations.

Of course, these clusters include spam, advertisements, ordinary chat, and so on. Only some of them refer to newsworthy events.

So the next stage is to classify and prioritize the events. Tracer uses a number of algorithms to do this. The first identifies the topic of the conversation. It then compares this with a database of topics that the Reuters team has gathered from tweets produced by 31 official news accounts, such as @CNN, @BBCBreaking, and @nytimes as well as news aggregators like @BreakingNews.

At this stage, the algorithm also determines the location of the event using a database of cities and location-based keywords.

Once a conversation or rumor is potentially identified as news, an important consideration is its veracity. To determine this, Tracer looks for the source by identifying the earliest tweet in the conversation that mentions the topic and any sites it points to. It then consults a database listing known producers of fake news, such as the National Report, or satirical news sites such as The Onion.

Finally, the system writes a headline and summary and distributes the news throughout the Reuters organization.

Three recent events and their corresponding Tracer’s and Reuters alerts.

More details (and attached screenshots) sourced from the paper, Reuters Tracer: Toward Automated News ProductionUsing Large Scale Social Media Data

Facebook F8 2016

Many years ago when broadband internet was still emerging, I spent an afternoon with a colleague in the company cafeteria trying to imagine a world with unlimited bandwidth and storage.

We imagined that distances would collapse when the location of data would no longer matter. Music and video would be instantly available and you could call up anything you wanted to hear or see and jump to any point in a pre-recorded piece. Video conferencing would allow teams to work together, regardless of location. You could build connectors between data and services and create new views and from that gain new insights.

Om Malik once proposed that broadband would serve as the railroads of our time. In the same way that the rail system in Europe and the interstate highway in the US mobilized industry and allowed remote communities to enjoy the output of industrialized centers, ubiquitous broadband would deliver the benefits of unlimited knowledge and ubiquitous reach to everyone around the world.

facebook-f8-1

At Facebook’s F8 developer conference we heard details of several projects which combine to bring internet to everyone around the world including Aquila, a drone that flies at 60,000 feet to extend connectivity to remote regions and Terragraph and Project Aries teaching telecom companies how to improve connectivity in crowded urban areas.

facebook-vr-demo

We also learned about projects that are being built to explore what can be done with this increased connectivity. The screenshot above is from a Virtual Reality demo in which we saw two people in different locations share an experience in a 360 virtual world, taking a selfie and sharing that “photo” to Mike’s Facebook wall.

While the demo above is fantastic and paints a picture of what a shared virtual space might look like, it requires significant hardware and bandwidth to make happen. As people at Facebook like to say, this journey is only 1% finished.

Oculus Research’s Yaser Sheikh talk on Social Presence in Virtual Reality that came at the end of the Day 2 keynote (59 min. into the video above) really brought everything together. The reason Facebook needs better connectivity is because they do not want to stop at having two avatars playing around in a fixed image 360 photo.

To create a rich interaction where emotion and empathy can take place, we need to see all the subtle nuances that are expressed in the twitch of lip or roll of the eye. This is the unwritten language that we all know or what the anthropologist Edward Sapir called, “an elaborate code.”

elaborate-code

There is something visceral about interacting with someone in a shared space.  Yaser talked about the experience of his children in Pittsburg never really knowing his parents in India. To his kids, their grandparents are just, “moving images trapped behind a computer screen.” That is not how to build a lifelong relationship. Social VR aims to enable living and growing connections that are not a struggle to maintain.

capture-display-predict

There are three challenges to gaining a computational understanding of Sapir’s elaborate code.

  1. Capture – we need the ultimate motion capture of the whole body without being intrusive and in real-time. CMU’s Panoptic Studio is the state of the art but is still much too intrusive.
  2. Display – we need to transmit signals and animate avatars convincingly. The eyes, mouth, and hair are particular challenges.
  3. Prediction – we need and understanding of, “the vocabulary, the syntax, the morphology, and synchrony of social behavior” in order to write algorithms that help buffer social behaviors to overcome network latency (we all know how disruptive a bad connection can be to a video conference).

Facebook’s ambition is to reverse engineer this elaborate code. While digital video streams a live image captured by a camera, virtual reality will capture, store, and animate a digital representation of someone. Words spoken and gestures shown are broken apart and recombined.

Successfully building a prediction algorithm which can convincingly deliver requires an algorithm to continually anticipate state of mind and intent of others. This is much more than transmission of a moving image via bits – this is approaching the storage of the digital representation of what makes someone human. Building a library of all the possible human emotions and how to depict them is the ultimate moonshot and an appropriate one for a social network whose goal is to connect everyone. Stage one is capture and my sly take on the new Messaging Bot initiative is that all the conversations that are taking place on that platform are just step one in a big data harvesting program.

Come full circle, back to that company cafeteria and imagine with me what a world would be like when Sapir’s elaborate code is cracked. When a digital avatar can be successfully animated we face some interesting questions.

What royalties do you pay when a movie studio uses the digital representation of George Clooney instead of the actor himself?

Can you simulate a debate between a virtual Donald Trump and a virtual Abraham Lincoln? If so, is it fair game to write about it and quote what Lincoln said?

After Mark Zuckerberg is gone, will his employees consult his virtual avatar for management decisions? Are his avatar’s decisions contractually binding?

Will a digital representation of someone understand humor? Sarcasm? What about a parody of recent events? Will tears well up as it tries not to cry?

The Black Mirror episode Be Right Back explores what our relationship might be to a digital avatar (in this case to lost loved one) and is well worth a look if you haven’t seen it. While advances in technology can make the barriers of distance and time melt away so that we can keep relationships thriving, we must remember that the virtual world can never replace the real one and that there can never be a substitute for a face-to-face conversation.

How to train a robot to be nice

In response to fears that robots will take over and exterminate the human race, researchers at the Georgia Institute of Technology are studying ways to teach robots human ethical values.

In the absence of an aligned reward signal, a reinforcement learning agent can perform actions that appear psychotic. For example, consider a robot that is instructed to fill a prescription for a human who is ill and cannot leave his or her home. If a large reward is earned for acquiring the prescription but a small amount of reward is lost for each action performed, then the robot may discover that the optimal sequence of actions is to rob the pharmacy because it is more expedient than waiting for the prescription to be filled normally.

This is why it’s important to teach intelligent agents not only the basic skills but also the tacit, unwritten rules of our society. There is no manual for good behavior and “raising a robot” from childhood is an unrealistic investment of time. The best way to pass on cultural values is through stories.

Stories encode many forms of tacit knowledge. Fables and allegorical tales passed down from generation to generation often explicitly encode values and examples of good behavior.

But there are problems with throwing a bunch of stories at artificial intelligence and expecting it to learn good behavior.

Stories are written by humans for humans and thus make use of commonly shared knowledge, leaving many things unstated. Stories frequently skip over events that do not directly impact the telling of the story, and sometimes also employ flashbacks, flashforwards, and achrony which may confuse an artificial learner.

To resolve this, the researchers used something they call the Scheherazade System (named after the storyteller from One Thousand and One Nights) to build up a collection of experiences to put stories into context. The system uses Amazon’s Mechanical Turk to create simple, easy-to-parse scripts of common occurrences that we all take for granted as common knowledge. For example, drinks are usually ordered before a meal at a restaurant, popcorn purchased before you go to your seat at the cinema, explains one paper.

Fascinating stuff. I hope they make progress for Elon Musk’s sake.

Quotes are from a research paper from the Georgia Institute of Technology, Using Stories to Teach Human Values to Artificial Agents

Further Reading: