To me, it seems as if this new paper is demonstrating the results of Google’s recent comments on using “thought vectors” to train neural networks (as we blogged about a few weeks back). The researcher quoted in The Guardian Link, Geoff Hinton, intimates that Thought Vectors could be a breakthrough in AI. If I’m reading these papers right, it seems like “Thought Vector” is synonymous with “Sequence”. The Geoff fellow is mentioned in the papers’ ack section, so I’m guessing these are the same things. Or maybe there are two different teams in Google doing similar shit and not talking to each other (it wouldn’t be the first time–ahem, news/reader/wave/mail/plus).
What about the movie sub-titles, you ask? Well, the experiment used movie sub-titles as a training corpus. The paper stated: “On a noisy open-domain movie transcript dataset, the model can perform simple forms of common sense reasoning. As expected, the conversations are filled with a roller-coaster of emotions.” OK, I might have added that list bit. Anyhow, the results (snarky articles aside) seem pretty impressive:
Human: how many legs does a cat have ?
Machine: four , i think .
Human: how many legs does a spider have ?
Machine: three , i think .
Human: how many legs does a centipede have ?
Machine: eight .
Nailed it, Google. Nailed it.