It seems unlikely that Google and Microsoft’s battles would ever spill over into a game of Jeopardy! But if it did, then Google would be a narrow winner.
So says Stephen Wolfram, the physicist behind the Wolfram Alpha site that aims to answer questions rather than simply provide search results.
He was responding to the latest developments in the ongoing attempts by IBM to produce a computer capable of winning the TV quiz show. The company’s “Watson” machine recently competed in two games against former human champions, to be aired on Feb. 14 – 16.
Although the machine had played dozens of practice games against other human experts, a warm up for the TV contest earlier this month was the first held before a public audience and thus with the result public knowledge. Watson defeated Ken Jennings and Brad Rutter, though whether that means IBM’s computing is truly superior or the humans were just out of practice remains up for debate.
Wolfram has a natural interest in the subject being that his site and Watson both carry out a similar role: attempting to transform a search query into a definitive question and then provide a definitive answer. (Watson, of course, has the handicap that it must perform the process in reverse.)
In an effort to work out how much of the Jeopardy task is simply down to the quality of a database of facts (ie the index used by a search engine) and how much is down to the trickier task of figuring out the actual question, Wolfram took a sample of Jeopardy! clues and fed them into search engines, then checked whether the correct answer (or rather correct question) appeared on either the highest ranked page, or the pages linked to by the first page of results.
When it came to the first page, Google got 69 percent correct, just beating Ask with 68 percent and Bing on 63 percent. When it came to the first result, which is arguably closer to the Jeopardy! scenario (where giving 10 possible answers is usually frowned upon), Google remained in top spot on 66 percent, narrowly ahead of Bing on 65 percent. Ask fell back to 51 percent (so no TV appearance for you, Jeeves), with third place snatched by the lesser known Yandex on 58 percent.
To put that into context, the average human contestant gets 60 percent of answers correct, while champion Ken Jennings has a record of 79 percent. Of course, having the necessary information available is one thing, while processing the information to pick out the answer (or rather question) is another.
Another notable aspect of Wolfram’s figures is that Wikipedia came in last position in both categories, scoring 23 percent among the first page of results and 29 percent on the highest result only (though it doesn’t appear to make sense that the latter result would be higher.) My suspicion is that this isn’t so much to do with insufficient/incorrect information on pages as it is the comparatively poor mechanics of the Wikipedia search engine.