Google would (just) beat Bing at Jeopardy!

January 27, 2011

Google would (just) beat Bing at Jeopardy!It seems unlikely that Google and Microsoft’s battles would ever spill over into a game of Jeopardy! But if it did, then Google would be a narrow winner.

So says Stephen Wolfram, the physicist behind the Wolfram Alpha site that aims to answer questions rather than simply provide search results.

He was responding to the latest developments in the ongoing attempts by IBM to produce a computer capable of winning the TV quiz show. The company’s “Watson” machine recently competed in two games against former human champions, to be aired on Feb. 14 – 16.

Although the machine had played dozens of practice games against other human experts, a warm up for the TV contest earlier this month was the first held before a public audience and thus with the result public knowledge. Watson defeated Ken Jennings and Brad Rutter, though whether that means IBM’s computing is truly superior or the humans were just out of practice remains up for debate.

Wolfram has a natural interest in the subject being that his site and Watson both carry out a similar role: attempting to transform a search query into a definitive question and then provide a definitive answer. (Watson, of course, has the handicap that it must perform the process in reverse.)

In an effort to work out how much of the Jeopardy task is simply down to the quality of a database of facts (ie the index used by a search engine) and how much is down to the trickier task of figuring out the actual question, Wolfram took a sample of Jeopardy! clues and fed them into search engines, then checked whether the correct answer (or rather correct question) appeared on either the highest ranked page, or the pages linked to by the first page of results.

When it came to the first page, Google got 69 percent correct, just beating Ask with 68 percent and Bing on 63 percent. When it came to the first result, which is arguably closer to the Jeopardy! scenario (where giving 10 possible answers is usually frowned upon), Google remained in top spot on 66 percent, narrowly ahead of Bing on 65 percent. Ask fell back to 51 percent (so no TV appearance for you, Jeeves), with third place snatched by the lesser known Yandex on 58 percent.

To put that into context, the average human contestant gets 60 percent of answers correct, while champion Ken Jennings has a record of 79 percent. Of course, having the necessary information available is one thing, while processing the information to pick out the answer (or rather question) is another.

Another notable aspect of Wolfram’s figures is that Wikipedia came in last position in both categories, scoring 23 percent among the first page of results and 29 percent on the highest result only (though it doesn’t appear to make sense that the latter result would be higher.) My suspicion is that this isn’t so much to do with insufficient/incorrect information on pages as it is the comparatively poor mechanics of the Wikipedia search engine.

Be Sociable, Share!

6 Responses to “Google would (just) beat Bing at Jeopardy!”

  1. Stan:

    In any case, none of the search engines can tell you what the correct answer is to a question so none of them could compete on Jeopardy (a la Watson).

    In all cases, search engines require someone to look through the documents to see if an answer (right or wrong) is in the returned documents.

    Watson determines the answer and the confidence that it’s correct. A pretty huge difference.

  2. John:

    I think those bing numbers are wrong, 63% for first page and 65% for first entry.

  3. watsonrocks:

    Neither would beat IBM’s Watson unless you got from Google or Bing precise answers, worded in Jeopardy queston form, (e.g. Who is John Lister?) in five seconds or less.

  4. bo2:

    John, I think your figures are closer.

  5. black ops 2:

    I agree with Stan. Good point.

  6. Eloy Rosentrance:

    You can certainly see your enthusiasm in the article you write. The sector hopes for more passionate writers like you who are not afraid to say how they believe. At all times go after your heart.|

Leave a Reply:


Recent stories

Featured stories

RSS Windows news

RSS Mac news

RSS iPad news

RSS iPhone & Touch

RSS Mobile technology news

RSS Tablet computer news

RSS Buying guides

RSS PS3/Wii/Xbox 360

RSS Green technology

RSS Photography

Featured Content

Archives

Copyright © 2014 Blorge.com NS