Watson computer vs Humans on Jeopardy --- a set up for advertisement?
Somehow watching these games left a sour feeling of a setup.
Jeopardy is probably not a really interesting challenge since it is:
- about factoid questions ( Who, What, When, Where) which are the easiest in QA.
- questions are often of very structured type (what is asked, often appears after the key word "THIS")
- lots of factual information in the question -- just googling many questions would give the right answer at the top
Competition was unfairly organized:
-computer got the text of the questions immediately and could use all the time it takes the host to read the questions for thinking.
-computer had an "instant buzz" advantage over humans. Computer picked its lead on questions for which humans also knew the answers, they just could not beat the computer in the speed of pressing the button.
The amount of IBM advertisement during the show really went over the top. There is no way to check whether the contestants were not paid to fail, but it could be. Comparing this challenge to the "Deep Blue vs Kasparov" challenge is really far fetched, since these contestants had no reputation to loose. While for Kasparov everything was at stake.
A more interesting challenge would be to:
-give equal time to humans and computer (ex. 10 sec, 60 sec) and checking correctness of the answers not the speed.
-pose questions which require at least several queries with qoogle (i.e. require some knowledge extraction and analysis)
-try to solve "Chto-Gde-Kogda" style questions, not simple factoids. Checking reasoning rather than information.
Jeopardy is probably not a really interesting challenge since it is:
- about factoid questions ( Who, What, When, Where) which are the easiest in QA.
- questions are often of very structured type (what is asked, often appears after the key word "THIS")
- lots of factual information in the question -- just googling many questions would give the right answer at the top
Competition was unfairly organized:
-computer got the text of the questions immediately and could use all the time it takes the host to read the questions for thinking.
-computer had an "instant buzz" advantage over humans. Computer picked its lead on questions for which humans also knew the answers, they just could not beat the computer in the speed of pressing the button.
The amount of IBM advertisement during the show really went over the top. There is no way to check whether the contestants were not paid to fail, but it could be. Comparing this challenge to the "Deep Blue vs Kasparov" challenge is really far fetched, since these contestants had no reputation to loose. While for Kasparov everything was at stake.
A more interesting challenge would be to:
-give equal time to humans and computer (ex. 10 sec, 60 sec) and checking correctness of the answers not the speed.
-pose questions which require at least several queries with qoogle (i.e. require some knowledge extraction and analysis)
-try to solve "Chto-Gde-Kogda" style questions, not simple factoids. Checking reasoning rather than information.