{"id":1838,"date":"2011-11-15T23:41:08","date_gmt":"2011-11-15T23:41:08","guid":{"rendered":"http:\/\/dalelane.co.uk\/blog\/?p=1838"},"modified":"2012-01-16T23:18:58","modified_gmt":"2012-01-16T23:18:58","slug":"the-problem-with-jeopardy","status":"publish","type":"post","link":"https:\/\/dalelane.co.uk\/blog\/?p=1838","title":{"rendered":"The problem with Jeopardy!"},"content":{"rendered":"

The I’ve-been-blogging-for-five-years-but-am-still-paranoid-about-work-related-posts-being-misrepresented disclaimer:<\/strong>
\nI do work for IBM as a developer on Watson but that role doesn’t extend to this blog. My responsibilities for Watson are limited to writing code… so any ramblings here are my personal views, and not necessarily representative of IBM’s positions, strategy or opinions.<\/p><\/blockquote>\n

The first time most people saw or heard of Watson was on “America’s favourite quiz show”, Jeopardy! <\/p>\n

And that association seems to have stuck. For the moment, at least, Watson’s identity seems to bound up with Jeopardy’s. It’s “the computer system that won on Jeopardy!”<\/p>\n

<\/a><\/p>\n

I’m not sure that’s entirely a good thing. <\/p>\n

Jeopardy! was a great demonstration of Watson’s capabilities in a lot of ways. It showed the breakthroughs in interpreting natural language, the breakthroughs in coming up with evidence and rationale and identifying the level of confidence in it’s answer’s, and the power of machine learning systems. <\/p>\n

But… it does make it easy to misunderstand the future potential of the technology. <\/p>\n

The Jeopardy! challenge was about man vs machine<\/a>. <\/p>\n

It was about a machine trying to beat the humans. <\/p>\n

A machine that had to work entirely by itself to come up with answers. <\/p>\n

And if it couldn’t come up with an answer immediately, or when didn’t have enough confidence in the answer that it did come up with, it’d give up, stay silent and not buzz in. <\/p>\n

None of that describes the way a future Watson could work. In my view, Watson should be about man and machine working together. <\/p>\n

Watson could help to make it’s users smarter. <\/p>\n

When a Watson system returns it’s answers, it could be showing it’s users the sources that it found that were relevant to the question – showing the passages from documents that the user might not have read or even heard of, helping to make them more informed. <\/p>\n

It could show the reasoning for why it had it’s level of confidence in the answer, helping them to consider new perspectives. <\/p>\n

And it works both ways – the users will make Watson smarter. <\/p>\n

When a Watson system doesn’t have enough evidence to be confident in it’s top answer, it wouldn’t have to give up there. Every interaction wouldn’t have to be a single, stand-alone question. Watson could ask follow-up questions, working with the user to gather more information, more context and more evidence, until the level of confidence in the top answer improves. <\/p>\n

Not only that, the machine learning in Watson would mean that every time a user provided a correct answer, Watson would get that little bit better and smarter at handling questions like that in the future. <\/p>\n

Reciprocal. Collaborative. <\/p>\n

These are not the words that Jeopardy! brings to mind. But it’s the vision of the future that I look forward to.<\/p>\n

Watson doesn’t need to be about providing a definitive, take-it-or-leave-it answer without any additional information. It could be about helping decision makers to have the relevant stuff from terrabytes of text, right at their finger tips. <\/p>\n

Watson could be a well-informed assistant for human experts. <\/p>\n

More like a librarian who knows all of the text that has been written about your field of interest, and can work with you to find just what you need to answer the question that you have. <\/p>\n

<\/p>\n

Does that vision come across in a man vs machine challenge? <\/p>\n

IBM has uploaded a number of Watson-related videos<\/a> to YouTube. Some of the comments that get posted on them are positive.<\/p>\n

Others, however…<\/p>\n

<\/a><\/p>\n

That isn’t the only one. There are loads like this. <\/p>\n

<\/p>\n

Or…<\/p>\n

<\/a><\/p>\n

To be fair, not all of them are entirely serious.<\/p>\n

<\/a><\/p>\n

Okay… let’s be honest. I’m using comments from YouTube to back up an argument. <\/p>\n

This is kinda cheating. There isn’t a point of view too crazy that you couldn’t find some commenter on YouTube that would back it up. <\/p>\n

And I like I say, there are positive comments there, too, so I don’t want to blow this out of proportion. It’s not actually that bad. <\/p>\n

But I do think introducing Watson to people in an adversarial, man vs machine format, has had some impact on the way it has been perceived. And there are plenty of more thoughtful, considered pieces in the mainstream press that talk about Watson being better than doctors<\/a>, rather than how a Watson system could be a tool to make doctors more effective. <\/p>\n

It’s not just the Internet crazies that are looking at this through a lens of “Watson was better than Ken and Brad on Jeopardy, and it beat them. Next it’s gonna be better than my doctor and beat them at medicine.” And in an era of outsourcing, there are people writing that they fear Watson as an attempt to save money by replacing humans with computers<\/a>. <\/p>\n