openpolitics.com

Watson Won Jeopardy, but is it smart enough to Spin Big Blue’s AI into Green

Talking about Watson is a good way to trigger eye rolls from people in the machine learning and AI community. There’s widespread agreement that its triumph on the specific backward-question problem of Jeopardy! was notable. Making sense of language remains one of the biggest challenges in artificial intelligence. But IBM quickly turned Watson into an umbrella brand promising a bewildering variety of bold new applications, from understanding the emotional tone of Tweets to scouring genomes for mutations. It bought startups and rebranded their wares as Watson and touted cute but hardly lucrative projects like Watson-designed recipes and dresses. In one TV commercialWatson chatted with Bob Dylan, confessing “I have never known love.”

Overhyped

Critics say IBM executives overshot badly by allowing marketing messages to suggest that Watson’s Jeopardy! breakthrough meant it could break through on just about anything else. “The original system was a terrific achievement, there’s no question about that,” says Oren Etzioni, CEO of the Allen Institute for AI. “But they’ve really over-claimed what they can deliver in a big way; the only intelligent thing about Watson is their PR department.”

 .. In fact, like all the AI systems in use today, Watson needs to be carefully trained with example data to take on a new kind of problem. The work needed to curate and label the necessary data has been a drag on some projects using IBM’s system. Ashok Goel, a computer science professor at Georgia Institute of Technology, got written up in The Wall Street Journal and Backchannel after building a Watson bot to answer questions from students to his online course on artificial intelligence. But its performance was limited by the amount of manual labeling of data needed. “It had fairly high precision, but it did not answer a very large number of questions,” Goel says

.. MD Anderson had walked away from more than $62 million and four years spent on contracts promising a Watson system to help oncologists treat patients. An internal audit reserved judgment on Watson’s intelligence but said the center had struggled to connect it with an upgraded medical records system. IBM maintains the system could have been deployed if MD Anderson had kept going; the center is now seeking a new partner to work with on applying AI to cancer care.