Update: Problems with IBMs Watson

Update: The article below predicted that following IBM’s  Watson’s triump at Jeopardy there would more proposals for computers software playing a more  active role in business and organizations. Here are 3 recent articles on that continuing trend:
NYTimes – armies of expensive lawyers to be replaced by cheaper AI software.
Business Week – Virtual AI agents will replace more live customer service  representatives.
Springer – the fragility of AI and Intelligent agents.


Okay, lets be honest,  Watson … or rather its IBM masters cheated. IBM got a crucial  dispensation for Watson. The machine  did not have to read or listen  to the clues and decipher them in the same manner as  as Ken Jennings  and Brad Rutter did. Instead the input to IBM’s Watson was made “machine friendly”. The perhaps unintended deception was that IBM’s Watson responded with good English speech so viewers just assumed Watson was also listening and understanding  spoken words – reading the revealed close from the board like Ken and Brad.

Now understanding a spoken question is no mean feat -as viewers see in the annual National Spelling Bee.  In fact IBM used to have one of the leading PC software programs for parsing and then recording spoken  speech during  most of the 1990s. With voice training it could “understand” dictation with a 99 to 99.5% error-free rate. That means, on average,   only 1 in 150 words are misinterpreted by the IBM voice savvy program. A great starting point for IBM.

But  Watson did not employ this speech understanding on Jeopardy. Only  Ken and Brad had to listen+read to understand  – i.e. the non-trivial task of parsing the spoken meaning during Alex Trebek’s reading of the  clue or answer. Ken and Brad  do have the added help that they get to read the revealed clues so if they have any doubts, they can rectify them from the visual read of the clue. But again, IBM has had text scanning programs for decades – this is not new technology. So Watson could and should have had to match Ken and Brad on an equal footing – display the  ability to hear, read and understand the Jeopardy query/answer and not be just  given machine friendly input for a true apples to apples Jeopardy challenge.


NYTimes Demo Version of Jeopardy-IBM  Watson Challenge
In fact, as IBM moves forward on delivering Watson as a commercial product, this is the next challenge of “humanizing” Watson. The machine will have to be able to respond to verbal inputs and/or  read/scanned written commands as well as typed or “direct “input. In fact, it was this input aspect of the “challenge” that Ye Editor found most disconcerting.

Neither during  the Jeopardy program or the coverage of the contest by PBS’ Nova program did Watson’s inputs gets displayed and addressed. So many additional questions on potential “unfair advantages” for Watson were not clarified. What exactly was the nature of the inputs Watson received? Was Watson afforded special coding?  Or were the querys identical to what viewers plus Ken and Brad saw on the Jeopardy set? And how were the queries fed to Watson? As a command line instantaneously  transmitted to Watson? And when was input given to Watson – exactly when Alex started reading the query/answer or at the moment he finished reading?

Do You Detect  A  Bit of Automation Testiness Here?

Yes, Ye Editors qualms with Watson winning the Jeopardy Challenge Match goes well beyond the technical leger-de-main pulled off by IBM.  Back at the 1964 Worlds Fair in New York, fairgoers including a very young Editor were promised aerial cars and more leisure time for the population then any one would know  what to do with by the turn of the 21st century. Fast forward 46 years later and the Great Leisure has yet to arrive. In fact, it takes 1.35 people to earn the median  US familiy  income of $44,389 while two earners  just crack $90,000 per year. Look what has happened to the labour force participation rates:

If it weren’t for the Financial Meltdown  and other recessions the US would be on its way to 70% of all people working … or is that having to work to make a living? Because that is the problem now.First, wages are stagnant if not declining over the past 10 years. Second, a reflection of  the dwindling pay for work – the supply of jobs  is diminishing,  not meeting demand. The result is that  the worldwide unemployment rate is gradually inching upward as work no longer expands to fill the the increasing numbers seeking it. The CIA Factbook shows  this gradual increase in the World unemployment rate. And the Factbook does not measure the huge % of the World’s  population that is grossly underemployed – as  hundreds of millions in China, India, and other developing countries live on barely subsistence wages.  One can see the problem in the US where the downturn saw millions of jobs exported overseas to lower wage earners with likely few ever to come back. Hence the following chart:

Look whats happened now that  5 million jobs have been permanently exported from the US. The number applying for work exceeds the supply of jobs by 4 or 5 to 1. And the ratio constantly surges up and down as the number of people returning to the labor force surges with each tiny increase in available jobs. And now IBM through  Watson and “buddies” wants to take a sizable chunk of the high paying  professional jobs in medicine, legal work, even engineering and automate them. So between outsourcing and automation, even highly educated job positions become vulnerable to one or both pressures :
1)downward trend on wages associated with the profession;
2)downward trend in the number of jobs and  positions available in the profession available.

So no thank you IBM and Watson. Computing and the outsourcing it has enabled  have already taken enough jobs with no realistic planning for the consequences. Also there are severe problems of  a star system of compensation that utterly fails to take into account team players and network effects – the result has been a concentration of wealth among the few that plagues nations worldwide.  Now  Watson and ilk are being prepped to take  away well paying  professional job opportunities. Shaping a “Smarter Planet” means addressing these issues not ignoring them or papering them over and, in the worst case, outright denying them.

Ye Editor: Just a Sabotager?

Is the Takethe5th’s editor just a Luddite, a fearful neophyte in the world of computing trying to do in the Smarter Planet people at IBM? Not exactly.  A BSc in Communication and Computer Science plus three decades of work in software and web development would point in the other direction. Responsible for designing, building and maintaining Takethe5th.com – warts and all – would indicate some savvy with the contemporary web development and software scene. And note what is being called  for – a serious re-examination of the complex issues surrounding artificial intelligence and automation. Not just looking at whether more jobs are created by automation like IBM’s Watson and other AI-oriented technologies but also the quality and pay associated with any associated/downstream jobs. So Ye Editor is not throwing a sabot at  IBMs Watson; but rather asking for IBM and others who invest so heavily in  AI to plan for the broad spectrum of  consequences associated with Watson-like displacement of jobs demanding high human intelligence.

Do we have any contemporary precedence for caution regarding rapidly advancing computer science and technology? Well consider the Internet and some of the unanticipated consequences of outsourcing to India a, China and the developing world. Or making revolution as in the Middle East. Or consider what air travel has done for the unexpected transmission of disease or terrorism. Yes, technology, like best laid plans of mice and men can oft go agley. As a result  concern is  fourfold:
1)the number of wicked, hard to solve problems facing society are increasing at a rapid rate – witness Nuclear proliferation, global warming, inability of food production to keep pace with  population growth, plus a range of very difficult energy supply transitions are just a few at the top of the list;
2)the amount of slack time and buffer resources to allow for trial and error in solving these wicked problems has declined;
3)the ability to find even feasible let alone cost effective and consensus agreement solutions has shown our political institutions to be taxed to the limit. Witness Wisconsin and energy policy;
4)the number of economic and financial manipulators and zero-sum players at the highest levels in the game is the distressing real Sabotagers – witness the recent Financial Meltdown and more recently the Budget Deficit crisis in Wisconsin.

So this post is raising a perennial existence issue  – does our modern technical society have a robust set of problem coping and solving capacity? Equally important, does it have enough Mensch – highest stake players of goodwill in the game? Only the Shadow knows … and ye Editor is taking a 5th of the hard stuff for that very reason.

2 thoughts on “Update: Problems with IBMs Watson”

  1. Excellent article and well written o’ye Editor.

    This has been a concern of mine for some time as well regarding the pace of growth of technology.
    Doing more is simply not going to be enough to maintain the standard of living that people have been used to for the past few decades.
    Everyone is going to have to learn how to adapt and adapt efficiently… that is, everyone in the bottom 90% (give or take a few percentages) of the pyramid.

    On Watson, yah, that was a piece of work – some of us knew well that it was going to be slanted towards Watson. What, no finger to press the buzzer…. that alone was unfair.
    Now, if it was listening and responding to human speech, my jaw would’ve dropped.

    1. Amen about Watson’s ability to understand spoken words. Indeed, that would have been jaw-droppingly impressive.

      But I am convinced this was a “Rally the IBM troops” demo. Look at the evidence:
      1)IBM is long gone from the PC World;
      2)IBM has no presence in the mobile revolution;
      3)IBM is losing out to Oracle and Microsoft in the database world;
      4)IBM is loing out to Oracle in the packaged server game;
      5)IBM’s Lotus is losing bigtime to Microsoft Exchange and Sharepoint and Meeting Services;
      6)IBM is losing in the Cloud to Amazon, Google, Microsoft even Rackspace…
      And the list of hardware and software woes can go on.
      Sure the BI and consulting businesses are going well. But the bottom line is the troopers and even the executive suite must be concerened that the major breakthroughs in technology are passing the company by despite having a huge R&D budget and a patent locker full to the brims.

      So Watson and the Graphene Chip announcement were IBM’s wave to the troops and other stakeholders saying – “hey, we are still a player in info technology innovation”. And remarkably enough with well greased PR – the World and media tipped the hat to Big Blue and then moved on. And, as usual in the 24-hour media muddle, not asking the tougher follow on questions or investigating where and what is IBM presence and strategy beyond the next 6 months.

Leave a Comment

Your email address will not be published. Required fields are marked *

Pin It on Pinterest