In the fight between AI and oncology experts, human intelligence has been winning the war so far, and will continue to do so for a long time

By Dr. Purvish M. Parikh MD, DNB,FICP, PhD, ECMO, CPI, MBA Group Oncology Director & Professor of Medical Oncology – Shalby Cancer & Research Institutes, Shalby MultiSpecialty Hospitals Precision & Medical Oncologist, Asian Institute of Oncology, Somaiya Hospital, Mumbai

I t is well established that a supercomputer can, and have, consistently beaten world chess experts since the late 1980s. The most famous win was in 1997 when Deep Blue won against World Chess Champion Garry Kasparov. However, chess is nothing but pure mathematics and hence, based on logic. The bigger victory for computers occurred in the year 2017 when Libratus (an artificial intelligence robot) beat the best poker (card game) team consisting of Dong Kim, Jason Les, Jimmy Chou and Daniel McAulay at the Rivers Casino in Pittsburgh. This victory was huge since, in poker, players don’t get to see each other’s hands and the computer had to bluff as well as counter misleading information to win.

So when IBM Watson was launched (in 2012) in the fight against cancer, there was a lot of hope. This artificial intelligence was touted to have the ability to swallow and digest the thousands of oncology studies that are published each year and also make sense of the big data generated in treating millions of patients. The AI was expected to go through a massive amount of genomic data being applied to precision medicine because no human can possibly sift through terabytes and petabytes of human genomic data without the assistance of computers to find hitherto hitherto-unrecognized unrecognized patterns. It was expected to spew out treatment recommendations customized to individual patients faster and more accurately than seasoned oncologists.

Read Other Articles by Cancer Experts

The Big Facts About Childhood Cancer
On The Brink Of Possibility
Multidisciplinary Team in Cancer Management
Experts, Now A Click Away

Hundreds of other companies joined the bandwagon and began developing health care products using artificial intelligence (Amazon, Microsoft, Google, and UnitedHealth Group). And IBM Watson continued its journey of machine learning by continually rejiggering its internal processing routines in order to produce the highest possible percentage of correct answers on some set of problems.

By 2017, there were doubts creeping in. And today we know that the truth is the exact opposite. So what happened? IBM Watson was ‘connected’ to the Memorial Sloan-Kettering Cancer Center (MSKCC), MD Anderson Cancer Center, Manipal Hospitals, and several other cancer institutes to offer its ‘expertise’. Watson Oncology was touted by MSKCC as a “cognitive computing system trained to interpret cancer patients’ clinical information and identify individualized, evidence-based treatment options that leverage our specialists’ decades of experience and research”. IBM Watson was supposed to reduce the time between the latest evidence generation and its application into real clinical practice in the community.

The company pitched Watson supercomputer as a “revolution in cancer care”. A handful of oncology physicians from MSKCC became the expert group of humans inputting their own recommendations into Watson, without substantial proof of the same. Said Dr. Sujal Shah, an oncologist from Florida: “Watson provided information, including medical journal articles, that was helpful.” However, it did not directly help him make that decision, nor did it tell him anything he didn’t already know.

Ultimately, IBM Watson became nothing more than Memorial Sloan Kettering in a portable box! This lopsided dependence on experts from a single institution led to significant bias. MSKCC neither sees patients from all walks of life nor does it see patients presenting to the community oncologists. Their interpretation and algorithms fed to the computer were based on affluent Caucasian patients who presented with advanced or relapsed cancers. And being the arrogant MSKCC, its doctors ‘who were training’ Watson refused to see the bias apparent under their nose. In fact, Dr. Andrew Seidman, MSKCC’s lead trainers of Watson, admitted to their “very unapologetic bias.”

It was the equivalent of eminence-based medicine (as opposed to evidence-based medicine). This “vast amount of experience” without it being based on clinical trials, propagated continuing treatments that did not work because the “beliefs” were never put to the test of robust evidence.

THE ROT WAS SO BAD THAT, WHILE IBM CONTINUED TO PROMOTE WATSON TO HOSPITALS AND DOCTORS GLOBALLY, ITS OWN INTERNAL DOCUMENTS SHOWED THAT THE SUPERCOMPUTER COMPANY’S OWN MEDICAL SPECIALISTS HAD ALREADY IDENTIFIED “MULTIPLE EXAMPLES OF UNSAFE AND INCORRECT TREATMENT RECOMMENDATIONS.”

Data from our own Manipal Comprehensive Cancer Centre (Bengaluru) data showed that concordance between IBM Watson and its multi-disciplinary tumor board ranged from 81% percent for colon cancer to 96% percent for lung cancer. Data from Gachon University Gil Medical Centre (Incheon, South Korea) showed a concordance rate of 73% percent for colon cancer and only 43% percent in gastric cancer. And Danish oncologists dropped IBM Watson altogether because they proved that agreement between local doctors and Watson was stood at a dismal 33 percent.

In the US itself, the Center had to face scathing criticism by the University of Texas System, accused of grossly mismanaging its splashy program with IBM ($62 million USD). The result? IBM Watson was prohibited from being used on that hospital’s patients.

Unfortunately, IBM had released Watson without proving that the technology worked. Its flaws slowly unfolded to the bewilderment of doctors, computer engineers and translational researchers in the most embarrassing manner. In fact, STAT found that the system doesn’t create new knowledge. Its highly-hyped artificial intelligence existed only in the most rudimentary interpretation of the term. It soon became apparent that oncology experts were needed to be trained “for decades” before they could properly label the information before feeding it to IBM Watson. No wonder Oren Etzioni (CEO of Allen Institute for AI) said, “IBM Watson is the Donald Trump of the AI industry—outlandish claims that aren’t backed by credible data.”

The rot was so bad that, while IBM continued to promote Watson to hospitals and doctors globally, its own internal documents showed that the supercomputer company’s own medical specialists had already identified “multiple examples of unsafe and incorrect treatment recommendations”. It was, therefore, no surprise that Deborah DiSanzo, Head of IBM Watson Health, was “resigning” amidst growing criticism and ridicule by doctors inside and outside of the company.

As Dr. Siddhartha Mukherjee said in Annals of Medicine (April 2017 issue), “The “black box” problem is endemic in deep learning. The system isn’t guided by an explicit store of medical knowledge and a list of diagnostic rules; it has effectively taught itself to differentiate moles from melanomas by making vast numbers of internal adjustments—something analogous to strengthening and weakening synaptic connections in the brain. Exactly how did it determine that a lesion was a melanoma? We can’t know, and it can’t tell us. All the internal adjustments and processing that allow the network to learn to happen away from our scrutiny. As is true of our own brains.”

The black box problem is endemic in deep learning. The system isn’t guided by an explicit store of medical knowledge and a list of diagnostic rules. Exactly how did it determine that a lesion? We can’t know, and it can’t tell us. A deep-learning system doesn’t have the power to explain.

So, is artificial intelligence dead in cancer management? Certainly not! Its role continues to be refined and will settle where it can magnifying human ability. However, the intuition and sixth sense of a cancer expert will continue to hold the upper hand, and a group of such cancer experts will be the best recommendation that anyone can expect for several years (if not decades) to come