An in-depth look at the potentials and pitfalls of online cancer survival calculators

By Dr. Purvish Parikh

With the incidence of cancer growing rapidly, it has become a major healthcare problem for all stakeholders. More than half the cancer burden exists in countries with low and middle income, countries ill-equipped to tackle the problem. India has undertaken a significant initiative with the focus on non-communicable diseases and development of infrastructure as well as trained personnel in different parts of our nation.

Ongoing efforts are already bearing fruit in tackling these serious illnesses (cancer is not one disease—it’s actually a group of more than 200 different types), by use of technology and recent advances to optimise chances of cure and improve outcome.

“However, patients and families are not easily satisfied because they know that cancer is a serious illness. They want to receive as much expert advice as possible, often taking second, third and even fourth opinions. This often leads to confusion, especially when combined with the family being bombarded by a plethora of information when they start consulting Dr Google’ (internet). Now the oncologists’ advice is pitted against the incomplete, inconsistent, biased, personal opinions, fake news and posts that every Tom, Dick and Harry put on the world wide web, usually without scientific basis or proof,” says Dr Bharath Rangarajan, Senior MedicalOncologist, Kovai Medical Center and hospital, Coimbatore.

Unrealistic expectations conflict with reality; miracle cure claims are in contrast to expert medical judgment, leading to an overall suspicion of anything that is bad news, usually from the healthcare professionals.

“I have personal experiences of highly-educated and otherwise sane and successful families coming to me with reams of printouts from the internet, several emails from overseas oncologists who have never seen the patient and a closed mind. They have already made up their mind what the cancer is, what is its stage, what is the best treatment option and simply want me to confirm that they are correct,” says Dr Purvish Parikh, Group Oncology Director, Shalby Cancer & Research Institute (11 hospitals across India), who also runs a very successful CancerExpert.in service. To add to this mix, we now have free online calculators that are advertised to show the chance of survival for most patients.

A quick online search (just like that done by patients and families accessing the net) revealed impressive figures (Table 1). “This is a true representation of the misleading and incorrect information available online. PubMed represents a search of high-quality medical journals where the manuscript is screened by other medical experts and only research or data that meets strict ethical and data integrity criteria are published. This, therefore, represents the most reliable information and even this is further classified into several levels based on the type of study evaluation (level one evidence, being the most robust),” says Dr Prasad Narayanan, Senior Medical Oncologist, Cytecare Cancer Hospital, Bengaluru. So while patients and families get swayed or confused by the information on ‘Cancer Survival Calculators’ available on Google (43,50,00,000 hits) or Google Scholar (22,300 results), the real valuable information is limited to the 170 articles published in PubMed-indexed medical journals of high-quality.

“Results of such cancer survival calculators can be faulty. Compounding the issue are lack of indigenous data and patient profiling, especially from the socioeconomic, ethnicor geographical area, which is not represented either in the training set or validation set.”

Dr Sumant Gupta, Senior Medical Oncologist, Sarvodaya Hospital, Faridabad

Such PubMed articles gradually increased from one each per year till 2009, 5 in 2010 and 2011, reaching double digits by 2014 and finally resulted in 43 last year (2019) (Figure1). Some examples of such cancer survivor calculators (predicting response, benefit or survival) include those for breast cancer, colon cancer, cancer surveillance, etc are shown in Table 2.

Table 2: Representative recently published cancer survival calculators for specific purposes

Sr. No	Title	URL
1.	Measurement of Residual Breast Cancer Burden to Predict Survival After Neoadjuvant Chemotherapy	https://bit.ly/39vTUQ6
2.	New models and online calculator for predicting non-sentinel lymph node status in sentinel lymph node-positive breast cancer patients	https://bit.ly/37sj1li
3.	ColoRectal Cancer Predicted Risk Online(CRC-PRO) Calculator Using Data from the Multi-Ethnic Cohort Study	https://bit.ly/2SqG6AU
4.	The Surveillance, Epidemiology, and EndResultsCancer Survival CalculatorSEER*CSC: Validationin a Managed Care Setting	https://bit.ly/37nN5Pe
5.	Predicting Cancer Prognosis Using Interactive Online Tools: A Systematic Review and Implications for Cancer Care Providers	https://bit.ly/2SLxnb5

What, really, are these calculators?

Such calculators are algorithms based on two or more parameters that could be patient-related, organ-related, cancer type and stage-related, and/or others. Thus, they are examples of the most simple objective manner, in analysing prognostic importance of individual variables that have the potential to influence how the disease will behave and the likely manner in which the patient will respond to various treatment options. So, if they really work as intended, they would be a great value to all stakeholders.

“I have personal experiences of highly educated and otherwise sane and successful families coming to me with reams of printouts from the internet, several emails from overseas oncologists who have never seen the patient and a closed mind. They have already made up their mind what the cancer is, what is its stage, what is the best treatment option and simply want me to confirm that they are correct.”

Dr Purvish Parikh Group Oncology Director, Shalby Cancer & Research Institute (11 hospitals across India); Also runs CancerExpert.in service

Why were they invented?

Cancer-related calculators were required to be invented to fill a vital gap, an unmet need. Let us take the simple example of calculating the dose of a chemotherapy drug for a patient. The effect of the drug would depend on several factors, the most important being the weight of the individual. However, it became quickly obvious that weight is only one of the factors that determine how the body is exposed to the effects of any chemotherapy. “For adults, data showed that the risk-benefit assessment was more accurately judged when using the surface area of the body to decide the dosing. Thus were born the first calculators in oncology—to easily find out the surface area of any individual in an accurate and reproducible manner. For instance, the simplest formula (calculator) for adult body surface area is the square root of (ht in cm x wt in kg/3600). We will come back to surface area calculation later in this article,” says Dr Padmaj Kulkarni, Senior Medical Oncologist, Deenanath Mangeshkar Hospital, Pune and Editor of Indian Journal of Medical & Paediatric Oncology.

No wonder Oncotype DX has growing competition with other calculators touting their superior benefit, including but not limited to CanAssistBreast, Mammaprint, Endopredict, Ki67 alone, Prosigna, Decipher and Polaris. Besides, most online survival calculators are fed by data from western population and their interpretation is very confusing for both doctors and their patients.”

Dr Randeep Singh, Director & Senior Consultant of Medical Oncology, NH Hospital, Gurugram

How are they invented?

The first step is identifying a need. For example, all breast cancer patients do not do well after surgery. A significant number would have the disease come back (relapse). So the need was to identify those patients who had a higher chance of relapse and then decide what additional treatment would benefit them and how much the benefit would be.

The second step is to gather the data from a sufficiently large number of similar patients and study all the variables in them, and correlate with actual outcome in them, and thus, attempt to see which factors predict high versus low risk of relapse. The first documented publication on this subject was by Blamey RW and others in Clinical Oncology in 1979.

Their paper described the importance of tumour size, tumour grade and regional lymph nodal involvement as prognostic factors in breast cancer, which they used to make a calculator, the formation of the Nottingham Prognosis Index (NPI). In 2001, the first such computerised calculator became available online—which became popular as AdjuvantOnline.

This was published in Ravdin and others in the Journal of Clinical Oncology in 2001, entitled ‘Computer program to assist in making decisions about adjuvant therapy for women with early breast cancer’. Their objective was to develop a computerised algorithm that would permit oncologists and their patients to understand and decide whether their early breast cancer required additional therapy and if so, how much would its benefit be. (It so happens that today access to this online system is limited to healthcare professionals alone, and not intended for direct patient use, the reasons for which we shall discuss later in the article). This was derived based on the input of data of thousands of patients with early breast cancer who had already been treated (with or without additional adjuvant therapy) from Surveillance, Epidemiology, and End-Results data from the USA. In addition, the outcome of adjuvant therapy randomised trials reported until 1998 were also included. Estimates were then subject to the Bayesian method of “adjustments” to generate this Prognostic Factor Impact Calculator.

“The very first article, on adjuvant online, clearly mentioned in its concluding statement that ‘ideal validation for all patient subsets withall treatment options IS NOT POSSIBLE.’ No wonder such calculators get refined at regular intervals, and today, there are 277 peer reviewed articles in high quality PubMed medical journals regarding this subject—adjuvant online.”

Dr Bhavesh Parekh, Professor and Head of Medical Oncology, Shalby Cancer & Research Institutes, Ahmedabad.

This included the input of the impressive number of variables like patient factors (age, menopausal status, other co-morbid illnesses) and tumour factors (size, grade, axillary node involvement, oestrogen receptor positivity).

Then five-year outcomes were matched for these patients with respect to whether they received additional therapy (after surgery) in the form of endocrine therapy (tamoxifen) or chemotherapy (like cyclophosphamide, methotrexate, 5FU or anthracycline combinations or taxane combinations). The overall survival and disease-free survival in these patients were mathematically assigned a value to come up with a model formula available online. “The very first article, on adjuvant online, clearly mentioned in its concluding statement that ‘ideal validation for all patient subsets with all treatment options IS NOT POSSIBLE.’ No wonder such calculators get refined at regular intervals, and today, there are 277 peer-reviewed articles in high-quality PubMed medical journals regarding this subject—adjuvant online,” says Dr Bhavesh Parekh, Professor and Head of Medical Oncology, Shalby Cancer & Research Institutes, Ahmedabad.

Are they applicable to all cancer patients?

There are more than 200 different types of cancers. In addition, the classification of these cancers is constantly evolving. New tests and new parameters allow finer refinement of the way we identify and sub-classify cancers—some categories disappear and new ones take shape. Hence, it is impossible to have calculators applicable to all types of cancers or all cancer patients. What is available today is limited to only some of the most common cancers and some subtypes that have robust data on known variables and treatment outcomes.

“Online calculators have failed generalisation over diverse population in many studies over the years. They give only a rough estimate of outcomes-based on data sets that are very specific and limited. They create immense stress on the patients and family when such tools are used indiscriminately and amateurishly,” says Dr Sachin Hingmire, Senior Medical Oncologist, Deenath Mangeshkar Hospital, Pune and editor of South Asian Journal of Cancer.

“For adults, data showed thatthe risk benefit assessment wasmore accurately judged whenusing the surface area of thebody to decide the dosing. Thuswere born the first calculatorsin oncology—to easily findout the surface area of anyindividual in an accurate andreproducible manner.”

Dr Padmaj Kulkarni Senior Medical Oncologist, Deenanath Mangeshkar Hospital, Pune, and Editor of Indian Journal of Medical & Pediatric Oncology

Do such calculators take into consideration all variables affecting a specific type of cancer?

“When first evaluated and created, the researchers take effort to include all known variables for which sufficient data is available. Unfortunately, this means that any missing data on these individual variables have the potential to create bias. This may lead to unrealistic (and misleading) outcome projections, which lead to unrealistic expectations from patients and families,” explains Dr Nikhil Ghadyalpatil, Senior Medical Oncologist at Yashoda Hospital, Hyderabad, and director of Optimus Oncology. Taking the example of the UKPREDICT calculation model, let us explain the development and validation steps. Initially, the calculator was developed on the basis of data from 5,694 women treated in East Anglia between 1999 and 2003 (details available in publication of 2010). Once developed, such calculators need to be validated to verify their accuracy (false positive and false negative levels). In this instance, the validation was based on data from 5,000+ breast cancer patients from the West Midlands as well as another group of data from British Columbia.

Why do some calculators keep changing their parameters?

Says Dr Govind Babu, Senior Medical Oncologist at HCG, Bengaluru and president of SAARC Federation of Oncology, “The fact that science is constantly evolving, more data is being generated and additional variables become known (new tests) automatically indicates that calculators also need to evolve to keep up. So it should not be surprising to anyone that online survival calculation tools need to be updated and change their parameters.” Let us take the example of OncotypeDx (first published in 2004), a model for cancer patients based on 16 cancer-related molecular gene testing to give a scoring system that would predict whether a particular subset of breast cancer patients have higher risk of recurrence or not, for ladies with early breast cancer whose tumours are oestrogen receptor-positive (ER+ve), lymph node-negative (LN−ve) and human epidermal growth factor-receptor 2 negative (HER2−ve). The recurrence score (0 to 100) and was-specified to indicate low risk if the score was less than 18, intermediate risk (18 to 31) and high risk if the score was more than 31.

“The fact that science is constantly evolving, more data is being generated and additional variables become known (newtests) automatically indicates that calculators also need to evolve to keep up. So it should not be surprising to any one that online survival calculation tools need to be updated and change their parameters.”

Dr Govind Babu, Senior Medical Oncologist, HCG, Bengaluru, and President, SAARC Federation of Oncology

The selection of the final 16 cancer genes was on the basis of three independent studies involving a total of 447 patients. While decision-making was clear in the low and high-risk categories, the intermediate group remained a grey area.

Hence, trials were quickly commenced on how to attenuate the intermediate range of patients. But even amongst the clear categories of low and high risk, there were problems.

Table 3: Oncotype DX Recurrence Score – criteria changing with time

	Low risk	Intermediate risk	High risk
Initial cutoff	Less than 18	18 – 31	More than 31
New cutoff	Less than 11	11 – 25	More than 25

For instance, estimating OncotypeDX recurrence score using the Mangee Equations led to varying results, based on the equation used. For instance (Figure 2) if for a patient, the Nottingham score is 6, ER score is 150, PR score is 150, Her2 status is equivocal, tumour size is 2cma and Ki-67 expression is 15%, the output is different. Equation one and three give similar results, 20.81 and 20.98 respectively, whereas Equation two will give a recurrence score of 24.55. The initial Magee Equation was published in 2008. Subsequent modifications of these equations were conducted based on data from a larger group of patients, initially 800 and then an additional set of 200 cases, including clinical Oncotype DX testing by Genomic Health.

In fact, the cutoff of the recurrence score in Oncotype DX was changed in 2018 (Table 3), based on data from10,273 breast cancer females recruited between 2006 and 2010, as part of the TAYLORx study. But did the change in the cutoff provide any value to the clinical patient management decisionmaking progress?

Table 4: Recurrence Score risk groups and outcome

Recurrence Score (RS)	Invasive Disease–Free Survival	Overall Survival
Less than 11	84.0%	93.7%
11 to 25	83.3%	93.8%
More than 25	75.7%	89.3 %

Table 4 shows the invasive disease-free and overall survival for patients classified according to the new cutoff of Oncotype DX recurrence score. The differences between low and such as lack of information on RNA amount used, missing units in the single gene expression report, missing references cited in the original study regarding the determination of the recurrence score formula, and vague information on the normalisation of the gene expression.”

They also stated that the Oncotype Dx assay itself could not be confirmed when using customized RT-profiler assay! “No wonder Oncotype DX has growing competition with other calculators touting their superior benefit, including but not limited to CanAssistBreast, Mammaprint, Endopredict, Ki67 alone, Prosigna, Decipher and Polaris. Besides, most online survival calculators are fed by data from western population and their interpretation is very confusing for both doctors and their patients,” says Dr Randeep Singh, Director and Senior Consultant of Medical Oncology at NH Hospital, Gurugram.

“Online calculators have failed generalisation over diverse population in many studies over years. They give only a rough estimate of outcomes based on data sets that are very specific and limited. They create immense stress on the patients and family when such tools are used indiscriminately and amateurishly.”

Dr Sachin Hingmire, Senior Medical Oncologist, Deenath Mangeshkar Hospital, Pune and Editor of South Asian Journal of Cancer.

Will it give me the best idea about my chance of survival?

Let’s go back to the Adjuvant Online in breast cancer example, first published in 2001 and now in its 9th version! There are several scientists like HessV who have demonstrated pitfalls in this calculator (published in 2008 and2014). Data from 2012 patients in a population-based Dutch cohort were included (1997 to 2004), of which 904 died and 326 patients had a cancer recurrence. They found that the first version of Adjuvant online calculator had overestimated 10-year overall survival by as much as 9.8%. And the second version of Adjuvant online had underestimated the same 10-year overall survival by 17.1%. No wonder such calculators need constant refining and recalibration. Yao Lung and colleagues from Taiwan did a similar evaluation in their patients (published in 2012).

“Cancer survival calculators take into consideration only a small fraction of the information needed to decide the best course of action. It is wrong to concentrate on the exact number or fraction— missing the woods for the trees.”

Dr Prashant Mehta, Senior Medical Oncologist at Asian Institute of Medical Sciences, Faridabad, & President of OGS Stars

They used data from their 559 patients with breast cancer from 1992 to 2001. They also found significant differences between calculator based prediction of breast cancer-specific survival and actual outcome in the entire group. For instance, in patients considered as high-risk, adjuvant online calculation overestimated outcomes—the calculator predicted 21.99%, whereas it actually was 17.46% in their patients.

“So results of such cancer survival calculators can be faulty and lead to treatment decision making that will not give an intended benefit. Compounding the issue are lack of indigenous data and patient profiling, especially from the socio-economic, ethnic or geographical area, which is not represented either in the training set or the validation set,” says Dr Sumant Gupta, Senior MedicalOncologist at Sarvodaya Hospital, Faridabad.

Earlier this year, Wang F and colleagues even evaluated the role of OncotypeDX in male patients with breast cancer. They found that in their group of 848 male and 110,898 female patients studied between 2010 and 2014, Oncotype DX score correlated male patient mortality only up to a score of 21, after which it flattened out. Whereas among their female patients, an increase in mortality correlated with recurrence score only after it reached 23.

What if patients are still confused about the result shown in the online calculator? Is there any other online resource that can help?

It is possible for patients to go online and chat with complete strangers (in a similar situation) and use their experience/feedback to make decisions. Let us take the example of an online conversation among patients with breast cancer ( https://www.cancerresearchuk.org/about- cancer/cancer-chat/thread/Oncotype-dx-score-18-chemo-orzoladex)

“When first evaluated andcreated, researchers take effort to include all known variables for which sufficient data is available. Unfortunately, this means that any missing data on these individual variablesh as the potential to create bias. This may lead to unrealistic (and misleading) outcome projections, which lead to unrealistic expectations from patients and families.”

Dr Nikhil Ghadyalpatil, Senior Medical Oncologist, Yashoda Hospital, Hyderabad, and Director, Optimus Oncology

The discussion thread was titled OncotypeDX score 18—Chemo or Zoladex?

Mrs C: I’m new to the forum and need advice. In 47, a lumpectomy was done and my Oncotype DX score has come as 18. In my head, I had thought 15 or lower is no chemo and 20 or above means chemo. My oncologists said its totally my decision on whether I want to take additional (adjuvant) treatment or not. They also said that if I don’t have chemo, another alternative is to have Zoladex injections. I’m so confused—up until now everything has been fairly straight-forward, but I’m so afraid of making the wrong decision. Although I know there is no right and wrong. Responses included the following:

Ms X: Go for the chemo. I had stage 2 cancer with micrometastasis in one node. My view was doctors would not suggest it if it wasn’t helpful. I chose to throw everything at my cancer.

Ms L: My score was 25. I didn’t want to take such a risk and end up regretting not having the chemo. I personally want to give myself the best chance possible as I’ve got a family too I want to be here for. So chemo.

Ms S: My score was 19 with 16% chance of recurrence outside of the breast. I’ve been doing lots of research and basically I too need to choose whether I do chemo and go bald orZolodex and get fat!!

Reply from Mrs C (patient who started the discussion): Thank you. It’s helped a great deal. After 2 nights of sleeping on it, I am definitely leaning towards chemo.

What should I do if the online calculator gives information that is significantly different from what my oncologist discussed with me?

Says Dr Prashant Mehta, Senior Medical Oncologist at Asian Institute of Medical Sciences, Faridabad and President of OGS Stars: “Online cancer survival calculators, at best, only give a very general idea of what might lie in the future. Medicine, including oncology, is not mathematics. One needs to get the bigger picture. Cancer survival calculators take into consideration only a small fraction of the information needed to decide the best course of action. It is wrong to concentrate on the exact number or fraction—missing the woods for the trees.”

Take the example of one survival calculator that is currently being propagated on social media. It gave a 5-year survival of 107% (Figure 3). That would mean that if 100 patients of that disease were diagnosed today, 107 would be alive after five years! An impossible and ridiculous calculation! It clearly outlines the inherent problems with half-baked ideas and untested material available online