The Problem with Outcome Metrics or The VIP Syndrome or Why Bill Clinton is Alive and Steve Jobs is Not.
A recent debate on using transplant center outcomes to decide accreditation with insurance and reimbursement reminded me of this article I had written on Quora some time ago. Copied and pasted.
Vinay Kumaran, Liver Transplant Surgeon
Transparency, the intention to treat and analysing data.
or
Why Bill Clinton is alive and Steve Jobs is not.
How do you select a surgeon or a Hospital? How, assuming you are sufficiently rich and powerful that you want the "best in the world", do you find the best (or hire the best finder to find the best)?
If you need a liver transplant, you can go to a website called SRTR -- Scientific Registry of Transplant Recipients. At this website, you can look up the mortality, one-year survival, five-year survival, waiting list mortality and other such statistics for various liver transplant centers. You can also look at waiting times for getting a liver and, if you have a private jet at your disposal which can take you anywhere, you can get listed at multiple centers which have a low mortality and a short time to transplant. Perfect, isn't it? I'm sure the team which did Steve Job's research for him did something like this.
To reiterate some of the information available in the public domain, Steve Jobs was initially diagnosed with a tumor in the pancreas. At the time of diagnosis, there was no evidence of the tumor having spread anywhere else and he was advised to undergo surgery to remove the tumor. Surgical removal is, for all practical purposes, the only treatment which can potentially cure a pancreatic malignant tumor. Steve Jobs, often prone to magical thinking and a personal 'reality distortion field' decided to treat the tumor with various quackeries including a vegan diet, acupuncture, herbal remedies and the services of a psychic. The tumor proved resistant to such measures and 9 months later he underwent surgery. It is possible that in that 9 months, the tumor spread to the liver. It was a relatively indolent tumor called a neuroendocrine tumor.
We do not know when it was discovered that the tumor had reappeared in the liver or what treatment he received. However, metastases from a neuroendocrine tumor to the liver do offer another opportunity for cure if the liver is removed and replaced by another transplanted liver. It is essential to ensure in such situations that there is no tumor anywhere else but the liver.
We know that he underwent a liver transplant at Methodist University Hospital Transplant Institute in Memphis, Tennessee. This center had a short waiting time to transplant and Steve Jobs, with a private jet at his disposal, was able to reach the hospital quickly when a liver became available. We also know that when he had the transplant, he had tumor deposits in the peritoneum. It is likely that if the patient had been anyone other than Steve Jobs the transplant would have been called off and the liver given to the next patient on the waiting list.
One of the things which keeps tumor under control is the immune system of the patient. It is known that tumors progress quicker in immunosuppressed patients and this is true of neuroendocrine tumors as well. In any case he died about 2.5 years after the transplant. He got what he wanted rather than what he needed.
Bill Clinton, on the other hand, seems to have done his research better. In 2004, Bill Clinton was discovered to have fairly severe coronary artery disease affecting multiple arteries. This would require a fairly complex cardiac operation. He underwent the operation successfully in the hospital that had the highest mortality for this operation in New York State, almost double the average for the State.
Why did Clinton pick this hospital and this surgeon?
One of the problems with having mortality rates and survival rates in the public domain is that it discourages surgeons from taking a risk.
If the median survival rate from liver transplantation (for instance) is 90% and the transplant team encounters a patients who is sicker than usual, say with a 70% chance of surviving the operation, the team becomes understandably reluctant to take on that patient. One option is to reject the patient before listing as 'too sick to transplant'. If the patient was okay at the time of listing and becomes sicker while waiting for a liver then he can be removed from the waiting list as 'too sick to transplant'. Once he is off the waiting list, he does not show up on the 'waiting list mortality' statistics. The patients who actually make it to transplant are those who have already passed this test by 'survival of the fittest'. A center which decides to give the 70 percenters a chance at life will inevitably find their survival statistics drifting away from the median. When they move two standard deviations away, the program is on probation and soon insurance will stop paying for transplant at that center. However, one must remember that for the patient denied transplant, there is only death to look forward to.
Now let's look at this from an 'intention to treat' perspective. We begin with 100 patients who need a liver transplant. At one extreme we deny all of them transplant and all 100 die.
In a more realistic scenario, we decide to assess the risk of transplant and we find:
10% mortality: 50 patients
20% mortality: 20 patients
50% mortality: 20 patients
90% mortality: 10 patients
Now Center A, which wants to keep up with the 90% survival statistics offers transplant to only the 50 patients whom they judge have a 90% chance of making it through the transplant. So they do 50 transplants, 45 of the transplanted patients survive. The remaining 55 patients die (5 after transplant and 50 without). The center transplant survival rate is 90%.
Another Center B, willing to be bit more aggressive and take on patients with 80% or more chance of survival transplants 70 patients. Of these, 45+16=61 patients survive and 39 patients die (9 after transplant and 30 without). The center transplant survival rate is 61/70=87%.
Center C doesn't give a damn about their statistics. They just want to give every patient who comes to them a chance. They transplant all 100 patients. 45+16+10+1=72 patients survive. The center transplant survival statistic is 72%. 28 patients die.
If we judge each center by survival outcomes alone we see A:90%, B:87% and C:72%. The choice is easy isn't it?
On the other hand if we look at how many of the 100 patients who presented with liver failure are alive at the end then it looks very different. A: 45%, B: 61% and C: 72%.
I'm not sure how Clinton managed to figure it out but he realized that the hospital in NY with the highest mortality was the one which was taking on the most difficult and high risk cases. They were used to managing such cases and gave him the best shot at recovery. Or maybe he just 'went with the flow'. Either way it turned out to be a good decision.
Comments
Post a Comment