Researchers Inappropriately Blast Innovation Center

An article in the New York Times recently criticized Medicare's Innovation Center for not using placebo controlled double blind study design. According to the article:

"But now that the Center has gotten started, many researchers and economists are disturbed that it is not using randomized clinical trials, the rigorous method that is widely considered the gold standard in medical and social science research. Such trials have long been required to prove the efficacy of medicines, and similarly designed studies have guided efforts to reform welfare-to-work, education and criminal justice programs.

But they have rarely been used to guide health care policy — and experts say the center is now squandering a crucial opportunity to develop the evidence needed to retool the nation’s troubled health care system in a period of rapid and fundamental change".

Patrick Conway M.D., a respected academic physician and director the Innovation Center, defended its reliance on demonstration projects, saying they allowed researchers to evaluate programs in the real world and regularly adapt them. “Does it look like it is working?” he asked. “If it does not look like it is working, we can stop.”

He said that the Center has had trouble getting such studies to yield solid results because those in the control groups — who do not get the innovation being tested — tend to drop out.  “We will use randomized designs when we can and when it’s appropriate,” he said".

Dr. Conway has it absolutely right. Human factors research is a completely different animal than testing what impact a molecule may or may not have in a drug study. The fact is molecules don't talk back or resist what a study tells it to do. Let's take a simple example: improving preventable mortality in a hospital. How do we randomize hospitals into groups that aren't going to work on preventable mortality versus hospitals that are going to work on it? The reality is if a hospital has a high preventable mortality rate the doctors and nurses are going to do everything they know how to reduce it. To tell them they can't work on interventions that will save lives for three years is just ludicrous. Studying an intervention with so many variables that includes many human factors is, to say the least, difficult and might take many years if it could be done accurately at all.

It's also important to realize we have a burning platform in this country to reduce cost and improve care quality. We can't wait 10-15 years for all the evidence, even if we could get it. The fact is the Innovation Center's activity is already significantly effecting the industry in positive ways. Take the Pioneer ACO program. In the first year alone Pioneer Fee for Service Medicare beneficiaries, as a whole, were cared for more efficiently than the non-pioneer population. A drop in care cost on average of $20 per member per month was obtained. In some markets the improvement was much larger at over $102 per member per month. (Read here.)

The best performing Pioneer ACO Bellin Thedacare Health Partners reduced the total cost of care for 20,000 Medicare beneficiaries by 4.6% year over year, others had similar results . This, all in the face of improved quality, as indicated by the 33 quality measures the Innovation Center established before the program started. The bottom line? If every market in the U.S. could reduce the cost of care by 4.6% we would not have a healthcare crisis. So, the issue at this point is how to move every healthcare organization to the best practice established by Bellin ThedaCare, not how to design 10 year randomized trials.