Lessons Learned: What the BMP Trials Controversy Has Taught Us

By: Terry Stanton

Terry Stanton

In an Orthopaedic Research Society (ORS) forum devoted to “avoiding wrong turns” in the development of new products and therapies, the inclusion of recombinant human bone morphogenetic protein–2 (rhBMP-2) among the session topics would come as no surprise. In 2011, rhBMP-2 came into a less-than-flattering spotlight over questions not only about complications associated with its use—much of it off-label—but also about how the original industry-sponsored trials could have reported essentially no adverse events.

According to the man most responsible for bringing that question to the fore, Eugene J. Carragee, MD, of Stanford University, the published findings by researchers exhibited the “warning flag of warning flags: when it is too good to be true.”

He pointed to the eight large industry-sponsored trials, involving about 600 patients treated with “an enormous dose” of rhBMP-2, without any complications. “If that’s true,” Dr. Carragee said, “then rhBMP-2 would be 20 times safer than a short course of Motrin and penicillin. I think that treating a thousand people with a dose of a very powerful growth factor and without any side effects is beyond Euripides’ proverb: ‘Man’s most valuable sense is a judicious sense of what not to believe.’”

In an issue of The Spine Journal devoted to rhBMP-2 questions, Dr. Carragee’s editorial expressed his view that “the safety profile has devolved, with troubling rapidity” because of process flaws including poor trial design and reporting bias peer review/publication shortfalls “that may have promoted widespread, poorly considered on- and off-label use, eventual life-threatening complications, and deaths.”

In his view, the trials conducted by investigators who received large sums of money from the manufacturer failed to address or accurately report complications that had been detected in previous research. These complications included neuroinflammatory reactions of retrograded ejaculation and radiculitis, infection and wound problems, and cancer promotion or carcinogenicity. The studies continued to depict impressively positive results even though, Dr. Carragee said, multiple reports of complications arose as soon as BMP came into general use in 2004.

He noted that stories about BMP were on the front page of major newspapers. “That’s too late for warning signs,” he said. “That’s a head-in-the-sand sign.”

The root of the problem with the research, he said, might have been the premise of the studies: that for single-segment fusion, the use of iliac crest bone graft (ICBG) “is so morbid that it had to be avoided.” Yet none of the data supported that premise, he said. “The early data, through 3 months, did not show improvement, in multiple randomized clinical trials. If the back pain from ICBG harvesting was so bad, you at least should see it at 6 weeks. Instead, this seemed to indicate that the inflammatory morbidity of the BMP was at least equal to or greater than the ICBG.”

Dr. Carragee related the experience and actual role of The Spine Journal, which published some of the studies at issue, in the rh-BMP-2 contretemps. He said the journal had amassed a backlog of 39 articles on BMP complications. He noted that on his return from active military duty, he found the journal’s editorial process “was being widely criticized.” Some of the complaints were from surgeons who were involved in the study. They claimed that the published data were inaccurate and they had encountered multiple complications that somehow didn’t make it into the particular article. Disclosure problems, poor reporting of adverse events, and data that didn’t agree with documents submitted to the U.S. Food and Drug Administration (FDA) were among the problems.

He even received a visit from a federal investigator, who read to him the racketeering statutes and “wanted to know whether the journal reviewers and the editors were dumb or complicit.”

In a systematic review of available data from the FDA and other sources, the editors found more major problems in the favorable studies, including the following:

  • Even in the larger trials, FDA data indicated complications with rhBMP-2 “may be common and serious,” but went unreported.
  • The presence and the magnitude of the authors’ conflicts were “astounding,” involving millions of dollars, even with the limited information the editors had about the conflicts. The disclosures
    were often “unintelligible or inconsistent.”
  • The pathology used to assess the ICBG harvesting was poor and seemed biased against the control group. “This in turn may have exaggerated the benefit or underestimated the morbidity of rhBMP-2 in the clinical situations tested,” said Dr. Carragee.
  • The control group methods and technique, as selected for both posterior approach methods, were potentially handicapped by design bias against the controls.

Among the lessons learned, Dr. Carragee said, were to beware of original safety concerns that are not systematically evaluated in the study and to beware of confusing a drug trial with a device trial. He also said to watch for adverse event reporting “that looks primarily at a statistical confidence in ‘proving harm’ rather than statistical confidence in safety. These are the flip side of each other, a bait-and-switch technique in which you talk about how safe something is but you report the statistics as harm.”

Surgeons should avoid extrapolating massive off-label product use from extremely limited safety data, should be suspicious if original datasets or even summaries were not available to reviewers or readers, and should be wary of “vague, bizarre, or even comical disclosures of funding and potential conflicts of interest,” Dr. Carragee said.

Major concerns that exist before a study is conducted should be systematically reported on in the trials. For example, comparative rates for retrograde ejaculation, a concern associated with an anterior approach, were not reported in any of the five papers based on the FDA trials, even though the data showed five times the rate of the complication in the BMP group versus the controls.

Similarly, the fact that concern about malignancy was not mentioned in the articles “should have been a big warning sign.”

Dr. Carragee noted that drug trials look at acute events versus cumulative occurrences, which are subject to “be wiped out by noise. Even large adverse event effects are going to be missed if you are going to add up everything over 5 years.” This problem was detected in reporting on urinary retention, delayed infection, and back and leg pain.

Panelist Dan M. Spengler, MD, of Vanderbilt University, asked: “Can we accept new products for human use based solely upon studies that have been supported by the producers of the products? Do lead institutions and investigators need to be non-conflicted?” In responding to his own question, he said, “Answers will need to consider financial and personnel resources as well as time delays to market.”

Jeffrey C. Wang, MD, of UCLA, observed that “Biologics will play a key role in the future of orthopaedic surgery. Scientific advances are made by scientists. We must critically review the evidence and make decisions based on good science.”

Disclosure information: Dr. Carragee—Intrinsic Therapeutics, Simpirica; Dr. Spengler: Journal of Bone and Joint Surgery–American, Musculoskeletal Transplant Foundation; Dr. Wang—Bone Biologics, Axiomed, Amedica, Corespine, Expanding Ortho, Pioneer, Axis, Syndicom, VG Innovations, Pearldiver, Flexuspine, Fziomed, Benvenue, Promethean, Nexgen, Electrocore, Surgitech, Medtronics, Biomet, Stryker, Seaspine, Aesculap, Osprey, Alphatech, CSRS, NASS.