Poorly executed analyses of drug research and development is useless

A recent study makes the bold claim that the most generally accepted estimate of the cost of drug development is overstated by a factor of almost 2:1.

This would be important news, if it were accurate.

The study  examined 10 cancer drugs from 10 pharmaceutical companies over a period of nine years. Using data available from public filings, the study extrapolates what it terms the “true” cost of bringing these drugs to market.

{mosads}While it is easy to understand the author’s’ interest and motivation for undertaking their analysis — many people simply find it hard to believe that it actually costs billions of dollars to bring a successful drug to market — their work simply fails to ask enough questions or to use proper methodology. In fact, if they had used appropriate methods, their analysis would likely have confirmed rather than challenged the finding they criticize.

The study has major methodological flaws. Perhaps most important, it lacks a scientist’s appreciation for the existing literature on the topic. The authors criticize DiMasi and others for lacking “transparency and independent replication.” This is simply untrue. They have published several peer-reviewed studies on the costs and uncertainty associated with drug development over more than 25 years, finding that the cost of drug development has grown substantially over this time. Moreover, the two authors, Vinay Prasad, M.D. and Sham Mailankody ignored independent peer-reviewed research that has confirmed the DiMasi results.

Against this substantial literature, the authors prefer an alternative estimate from a non-peer-reviewed policy brief that referenced drugs introduced in the 1980s. The comparison is simply not good science and is terrible economics.

No one acquainted with the facts disputes that the odds of successfully bringing a new drug to market are low. Because there are so many failures for each success, calculating the average cost of success necessitates a careful estimate of the probability of success. This is the first major shortcoming of Prasad and Mailankody’s analysis.

They reason that looking at companies that have only one success must be representative of the overall rate of success. They ignore the fact that many other companies fail to ever achieve a successful drug launch, so the denominator they use to calculate the likelihood of success is far too low.

The study’s data indicates a clinical success rate of around 23 percent. A recently published analysis of clinical development success rates suggests that the probability of bringing a new drug from Phase 1 to approval is 9.6 percent and for oncology specifically, it is 5.1 percent.

The likelihood of bringing a new drug to market from the pre-clinical stages is lower still. Simply adjusting for this overstatement in the implied probability of success, Prasad and Mailankody would have found numbers in the same ballpark, and most likely above, those of DiMasi et al.

Next, the authors call out what they see as a large disparity between R&D costs and product revenues. They report sales in excess of 1000 percent of calculated R&D expenditures for four of the products in their sample. The others have sales as a percentage of R&D ranging from 171 percent to 115 percent, with one earning just 17 percent of calculated R&D expended in revenue by the end of 2016. Such a skewed distribution of revenues is not unusual in pharmaceutical markets. The blockbuster model has fueled the industry for decades.

And while that is as far as the authors go, they should go much farther. A spot check of the data again reveals serious questions. Three of the companies with the largest revenues have been acquired by other companies and are no longer stand alone companies, but the data for the fourth (Eculizumab, from Alexion Pharmaceuticals) raises some interesting questions.

Eculizumab (brand name Soliris) is the third highest revenue generating drug of the ten the authors study, earning 1,588.5 percent of R&D costs through 2016 as reported by the authors. Alexion launched Soliris in 2007, but the company’s 2007 10-K report indicates that it had been investing in drug development efforts since its founding in 1992, earning no revenues from product sales over a nearly fifteen year period.

According to the company’s 10-Ks, it had accumulated revenues from worldwide sales of Soliris of about $12.4 billion by the end of 2016. However, the company’s accumulated net income — i.e., the profits earned over the life of the company after covering expenses, including R&D expenses for other potential products — was only about $1.6 billion. That represents about 13 percent of total revenues earned — a strikingly low profit rate for one of the claimed “winners” of the drug development lottery. These numbers represent a different reality than Prasad and Mailankody portray.

The authors also misunderstand the time-frame over which investments in new drug development occur, and they are not aware of the economic realities the companies they evaluate face. In fact, the risk that pharmaceutical companies, and their investors, assume with research and development, is substantial. The vast majority (90 percent) of publicly-traded biopharmaceutical companies in 2014, for example, reported no profit.

Certainly, if R&D could be made less costly, more companies would be willing to enter the search for cures. That would lead to more products coming to market and more cost-saving price competition in the end. So, finding a way to lower the cost of R&D is an admirable goal. It would have benefits to patients and to health care systems. However, poorly executed analyses of R&D costs do nothing to make progress toward that goal.

Richard Manning, PhD, is a partner in the Life Sciences Practice at Bates White, an economics consulting firm in Washington, DC.