The Success Rate of New Drug Development in Clinical Trials: Crohn’s Disease

- Purpose . To determine the risk of drug failure during clinical trial testing in Crohn’s disease and determine what steps can be taken to improve outcomes. This is the first study to quantify such risk for a single disease. Methods . Moderate to severe Crohn’s disease was investigated by reviewing Phase I to III clinical studies conducted during 1998 to June 2008. Clinical trial failure causes were classified as commercial or clinical and compared with industry expectations. Drugs were not eligible for inclusion in this study if: their Phase I study occurred before 1998; targeted mild Crohn’s disease patients; did not test for remission or Crohn’s disease activity index (CDAI) score reduction reflecting disease burden; did not involve industry support or were not conducted in the United States. Results . A search of clinicaltrial.gov yielded 37 drugs that met our search criteria. The cumulative success rate for drug development in Crohn’s disease is 18%, from start to finish of clinical trial testing. New drug approvals are dominated by protein based therapeutics in this indication. Commercial and clinical failures both contributed substantially to the failure rates of new drugs. Phase I clinical testing appeared to offer little risk mitigation with pass rates at 92%. Conclusion . Funding intended to advance Crohn’s disease must take into account the disease specific historical failure rate of drug development in forecasting any reasonable expectation of producing new therapies. As it currently stands, one in five drugs will be successfully approved that enter clinical trial testing in this indication. To manage this risk, continued development of biologics over small molecule drugs may be warranted in this disease. __________________________________________________________________________________


INTRODUCTION
After decades of comparative "slow-down" in innovative therapies, the human genome project promised to harbor unprecedented advances. Pharmaceutical industry experts believed that more new drugs would be approved from 1997 to 2007 than the previous 100 years 1 . The reality has fallen short of this, with many in industry surprised by extent of failures of promising new compounds during clinical trial testing 2 accompanied by a sharp rise in clinical development costs 3,4 . In this study, for the first time we quantify this problem by examining a specific disease.
We compare the risk estimates for Crohn's disease described here against industry wide risk estimates recently reported 5 . Importantly, understanding the risk of failure for developing a new drug is valuable to both private and public sector as it makes decisions on the allocation for funding to support drug development across disease areas. Such risks are a product of factors that may be specific to the disease area itself, such as its unique etiology, accepted primary endpoint for drug approval, current treatment options and the extent of medical need.
Crohn's disease is a chronic inflammatory condition that is likely an autoimmune reaction by the patient to their gastrointestinal tract, that reduces life expectancy and can result in the need for surgery if left unchecked 6,7 . We chose Crohn's disease because of its comparative success relative to other disease areas, which has seen more drugs approved for this patient population (moderate to severe) than most other disease areas by a wide margin. However, Crohn's disease itself remains a serious condition, with the annual costs of therapy with any of new therapeutics exceeding ten thousand dollars USD per patient 8

NOVELTY OF THE WORK
Drug development is under siege with poor success rates over the last ten years. Despite this trouble there have not been systematic efforts to quantify the risk of drug failure at the level of a specific disease. Further, this work looks at a range of firms large and small, which allows us to examine the "commercial" contribution to drug failure, in addition to clinical considerations.

Crohn's disease study eligibility:
Clinical studies in Phase I, II or III in moderate to severe Crohn's disease conducted in the United States during 1998 -June 2008 were examined. Drugs were excluded if: they included mild Crohn's disease patients for treatment; did not assess patient outcomes such as remission or a reduction in disease burden such as the Crohn's Disease Activity Index (CDAI); failed prior biologic therapy; did not involve industry support or if their Phase I study in this indication was conducted before 1998; did not belong to one of the clinical trial testing Phases (I, II or III) or if the Phase I trial for the compound is ongoing. Clinical trial.gov was searched with "Crohn's disease" and then each trial result was examined with the above criteria in mind Databases and online tools: In addition to www.clinicaltrial.gov, for searches as early as 1998 press releases were also examined as a supplemental tool, and the following search terms were used: Crohn's disease + press release; Crohn's disease + Phase + press release; Crohn's disease + trial + press release; Crohn's disease + Phase + news and Crohn's disease + trial + news. Online tools used included: www.archive-it.org; PR Newswire (Factiva, accessed through University of Toronto Libraries); Business Wire (ProQuest 5000, accessed through University of Toronto Libraries); www.findarticles.com.

Clinical trial success classification:
Phase I clinical testing was classified as a "success" if the drug advanced to Phase II. Phase II clinical testing was a "success" if the drug was then advanced to Phase III. Phase III clinical testing was classified as a success if the US Food and Drug Administration (FDA) approved the compound and it remained on the market. For all drugs in this study we performed follow-up to March 1 st 2010 to determine what additional activity occurred for the compound in question and whether it advanced to the next phase of clinical trial testing, was approved, or withdrawn.
If the drug had momentarily been withdrawn from the market, but then allowed back onto the market (usually following a label change) this was still classified as a success.

Classification of clinical trial failures:
This was broken down into two components: medical failures and commercial failures. Medical failures refer to indications where the drug failed to meet its primary endpoint (Phase II and Phase III ) or had significant safety issues (all phases). However, there were examples of drugs that completed their trials "successfully" based on clinicaltrial.gov disclosures, press releases and conference proceedings, which showed no further signs of any clinical development. In cases where drugs showed no signs of any further development whatsoever for 2 years or more, these drugs were classified as commercial failures. By this it is inferred that the firm has decided further development of the compound may not provide sufficient return on investment compared to other clinical candidates. Alternatively, the firm itself may not have been able to raise the financing to continue with the development of the compound, which would also result in a commercial failure.

Drug and Company Classifications:
In general biologics were defined in accordance to classification by the FDA as "biological products are generally derived from living material--human, animal, or microorganism--are complex in structure, and thus are usually not fully characterized" 9 . Firms classified as "large firms" followed industry index classification (e.g. Standard and Poor's Pharmaceutical Index) and classified as small biotechnology companies if found in the index classification for that group of companies (Nasdaq Biotechnology Index).

RESULTS
Clinicaltrial.gov yielded more than 300 possible clinical programs, many of which did not satisfy our eligibility criteria. A single drug has many clinical studies and will account for many such "hits" in the database. Many studies were in mild Crohn's disease or did not belong to one of the three phases of clinical testing, and were thus excluded. Some clinical studies were not testing the ability of a drug to induce remission or reduce the CDAI score. In summary, a search of drugs that had completed at least Phase I clinical development in our time window revealed that 37 drugs had at least entered Phase I and 24 drugs passed Phase II, which includes an additional 5 drugs that had subsequently completed Phase III. Infliximab, one of the approved monoclonal antibodies for the treatment of moderate to severe Crohn's disease, was not included in this study as its' trials for this indication started outside our time window. Three new treatments approved for Crohn's disease were included in this data set.
In Figure 1, the clinical trial success rate in Crohn's disease is depicted from this data compared against previously reported rates for the industry as a whole (expressed as cumulative rates, taken from 1999-2004 data for a range of diseases 5 ). The probability of a compound transitioning into the next phase of clinical development is indicated. Cumulative pass rates refer to the probability of completing the current clinical trial and any preceding clinical trial phase successfully (e.g. the product of probabilities). Interestingly, in Crohn's disease there is an extremely high success rate (92%) in Phase I trials compared against industry expectations, suggesting little risk mitigation from this clinical testing phase. In examining Phase II trial success rates, we see a substantial drop to 40% in Crohn's disease. At this point the cumulative success rate does not change much given our high performance in Phase I. In Phase III we see a success rate of 50%, which gives us a total cumulative success rate of only 18%, which is inline with overall industry estimates of success. 5 In our next step of our analysis we looked at broad causes of clinical trial failure (Fig. 2). In Phase I, there were three failures, and based on our methodology these were classified as commercial in nature. In Phase II, we see roughly an equal distribution of trial outcome among medical failures (6 drugs) and commercial failures (7 drugs). If we then excluded commercial failures, this resulting smaller data subset, showed a trial success rate of 34 of 34 drugs having been successful in Phase I, 11 of 17 drugs successful in Phase II and 3 of 5 drugs successful in Phase III. Further, the cumulative success rate allowing only failures that are medically related is 39% (Fig. 3), is accordingly higher than industry expectations of 17% for cumulative success rates (Fig. 1). Notably, there are no commercial factors involved with failure during Phase III in Crohn's disease, as they appear to be most prevalent during Phase II.

Crohn's Cumulative Pass Rates
The performance of small firms ("biotechnology companies") and large firms ("big pharma") with respect to cumulative probability of success was examined (Fig. 3). Based on the parameters of our analysis used in this study for this disease area, it appears that large pharmaceutical companies do better than smaller more "innovative" firms in this indication. Confounding variables to this observation are discussed later.
Lastly, we noticed a significant amount of biologics compared to small molecule drugs, in Crohn's disease that appeared quite dominant in successful new product development (Table 1). In the time interval we examined, all three product approvals for moderate to severe Crohn's disease were biologics. Not a single small molecule drug has been approved. This is not due to lack of effort, as we came across almost as many small molecule drugs that completed Phase II (10 drugs including failures and compounds advanced to Phase III) as biologics (14 drugs including failures and compounds advanced to Phase III).

DISCUSSION
New drug development has experienced significant difficulty in the past several years that has weighed heavily on the industry as discussed in the US Food and Drug Administration's Critical Path Initiative 4 . While industry risk estimates have been reported 5,10,11 , what is currently missing are estimates for specific indications such as Crohn's disease, as reported here. There may be important differences across disease areas in determining clinical trial risk that cannot be determined by looking at risk estimates that factor several different disease areas simultaneously. We chose Crohn's disease to determine drug development risk, given its remarkable commercial success of new drugs compared to other disease areas. The cumulative success rate for Crohn's disease is comparable to that expected by industry (18% vs. 17%) where the same estimate is applied to all disease areas. In other estimates of industry risk, 18% seems favorable compared to value of 11% for the industry as a whole 10 . Drug development in other disease areas (e.g. Alzheimer's disease, cancers, Lupus) is nowhere near as successful as Crohn's disease for new drug approvals. Therefore, these other disease areas likely possess cumulative success rates well below 18%. In Crohn's disease alone, this failure rate estimate implies that 5 drugs need to have their development funded for one drug to be approved by the FDA on average. This number is worse if the therapeutic in question is a small molecule rather than a protein, consistent with earlier reports for the industry as a whole 12 .
Crohn's disease has seen 3 new drugs (natalizmab, certolizumab and adalimumab) approved in the time window we examined, that met our screening criteria. This made Crohn's disease an ideal choice for this study, given the small sample sizes encountered with this kind of research, a disease area was needed with as many drug approvals possible to provide some meaningful baselines for comparisons. While there is not enough data in this study to support the theory, we are inclined to speculate that first in class drug compounds (e.g. infliximab) carry a significant burden of clinical trial risk compared to later follow-on compounds that belong to the same class (e.g. adalimumab). Should government sponsored reimbursement practices disproportionately reward first in class drugs that are approved, and offer less reimbursement to follow-on drugs that have confronted less clinical risk in their development?
A deeper understanding of the risk of new drug development for a specific disease area is needed information by both public and private sector. Understanding the risk associated with developing a new drug for a specific disease area can be explicitly used in corporate financial modeling to determine the required level of return for the investment and arrive at a "go / no go" decision on whether to embark on the research. A drug with a large market can be a poor investment if the risk of failure for that disease is so high that other smaller market opportunities look more appealing given their reduced risk. From the standpoint of public policy, if we are to support new drug development for serious unmet medical needs, a framework needs to be erected to incorporate clinical trial risk and determine the appropriate level of support from such data.
While commercial risk is presented in this study, it has many assumptions in its identification and should be viewed with caution. There is always the distinct possibility that the reason a drug undergoes no further development, may be due to clinical concerns not disclosed by the firm. Some may have concerns regarding the inclusion of commercial risk in the calculation used to arrive at cumulative probability of new product approval. In the private sector, clinical development and the ability to move forward with a new compound is the composite of both medical and commercial considerations. If we want to gauge clinical development risk in the context of private sector activities, we believe this is the relevant estimate, which for Crohn's disease shows an 18% cumulative probability that a new drug will be successfully approved.
Large pharmaceutical companies appeared to outperform small biotechnology companies in this study showing very high Phase III success rates. However, it has to be kept in mind that the sample size is small for this analysis and longer-term comparisons will be necessary. However, if we accept this finding prima facie, the experience large pharmaceutical companies have in conducting large multi-centre studies may explain this outcome. This may also reflect greater selectivity on the part of large pharmaceutical firms in choosing the most promising candidates to advance into final clinical testing compared to small biotechnology companies. The dominance of biologic success in Crohn's disease is consistent with observations by others regarding the comparative success of such large molecule compounds when contrasted against small molecule compounds 2,5,12 . The superiority of biologics may be specific to Crohn's disease or indicate that there may be an inherent benefit to developing biologics over small molecules. In contrast, compared to small molecule compounds, biologics show inferior performance in other indications such as HIV or non-Hodgkin's lymphoma.over the same time interval reported here (unpublished data).
While aggregate industry numbers are useful, a real answer regarding the success of biologics over small molecules has to be examined at the disease level as performed here, in our view.

CONCLUSION
Our main conclusion is that the risk of clinical trial development for new drugs in Crohn's disease is in line with industry expectation, with a cumulative success rate of 18%. This is alarming, considering that Crohn's disease area is an industry success story for new drug approvals compared to other diseases, and suggests that the drug development failure rates for other disease areas, particularly in areas where there is limited research, are much worse. Future research will need to examine other disease areas so that both public and private sector funding may be based on disease specific systematic risk estimates for funding and reimbursement practices to reward new drug development.