Review of quality deficiencies found in active pharmaceutical ingredient master files submitted to the WHO Prequalification of Medicines Programme.

PURPOSE
The aim of this work was to determine the number and type of active pharmaceutical ingredient (API) quality deficiencies in API Master Files (APIMFs) as submitted to the World Health Organization (WHO) Prequalification of Medicines Programme (PQP).


METHODS
We conducted a retrospective review of API quality deficiencies identified following the assessment of new APIMFs for non-sterile APIs during a 6-year period from 1 January 2007 to 31 December 2012. All deficiencies were collected, classified and quantified according to the Common Technical Document (CTD) sections and subsections and as groups of commonly raised questions.


RESULTS
There were 5446 deficiencies collected from 159 APIMF deficiency letters by CTD section, by selected CTD subsections and by selected CTD subsections and year. More than 50% of the total number of deficiencies related to the manufacturing sections of the CTD, followed by deficiencies concerning the impurities, the API specification and the stability sections of the CTD. A pattern of API deficiencies across the different CTD subsections and over time was identified.


CONCLUSIONS
The most frequent critical deficiencies were related to how the specific manufacturing process and the key materials used, in particular the API starting material, impact the API impurities content. The number and pattern of APIMF deficiencies did not change over time. The results are compared to the findings in similar studies as reported by the United States Food and Drug Administration (USFDA), the European Directorate for the Quality of Medicines (EDQM) and the European Medicines Agency (EMA) and similarities and differences are discussed. Our findings highlight the need for greater guidance and technical assistance for API manufacturers submitting APIMFs to the PQP.


INTRODUCTION
The World Health Organization (WHO) Prequalification of Medicines Programme (PQP) has assessed the quality, safety and efficacy of medicines, in particular generics, since 2001 (1). WHO PQP was created at the request of United Nations Organizations and other parties to ensure that the quality of priority essential medicines procured are verified by an independent body, since typically these medicines have not previously undergone assessment by a stringent regulatory authority (SRA) (2). The WHO PQP assesses medicines in close cooperation with experts from national regulatory authorities from both developed and developing countries and relies on specific WHO, as well as other stringent international guidelines. PQP covers a few therapeutic areas of critical importance (malaria, HIV, tuberculosis, influenza, reproductive health, neglected tropical diseases and zinc for the treatment of diarrhoea) and most of the prequalified medicines are manufactured and sold in developing countries. More than 350 medicinal products have been prequalified to-date, most of them multisource products (generics). The list of prequalified medicines is widely used at national and international level by agencies or organizations that supply medicines in environments where national regulatory systems are weak or non-existent, to ensure that resources are not spent on medicines of unknown quality, safety and efficacy (3). WHO is not a supranational regulatory authority and cannot bypass national regulatory agencies. Specifically, in prequalifying a product WHO only determines the acceptability of the product for procurement in terms of the product's quality, efficacy and safety. WHO does not issue a marketing authorization for the prequalified product. ________________________________________ To be marketed in individual countries a product still needs to be registered by national regulatory authorities, although previous prequalification of the product can facilitate its national registration. Within PQP there are specific initiatives to further facilitate national registration of prequalified products for the purpose of increasing access to quality medicines and avoiding duplication of regulatory effort (4).
The assessment of the quality part of a medicine application submitted to PQP includes the review of two critical components: the active pharmaceutical ingredient (API) and the finished pharmaceutical product (FPP). This is the approach adopted by all SRAs since a goodquality medicine starts with a good-quality API.
A recent study performed by PQP surveyed the type and extent of FPP deficiencies, but the API component was only partially addressed (5). The number of API deficiencies observed within PQP is high. Improving the quality of the API component of an application can reduce assessment time and ultimately speed up access to prequalified medicines. Identifying trends or patterns in API deficiencies might therefore help improve generic medicine applications submitted to the PQP.
In a generic medicine application to PQP, the quality of APIs can be supported in different ways, but the most common option utilises the submission of API master files (APIMFs) (6)(7). Like in WHO PQP, the use of API master files, also known as Drug Master Files (DMFs) or Active Substance Master Files (ASMFs) has been a common way of presenting API information to SRAs for many years, primarily because this type of procedure preserves the know-how of the API manufacturer and the related confidential information (8)(9).
The APIMF procedure started at the end of 2006, but its use has increased significantly and most of the API information assessed as part of the applications for prequalification is now in the form of APIMFs. In addition, since late 2010 PQP also started accepting APIMFs in conjunction with applications to prequalify APIs (10). In the framework of this scheme the acceptance of an APIMF means that if the specific API is manufactured in a site compliant with Good Manufacturing Practices (GMP), it is considered of acceptable quality in all aspects described, except in those that need further assessment in the context of the FPP such as the API properties that can influence the product performance.
The objective of the present study was to review the API quality deficiencies observed in the assessment of APIMFs received by the WHO PQP to identify the types of deficiencies observed and any trends over time. It was hoped that this review could guide PQP efforts in providing specific information to applicants on assessment requirements, or in undertaking other measures, to reduce the API deficiencies in future dossiers.
The study presents an exhaustive review of API quality deficiencies quantified to the level of the Common Technical Document (CTD) subsection. It covers 6 years of activity of the APIMF procedure within the WHO PQP. We believe that such a detailed review in the field of APIs is without precedent. Furthermore, to our knowledge this is the first time a study on quality deficiencies has been performed on APIs intended for generic medicines from emerging markets.

MATERIALS AND METHODS
The information contained in submitted APIMFs is compiled by the manufacturers in the format of module 3 of the CTD (11-12). After each assessment, a letter containing a list of the API deficiencies is sent to the applicant. The applicants can then respond and update their dossiers with the requested information necessary to verify the quality of their API. There can be multiple rounds of questions.
We investigated the type and extent of deficiencies raised in the first deficiency letter following the APIMF assessment. The decision to review the first PQP deficiency letter was made since this represents the outcome of the assessment of the initial submitted APIMF. This is advantageous since the initial APIMF covers all aspects of the API quality and is "freshly" compiled by the manufacturer in a standardized document. The review of the first deficiency letters, therefore, offers valuable information on the applicant's knowledge of the requirements for APIMFs submitted to PQP.
The study covered deficiency letters sent in a 6-year period from 1 January 2007 to 31 December 2012 and included all new APIMFs for non-sterile APIs that were assessed for the first time. A total of 7 APIMFs received in this period were excluded from the study since some of the quality deficiencies had been communicated to the applicant outside the first deficiency letter (as part of the assessment of the FPP or as part of the assessment of sister files).
Deficiencies were collected from a total of 159 APIMF letters of which 128 were for APIs of chemical synthesis, 30 for semi-synthetic APIs and one for an API prepared by fermentation. The overwhelming majority of these APIMFs were submitted in support of FPPs seeking prequalification, but also included were APIMFs submitted in support of applications to prequalify the API itself, since the quality assessment requirements are identical. The 159 APIMFs had been submitted by 51 applicants (23 from India, 20 from China and 8 applicants from other countries).
All quality deficiencies were collected and classified according to their CTD sub-section relevance. Each question raised in the letters to the applicants was analysed and the individual requests were extracted, recorded as deficiencies and classified according to the CTD subsection.
Only the quality-related deficiencies were classified, other deficiencies related to administrative aspects of the APIMF procedure were not tabulated. In addition, deficiencies related to process validation/evaluation and manufacturing process development were excluded from this study. Neither of these sections commonly form a substantive part of an APIMF application to PQP. In the first instance this is because the majority of APIs submitted for assessment are non-sterile and therefore the process validation/evaluation section is optional according to the CTD guideline (11). Secondly manufacturing process development sections in the APIMFs submitted to PQP typically contain very limited data. Thus, the type and number of questions raised in these sections were not representative of APIMF submissions in general.
An excel database was created with all the collected quality-related deficiencies and these were classified in four ways. and analytical procedures, respectively were grouped together for reporting purposes since the questions raised often referred to the same quality deficiency and were interrelated. The PQP generic guideline and international and European guidance were taken into account to assign the different deficiencies to the various CTD subsections (6,11,13).
Third, deficiencies (according to the subsections identified in Table 1) were also assigned by the year in which the deficiency letter was sent in order to identify any trends over the time period studied.
Mean, median, minimum, maximum (range), standard deviation and percentage of the number of specific deficiencies (with respect to the total number of deficiencies) were calculated for each CTD section, CTD subsection and by year.
Finally, within each CTD subsection "specific" deficiencies were listed descriptively in terms of commonly raised groups of questions and quantified. Their frequency was also quantified with respect to deficiencies within the particular subsection and with respect to the total number of deficiencies. For some of the CTD subsections (e.g. concerning the API manufacture) deficiencies were pooled in broad groups due to the huge diversity of the reported deficiencies.

RESULTS
We collected a total of 5446 quality-related deficiencies from 159 deficiency letters. The mean number of APIMF deficiencies was 34 (standard deviation: 12.3) and the median was 34 (range: 9-77 deficiencies). Table 2 shows the median and range of the number of APIMF deficiencies raised; the mean number of APIMF deficiencies and standard deviation; and the number of questions raised in each section as a percentage of the total number of deficiencies. Figure 1 graphically presents deficiencies by CTD subsection as the percentage of the total number of deficiencies raised.

Description of frequent deficiencies
The most frequent specific deficiencies within each subsection are shown in Table 3.

DISCUSSION
In this study we reviewed the API deficiencies identified over a period of 6 years since the initiation of the APIMF procedure in the PQP. To our knowledge this is the first time a review of API quality deficiencies covers all dossiers over such a long time period; the European Directorate for the Quality of Medicines (EDQM) top ten deficiencies documents are based on the contents of a sample of the deficiency letters in each year and the European Medicines Agency (EMA) quality deficiencies publication included questions raised over a period of only two years (16)(17)(18)(19)(20)(21). In addition our study includes the quantification and description of all deficiencies to the level of the CTD subsection; the EDQM reported the ten most frequent groups of deficiencies and in the review of deficiencies raised by EMA these were quantified to the level of the CTD section only (16)(17)(18)(19)(20)(21). Furthermore, to our knowledge this is the first time a study on API quality deficiencies has been performed on APIs intended for generic medicines from emerging markets. Our study allows an in depth exploration of the most common issues arising in APIMFs within PQP and their evolution over time.  The information on the API manufacturing process was incomplete. For example: -The process description was not sufficiently detailed or the flow sheet of the scheme of synthesis did not include molecular formulae and weights, chemical structures of starting materials and intermediates or the use of reagents, catalysts and solvents in each step.
-Solvents used were not clearly identified in the process.
-The maximum batch and scale of manufacture were not clearly stated.
-Neither the blending of API batches was clearly stated nor was it confirmed that each batch incorporated into the blend was individually tested and found to meet specifications set for the final API prior to blending.
-Additional steps such as milling and micronization of the API were not clearly stated or described.
-The use of API alternative processes was not clear.

375
47.5 6.9 The recovery of materials was not clearly stated or deficiencies related to these practices were identified. The need for good quality APIs in a globalized world is an increasing focus of pharmaceutical regulators' international forums and increasing complex API chains are a challenge (14). SRAs continue their efforts to publish guidelines for applicants focusing on the quality of APIs (15). Limited information is available in the public domain with respect to the actual API deficiencies found by assessors in SRAs, but specific studies have been published by the EDQM (16)(17)(18)(19)(20). There are also two publications on the review of API deficiencies raised by the United States Food and Drug Administration (USFDA) and the EMA (21)(22).
The WHO PQP also faces problems regarding the quality of APIs and their associated APIMFs. Since a medicine cannot be prequalified if the quality of the API component is not satisfactory, the high number of API deficiencies causes delays in the acceptance of the APIMF and by extension delays in the medicine's prequalification. This is not in the interest for neither PQP, the API manufacturer, the FPP manufacturer, nor ultimately the patient.

By CTD section
More than 30 deficiencies were found on average for each of the APIMFs. Deficiencies were not uniformly distributed within the CTD sections (Table 2). This was expected since in an APIMF some sections, such as the ones for general properties (3.2.S.1), reference standards (3.2.S.5) and container/closure system (3.2.S.6), contain less information than others and therefore the number of deficiencies was also anticipated to be lower.
We observed that more than half of the total deficiencies were related to the API manufacturing section (3.2.S.2). In fact, this is also the only section where all APIMFs showed at least one deficiency and the upper limit of the range was also significantly higher compared to the other sections. The API manufacturing section is critical. This confidential section is where the API manufacturing process is fully described along with all the controls applied to the process and to the materials used and needs to be carefully reviewed since it has an impact in almost all other sections. This section ultimately supports the API efficacy, since obtaining the desired API structure through the method of preparation determines the pharmacological effect, but most importantly, this section is critical to establish the impurity profile of the API and therefore its safety (23). In addition to related substances and degradation products, every material used in the API preparation is potentially capable of being carried over into the final API and occur as an impurity. Similarly, the quality of the materials used in the API preparation, and in particular their purity, also may have an impact on the final API impurity profile. Since impurities do not provide any therapeutic benefit, they need to be controlled and reduced to a minimum (24).
In addition to the manufacturing section, CTD sections concerning the API control each accounted for more than 10% of the total number of deficiencies. In order to ensure that the properties of the API affecting the efficacy and safety are consistently reproduced and maintained during the API retest period or shelf-life, there needs to be adequate specifications and analytical procedures to verify these characteristics when the API is released and throughout the retest period (25). This itself is only possible if the analytical procedures are validated for the intended purpose and are able to detect those impurities defined by the manufacturing process as well as degradation products (26). Therefore, most of APIMF deficiencies, when not related to section 3.2.S.2 were related to the impurities, specifications, analytical procedures, and stability.

By CTD subsection
When viewed by subsection, the pattern of deficiencies (i.e. the percentage of APIMF deficiencies in the different CTD subsections of the total number of deficiencies) was as stated in Figure 1. The 5 most affected CTD subsections were characterized and are discussed in more detail below. Comprehensive descriptions of all deficiencies along with their specific frequency can be found in Table 3.
We found that the most frequent deficiencies reported by PQP were related to the control of materials (subsection 3.2.S.2.3). The majority of these deficiencies were due to insufficient information on the methods of preparation and sources of the starting materials. Sufficient information on the preparation of the starting material needs to be provided in an APIMF so that assessors can verify that impurities arising from the manufacturing process are controlled. However obtaining this information from third parties is often difficult for API manufacturers.
Redefinition of the starting material was requested for a third of the APIMFs included in our study. In recent years there has been a global tendency from API manufacturers to propose shorter synthesis and more complex starting materials (27). It is typically cheaper for the API manufacturers to buy reaction intermediates from secondary manufacturers that do not operate under API GMP, but this makes it difficult for regulatory authorities to assess if potential impurities arising during preparation are well controlled or removed. When the starting materials proposed by the applicants are rather considered as intermediates, the assessors will request a redefinition to an earlier molecule. The request for redefinition of the starting material has two important consequences. First, significantly more detailed information is needed on the earlier steps of the synthetic process and the materials used in order to assess the potential impurities, leading to significant revisions in the APIMF. Second, since the initially proposed starting material is now rather considered to be a reaction intermediate, its preparation should follow GMP requirements, which may present a challenge (15, 28).
As well as the problems with the starting material quality, a further group of issues within section 3.2.S.2.3 (~20%) related to the unsatisfactory specifications of raw materials, reagents and solvents. For example, not all raw materials were tested for identity and assay, and the purity of some materials was also an issue, in particular when their impurity content represented a risk; a common example being the control of benzene in solvents, since it can be potentially carried over into the final API.
The second most frequent group of deficiencies after the control of material subsection were related to the description of the API manufacturing process (subsection 3.2.S.2.2). Information on the API preparation was often incomplete making it difficult to determine the impurities introduced or formed in the process or the final physical properties of the API. The scale of manufacture was often not stated and information on the procedures for the recovery of materials, which may have a potential impact in the impurity profile, was unsatisfactory in most APIMFs. The reuse of materials is permitted, but it was not always declared in the APIMFs and only subsequently identified by PQP inspectors following inspections of the related manufacturing sites. Therefore, when they exist, they need to be declared and documented so that the impact on the quality of the API can be assessed (6,29).
Stability deficiencies (section 3.2.S.7) were the third most frequent deficiencies observed, in particular relating to the acceptance of the retest periods proposed by the applicants, which in many cases were inadequately supported. In addition, for almost all APIMFs, either a storage statement was not proposed for the API labelling or the recommended storage temperature statement was considered unsatisfactory taking into account the climatic zones in which the product was expected to be marketed (30).
The discussion on impurities (subsection 3.2.S.3.2) was deemed insufficient in most of the APIMFs studied and was the fourth most affected CTD subsection in terms of number of deficiencies. The profile of impurities of the API was not well characterized in most of the APIMFs reviewed. Applicants often did not discuss the carry-over into the final API of reagents, solvents or catalysts used in the process or take into account potential impurities such as by-products and intermediates, which vary depending on the route of synthesis, or degradation products. This is considered critical since it impacts directly on the acceptability of the API specification (subsection 3.2.S.4.1), which establishes what impurities need to be controlled and their maximum acceptable level, or the analytical methods (subsections 3.2.S.4.2 and 3.2.S.4.3) that are needed to detect them. Further, a specific discussion on genotoxic impurities was absent in almost one third of the dossiers.
Fifth were deficiencies related to the API specification subsections (3.2.S.4.1 and 3.2.S.4.5). As stated above these were predominantly due to the fact that the limits proposed for impurities (including residual solvents) were not acceptable according to international requirements or due to the incompleteness of the specification. In addition, the format of the API specification provided in the dossier was unsatisfactory for more than half of the APIMFs of the study. This included copies of the API specifications that were not signed, dated and version-numbered or that did not include reference to analytical procedures.

Comparison with API deficiencies reported by USFDA
Delays in the approval of generic drugs due to deficiencies in the DMFs have been published by the USFDA (22). Illustrative examples of DMF deficiencies were reported by USFDA with respect to the manufacture section, the quality of starting materials, reagents and solvents, the setting of impurities controls, the API specifications and the stability (22). Since the USFDA review did not report numbers, no quantitative comparison can be made but similar to PQP, the issue of starting materials was stated in the USFDA article as being one of the most common problems.

Comparison with API deficiencies reported by EDQM
Reports on the top ten API deficiencies were published by the EDQM each year in the period 2007-2011 (16-20). The absolute numbers of questions raised were different from those reported by us, and cannot easily be compared since the method of collation for reporting by the EDQM is unknown. It is apparent however that when comparing the average number of questions raised, or the lower limit of the range of the number of questions raised, the number of deficiencies identified by PQP is higher than that reported by the EDQM.
One reason for the larger number of deficiencies reported by us could be the methodology used (i.e. how deficiencies were collected and classified). Another reason is that in PQP the screening of new APIMFs is administrative only and therefore all technical issues are communicated in the first deficiency letter, whilst the assessment of a new Certificate of Suitability to Monographs of the European Pharmacopoeia (CEP) applications in the EDQM is preceded by a technical validation. This validation at the EDQM can block the application until identified issues, such as those related to the API starting material method of preparation and specifications or the discussion of the presence of impurities and solvents in the final API, are resolved (31). Another reason for the quantitative difference could be the population of manufacturers submitting APIMF applications to the PQP and their specific APIs. PQP actively encourages APIMF submissions to the Programme by manufacturers that may have limited exposure to SRA assessment in order to address gaps in the availability of priority medicines. Further, capacity building of such manufacturers is an important aspect of PQP's activities. To this extent many manufacturers use the experience gained as a means to prepare themselves for submission to SRA jurisdictions. Hence it is likely that, on average, the applicants included in our study were less experienced (at least in terms of submitting a dossier to an SRA) than the ones presenting CEP applications. This naivety of expectation together with the absence of a technical screening would contribute to a greater number of deficiencies in each section.
In contrast, qualitative analysis between PQP and EDQM deficiencies showed that when the types and relative occurrence of deficiencies were compared, the areas of greatest deficiencies were similar. In both studies the subsection on the control of materials 3.2.S.2.3 was confirmed as critical. For example, in the year 2011, the top EDQM deficiency was the absence of discussion on the carry-over of impurities and by-products from key materials in the process into the API. The EDQMs second, fourth and fifth most common deficiencies were also related to the nonacceptability of the proposed starting materials, their suppliers or their specifications. All four of these deficiencies were covered in our study by deficiencies under section 3.2.S.2.3, our most common deficiency (20.2% of the total number of API deficiencies). These similarities suggest that manufacturers applying to either PQP or EDQM experience the same difficulties in meeting or understanding international requirements. In fact, worldwide the sourcing of APIs by medicine manufacturers is largely concentrated in a few countries such as India and China and therefore all regulators face the same challenges to ensure their quality (14).

Comparison with EMA reported API deficiencies
In the EMA study the API assessment was part of the assessment of the finished product dossier, because the API information was provided as part of a full dossier submitted by the marketing authorization applicant (21). Still the API deficiencies were broken down into CTD sections, which allowed a quantitative and qualitative comparison.
From a quantitative point of view the absolute numbers of API deficiencies were higher for PQP results suggesting that our applicants were also less experienced than the ones applying for the European Centralized Procedure. This is only an approximation however, since the methods used to record deficiencies are likely to be different and we could not compare ranges.
Interestingly, and in contrast with our findings compared to the EDQM deficiencies, we found clear differences in comparison with the pattern of the deficiencies reported by EMA in the same years (2007)(2008). In the EMA study the deficiencies related to the API control section (3.2.S.4) were more frequent than the ones related to the manufacturing section (3.2.S.2), as opposed to our study where the deficiencies concerning the API manufacture were the most common. In the EMA study there was only one major objection referring to the quality of the starting materials and the complexity of starting materials as such was not reported as a deficiency. This is unlikely to be due to differences in the assessment policies being applied because similar international guidance is used in the PQP. Most of the medicines assessed in Europe through the centralized procedure are not generics and rather include new APIs (32). It is likely that the majority of medicines covered by the EMA study were innovator products for which APIs would have been manufactured from simpler molecules. Companies dealing with this kind of medicines are generally well resourced and experienced firms that are responsible for the synthesis of the API from simple molecules thereby avoiding issues related to redefinition of starting materials and can provide full information on the starting materials. Moreover, since pharmacopoeial monographs are not available for new APIs included in innovator products applicants need to justify their specifications in depth, in particular the limits for impurities and their toxicological qualification (often based on non-clinical studies). This would explain a larger relative number of questions related to the control of the API with respect to the API manufacture. In our study though, at least 70% of the APIs were described in one of the PQP recognized Pharmacopoeias, as defined in the PQP quality guideline, and therefore the proportion of deficiencies on the specifications, analytical methods and acceptance limits for impurities was relatively lower with respect to issues related to section 3.2.S.2.
In any case, despite this important difference, most of the API deficiencies identified as part of the EMA study were also reported in the APIMFs included in our study.
We were unable to find any studies on API deficiencies in European ASMFs used for registration of generic medicines through national or European decentralized procedures.
Overall, from the data reviewed it was found that the pattern of API deficiencies as reported by us was consistent with the findings by SRAs, for existing APIs used in generic medicines. Most notably the most common deficiencies observed by PQP and EDQM both related to how the impurity profile is impacted by the specific manufacturing process and the key materials used.

Pattern of deficiencies over time
The quantification of deficiencies in our study encompassed 159 APIMFs assessed by PQP over a period of 6 years.
We found that the pattern (i.e. the average number of APIMF deficiencies in the different CTD subsections by year) was consistent across the study period ( Figure 2). Nevertheless, there were some differences between years in terms of absolute number of deficiencies. The total number of deficiencies per APIMF was relatively constant in the period 2008-2011, but differences were observed with respect to the years 2007 and 2012.
In 2007, the number of deficiencies per APIMF was lower compared to all other years, as was the number of applications studied (n=10). This was visible in particular for subsections 3.2.S.2.2 and 3.2.S.2.3. This was due to the fact that the APIMF procedure within the PQP had only just started and was in a pilot phase.
An increase in absolute number of deficiencies in the control of materials subsection was observed in the year 2011 compared to previous years. In 2012 an increase was observed not only for the control of materials subsection but also for the manufacture and the impurities subsections and was reflected further in a higher total number of deficiencies.
Investigations into the the specific details regarding the deficiencies revealed that this was not due to a lowering of APIMF quality during those two years. By 2011 and 2012 WHO PQP had already acquired significant experience with the communication of deficiencies to the manufacturers, in particular with respect to their responses to the issues relating to the quality of the starting materials, which were generally not completely resolved at the first attempt. Therefore, deliberate changes were made to the way requests for information were made to applicants. Essentially deficiencies raised were presented more precisely and in greater detail commencing in 2011, in order to anticipate any further issues and in the hope of reducing the time for the APIMF acceptance. Consequently, the higher number of deficiencies in 2011 and 2012 should not be seen as a trend related to dossier quality.
Overall it can be concluded that the pattern of deficiencies reported by year remained constant over time. To further confirm this, and given that the APIMFs were submitted by a total of 51 applicants, we decided to investigate selected applicants that had submitted more than one APIMF during the study period. This was done to determine if involvement with PQP had generated any improvements in the type and number of deficiencies, as a result of an improvement in the preparation of their APIMFs. Three applicants representing 36% of the total received APIMFs were investigated (data not shown). Both the pattern of deficiencies and the average number of APIMF deficiencies for a given manufacturer were similar over time, suggesting that the quality of the APIMFs submitted by these applicants did not improve over time. It must be pointed out however that the material was limited in that the number of APIMFs and companies was small.
The APIMF may not be a priority for the company however, since the quality of APIs is not assessed in all jurisdictions and therefore the APIMF acceptance is not necessarily a prerequisite for FPP manufacturers seeking to register their medicines in countries with weak national regulatory systems. Nonetheless, it seems unlikely that applicants are unwilling to improve. For example a clear learning effect has been seen for finished product dossiers in PQP over time for a number of applicants (data not shown). The number of applications received for the APIMF procedure shows that there is an interest in prequalification and therefore a prompt questionfree assessment leading to a quick acceptance of the APIMF must necessarily be in the interest of the API manufacturers. Overall it is more likely that the reason why APIMFs did not improve in general is that the available guidance for applicants is insufficient, either because the guidance is not sufficiently clear, or because there is insufficient guidance on APIMF content expectations.
With respect to the needs of greater guidance to industry, WHO PQP published in May 2012 its generic quality guidance, adopted by the WHO Expert Committee on Specifications for Pharmaceutical Products (6). This guideline is specifically addressed to PQP applicants and should be an instrument for applicants to decrease the number of deficiencies. The long-term effects of its implementation in the API field will need to be determined to see whether it has been helpful for applicants as regards the technical content of APIMFs presented to PQP from 2013 onwards. Similarly the finalization of the International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) Q11 guideline on development and manufacture of drug substances in May 2012 could have an influence on the number of deficiencies, especially in the area of the selection of starting materials (15).
PQP includes as many details and clarifications as possible in the deficiency letters to guide applicants through the actions that they need to undertake to ensure the acceptability of their APIMFs. Deadlines for the additional data to be provided are not set and the number of rounds of communication is not restricted. Furthermore, PQP also arranges technical assistance to manufacturers of APIs of particular priority to the programme and this includes assistance on the compilation of an APIMF. PQP also undertakes training seminars for API manufacturers. In addition to the provision of updated guidance, greater opportunities for training on PQP requirements would assist manufacturers' understanding of PQP guidance and ultimately reduce the number of deficiencies. The primary goal of these capacity building activities focusing on API manufacturers is to increase the availability of priority APIs of acceptable quality in the shortest time possible.
It is critical to have quality-assured APIs available for medicines manufacturers. The type and extent of API deficiencies found in our study show that for most of the APIMFs assessed within PQP the API did not meet international quality standards initially. Substandard APIs may have an impact on the FPP quality and safety that can lead to substandard medicines. Prequalified medicines have shown a low failure rate in quality monitoring surveys and this to a certain extent is due to the use of quality APIs (33). By engaging with API manufacturers to make quality API sources available, PQP also raises market expectations regarding API quality to the detriment of substandard products.

CONCLUSION
Our review of active pharmaceutical ingredient deficiencies in active pharmaceutical ingredient master files assessed within the World Health Organization Prequalification of Medicines Programme during a 6-year period from 2007 to 2012 identified deficiencies with respect to all Common Technical Document subsections. The majority of the deficiencies were related to critical areas of the dossier such as the control of materials, the active pharmaceutical ingredient method of preparation, the stability and the impurities and their control. The results also indicate that the active pharmaceutical ingredient master file procedure, initiated at the end of 2006, is a mature procedure for the assessment of the quality of active pharmaceutical ingredients, since it allowed for the detection of a rather constant pattern of active pharmaceutical ingredient deficiencies over a long period time.
The pattern of deficiencies in active pharmaceutical ingredient master files was similar to the findings of stringent regulatory agencies such as the European Directorate for the Quality of Medicines that also assess active pharmaceutical ingredients for generic medicines showing that equivalent requirements are applied in the Prequalification of Medicines Programme and that manufacturers applying to either the Prequalification of Medicines Programme or the European Directorate for the Quality of Medicines experience the same difficulties in meeting or understanding international requirements. However applicants to the Prequalification of Medicines Programme seem to be less experienced and moreover continued to submit active pharmaceutical ingredient master files of the same deficient standard, since the number and type of deficiencies did not appear to change over time.
Our findings stress not only the need to promote the use of prequalified medicines, for which the active pharmaceutical ingredient component necessarily needs to be assessed and accepted but also the need for additional guidelines for active pharmaceutical ingredient manufacturers in the specific areas identified and continued investment in resources to help the manufacturers compile their active pharmaceutical ingredient master files.
In addition to the provision of public guidance documents, technical assistance for manufacturers in the areas of active pharmaceutical ingredient quality requirements and the compilation of a new active pharmaceutical ingredient master file should continue and be intensified so that common deficiencies can be avoided. This activity would support the goal of the Prequalification of Medicines Programme of medicines for everyone, since reducing the number of deficiencies in active pharmaceutical ingredient master files will lead to acceleration of active pharmaceutical ingredient master file acceptance and ultimately speed the prequalification of generic medicines using the active pharmaceutical ingredient master file procedure.
Finally, the results are an important reminder that the quality of an active pharmaceutical ingredient cannot be assumed or taken for granted, since the same types of deficiencies were persistently observed both within the Prequalification of Medicines Programme and by other regulators. The approval for sale of medicines in which the active pharmaceutical ingredient has not undergone sufficient assessment jeopardises the quality, efficacy and safety of these medicines and as a consequence presents a challenge to national regulatory agencies in resource limited settings. In addition, repeated active pharmaceutical ingredient deficiencies across applications unnecessarily increase the time for approval of the active pharmaceutical ingredient and as a consequence the time-to-market for life saving medicines. It is essential therefore that technical guidance and assistance is available to active pharmaceutical ingredient manufacturers in order to increase the availability of quality-assured active pharmaceutical ingredients and to avoid unnecessary delays to medicine approvals.