Methodological Characteristics of Clinical Trials: Impact of Mandatory Trial Registration

- Purpose : Numerous studies across multiple specialties have evaluated the impact of trial registration on quality of study reports and found significant improvements over several domains. However, the impact of mandatory trial registration on the quality of clinical trial protocols remains hitherto unexplored. Methods: We carried out a retrospective cohort study of clinical trial applications submitted to drug regulatory authority of India for initial review with the objective of comparing methodological characteristics of their protocols. Since trial registration was made mandatory in the country in June 2009, we selected two study periods as between January 2007 to May 2009 (Period I) and July 2009 to December 2011 (Period II). Seventy-five protocols were randomly selected using a computer-generated list for each study period, making a total of 150 protocols. Data on twelve key methodological characteristics were collected including clearly defined primary outcomes, randomization, blinding, use of control group, statistical methods, handling of withdrawals amongst others. Results: More than 3/4th of the trial applications in the two study periods were for new chemical entities and nearly 90% were pharmaceutical industry sponsored studies. Comparing the period before and after implementation of mandatory trial registration, description of clearly defined trial outcomes improved from nearly 42% to 80% (p<0.001), sample size justifications increased from 38% to 70% (p<0.001) and use of allocation concealment improved from 24% to 49% (p=0.001). Marked improvement was also noted for blinding, description of statistical methods and handling of withdrawals and dropouts. Remaining characteristics did not change significantly between the two study periods. The mean cumulative scores for the study protocols improved significantly from 7± 0.296 in the first period to 8.93± 0.346 (p<0.001) in the second period. Conclusions : Our study found a significant improvement in the methodological quality characteristics of the protocols particularly in elements related to minimization of bias and statistical methods, which could be attributed to mandatory trial registration. Overall, the significant improvement was limited to global clinical trials, and room for improvement was noted for two quality characteristics – proportion of randomized studies and trials adequately describing the generation of allocation sequence. __________________________________________________________________________________________


INTRODUCTION
Modern drug discovery and development is an increasingly time-consuming, risky and an expensive process. The twin objectives of saving of time and money have long resulted in globalization of clinical trials to developing countries. According to a recent analysis, 90 % of new drugs approvals in 2017 were based on data from clinical trials conducted at least in part outside the U.S. and Canada (1).
Factors that have favored the globalization of global clinical trials to the developing world include lower overall operational costs, ease of recruitment of treatment naive subjects in a timely manner, availability of contract research organizations with extensive overseas connections, and less publicized reasons including often less stringent regulations and ease of conduct of placebocontrolled trials (2,3). Adherence to Good Clinical Practice (GCP) standards ensures that rights, safety, and confidentiality of trial participants are protected, and the results reported are credible and accurate. It also serves to ensure that the investigators follow uniform scientific, ethical and legal norms for studies conducted across multiple national and international sites (4).
As per GCP guidelines, ensuring trial quality is a continuous process, which begins with designing of study protocol, is critical during conducting, monitoring, recording, analyzing and continues during the reporting of trials. Hence, it is easy to foresee that a deficiency in the quality of protocol would adversely affect virtually all aspects and the eventual quality and legitimacy of the clinical trial process. It is widely believed that an important measure to strengthen the credibility of clinical research is to ensure better designed and more transparent protocols. However, it is also true that generally, protocols are not completely respected and adhered to (5). Several audits in the past have compared accepted protocols with the subsequent published articles and found that adherence to protocols was suboptimal and selective (6)(7)(8). This could be ascribed to the fact that while there are several guidance documents available to aid reporting of clinical trials and other study designs, there is little to guide the composition of a robust protocol (5).
The Clinical Trials Registry of India (CTRI) was launched in 2007 and since then has grown significantly as a publicly accessible resource for clinical trial data. Although, initially launched as a voluntary exercise, trial registration in the CTRI was made mandatory by the Indian drug regulatory authority, Central Drugs Standard Control Organization beginning from 15 th June 2009 (9). Since April 2018, CTRI is accepting and registering studies only prospectively, that is prior to the enrolment of first patient (10). The benefits of trial registration are substantial including improved transparency of trial designs and methods, enhanced accountability of researchers, reduction of the potential for patient harm, duplication of studies, selective reporting and publication bias and improvement of trial methodological quality by providing open review of protocols (11,12). Several studies across various medical and surgical specialties have evaluated the impact of trial registration on quality of study reports and found significant improvements over several domains and accepted standards of reporting (13)(14)(15)(16)(17). These studies have found substantial differences in terms of adherence to methodological standards of the CONSORT statement between studies that reported trial registration and those that did not. In fact, a study found better reporting quality of trial registrations as compared to respective published articles in journals (18). However, no study in our knowledge has investigated the impact of trial registrations on the quality of clinical trial protocols. We hypothesized that the introduction of mandatory trial registration has the possibility of positively influencing trial protocols by enhancing transparency and presenting crucial trial elements in the public domain. Therefore, the current study was carried out to evaluate the impact of mandatory Clinical Trials Registry-India (CTRI) registration on the methodological characteristics of clinical trial applications submitted to the drug regulatory authority for trial approval.

METHODS
The study was conducted as a retrospective cohort study of a subset of original clinical trial applications submitted for initial review to the drug regulatory authority of India. Prospective interventional studies involving evaluation of new chemical entities (NCE) as well as biological products like vaccines and recombinant therapeutic proteins were included. The objective was to compare the methodological characteristics of these protocols submitted before and after Clinical Trials Registry-India (CTRI) registration was made mandatory in the country. A total of one hundred and fifty clinical trial protocols submitted for review to the Investigational New Drug (IND) committee and the New Drug Advisory Committee (NDAC) of the drug regulatory authority, Central Drugs Standard Control Organization (CDSCO) were evaluated. These committees were constituted to advise the regulatory authority in matters related to review and regulatory approval of clinical trials and new drugs. Seventy-five clinical trial protocols were randomly selected from the dossiers submitted for review in the period January 2007 to May 2009 and the rest of the 75 protocols were randomly selected from submissions made during the period July 2009 to December 2011 using Microsoft Excel software. These drug trial applications were selected from the research proposals sent to the Department of Pharmacology for review by the subject expert, YKG and were kept in the "IND and new drug clinical trial documents" archive of the department. We excluded amended protocols that were resubmitted by the trial sponsors in response to comments and observations of the drug advisory committees, other protocol amendments initiated and submitted by the sponsor and any protocols that were voluntarily withdrawn by the sponsors.

Selection of methodological characteristics
In the absence of a generally accepted method for the assessment of quality of clinical trial protocols at the time of submission of these trial applications, a set of 12 methodological characteristics were considered for evaluation of clinical trial protocols. The selection of these characteristics was based on the following instruments/ guidelines: i.
CONSORT statement checklist ( Definitions of methodological quality criteria used in the study Based on assessment of above mentioned instruments, twelve methodological quality characteristics were identified and defined as follows: 1) eligibility criteria of the study population, 2) controlled clinical trial, 3) randomization, 4) method to generate randomization sequence, 5) blinding, 6) allocation concealment, 7) clearly defined outcome measures, 8) sample size calculation, 9) description of statistical analysis, 10) methods to handle withdrawals and dropout patients, 11) plans for monitoring and 12) evaluation of patient compliance. These criteria were defined in detail as specified in table 1.
Data extraction was carried out by using forms created in Microsoft Excel to record protocol details and their methodological characteristics. For the presence of each of these 12 methodological characteristics one (1) point was given. In case a particular criterion was not specified, zero (0) points were awarded for that element. Thus the minimum possible score was 0 while the maximum achievable score was 12.

STATISTICAL ANALYSIS
Data entries made into the Excel spreadsheet were cross checked for errors. Analysis was done using Stata ver. 15.1 (StataCorp LLC, TX, USA) software. Chi square test was used to test for significance between proportions in the two time periods and the independent samples Student's t test was used for continuous data. Data were expressed in percentage (%) or as mean scores. P values of less than 0.05 were considered to be statistically significant.

RESULTS
For the sake of simplicity, the period from January 2007 till May 2009 will be referred to as "Period I" while the period from July 2009 till December 2011 will be termed "Period II" in the following sections.

Characteristics of drug trial applications
In the Period I, out of 75 trial applications analyzed, 61 (81.3%) were of new chemical entities while 14 were of biopharmaceuticals. In the study Period II, 56 (74.7%) applications were of NCE and the rest 19 were of biopharmaceuticals ( Table 2). The clinical trial protocols analyzed for the Period I revealed that 8 (10.7%) studies were from academia and 67 (89.3%) were from pharmaceutical industry. Similarly, in the latter study period majority of the clinical trials were industry sponsored as compared to academia [71(94.7%) vs. 4 (5.3%)] and there was no statistical difference between the source of drug trial applications analyzed in this study. On evaluating the distribution of the new molecular entities according to the Anatomic Therapeutic Chemical (ATC) classification (first level) it was found that the top five categories in the Period I were nervous system, antineoplastic and immunomodulating agents, anti-infectives for systemic use, systemic hormonal preparations (excluding sex hormones) and blood & blood forming organs. In Period II the top five ATC groups were anti-infectives for systemic use, systemic hormonal preparations (excluding sex hormones), cardiovascular system, musculoskeletal system and antineoplastic and immunomodulating agents ( Table  3). As for the location of clinical trials proposed in the investigational drug trial applications, in Period I, 44(58.7%) proposed trials were global while 31(41.3%) were planned for India only. In Period II the proportion of global {50(66.7%)} trials was slightly more but this difference was not statistically significant (Table 2).

Methodological quality characteristics of clinical trial protocols
During the period Jan 2007-May2009, more than 40% clinical trial protocols had clearly defined primary and secondary outcome measures which significantly increased to 80% for the period July 2009 to December 2011. All the protocols in both the study had listed specific inclusion and exclusion criteria that qualify participants to enter the study (Table 4). Table 1. Definitions of methodological characteristics evaluated among the study protocols.

Methodologic Characteristics
Definitions Eligibility criteria of the study group This criterion included specific inclusion and exclusion criteria that qualify participants to participate in the study.

Controlled trial
The proposed trial was considered to be controlled if the study methods listed atleast one control group. The type of controlled trials included parallel group studies, crossover trials and others like factorial designs. Randomization The proposed trial was considered to be randomized if there was explicit use of terms like random, randomly or randomization in the study methodology Method to generate randomization sequence This characteristic included the method used for generating the allocation sequence. Acceptable methods included predetermined lists, such as computer/ software generated lists of random numbers, use of random number tables and randomization performed by external, statistically competent individuals. Methods of assignment that are prone to bias include alternating assignment, use of even/ uneven dates or assignment by date of birth was considered unacceptable. Blinding This criterion included the fact whether the study was described to be double blind whereby either both the participants and the study investigators and/or outcome assessors are unaware of the group assignment of the patients.

Allocation concealment
The method of ensuring concealment of patient assignment until the point of treatment allocation was evaluated. Acceptable methods included the use of Interactive voice response system (IVRS), or pharmacy/ centrally controlled participant allocation with opaque envelopes that were sealed and sequentially numbered. Clearly defined primary and secondary outcomes This characteristic included the presence of clearly defined primary and secondary outcomes in the methods section of the trial protocol. Sample size calculation A clear description of the statistical method used for calculating the power of the study or the number of participants required in each study group, to attain adequate statistical power for answering the research question should be included.

Description of statistical methods
This criterion included a detailed description of statistical methods to be employed for assessment of trial data.

Handling of withdrawals and dropouts
The protocol was evaluated for the provision of predefined methods of dealing with missing values in the data set and handling withdrawals and dropouts in the study population Monitoring plan This characteristic comprised of a study monitoring plan listed in the protocol. The monitoring plan should seek to ensure study compliance with protocol, GCP as well as regulatory requirements such as ADR reporting. A description of study monitoring policies and procedures should have been provided along with the proposed frequency of site monitoring visits.

Assessment of patient compliance
This criterion consisted of a statement describing assessment of patient compliance using methods such as pill counts, self/parental reports, drug assays, directly observed therapy, patient diary and other methods.

Characteristics minimizing the risk of bias
The clinical trial protocols submitted to the regulatory authority during both the periods were analyzed for details about items aimed at minimizing the risk of bias. On evaluation it was found that in Period I, nearly 70% of proposed trials were controlled and described as randomized as well whereas in Period II, 80% of protocols described a control group and 60 (80%) had used the term random, randomly or randomization in the study methodology.
In Period I, 35 study protocols had adequately described an acceptable method used for generating the allocation sequence, 33 (44%) were double blind and 18 (24%) had listed an acceptable method of allocation concealment. When the same assessment was made for protocols under Period II, it was found that 60% of them had described random allocation sequence generation. However, when compared to previous period, a significantly high numbers of protocols reported double blinding and described concealment of allocation using acceptable instruments (Table 4).

Characteristics related to statistical methods
On evaluation of the clinical trial applications belonging to Period I, it was found that nearly 38% protocols provided statistical justification for the choice of sample size or calculation of statistical power and approximately 61% provided a detailed description of statistical methods to be employed for assessment of trial data. In Period II while 78.7% protocols stated methods of statistical analysis, a significantly high number of early 70% study proposals had provided sample size calculations (p< 0.001). Analysis of the protocols for provision of handling of withdrawals and dropouts revealed that while only 44% protocols described predefined methods of handling missing data in Period I, 68% trial protocols in period II had mentioned these details (p< 0.05) ( Table 4).

Characteristics related to good clinical practice (GCP)
The share of trial protocols in which GCP monitoring was intended was 85% in Period I and somewhat lower at 83% in Period II. Compliance assessment measures were stated for 57 (76%) protocols in Period I, which improved by 8% in Period II although this increase was not statistically significant.
Mean cumulative scores on the basis of methodological characteristics of clinical trial protocols The mean cumulative scores for the study protocols improved significantly from 7± 0.296 in Jan 2007 -May 2009 period to 8.93± 0.346 (p<0.001) in the July 2009 -December 2011 period. The mean cumulative scores of protocols submitted as a part of global trial applications increased significantly from 7.5±0.367 in Period I to 10.18±0.279 in Period II (p<0.0001). The mean scores of clinical protocols for trials proposed to be conducted exclusively in India also improved marginally from 6.29±0.469 in Period I to 6.44±0.63 in Period II. The difference however, failed to reach statistical significance. For academic protocols, the cumulative score improved from 4.13±0.90 to 7.5±1.55 across the two study periods while for industry sponsored studies, the mean scores increased significantly from 7.34±0.29 to 9.01± 0.35 (p=0.0004) (Fig. 1).

DISCUSSION
The trial protocol is the core document that serves as the foundation for planning, conduct and reporting of clinical studies. The protocol determines the structural integrity of any research work and good quality protocols provide readers an assurance on the probability that the study results are a valid measure of the truth. Public availability of key elements of protocols has become increasingly important for ensuring transparency, integrity and quality control of research and for evaluation of trial results. Adequate knowledge of a trial protocol allows an appropriate ethical evaluation before initiation, prevents unethical modifications in methods and reporting of results and a proper contextual understanding of the results after study completion (5,25). To enhance the transparency and accessibility and to strengthen public trust in clinical research, registration of trials in the Clinical Trials Registry -India (CTRI), was made mandatory by the drug regulatory authority with effect from June 15, 2009 with recent regulations mandating only prospective registration of studies. An important purpose of CT registration is to improve clinical trial design and refine research methods (11). Though several investigations have suggested that registration of trials is associated with an improved quality of trial reporting, its impact on trial designs and study protocols remains unknown. The present study analyzed clinical trial protocols for a set of 12 methodological characteristics based on well-known and universally accepted guidelines and checklists. Since some of these instruments have been actually devised to guide reporting of clinical trials, this precluded their use in toto to assess quality of trial protocols, the principle focus of our study. These selected criteria have been previously utilized in studies and are considered critical for deriving conclusions from research findings that are meaningful to actual clinical practice (26,27). A scoring method comprising of both methodological characteristics based on the design of study as well as concordance with GCP guidelines was used in this study. Such a scoring system was expected to bring objectivity to the assessment of the protocols.
Broadly there was a significant improvement in the methodological quality characteristics of the protocols as evidenced by an increase in the mean cumulative scores between the two time periods. Notably there was a significant increase in the number of protocols reporting well defined primary and secondary outcome measures in Period II as compared to those in Period I [OR (95% CI): 5.68 (2.75, 11.70), p <0.001].
A previous study by Berendt et al. has demonstrated similar improvement trends in the reporting of primary outcomes in academic clinical trial applications submitted to drug regulatory authority. The study showed an improvement from 60% protocols reporting primary/secondary endpoints in 1995 to 92% in 2005 (27). The study examined the impact of European Clinical Trials Directive introduced in 2004. The reporting of primary outcomes has been found to be somewhat suboptimal even in published clinical trials as well as clinical trial registries with studies reporting improvements over time following introduction of measures such as CONSORT statement (28,29 (29). A study designed to evaluate the reporting of key methodological study characteristics in trial registries found that the weighted overall proportion * * in the WHO ICTRP registry for adequate reporting of primary outcomes was 66% (95% CI 60-72%) (30). A significantly higher number of protocols provided justification of sample size of subjects to be enrolled in the study in Period II as compared to those in Period I [3.80 (1.93, 7.45), p< 0.001]. Lee and Tse investigated the quality of reporting of sample size calculations in trials published in PubMed in December 2014 (n=451) that did not provide key information about sample size calculations and found that less than 60% papers reported a priori sample size calculations (31). Another systematic review of reporting of sample size calculation in anesthesia journals found that basic elements for calculating sample size are not consistently provided and in nearly a third of randomized controlled trials reported and replicated sample sizes failed to match (32). Similar pitfalls have been reported for RCTS in other specialties as well (33)(34)(35).
As compared to Period I, a significantly higher number of protocols described predefined methods of handling missing data in Period II [2.70 (1.40-5.25), p=0.003]. Missing data are a potential source of bias and can seriously compromise the ability to draw definitive inferences from clinical studies (36). A review of articles published in leading medical journals found that 95% of trial reports reported missing outcomes data with inconsistency noted in the definitions of intention to treat approach across trials (37). As far as description of the study as double blind is concerned, there was a significant improvement in period II as compared to previous period (44% vs. 61.3%, p<0.05). Although these figures depict an improvement but even the improved data cannot be concerned acceptable since blinding is a critical measure to limit bias in a RCT. Although the nature of certain comparator treatments may preclude blinding, this hardly explains a significant proportion of unblinded studies seen in our study. It has been shown that trials not following a double-blind design are more likely to show advantage of novel interventions over existing treatments (38). An analysis of 250 controlled trials found that RCTs not employing double blind designs yielded significantly larger treatment effects with odds ratios being exaggerated by 17% (39). Among protocols submitted to Danish Medicines Agency from 1993-2005, only 54% of controlled studies were double blind (27). Similar to blinding, allocation concealment is an important element to control bias in a clinical trial. A study examining a sample of RCTs published in major medical journals in 2015 found that nearly 40% of these studies did not report satisfactory methods for allocation concealment (40). However, improvements have been noted as far as reporting trends are concerned. A longitudinal study examining RCT reports published between 1995 -2010 found significantly higher fraction of trials published with adequate reporting of allocation concealment over time (41). In our study a highly significant improvement (p<0.001) in the proportion of trial protocols reporting allocation concealment was evident in Period II as compared to Period I and number of compliant trial applications nearly doubled between the two time periods. This was primarily attributable to the use of centralized allocation using interactive voice or webbased response system (IVRS/IWRS) and pharmacy-controlled allocations in global clinical trials.
In a study assessing the impact of trial registration on quality of reporting of randomized trials it was found that registered RCTs when compared with unregistered ones, were having lower Risk of Bias (RoB) (42) in each domain of the RoB tool and significant difference was reached for both blinding as well as allocation concealment (43). Similarly, significant differences in the quality of reporting have been noted for baseline characteristics, flow of participants, as well as implementation of randomization between the registered and unregistered RCTs (13).
An important finding of our study is that two important methodological aspects of clinical trialsthe proportion of randomized trials and trials adequately describing the generation of allocation sequence, failed to improve significantly between the two time periods. The finding of 20% nonrandomized trials in Period II remained unacceptably high. In the study by Berendt et al.. only 43% of 264 randomized trial protocols had described an acceptable method of generation of allocation sequence (27).
The mean cumulative scores for the study protocols improved significantly during Period II as compared to Period I. The increase in cumulative scores across most of the considered methodological characteristics could possibly be the favorable outcome of clinical trial registration which was made mandatory in India since June 2009. Schedule Y of the Drugs and Cosmetics Act (1940), the key document that governs the conduct of clinical trials in India was amended in 2005 and Indian GCP was first introduced in 2001. An important initiative for improving transparency in the conduct and reporting of clinical research was provided by the International Committee of Medical Journal Editors (ICMJE) in 2005, when it introduced its policy requiring mandatory registration of clinical trials as a condition for manuscript consideration (44). In 2007, the Food and Drug Administration Amendments Act of 2007 (FDAAA) was passed which mandated more types of studies to be registered, additional registration information and disclosure of summary results for certain trials (45). Although the focus of this legislation was US but its impact on global clinical trials was expected. However, the protocols submitted to the drug regulatory authority in India which were considered for inclusion in this study belonged to time periods January 2007-May 2009 and from July 2009-December 2011. Therefore, an improvement in the overall methodological quality of clinical trial protocols could be attributed to mandatory trial registration on ctri.nic.in which is the only significant legislation that applied specifically to Indian studies, both global as well as local during the Period II.
Several studies, as noted earlier, have demonstrated the favorable impact of trial registration on quality of trial reports. However, to the best of our knowledge, this is the first study which assesses the impact of mandatory trial registration on methodological quality of protocols. The limitations of the present study could be small sample size of protocols in the two time periods, however, lack of previous data precluded formal power calculation. Secondly it can be argued that another set of criteria could have been used. But the criteria chosen in this study are based on well recognized and globally accepted instruments. The Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guidelines, a checklist that aims to facilitate composition of good quality RCT protocols has since been introduced in 2013 (46). However, the protocol documents considered in our study belong to the period prior to its introduction. Thus we refrained from using SPIRIT checklist to analyze trial protocols submitted for regulator's consideration during the study years. In conclusion, the overall impact of mandatory trial registration on quality of trial protocols was positive and several key indicators of trial quality improved following introduction of mandatory registration in the country. However, the results of our study highlight that improvement was largely limited to industry sponsored studies and to global clinical trials and there is scope for further enhancement of study quality in academia and for the trials planned to be conducted exclusively within the country. Since April 2018, India has introduced mandatory prospective trial registration. This move is expected to further improve the quality of protocols and strengthen the ultimate purpose of trial registration,that is to allow better healthcare decision making in the light of all the available evidence.

REMARKS
Authors have no conflicts of interest with the contents of the paper. No funding was received for the study.