High-quality reports and their characteristics in the Japanese Adverse Drug Event Report database (JADER)

-- Purpose: Spontaneous adverse drug reaction reporting is the foundation of postmarketing drug safety monitoring. The present study aimed to analyze and clarify the quality and characteristics of the Japanese Adverse Drug Event Report database (JADER) using the World Health Organization (WHO) documentation grading scheme and the vigiGrade completeness score. The characteristics of reports were described using both schemes simultaneously. The way of proper use of these two schemes was explored. Methods: The WHO documentation grading scheme and the vigiGrade completeness score were applied to the same dataset (JADER202001 dataset). Reports classified as high-quality under both assessment criteria were extracted, and the characteristics of these reports were analyzed. Results: Of the 607,361 adverse drug reaction reports analyzed, 52.8% were ‘well-documented reports’ with a vigiGrade completeness score >0.8. Under the WHO documentation grading scheme, 328,702 reports (54.1%) were Grade 2 and 5,178 (0.9%) were Grade 3 (including rechallenge information). Among well-documented Grade 3 reports, classified as the highest quality, a high proportion of the adverse drug reaction reports were related to disorders of hematopoietic function resulting from anticancer drugs. Because a high proportion of the reports with rechallenge information were for anticancer drugs as suspect drugs, the WHO documentation grading scheme tended to extract reports regarding anticancer drugs as high quality. Conclusions: We conclude that the two schemes need to be used appropriately, depending on the purpose of analysis, the target adverse drug reactions, and suspect drugs.


INTRODUCTION
The collection, evaluation, and use of drug safety information by healthcare professionals are essential for the safe and proper use of drugs in clinical settings. Healthcare professionals, pharmaceutical companies, and regulatory authorities are required to carry out pharmacovigilance activities as part of a consistent set of safety measures from the drug development to postmarketing stages. Spontaneous reporting of adverse drug reactions (ADR) is the foundation of postmarketing drug safety monitoring (1,2). It can be defined as a system for collecting and collating case reports of suspected ADRs to detect the potential for unknown drug-related harmful effects. Some of the ADR reports collected in Japan are made public via the Japanese Adverse Drug Event Report database (JADER database). They can be downloaded from the Pharmaceuticals and Medical Devices Agency (PMDA) website (3).
Similar databases are maintained in other countries and organizations around the world, including: the VigiBase, maintained by the WHO (4); FAERS (FDA Adverse Event Reporting System), maintained by the United States Food and Drug Administration (FDA) (5); the Lareb database, maintained by the Netherlands Pharmacovigilance Centre Lareb (6); and Eudravigilance, maintained by the EMA (European Medicines Agency) (7). Through signal detection and other tools, these databases play an essential role in postmarketing drug safety.
The quality of ADR information used for causality assessment and signal detection can be assessed by a pharmacoepidemiology expert or by evaluating the data elements' completeness in each case report. Completeness assessment tools include: the WHO's documentation grading scheme (8); vigiGrade (9), which was created to assess the quality of the information in VigiBase; the ATHE score (10); amongst other methods of calculating completeness scores (11)(12)(13). Here, we have analyzed the quality and characteristics of the JADER, using the WHO documentation grading scheme (14) and the vigiGrade (15). The WHO's documentation grading scheme, which was noted if essential elements (e.g., date of onset of ADR, duration of administration of suspected drugs, dose, and frequency of administration, rechallenge information) were given in the reports, could divide ADR reports into four grades. The vigiGrade completeness score was developed to measure the amount of clinically relevant information in a structured format without reflecting whether the information establishes causality between the drug and ADR (15). In this study, we compared and contrasted the characteristics of reports using the two schemes. Besides, the proper use of these two schemes was explored. This study aimed to find a way to evaluate the quality of ADR reports for postmarketing safety measures effectively.

METHODS
For this study, the WHO documentation grading scheme (8) and the vigiGrade completeness score (9) were applied to the same dataset (JADER202001). Reports classified as high-quality under both assessment criteria were extracted, and the characteristics of these reports were analyzed. The JADER202001 dataset containing ADR reports from April 2004 to September 2019 was used. JADER uses a report format based on ICH E2B-M2 (R3) and consists of four tables: 1) case list  (1,231,770 entries). From the JADER202001 dataset, the data for the first listed suspect drug (in 607,361 reports) and the data for the first listed ADR (in 611,336 reports) were extracted and connected. All 607,361 connected cases were subsequently analyzed in a similar manner. All ADRs and underlying diseases were coded using the Medical Dictionary for Regulatory Activities (MedDRA) terminology (Version 22.1) preferred term (PT).
The WHO documentation grading scheme (8) classifies ADR reports using a 4-grade scheme, allowing classification from Grade 0 (lowest quality) to Grade 3 (highest quality); the algorithm for this scheme is shown in Figure 1. A report is classified as Grade 3 if it includes information on rechallenge (re-administration of the suspect drug after the onset of the ADR), the suspect drug start date, ADR onset date, the reason for the use of the suspect drug, and outcome of ADR. In the vigiGrade completeness score (9), a score of the completeness of a fixed number of elements included in the ADR report, can be calculated using the formula shown below: Where P i is a numerical penalty for the elements i shown in Table S1.
The elements included in the JADER were used for calculating the vigiGrade completeness score. Time-to-onset was calculated from the suspect drug start date and the ADR onset date and used in the vigiGrade completeness score calculation ( Figure  S1). As with our previous research (15), some of the assessed elements and penalties used in this study were modified from the original vigiGrade completeness score (Table S1). In previous research by the WHO Uppsala Monitoring Centre (9), a vigiGrade completeness score of >0.8 was defined as a 'well-documented report'. The same definition was used in this study. Significance was defined as p<0.05 for all statistical tests, and SAS version 9.4 (SAS Institute Inc., Cary, NC, USA) was used for all analyses.

RESULTS
Of the 607,361 ADR reports analyzed, 320,657 (52.8%) were 'well-documented reports' with a vigiGrade completeness score >0.8 (9). Under the WHO documentation grading scheme (8), 328,702 (54.1%) were Grade 2 and 5,178 (0.9%) were Grade 3. Table 1 shows the relationship between the vigiGrade completeness score and the WHO documentation grading. According to the results, 315,616 reports (52.0%) were classified as welldocumented and Grade 2, and 5,041 reports (0.8%) were well-documented and Grade 3. Moreover, all 'well-documented reports' according to vigiGrade were classified as Grade ≥2 under the WHO documentation grading scheme. There were 233,569 not well-documented Grade 0 reports, representing 38.5% of the overall reports.
The report characteristics based on the combined vigiGrade completeness score and the WHO documentation grading scheme are presented in Table 2. In addition, the percentages of missing data are presented in Table 3. Among reports classified as both well-documented and Grade 3, a physician was included among the reporters in 4,260 reports (84.5%), and 4,889 reports (97.0%) came from a pharmaceutical company. The report status was 'investigation complete' in 5,040 reports (99.9%), and spontaneous reports accounted for 3,468 (68.8%) of all well-documented Grade 3 reports. Analysis of missing elements showed that there were almost no omissions in well-documented Grade 3 reports. In contrast, among the not well-documented Grade 0 reports, none allowed time-to-onset to be calculated, and all were unclear about whether the suspect drug was used before or after the ADR occurred. The not well-documented Grade 1-3 reports might omit more than one essential (penalty 50%) or important (30% penalty, if missing) element, such as time-to-onset, sex, age, etc.
In the overall reports, the top ten primary suspect drugs included immunosuppressants such as methotrexate, tacrolimus, prednisolone, and cyclosporine, and anticoagulants rivaroxaban and apixaban (Table S2). However, among welldocumented Grade 3 reports, eight of the top ten primary suspect drugs were anticancer drugs, and the other two were nafamostat mesylate and irradiated platelet concentrate. ADRs common in both classifications were cytopenias (such as neutropenia and thrombocytopenia), anaphylactic shock, hepatic impairment, and drug eruption. Although interstitial lung disease and pneumonia were two of the top ten in the overall reports, these ADRs were not in the top ten in well-documented Grade 3 reports. Pyrexia, diarrhea, and decreased blood pressure were not in the top ten in the overall reports (Table S3).
Among well-documented Grade 3 reports, the most common pair of the suspect drug and ADR was paclitaxel and leukopenia (n=61, 1.2%; Table S4). Among not well-documented and Grade 0 reports, classified the lowest quality reports, immunosuppressive drugs, anticoagulants, and an antiepileptic drug (carbamazepine) were in the top ten suspect drugs similar to the overall trend. The top ten adverse drug reactions included interstitial lung disease, pneumonia, renal dysfunction, lymphoproliferative disorders, and death.Methotrexate and lymphoproliferative disorders were the most common pair of the suspect drug and ADR among not well-documented and Grade 0 reports (n=2,036, 0.9%; Table S4).

High-quality ADR reports across the two assessment criteria
In the present study, the WHO documentation grading scheme (8) and the vigiGrade completeness score (9) were used to extract high-quality reports in common across the two assessment criteria and explore their characteristics. All of the 'welldocumented reports' in the vigiGrade were classified as Grade 2 or above in the WHO documentation grading scheme. The WHO documentation grading scheme only evaluated the presence or absence of information, but not the information's granularity or accuracy (as evaluated by vigiGrade). Welldocumented reports judged by the vigiGrade could have enough information such as outcome or indication, which were essential to be classified into grade 3 or 4 by the WHO's grading scheme (8). This was a potential reason explaining why a 'well-documented report' was considered high quality across the two evaluation criteria. Whereas, because the vigiGrade completeness score focused on time-to-onset for quality evaluation, some ADR reports with fewer missing elements were classified as not well-documented due to omitting time-toonset information.

The characteristics of 'well-documented' Grade 3 reports
Well-documented Grade 3 reports were analyzed to extract the highest-quality reports. Under the WHO documentation grading scheme (8), Grade 3 requires information on rechallenge, in addition to the criteria required for Grade 2( Figure 1). Of the 607,361 reports analyzed, 5,041 (0.81%) met all of these requirements. An analysis of these reports' basic characteristics showed that 99.9% of welldocumented Grade 3 reports were 'investigation complete'. Detailed investigations were typically conducted by the regulatory authorities or pharmaceutical companies, suggesting that high report quality was obtained by supplementing information that had been missing. Analysis of report type showed that study reports accounted for 1,396 (27.7%) of well-documented Grade 3 reports. Study reports are ADRs obtained from solicited studies such as drug-use results surveys and postmarketing clinical studies under the Japanese postmarketing safety measures. The quality of study reports may be affected by diverse factors, including whether a rechallenge was completed, the availability of detailed survey questionnaires, and visits by the company's medical representatives. The majority of the reporters and senders of welldocumented Grade 3 reports were physicians and pharmaceutical companies. This is because, in Japan, routinely, drug-use results surveys and/or other safety measures are requested by pharmaceutical companies from physicians.

Comparison of the top ten adverse drug reactions and suspect drugs
Comparing the top ten ADRs in well-documented Grade 3 reports with those in the overall reports revealed that disorders of hematopoietic function, such as leukopenia, neutropenia, and thrombocytopenia, occupied 4 of the top 10 places in both classifications. Interstitial lung disease and pneumonia were only observed in the overall reports. In contrast, fever, diarrhea, and decreased blood pressure were only observed in well-documented Grade 3 reports. In the overall reports, the top positions for suspect drugs were occupied by immunosuppressants. In contrast, among welldocumented Grade 3 reports, 8 of the top 10 suspect drugs were anticancer agents (the other 2 were nafamostat mesylate and irradiated platelet concentrate). The higher proportion of ADR reports related to disorders of hematopoietic function in well-documented and Grade 3 reports can be attributed to the higher proportion of reports with cytotoxic anticancer drugs in particular accounting for half of the top ten drugs. During treatment with cytotoxic anticancer drugs, leukopenia, neutropenia, or thrombocytopenia may occur due to bone marrow suppression.
If a drug holiday leads to recovery from bone marrow suppression by anticancer drugs, it is common practice to resume treatment at a reduced dose of the suspect drug. It is often difficult to replace anticancer drugs with other drugs because of differences in indications and evidence (even if allogeneic drugs with similar effects are available). They may be a factor favoring a rechallenge after the appearance of adverse events. In addition, new anticancer drugs are often the subject of specific use outcome studies. The collection of detailed safety information based on contracts between healthcare providers and companies may cause higher reporting quality.   Conversely, for interstitial lung disease, druginduced pneumonia, and lymphoproliferative disorders, it is generally not recommended to readminister the suspect drug once an adverse reaction has occurred from a safety standpoint. As a consequence, it is difficult to obtain information on rechallenges after these ADRs, and causality is often estimated by confirming symptom recovery with discontinuation of the suspect drug (16-17) (dechallenge positive).
The lowest quality, not well-documented Grade 0 reports tended to include immunosuppressive and anticoagulant drugs, as suspect drugs, and interstitial pneumonia, pneumonia, and lymphoproliferative disorders, as ADRs. Rheumatoid arthritis, methotrexate (MTX), and lymphoproliferative disorders (LPD) were the most common combination of the reason for the use of the suspect drugsuspect drugand ADRs, with 89% of reports of this combination being not welldocumented Grade 0 (data not shown). MTXassociated lymphoproliferative disorders (MTX-LPDs) occur during immunosuppressive drug treatment for diseases such as rheumatoid arthritis. MTX-LPDs may occasionally cause death. The 2011 edition of the Japan College of Rheumatology guideline for MTX use in patients with rheumatoid arthritis (17) described MTX-LPD. After the publication, the number of reports of MTX-LPD may have increased due to the increased public attention. Although over 90% of MTX-LPD related reports lacked the start date of suspect drug or onset date of ADR, the patient's sex, age, and outcome were recorded in 60-80% of these reports. MTX-LPD is a late-onset disease, and time-to-onset among Japanese rheumatoid arthritis was reported as approximately five years (18). Moreover, because MTX-LPDs include malignant diseases like cancer, it was challenging to identify the exact date of onset. Therefore, the lack of information concerning timeto-onset may be a consequence of these reasons. Nonetheless, time-to-onset information is indispensable for evaluating the causal relationship between the suspect drugs and ADRs, and other drug safety information.

The proper use of two schemes
Evaluation of the quality of reports in the JADER revealed that many reports lacked time-to-onset information as with other ADR databases (19,20). Furthermore, the quality of reports varied depending on the reason for the suspect drug usesuspect drug -ADR combination. When a causal relationship between suspect drugs and ADRs is assessed individually, it may be more efficient to extract and use reports classified as Grade 3 in the WHO documentation grading scheme because they include information on rechallenges. However, from our results, it was also demonstrated that the WHO documentation grading scheme, which required information on rechallenge for high-quality reporting, may result in the extraction of reports biased towards anticancer drugs and the ADRs associated with them. The vigiGrade completeness score would classify some ADR reports as low quality because of the lack of information on time-toonset, even if the reports contained beneficial patient characteristics. Moreover, the ADR reports without time-to-onset information included reports that had a long time-to-onset or secondary cancer for which the date of onset could not be determined. In summary, it was revealed that the two schemes need to be used appropriately, depending on the purpose of analysis, the target ADRs, and suspect drugs. For example, when a causal relationship between an anticancer drug and an ADR would be assessed, we can use well-documented reports in the vigiGrade due to their completeness. In all situations on postmarketing safety measures, information on time-to-onset is suggested to be essential to evaluate causality or to detect safety signals. In the JADER, a more advanced reporting system should be developed to gather this information precisely.
FUNDING. This research was partially supported by the Research on Regulatory Harmonization and Evaluation of Pharmaceuticals, Medical Devices, Regenerative and Cellular Therapy Products, Gene Therapy Products, and Cosmetics from the Japan Agency for Medical Research and Development (AMED).

CONFLICTS OF INTEREST. Masami Tsuchiya,
Taku Obara, Makoto Miyazaki, Aoi Noda, Takamasa Sakai, Ryohkan Funakoshi, and Nariyasu Mano have no conflicts of interest directly relevant to the content of this study. Makoto Miyazaki is also an employee of a pharmaceutical company, MSDKK. However, he has no conflicts of interest directly relevant to the content of this study, as this study focuses on all marketed drugs in Japan and not on any specific drug. Figure S1. Four examples of penalties when there is imprecise or missing information on time-to-onset. ADR adverse drug reaction, TTO time-to-onset Table S1. Dimensions (weighted by their relative importance for causality assessment and follow the listed principles) and penalties (for absence of information on specific dimensions reflecting the importance of that dimension for causality assessment) based on vigiGrade and the present study