Orthopaedic Surgical Content Associated with Resources for Clinical Evidence

Introduction: As evidence-based medicine is increasingly being adopted in medical and surgical practice, effective processing and interpretation of medical literature is imperative. Databases presenting the contents of medical literature have been developed; however, their efficacy merits investigation. The objective of this study was to quantify surgical and orthopaedic content within five evidence-based medicine resources: DynaMed, Clinical Evidence, UpToDate, PIER, and First Consult. Methods: We abstracted surgical and orthopaedic content from UpToDate, DynaMed, PIER, First Consult, and Clinical Evidence. We defined surgical content as that which involved surgical interventions. We classified surgical content by specialty and, for orthopaedics, by subspecialty. The amount of surgical content, as measured by the number of relevant reviews, was compared with the total number of reviews in each database. Likewise, the amount of orthopaedic content, as measured by the number of relevant reviews, was compared with the total number of reviews and the total number of surgical reviews in each database. Results: Across all databases containing a total of 13268 reviews, we identified an average of 18% surgical content. Specifically, First Consult and PIER contained 28% surgical content as a percentage of the total database content. DynaMed contained 14% and Clinical Evidence 11%, whereas UpToDate contained only 9.5% surgical content. Overall, general surgery, pediatrics, and oncology were the most common specialty areas in all databases. Discussion: Our findings suggest that the limited surgical content within these large scope resources poses difficulties for physicians and surgeons seeking answers to complex clinical questions, specifically within the field of orthopaedics. This study therefore demonstrates the potential need for, and benefit of, surgery-specific or even specialty-specific tools.


Introduction
Traditionally, physicians have relied heavily upon their own knowledge of human pathophysiology to determine the course of treatment for their patients. It is increasingly understood that this is not sufficient evidence or support upon which to make or base an effective treatment decision [1]. Today physicians base their decisions and actions on the best available scientific evidence in conjunction with their own expertise and the patient's values and preferences [2]. This allows clinical practice to evolve with the development of new practices and the acquisition of new medical knowledge.
Though this method of practice is superior to historical methods, it is not without accompanying challenges. For instance, research in adult internal medicine has demonstrated that physicians would need to read 17 articles each day to stay up-to-date with the current evidence [3]. This daunting task can be very difficult to achieve and is highly time consuming. Another similar challenge relates to the significant time required to access and appraise research findings by the physician before such findings can contribute to their overall clinical practice. Further, many physicians are not trained in such research methods and may not have the skills required to access and critically appraise relevant articles to determine the strengths of recommendations. Ultimately, this can lead to unreliable and incorrect treatment decisions if too much emphasis is placed on a poorly conducted study [4]. This latter consideration is especially problematic in surgical specialties given that there are already less stringent controls placed on the validation of new technologies and surgical techniques. More importantly, studies demonstrate that higher quality evidence, such as systematic reviews and randomized controlled trials (RCT), may only comprise 5% of the content of leading surgical journals [5] suggesting that more surgical practice may be based on lowquality evidence and therefore has a higher potential to negatively influence surgical treatment decisions. Furthermore, this percentage is significantly lower when specific surgical specialties such as orthopaedics are considered.
Several resources have been created to facilitate evidencebased practice by distilling and appraising the contents of medical journals into summaries with recommendations for practice. Examples of such resources include: First Consult, UpToDate, DynaMed, Clinical Evidence, and Physicians' Information and Education Resource (PIER) [6]. The need for these resources has been highlighted in a study which found that when physicians sought the answers to 46 clinical questions using their own choice of resources they did not always arrive at the correct answer [7]. These same physicians were also occasionally found to incorrectly change treatment decisions based on the evidence collected. In the same study, when physicians were supplied with reliable sources, there was an increased rate of correct answers, thus, the initial discrepancy was attributable to the improper collection and critical appraisal of evidence by the physician [7]. Tools such as UpToDate and DynaMed serve as sources of pre-appraised and summarized information to aid in the decision making process. The summaries of evidence that are provided by these tools include content from upwards of 150 medical journals [8,9]. Thus, these resources provide a broad scope for all medical practitioners; however, their true value to surgeons remains widely unclear. Furthermore, within surgical subspecialties such as orthopaedics, the content presented in these resources may be extremely limited.
It remains unclear whether medical specialties such as surgery and subspecialties such as orthopaedics are well represented in these large resources. To facilitate the adoption of such resources by surgeons, who represent approximately 17.76% [10,11] of the medical specialty and approximately 1 559 219 [10Á12] individuals worldwide, requires a demonstrated focus on surgical content to guide practice. The aim of this study was to comprehensively quantify the amount of surgical content within five major evidence summary tools: DynaMed, Clinical Evidence, UpToDate, PIER, and First Consult. We hypothesized that surgical content represented a small proportion of the available reviews in these resources.

Methods
We conducted a comprehensive search across five commonly used evidence summary resources (DynaMed, Clinical Evidence, UpToDate, PIER, and First Consult) to identify surgical and orthopaedic content from inception to February, 2012.

Eligibility criteria
The results for surgery were processed based on predetermined inclusion and exclusion criteria. The criteria outlined that each entry needed to be unique and contain surgery as part of an intervention. All mentions of surgery in citations, as risk factors, and as procedures not associated with a particular clinical condition were excluded. Additionally, incomplete or duplicate summaries were also excluded. All the included summaries were counted and recorded as a whole by surgical specialty and by orthopaedic surgery branch. Only orthopaedic classifications were made as this reflects the expertise of the authors. Table 1 provides a basic comparison of resources used in this study.

Resources and tools
DynaMed, published by EBSCOhost, is a reference tool designed for point of care use by physicians and health care professionals. It contains synthesized evidence summaries for an extensive range of topics. DynaMed uses a seven-step editorial process to maintain the currency of the summaries. As new content is found, it is added to the existing summaries and the conclusions are modified accordingly [13]. Additional information is provided through links to guidelines, articles, and patient care information.
First Consult, published by Elsevier, is a resource designed to provide evidence-based summaries to physicians for the purpose of facilitating clinical decision . First Consult also includes interactive differential diagnosis and procedures components as well as peer-reviewed practice guidelines and patient handouts [14]. PIER, published by the American College of Physicians, is a resource that provides appraised evidence summaries of a wide range of clinical topics, primarily targeting adult internal medicine (paediatric content is now being added). It is designed to improve the overall quality of patient care. Each summary provides links to the abstracts of the original journal articles as well as drug treatment information. Summary content is updated monthly to ensure that the content is based on the most current evidence available [15].
UpToDate, published by Wolters Kluwer, provides appraised summaries of a broad range of clinical topics and medical specialties. Treatment recommendations are graded to allow those accessing the resource to assess the strengths of the recommendation and corresponding evidence. It provides drug information and supporting graphics to facilitate clinical decision making and information abstraction [16].
Clinical Evidence, published by British Medical Journal (BMJ), provides evidence summaries of the literature concerning the treatment and prevention of the most common and most important clinical conditions in primary and hospital care. Each summary includes clinical questions and details regarding the benefits, harms, and uncertainty for the interventions to accompany the summaries, it does not provide treatment recommendations. It also links to additional information such as drug safety alerts, practice guidelines, articles, and updates [17].

Search terms
As each resource is unique in terms of organization and search options, we deliberately kept our search strategy simple. This resulted in a large number of potentially eligible reviews that were individually reviewed for eligibility and inclusion. We searched databases under the keyword ''surgery''. This term was piloted and tested with the assistance of a librarian at McMaster University. During the testing phase, it was determined that using a keyword search for ''surg*'', while providing a broader result set yielded a high number of irrelevant results, capturing entries that included references to surgical masks or surgical gloves but were not related to any aspect of surgery. Another consideration for excluding this term was that it was unsearchable in DynaMed. In addition, depending on the features of the resources search engine, a large number of duplicate summaries were found when searching with surg*. The search was current as of February 2012.

DynaMed
When the term ''surgery'' was used in searching Dy-naMed, the returned results were related to surgical considerations, surgical techniques, drug summaries, and specific disease summaries. The researcher identified where the term surgery was appearing within the summary, and if it pertained to a surgical treatment then it was considered for inclusion. If the relationship to surgery appeared elsewhere in the summary, for example in a citation or as a risk factor, it was excluded. Counts were done for total surgery-related summaries, total for each specialty, and total for each orthopaedic subspecialty.

First Consult
A search in First Consult using the key term ''surgery'' yielded medical topics, differential diagnoses, and procedures. Given that for the purpose of this study only medical topics were to be included, the search was restricted to only the medical topics generated. Within each summary, the researcher screened the conditions for surgical content and included only those that presented surgical treatments as the main focus. Counts were done for total surgery-related summaries, total for each specialty, and total for each orthopaedic subspecialty.

Clinical evidence
When searching this resource, it was noted that headings generated during the search were not unique diseases but rather subtopics within clinical condition summaries. As a result, the overall summary and all subsequent hits under the one summary were included as one unique entry if there was a relationship to surgery. Counts were done for total surgery-related summaries, total for each specialty, and total for each orthopaedic subspecialty.

PIER
The advanced search option was utilized whereby only ACP PIER and American Hospital of Formulary Services Drug Information (AHFS DI) Essentials tools were searched with the keyword ''surgery''. The results generated included management, evaluations, assessments, drug summaries, entire specialties, specific treatments, prophylaxis, and clinical conditions. Only the clinical conditions met the inclusion criteria, and therefore all other results were excluded. Within each of these summaries the researcher screened the conditions for surgical content and included only those that presented surgical treatments as the main focus. Counts were done for total surgery related summaries, total for each specialty, and total for each orthopaedic subspecialty.

UpToDate
At the time of searching (February, 2012) the term ''surgery'' could not be used for searching this resource as clinical conditions were not generated. As such, it was assumed that all surgery-related content was contained within the ''general surgery'' topic of the content sections of the tool. Within this topic there were 25 subtopics some of which contained clinical condition content and some of which did not. Subtopics that did not contain full and complete clinical condition summaries were excluded. Each clinical condition was considered as a unique entry even though there appeared to be several summaries discussing different aspects of the same condition. Clinical condition summaries were considered for inclusion if the content focused on surgical interventions.
A unique method to generate a total count of clinical conditions within this tool was used. The percentage of general surgery unique clinical condition summaries (determined from the initial search as outlined) was determined in relation to the total content of the general surgery topic. This percentage was then applied to the total content number for the tool.

Data abstraction
All the included articles were counted and recorded as a whole by surgical specialty and by orthopaedic surgery subspecialty. The surgical specialties utilized for this purpose included: cardiac, endocrine, general, gynecological, neurosurgery, obstetrics, maxillofacial surgery, ophthalmology, oral, orthopaedic, otolaryngology, paediatric, plastic, proctology, oncology, thoracic, transplant, urology, vascular, and trauma. The researchers noted that a particular surgical technique can often apply to more than one specialty and that many disorders may be treated with multiple surgical treatments of multiple surgical specialties. Each result generated from the search was therefore allowed to have multiple surgical specialties associated with it. We further examined the subspecialty content in orthopaedic surgery to explore the ''within specialty'' content. Our decision to use orthopaedic surgery was based upon the expertise of the authors. The orthopaedic subspecialities included: arthroplasty, foot and ankle, general orthopaedics, hand and wrist, metabolic disorders, paediatric orthopaedics, shoulder and elbow, spine, sports medicine, trauma, and tumour. All specialties were chosen by an orthopaedic surgeon at McMaster University. The researcher categorized all summaries based on surgical and orthopaedic specialty with the assistance of a medical professional.

Analysis
The total number of surgical summaries in each resource is presented as a raw value and as a percentage of the total number of summaries in that resource. Additionally, the number of orthopaedic summaries in each resource is presented as a raw value, as a percentage of the total number of summaries in the resource, and as a percentage of the total number of surgical summaries in each resource. These results can be seen in Table 2. The specialty-specific data is presented in Table 3 in its raw form as numbers of specialty-specific summaries in each resource. Totals for each specialty across all resources are also presented. Sums of all the specialty summaries in each resource were recorded; however, these are not reflective of the number of surgical summaries in each resource due to the specialty overlap in each summary. The percentage of specialtyspecific content compared with total surgical content is also provided for each specialty in each database and for the totals across all resources. Finally, orthopaedic subspecialty data is presented in Table 4 in the same manner as specialty-specific data: numbers of subspecialty summaries in each database, total across all resources, sums of all subspecialty summaries within each database, and corresponding percentages of subspecialty specific content were compared with total orthopaedic content.

Overall surgical content
We identified a total 13 268 reports across all five resources. Among these, surgical content averaged 18% (range 9.5%Á28%). The most common surgical specialty areas included general surgery representing 25% of the surgical summaries (range 20%Á29%), pediatrics representing 18% of the surgical summaries (range 8%Á27%), and oncology representing 14% of the surgical summaries (range 10%Á19%). Least commonly represented were maxillofacial surgery and oral surgery. The resources including the greatest surgical content included DynaMed with 798 summaries (14%) and UpToDate with 480 summaries (9.5%). PIER and First Consult had fewer surgical summaries at 186 and 430, respectively; however, together they had the most surgical content by percentage of total content at 28%.

Orthopaedic surgical content
We identified a total 13 268 reports across all five resources. Among these, orthopaedic content averaged 2.04% (range 0.2%Á4%) of the total database content and averaged 12.1% (range 2%Á20%) of the surgical content. The most common orthopaedic subspecialty areas included general orthopaedics representing 26% of the orthopaedic summaries (range 9%Á36%), trauma representing 24% of the orthopaedic summaries (range 7%Á 50%), and pediatric orthopaedics representing 21% of the orthopaedic summaries (range 7%Á25%). Least commonly represented were tumour and shoulder or elbow. The resources that contained the greatest amount of orthopaedic content relative to total content included First Consult with 64 summaries (4%) and DynaMed with 126 summaries (2%).

Summary of findings
Among the 13 268 summaries across all five resources, 1948 surgical summaries were found. This represented an average of 18% surgical content. DynaMed had the most surgical summaries with 798; however, PIER and First Consult had the most surgical content by percentage of total content with 28%. An average of 12.1% of the surgical content within each database was orthopaedic. General surgery was the most represented surgical specialty and maxillofacial was the least represented. General orthopaedics was the most represented orthopaedic subspecialty whereas tumor was the least represented.

Strengths and limitations
As with any study, this study has limitations that merit discussion. The dynamic nature of the tools themselves poses some limitations. Each of the tools updates content with some frequency and therefore the results provided in this study only reflect their content as of October 2011Á February 2012. Nonetheless, the changes in the amount of surgical content between this time period would likely be small given the already small number of surgery-related summaries and guidelines contained within these tools. Additionally, the procedure used to assess surgical content in UpToDate was limited by the structural setup of the database. This led the researchers to make the assumption that all the surgical content was contained within the general surgery topic and led the researchers to use a calculated estimate of the number of clinical conditions as opposed to generating a precise number.

Relevant literature
The practice of evidence-based medicine (EBM) requires the collection of relevant evidence to be efficient; otherwise it would be very difficult for clinicians to utilize top quality evidence in making everyday decisions for different patients with different needs. Third generation tools attempt to increase this efficiency by summarizing and appraising the available evidence; however, they too can be inefficient due to their large scope of content. When specialists are seeking evidence for specific interventions they are required to find, in essence, the needle in the haystack. This imposes particular challenges for surgeons who often do not have the time to search through large volumes of extraneous information to access their desired content. One study of surgeons noted this lack of evidence relevant to surgery and the time that is therefore required to find this relevant information [18]. Furthermore, the large scope of content in these resources has been shown to contribute to a length of time spent correctly answering a clinical question that is too long to be efficient in clinical care [19]. As EBM becomes a more integral part of all surgical practice, these hindrances will likely become increasingly problematic.
The results of this study demonstrate these issues by signifying the limited surgical content, overall and by specialty, in each of the tools. These results suggest that an orthopaedic surgeon desiring evidence for interventions related to shoulder or elbow injuries would have to search through the 5539 summaries to access the 11 potentially relevant summaries concerning shoulder or elbow orthopaedics. This study therefore shows the potential need for, and benefit of, surgery specific or even specialty specific tools.
As discussed previously, RCTs are considered to be the highest quality evidence but these resources do not routinely perform systematic searches to identify the best and most current evidence. For example, a First Consult summary of colorectal cancer was most recently revised 10 June 2010. Given the revision gap of nearly two years, this database cannot claim to contain the most current evidence with respect to this topic. Furthermore, a 2011 study found that the rate of citation overlap among UpToDate, DynaMed, First Consult, Clinical Evidence, and PIER was less than 1% [20]. This brings to light the question of which database is actually citing the best evidence. Together these aspects, currency and evidence content, question the quality and utility of these resources.
The general nature of the tools used in this study brings forth the question of whether or not the tools are doing justice to the surgical content that is actually presented and the ease with which surgeons can interpret it. For these tools to assist EBM, it is required that they provide information that can be easily interpreted and used to guide surgical practice. A study evaluating perceived content quality and user preference for ACP PIER, DISEASEDEX, First Consult, InfoRetriever, and UpToDate found no significant difference among resources in perceived content quality, however, they did find that UpToDate was superior in user preference [21]. Nonetheless, each tool provides their recommendations and corresponding evidence differently and their utility for surgeons merits further investigation.

Relevance
Our findings demonstrate a clear gap in resources for surgeons. The authors, therefore, have been involved in the development of a new surgery-specific evidence based summary resource call OrthoEvidence (www.myortho evidence.com). OrthoEvidence is unique in that it focuses particularly on presenting high-quality evidence through these summary reports. Each summary report efficiently highlights the important aspects of a study to orthopaedic healthcare professionals, thereby aiding in evidence-based practice. Designed by practitioners, resources such as OrthoEvidence may prove to be invaluable to the medical practitioner who does not have the large amount of time that is required to search for relevant and high-quality evidence within their field.
Our findings suggest that the limited surgical content within these large scope resources poses difficulties for physicians and surgeons seeking answers to complex clinical questions. This demonstrates a need for surgical subspecialty focused resources presenting the best available evidence. These resources will have a more narrow scope of content to facilitate navigation of the databases and relevant evidence extraction for surgeons.