A Comparison of Selected Bibliographic Database Search Retrieval for Agricultural Information


  • Stephanie Ritchie
  • Kelly Banyas
  • Carol Sevin




Search result retrieval was compared across eight research literature databases (AGRICOLA, AGRIS, BIOSIS, CAB Direct, FSTA, Google Scholar, Scopus, and Web of Science) for three topics from different agricultural disciplines to compare retrieval results based on searcher experience. Precision, recall, and uniqueness were analyzed by rating search results (~2400 citations) for relevancy. A generalized linear model statistical analysis determined that AGRICOLA ranked highest for precision and was statistically more likely to produce a relevant result than four other databases. CAB and Web of Science ranked highest for recall and both overlapped with AGRICOLA for statistical likelihood of producing a relevant result. Google Scholar retrieved the most unique content, but almost half of that content was not judged relevant. AGRICOLA, BIOSIS and CAB retrieved the most unique and relevant content. This study will help researchers and librarians working in the agricultural disciplines to select the bibliographic databases that will provide the most relevant search results and are most likely to meet their research need. It may also serve as a template for future bibliographic research in other disciplines.


Download data is not yet available.


Bethel, A. & Rogers, M. 2014. A checklist to assess database-hosting platforms for designing and running searches for systematic reviews. Health Information & Libraries Journal 31(1): 43–53. DOI: 10.1111/hir.12054.

Bramer, W.M., Giustini, D. & Kramer, B.M.R. 2016. Comparing the coverage, recall, and precision of searches for 120 systematic reviews in Embase, MEDLINE, and Google Scholar: A prospective study. Systematic Reviews 5(1): 39. DOI: 10.1186/s13643-016-0215-7.

Buck, W. 2017. Precision and recall: An ontological perspective. Canadian Journal of Information & Library Sciences, 41(1/2): 42–51. https://muse.jhu.edu/article/666448.

Clarke, S.J. & Willett, P. 1997. Estimating the recall performance of web search engines. Aslib Proceedings 49(7): 184–189. DOI: 10.1108/eb051463.

Clough, P. & Sanderson, M. 2013. Evaluating the performance of information retrieval systems using test collections. Information Research 18(2). Paper 582. http://www.informationr.net/ir/18-2/paper582.html.

Craswell, N. 2009. Precision at n. In: Liu L, Özsu MT, editors. Encyclopedia of Database Systems. Boston, MA: Springer US. p. 2127–2128. DOI: 10.1007/978-0-387-39940-9_484.

Crawley, M.J. 2015. Statistics: An Introduction Using R. 2nd ed. Chichester (UK): John Wiley & Sons. DOI: 10.1002/9781119941750.

Deka, S.K. & Lahkar, N. 2010. Performance evaluation and comparison of the five most used search engines in retrieving web resources. Online Information Review 34(5): 757–771. DOI: 10.1108/14684521011084609.

Freelon, D.G. 2010. ReCal: Intercoder reliability calculation as a web service. International Journal of Internet Science 5(1): 20-33. http://www.ijis.net/ijis5_1/ijis5_1_freelon.pdf.

Fleiss, J.L., Levin, B. & Paik, M.C. 2013. Statistical methods for rates and proportions. John Wiley & Sons. DOI: 10.1002/0471445428.

Gisev, N., Bell, J.S. & Chen, T.F. 2013. Interrater agreement and interrater reliability: Key concepts, approaches, and applications. Research in Social and Administrative Pharmacy 9(3): 330–338. DOI: 10.1016/j.sapharm.2012.04.004.

Griffith, B.C., White, H.D., Drott, M.C. & Saye, J.D. 1986. Tests of methods for evaluating bibliographic databases: An analysis of the National Library of Medicine’s handling of literatures in the medical behavioral sciences. Journal of the American Society for Information Science 37(4): 261–270. DOI: 10.1002/(SICI)1097-4571(198607)37:43.0.CO;2-6.

Gwet, K.L. 2 2014. Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement Among Raters (4th edition). Gaithersburg, MD: Advanced Analytics, LLC. http://www.agreestat.com/book4/.

Joseph, L.E. 2007. Comparison of retrieval performance of eleven online indexes containing information related to quaternary research, an interdisciplinary science. Reference & User Services Quarterly 47(1): 56–65. DOI: 10.5860/rusq.47n1.56.

Landis, J.R. & Koch, G.G. 1977. The measurement of observer agreement for categorical data. Biometrics 33(1): 159–174. DOI: 10.2307/2529310.

Lowe, M.S., Maxson, B.K., Stone, S.M., Miller, W., Snajdr, E. & Hanna, K. 2018. The Boolean is dead, long live the Boolean! Natural language versus Boolean searching in introductory undergraduate instruction. College and Research Libraries 79(4): 517–534. DOI: 10.5860/crl.79.4.517.

McCain, K.W., White, H.D. & Griffith, B.C. 1987. Comparing retrieval performance in online data bases. Information Processing & Management 23(6): 539–553. DOI: 10.1016/0306-4573(87)90058-6.

McHugh, M.L. 2012. Interrater reliability: The kappa statistic. Biochemia Medica 22(3): 276-282. DOI: 10.11613/BM.2012.031.

Ouzzani, M., Hammady, H., Fedorowicz, Z. & Elmagarmid, A. 2016. Rayyan—a web and mobile app for systematic reviews. Systematic Reviews 5: 210. DOI: 10.1186/s13643-016-0384-4.

Perry, J.W., Kent, A. & Berry, M.M. 1955. Machine literature searching X. Machine language; Factors underlying its design and development. American Documentation 6(4): 242–254. DOI: 10.1002/asi.5090060411.

Ritchie, S.M., Young, L.M. & Sigman, J. 2018. A comparison of selected bibliographic database subject overlap for agricultural information. Issues in Science & Technology Librarianship 89. DOI: 10.5062/F49Z9340.

Sanderson, M. 2010. Test collection based evaluation of information retrieval systems. FNT in Information Retrieval 4(4): 247–375. http://marksanderson.org/publications/my_papers/FnTIR.pdf.

Sewell, R.R. 2011. Comparing four CAB Abstracts platforms from a Veterinary Medicine perspective. Journal of Electronic Resources in Medical Libraries 8(2): 134–149. DOI: 10.1080/15424065.2011.576608.

Shafi, S.M. & Rather, R.A. 2005. Precision and recall of five search engines for retrieval of scholarly information in the field of Biotechnology. Webolog 2(2). http://www.webology.org/2005/v2n2/a12.html.

Ştirbu, S., Thirion, P., Schmitz, S., Haesbroeck, G. & Greco, N. 2015. The utility of Google Scholar when searching geographical literature: Comparison with three commercial bibliographic databases. Journal of Academic Librarianship 41(3): 322–329. DOI: 10.1016/j.acalib.2015.02.013.

Stokes, P., Foster, A. & Urquhart, C. 2009. Beyond relevance and recall: Testing new user-centred measures of database performance. Health Information & Libraries Journal 26(3): 220–231. DOI: 10.1111/j.1471-1842.2008.00822.x.

van Rijsbergen, C.J. 1979. Information Retrieval. 2nd ed. London: Butterworths. http://www.dcs.gla.ac.uk/Keith/Preface.html.

Voorhees, E.M. 2002. The philosophy of information retrieval evaluation. In: Peters C, Braschler M, Gonzalo J, Kluck M, editors. Evaluation of Cross-Language Information Retrieval Systems. CLEF 2001. Vol. 2406. Berlin: Springer. (Lecture Notes in Computer Science). p. 355–370. DOI: 10.1007/3-540-45691-0_34.

Walters, W.H. 2009. Google Scholar search performance: Comparative recall and precision. portal: Libraries & the Academy 9(1): 5–24. DOI: 10.1353/pla.0.0034.




How to Cite

Ritchie, S., Banyas, K., & Sevin, C. . (2019). A Comparison of Selected Bibliographic Database Search Retrieval for Agricultural Information. Issues in Science and Technology Librarianship, (93). https://doi.org/10.29173/istl48



Refereed Articles
Share |

Most read articles by the same author(s)