Evidence Summary

Secondary School Students Ascribe Value to Presentation, Accuracy, and Currency in their Evaluation of Web-Based Information

A Review of:

Pickard, A. J., Shenton, A. K., & Johnson, A. (2014). Young people and the evaluation of information on the World Wide Web: Principles, practice and beliefs. Journal of Librarianship and Information Science, 46(1), 3-20. http://dx.doi.org/10.1177/0961000612467813

Reviewed by:

Kimberly Miller

Learning Technologies Librarian

Albert S. Cook Library

Towson University

Towson, Maryland, United States of America

Email: kimberlymiller@towson.edu

Received: 20 Nov. 2015 Accepted: 10 Feb. 2016

2016 Miller. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

Abstract

Objective – To measure the importance students place on criteria used to evaluate Web-based information.

Design – Online, self-report questionnaire.

Setting – Secondary school in the United Kingdom.

Subjects – 149 students aged 13-18 years, representing a response rate of approximately 21% of the 713 students sampled.

Methods – The authors used themes generated in a previous study of Web-based information evaluation (Pickard, Gannon-Leary, & Coventry, 2010) to create a 10-item questionnaire about the importance of criteria used to evaluate Web-based information. Criteria represented in the questionnaire included accuracy, authority (2 statements), currency (2 statements), coverage, presentation, affiliation, source motivation, and citations. Students used a four-point scale from “Very important” to “Not at all important” to indicate how significant they considered each criteria to be when they evaluated websites.

Students received an email invitation to participate in the study, with a link to the questionnaire in the school’s SharePoint environment. Two subsequent email reminders were sent approximately 8-10 weeks after the initial invitation to participate. Teachers at the school were also asked to promote the questionnaire in their classes.

Main Results – Over 75% of the 149 student respondents rated statements about presentation (n=116), accuracy (n=114), and currency (n=116) as “Very important” or “Quite important.” A majority of students (over 50%) rated the two statements about website authorship as being only “A little important” or “Not at all important” (n=92, and n=86). However, 62% of students (n=92) indicated that a website’s sponsoring organization is “Very important” or “Quite important.” The authors suggest there were some differences between responses from older and younger students, with older students more likely to rate statements about coverage, citations, organization sponsorship, and source motivation as “Very important” or “Quite important.”

Conclusion – The authors recommend that instruction about information evaluation for teenagers does not need to take a “back to basics” approach (p. 16), as most questionnaire respondents indicated they already find several criteria to be important when evaluating information. Instead, instruction should address student opinions and misconceptions about Web-based information in the context of their school assignments or other information needs. For example, students may be more motivated to learn about and apply evaluative criteria that are generated through discussion with their peers. Students may also be more receptive to expanding information evaluation criteria when they are researching topics they find interesting or important. Finally, the authors recommend that instruction should take into account the context or situations in which various evaluation criteria may be most important.

Commentary

Evaluating Web-based information remains a key skill in today's technology-saturated world. Documents such as the Association of College and Research Libraries’ Framework for Information Literacy for Higher Education and the American Association of School Librarians’ Standards for the 21st Century Learner place evaluation skills at the heart of modern information literacy education and practice.

This study begins with an engaging literature review that describes the challenges underlying contemporary information evaluation. This review includes the historical significance of evaluation, the complex cognitive abilities required to formulate complete evaluations, and the centrality of evaluation in critical thinking and information literacy. The review provides insights into the complicated nature of studying, measuring, and teaching information evaluation skills.

Material design, data collection, and data analysis methods are described in a clear and replicable manner, but questionnaire validity, data reporting, and result analysis are more difficult to evaluate when reviewed against Glynn’s (2006) critical appraisal checklist. The data collection instrument was constructed using findings from a previous study of information evaluation criteria in higher education (Pickard et al., 2010). Constructing a questionnaire based on themes from a previous study signals instrument validity and allows the current study’s results to be compared to previous research results. There is less discussion about why teenagers’ “internal cues” (p. 9) are expected to match those of higher education users’ aside from the authors’ observation that existing information literacy instruction usually targets similar criteria.

The study results are presented in a table of descriptive statistics per questionnaire item and response option, while the narrative outlines underlying patterns in evaluation cues students deemed more or less important. A visual representation of other results, such as a list of the top three most and least important internal cues, or a table summarizing results by cue, would have helped the reader quickly interpret trends. Sample size limitations restrict the study's ability to detect differences between students’ grade levels. This means the influence of confounding factors, like students’ developmental and ability differences, is unclear in the analysis. The study authors acknowledge these limitations and do not overextend their interpretation of the evidence.

Librarians should find the study’s themes useful when designing information evaluation instruction. In particular, a study observation about how the classroom curriculum may influence information evaluation is worth further consideration in practice. For example, the study authors hypothesize that students value evaluation criteria presented in their regular course work, such as the emphasis teachers place on attention to grammar and presentation. More often than not, library or information literacy instruction takes place embedded within existing classes. This means understanding the greater curricular context in which students develop and internalize information skills and attitudes is essential for coherent, meaningful information literacy instruction.

References

Glynn, L. (2006). A critical appraisal tool for library and information research. Library Hi Tech, 24(3), 387-399. http://dx.doi.org/10.1108/07378830610692154

Pickard, A. J., Gannon-Leary, P., & Coventry, L. (2010). Users’ trust in information resources in the web environment: A status report. Retrieved from http://ie-repository.jisc.ac.uk/470/2/JISC_User_Trust_final_report.pdf