Undergraduate Students Perceive Reference Encounters to be Teaching and Learning Activities

Abstract

A Review of:
Gremmels, G. S., and K. S. Lehmann. “Assessment of Student Learning from Reference Service." College & Research Libraries 68.6 (2007): 488-501.

Objective – The study explores the instructional nature of reference encounters from the perspective of students and librarians. Specifically, the study asks: 1) whether students perceive reference interactions to be instructional, 2) whether what they learn is the same as what the librarians intended to teach, and 3) whether they connect reference-based instruction with any formal information literacy classes in which they may have participated.

Design – Survey questionnaire with two parts: one for students and the second for reference librarians, administered twice (revisions to the study and to the questionnaire were made in between).

Setting – Wartburg College in Iowa, United States. Wartburg is a 1,800-student private, residential, coeducational college with a strong course-integrated information literacy program.

Subjects – An unknown number of undergraduate students who were perceived by reference librarians to have asked instructional questions at the reference desk (264 surveys were collected from students, but some students may have completed more than one survey as the first implementation of the study allowed repeated participation for students with more than one instructional reference encounter) and four librarians.

Methods – The study was conducted two times during the years 2003 and 2004 (referred to here as study A and study B) and findings are reported for each study separately. The data collection instrument in both implementations was a paper survey that was divided into two sections and perforated to collect information from two perspectives: the student and the librarian. The surveys were numbered to facilitate matching between the two sections after the survey was completed and returned by the student/librarian pair.

Potential student participants in the study were identified at the reference desk: each time a librarian deemed a reference question to be instructional, he or she invited the student to complete a short survey. If the student agreed, the librarian tore off and kept the librarian portion of the survey and gave the second section to the student to complete. Students and librarians deposited their sections into a box and sections were re-matched and moved to a secure location every few days.

On their section of the survey, students were asked whether the librarian who assisted them taught them anything while answering their question, and if so, to describe what the librarian taught them in their own words. Additionally they were asked if what the librarian taught them built upon skills learned in a library session held for the present assignment or from a previous class, if applicable. On the librarians’ section of the survey, there were two questions: what the librarian intended to teach the student (to be chosen from a checklist) and whether the librarian thought the student understood that intention during the reference encounter. Comments were allowed.

Some revisions were made to the survey instrument between study A and study B, including two of note: a check list matching that on the librarians’ survey was added to the student survey question about what the librarian taught the student, and the student participant selection criteria were changed to limit only one response per student.

At the end of both studies, data analysis was done, including entering data into a spreadsheet and transcribing both student and librarian descriptions of learning into a narrative document (for study A) or Qualrus, a software program for qualitative research (for study B). The study authors independently compared the librarians’ and students’ descriptions of what was taught and labelled the match as either “related”, “inconclusive” or “not related” (for study A) or “strong match,” “acceptable match,” or “no match” (for study B). Disagreements were discussed and authors came to an agreement for each.

Main Results – Response rates for study A (85%) and study B (78%) were high. Most students indicated that they believed the librarian taught them something (94% for study A and 98% for study B).

Findings on whether the students learned what the librarian intended to teach are mixed. For study A, 60% of the student responses were deemed matches, 20% were not related and an additional 20% were inconclusive. For study B, the authors report their findings in a different manner since the student survey included a checklist that matched the librarian survey in addition to a narrative description of what the librarian taught them. The findings therefore include whether the librarian and student surveys matched in the open-ended descriptions, in the category checklist, in both, or in neither. In this second study, 21% matched in the description only, 36% matched in the category only, 21% matched in both the description and the category, and 22% showed no match at all. This puts the overall match rate for study B at 78%.

Surveys were analyzed to determine which categories were most likely to be matched. For study A, 62% of the matches were in the “tool” category, 4% in “terms,” 16% in ”strategy,” 10% in ”database,” and 8% in ”other.” For study B, 42% of the matches were in the ”tool” category, 22% in “terms,” 16% in “strategy,” 16% in “database” and 4% in “other.”

Additional findings relate to the connections between the reference encounters and previous information literacy sessions. Approximately one third of students had participated in an information literacy session (33% for study A and 34% for study B) for their class. Of these students, most connected what the reference librarian taught them and what they had learned in their in-class instruction (89% for study A and 95% for study B). The rate for how many students connected reference-based instruction with a prior information literacy class was lower, but still quite high (77% for study A and 74% for study B).

Conclusion – The vast majority of student participants perceived that the reference encounters were instructional and most also connected what they learned from the librarian in the reference encounter to librarian-led information literacy sessions in their current or previous classes. This suggests that post-session reference assistance could help reinforce information literacy principles in a one-on-one situation, perhaps closer to the time of need. For this follow-up instruction to be effective, reference librarians may want to refer directly to what was taught in the information literacy sessions, which could help place new skills in the context of those with which the students are already familiar. The authors also suggest that short reference encounters at the desk may not be appropriate for this kind of instruction and propose that scheduled consultations of up to 30 minutes may better meet students’ needs.

Rates for matching what the student thought the librarian taught them and what the librarian intended to teach them were lower, if still relatively high at 60% (study A) and 78% (study B). The authors attribute this lower rate in study A to conservative coding, the inability of researchers to follow up on what the students meant by their answers, and the difficulty students might have in describing what they refer to as “tacit knowledge.” Undoubtedly, the open-ended nature of the students’ responses led to difficulties in matching how they might describe a skill or tool to how a librarian would describe it. Adding a checklist of categories for students to select from for study B certainly made it easier to overcome the problem of students not using (or perhaps even knowing) the same terminology to describe skills as librarians and helped to achieve a higher match rate.

“Tools” was the category that was most likely to be matched in both study A and study B. The authors did not speculate on the reasons for this finding.