Introduction

For all practitioners, evidence based practice is challenging. For those who teach information literacy instruction (ILI), evidence based librarianship becomes particularly difficult when teaching sessions are “one shot.” Many librarians have a single session with a group of students that will last between 50 and 80 minutes, and a significant amount of content to cover. As student learning is the ultimate goal for ILI, the choice of what content to cover (e.g., teaching concepts or skills) and how to teach that content (e.g., lectures or hands-on practice) is of the utmost importance.

Collecting evidence for any research is challenging; however, when attempting to practice evidence based librarianship in this restricted context, data collection must be quick and unobtrusive, not taking up precious class time. One way to incorporate data collection into one-shot sessions is to use what already takes place in the class. Many librarians do this by collecting assignments, either research assignments that have been assigned by the course lecturer (Webster & Reilly, 2003) or worksheets that have been assigned by the librarian in the ILI session (Fain, 2011), or a combination of these two methods. For other librarians, course assignments are not available and they may decide worksheets take up more class time than they are willing to give.

The author was interested in questions of pedagogy – how best to use class time in the ILI session. Typical classes were short lectures followed by lengthy periods of time to search independently, combined with one-on-one help termed scaffolding. Scaffolding is a technique in which a teacher works with students individually to give them support, gradually removing that support as the student is able to work more independently. (Larkin, 2008). The evidence based librarianship project was designed to determine if current practices were helpful to students. The research question was whether giving students time to research independently during a one-shot information literacy instruction session, combined with scaffolding, is an effective use of class time.

Literature Review

Information literacy instruction is a large part of many librarians’ work. As such it has been the subject of much study. To better understand what takes place in the classroom, it is important to examine both student behaviour and classroom pedagogy.

Information literacy instruction is a complex research topic, with a multitude of factors impacting the learner, the learning environment, and the instruction. For those who teach searching, understanding how students search for online information is an important aspect. In their Information Behavior Model, Urquhart and Rowley (2007) identified many micro and macro factors that impact student information behaviours, including information literacy, search strategies, discipline and curriculum, pedagogy, support and training, information resource design, and access. Many students typically begin searching with Google (Griffiths & Brophy, 2005; Urquhart & Rowley, 2007), relying less on academic resources (Griffiths & Brophy, 2005). Starting searches with Google and limiting use of resources to those that are well known is typical, as ease of use and familiarity are important factors in their choice of resource (Dervin & Reinhard, 2007; Griffiths & Brophy, 2005; Urquhart & Rowley, 2007). Searchers in online environments typically include few terms in their search queries, infrequently use advanced search techniques such as Boolean operators, and infrequently use advanced search features such as limiters (Lau & Goh, 2006; Markey, 2007a, 2007b; Wang, Berry, & Yang, 2003; Willson & Given, 2010). In addition to simple searching, students often expect online public access catalogues and databases to work like a search engine (Griffiths & Brophy, 2005; Novotny, 2004). Searchers with less search experience spend less time thinking about and planning searches, in addition to using fewer self-aware, metacognitive strategies than do searchers with expertise (Tabatabai & Shore, 2005). Overall, undergraduate students use familiar sources in fairly rudimentary ways to satisfy their information needs.

Many researchers have examined the effectiveness of ILI. Some research has focused on the general effectiveness of ILI. Portmann & Roush (2004) found that ILI increased library usage, though not library skills. One of Koufogiannakis and Wiebe’s (2006) findings from their meta-analysis of ILI studies was that, overall, instruction of any variety was better than no instruction. Other research has focused on the effectiveness of particular interventions. Buhay, Best, and McGuire (2010) found that student scores on post-tests were statistically significantly higher when they used clickers in ILI. Marcus and Beck (2003) found that students who took part in a treasure-hunt style self-orientation to the library scored higher on questionnaires and rated the tours more positively than those who were in librarian-led orientation groups. Bren, Hillemann, and Topp (1998) found that using a guided, hands-on instructional method increased undergraduate students’ retention of information provided during an ILI session.

These studies indicate that ILI is effective and that particular interventions can be used. What is missing from studies of ILI effectiveness is an examination of scaffolding and independent search time. One article mentioned scaffolding as part of the ILI instruction (Johnson et al., 2011). No articles were found that addressed independent search time. It is difficult to determine whether this apparent lack of literature is due to these pedagogical practices not being researched, or due to differences in terminology that make the literature difficult to find.

Context and Aims

Context

Mount Royal University is a four-year undergraduate university. The student body is 10,551 full-time students (Mount Royal University, n.d.a). The institution has four categories within its Aims of an Undergraduate Undergraduate Education, with information literacy listed under Intellectual and Practical Skills (Mount Royal University, n.d.b). In 2009/2010, the library taught 710 ILI sessions to over 12,000 students. Typically ILI sessions are one-shot sessions that are either 50- or 80-minutes long and tend to be focused on a specific research project. The vast majority of ILI sessions are hands-on and take place in computer labs.

Aims

As part of evidence based practice, the author was interested in examining the information literacy instruction she provides, with a view to improving understanding of whether the independent time to search in the ILI session is used effectively and how teaching could be altered to improve learning. Feeling that she was trying to include too much content into one-shot sessions, she cut down on the amount of content presented in class in the lecture format, focusing on the specific research assignment and providing as much time as possible for hands-on work. The focus of these ILI sessions became the students’ research assignments and the individual help (scaffolding) provided to students by the librarian and the class instructor.

The researcher felt that giving students time to work on their research assignment is a more active learning technique and that students would have the opportunity to try what had been discussed in class, along with scaffolding. Part of scaffolding is to work within students’ “zone of proximal development,” the gap between what a student can achieve on their own and what they can achieve with help. By focusing on individual time with students to scaffold their work, the researcher believed that instruction could be better tailored to students’ specific needs (e.g., working within their zone of proximal development), focus on those needing more help and support by providing additional time and attention, while more confident students could get research done during class time.

While this was the reasoning behind the original changes made to the ILI sessions, the assumption being made was that independent time to work would be beneficial for students. The author received challenges to this assumption by colleagues in the scholars’ program who wondered if students were prepared for time to work independently and whether giving students an assignment that introduced concepts might be a more beneficial use of class time. While much research has been done into which method of delivering information literacy instruction (e.g., computer-assisted instruction vs. traditional instruction), fewer librarians have researched what specific aspects of a method of instruction make it beneficial.

Methods

Participants

Students from 10 ILI sessions were the participants in this study. The classes were at the first-, second-, and third-year level from the religious studies, psychology, and general education disciplines. Class sizes ranged from 20-30 students, meaning the participants were drawn from an overall sample of approximately 200-300 students. The classes focused on searching for sources for their assignment, basic search strategies (Boolean operators, truncation, and phrase searching) and database searching. In two sections of a third-year psychology course, students were taught to use MeSH. The inclusion criteria for the study were that students were attending an ILI session that was 80 minutes or longer, as 50-minute sessions were too short to include a Web survey. All students in ILI sessions were asked to perform the same tasks, fill out a research log during independent searching, and to complete a Web-based survey at the end of class. All students’ responses were examined to inform pedagogy and student learning. Only the data of those students who agreed to participate were included in the study. In total, 73 students agreed to include their Web survey in the study and 83 students agreed to include their research log in the study. This study received approval from the Human Research Ethics Board at Mount Royal University.

Web Survey

This quantitative study included both a Web survey and research logs. To examine the research question, the researcher designed a Web survey and research log. Section 1 of the Web survey asked whether students had attended a previous ILI session (and if so, how many), trying to gauge students’ prior experience (see Appendix A). Section 2 of the Web survey was designed to determine the specific aspects of the ILI session – the different pedagogical tools used in the classroom – that students perceived to be most useful by comparing one aspect to another. Students were asked to rate the activities that took place during the class from most useful (1) to least useful (9). Finally, in Section 3, students were also asked to rate their preparedness before and after the ILI session on a four-point Likert scale. The research referred students to the Web survey URL via the online subject guide for the class. The Web survey was administered at the end of the ILI session, taking approximately five minutes to complete.

Research Logs

The research logs attempted to uncover students’ information behaviours – which resources they use, how they search, how they modify their searches, what they think about their searching (see Appendix B). The research logs were examined to determine if students used the skills taught in class during independent searching. Other research has described research journals as part of an ongoing research process throughout a class (Smith, 2001; Warner, 2003). The research log used in this study is intended to capture students’ searching at a particular time and to aid students in recording their search process, similar to Kuhlthau’s search logs (2004, pp. 32-33). Bates’ Berrypicking model (1989) and Kuhlthaus’ Information Search Process model (2005) were used in the creation of the research log – to help students track their search progression and the change in their thinking that leads to search modification.

During the independent search time, students were asked to record their work. The research log was on carbonless paper; students kept the top copy while the author kept the bottom. It is important for students to keep track of their searches to understand where they have searched, to understand what they have searched, and to examine how their research might progress. In addition to being a form for data collection, the researcher employed the research log as a pedagogical tool to try to increase students’ awareness of their searching. The research log was used in preference to computer logs, which do not involve student thought. Students were asked to record their topic/thesis statement and ideas/concepts related to their topic. Students were also asked to record their searches: the date, the resource searched, the search query, what was found, and notes to self. If students asked questions during the ILI session, they were also asked to record their questions: what the question related to, if their question was answered, and what questions they felt might come up later.

Analysis

The researcher analyzed the Web surveys using descriptive statistics (frequencies and percentages), which were chosen to summarize the responses from the students in the sample and to provide basic information about the responses. The researcher examined research logs to determine if there was evidence of what took place during the independent search time, while recorded searches were examined to determine where students searched, the search queries students created, the search strategies students used, the ways in which searches changed, and what students wrote about their searching. From looking at how students’ modified their searches and what they wrote about their searches, the researcher developed categories to describe the commonalities seen. After developing operational definitions for the categories, the author categorized the searches. The categories were: search complexity (searches with two or more terms or use of specific search strategies), search progression (series of searches in which students increase the complexity or precision of their searches), use of feedback (series of searches in which students use the results of previous searches to modify or improve searches), and mode of search modification (incremental modifications or jumps from one strategy to another). The researcher used the categories to produce descriptive statistics about how students used their independent search time.

Results

Usefulness of ILI Sessions

The Web survey asked students to rate the usefulness of the different aspects of the ILI session from most useful (1) to least useful (9). Students rated individual help from the librarian as the most useful aspect of the ILI session with a rating of 3.89 (see Table 1). The second most highly rated aspect of the session was discussions of how to use resources (4.46). After the top two rated aspects of the session there was little variance between ratings. The author had hypothesized that students would rate time to work independently as the most valuable aspect of the ILI session. Instead, time to work independently was tied for the eighth most useful aspect of the session. Rather than simply valuing the time to work, students rated the one-on-one help they received during that independent search time as more useful. While this survey only asked for student perceptions of usefulness by comparing different pedagogical activities, students valued the scaffolding that took place in the session. As the results from the Web survey were too small to run a test for statistical significance, the ratings should be treated with caution and more research is needed to confirm these findings.

Table 1

Average Rating of Aspects of ILI Sessions from Most Useful (1) to Least Useful (9)

Answer Options	Rating Average
Individual help from librarian	3.89
Discussion of how to use the resources	4.46
Discussion about search difficulties	4.86
Citation discussion	4.93
Individual help from class instructor	4.94
Discussion of the resources to use	5.00
Time to work independently	5.23
Working/discussing with class mates	5.23
Explanation of the assignment	5.75

Table 2

Evidence, from Three Different Students, of Using What Was Taught in Class

Evidence of using what was taught in class	Use (%) N=77	Example of evidence
Clear evidence	43 (56%)	Bipolar Disorder in MM, review articles, linked full text
Unclear evidence	21 (27%)	Bipolar Disorder, youth
No evidence	13 (17%)	Bipolar Disorder

The researcher examined the research logs to determine if students used what was taught in class. Of the 83 research logs included in the study, 77 (93%) contained recorded searches; 43 of those 77 (56%) showed clear evidence of using what had been taught, while 21 (27%) showed some evidence, and 13 (17%) showed no evidence (see Table 2). There is evidence that students understood the content of the lesson well enough for them to use it to search during independent searching within the context of the ILI session. (Table 2)

How Students Are Searching

The author also analyzed the research logs for evidence of how students search during the time given for independent searching. In total 237 searches were recorded, representing an average of 3.1 searches per research log (n=77). Searches averaged 3.7 words per query. Of the 77 research logs containing searches, 52 (68%) included use of Boolean operators, 23 (30%) included use of truncation, 19 (25%) used phrase searching, and 16 (21%) recorded the use of a search limit. Students used Boolean operators most frequently of the search strategies taught during the ILI session. Of those using Boolean operators, 1 (2%, n=52) used them incorrectly; for those using truncation, 4 (17%, n=23) used them incorrectly; and for phrase searching, 6 (32%, n=19) used them incorrectly (see Table 3). The percentage of incorrect uses for each of the search strategies may indicate that students are most comfortable using Boolean operators and least comfortable using phrase searching.

Table 3

Search Strategy Use and Examples of Incorrect Use

Search Strategy	Use (%) N=77	Inappropriate Use (%), N	Example of Incorrect Use
Boolean	52 (68%)	1 (2%), n=52	Divorce and children and childhood
Truncation	23 (30%)	4 (17%), n=23	Immigration and poverty and Canada*
Phrase	19 (25%)	6 (32%), n=19	“abuse” and “elder”, specific to 65+¹

¹Referring to database-specific age limit

Table 4

Search Complexity and Examples of Complexity

Search Complexity	Use (%) N=77	Example of search complexity
Complex search	34 (44%)	Globalization and relig* and identity
Not complex search	23 (36%)	Walmart and globalization
Elements of complexity	15 (19%)	Eat* local*

Searches were also rated on their complexity. A complex search had more than two ideas, or had two ideas in addition to employing specific search strategies – Boolean operators, truncation, subject heading searches, etc. The author found that 34 of 77 (44%) research logs included complex searches, that 28 (36%) did not have complex searches, and that 15 (19%) had elements of complex searches but could not be fully categorized as complex (see Table 4). When looking at the number of students that used more than two ideas in their search and different search strategies, the author discovered that the overall searches were relatively simple. While different assignments required differing levels of search complexity, many of the topics students were exploring would retrieve results too great in number or lacking in precision.

The researcher also examined how searches were modified, whether in increments, by making small modifications to search strategies or an aspect of a term, or in jumps,

such as by changing vocabulary, topics, or resources entirely. Of the 60 research logs with multiple searches, 37 (62%) made search modifications using increments, 13 (22%) made search modifications using jumps, and 10 (17%) made search modifications using both. In making incremental modifications, students were searching in a more focused way, testing how small changes to a search will affect the search results. In making jumps in search query modifications, students were searching in a broader way, by exploring what is available or exploring their topic.

The researcher examined search modifications (changes made to search queries and/or resources in which the search was carried out over successive searches) to determine the specifics of how such changes were made: keywords used, resources used, and search techniques. Changing the keyword used was the most common modification, followed by adding or subtracting keywords, then changing the resource in which the search was carried out, followed by changing search techniques, such as using operators, truncation and/or phrase searching (see Table 5). Of 60 research logs with recorded search modifications, 19 (32%) revealed the use of multiple search modifications during the search. The data suggest that students view keyword terms as the primary way to change their searches.

Table 5

Types of Search Modifications by Number and Percentage of Research Logs

Search Modification	Number (%) (n=60)*
Changing keywords	46 (77%)
Adding or subtracting keywords	37 (62%)
Changing resource used	22 (37%)
Putting on or taking off database limits	9 (15%)
Adding or subtracting Boolean operators	5 (8%)
Adding or subtracting truncation	3 (5%)
Adding or subtracting phrase searching	2 (3%)
Other	2 (3%)

*Multiple search modifications could be used in one research log.

Student Preparation to Search Independently

At the end of the ILI session students filled out the Web survey, rating their preparedness to do research before and after the session. Looking back, 41% of students rated their preparedness before the session as “prepared” or “somewhat prepared,” while 100% of students rated their preparedness after the session as “prepared” or “somewhat prepared” (n=73). The author examined research logs to see if this perception was corroborated in behaviour.

Examining changes in the research logs allowed the researcher to observe how students adapted their searching during time given in class to search independently. Searches were examined to determine if they showed progress, which was defined as a series of searches in which students increased the complexity or precision of their search. An example of a student search that demonstrates progression is shown in Table 6, while an example that does not demonstrate progression is found in Table 7. A total of 60 research logs contained more than one recorded search and were examined for search progression. Of those 60 research logs, 41 (68%) showed a progression while 19 (32%) showed no progression. That evidence of progression appeared in the majority of research logs with multiple searches indicates that students were able to use their independent search time to adaptively change their searches.

Effective changes must use feedback from the results of previous searches. Evidence of use of feedback was defined by a series of searches in which students used the results of previous searches to modify and improve their searches, as evidenced by discussion of changes in the Results or Notes to Self fields or the modification of search terms. Of the 60 searches that had multiple recorded searches, 29 (48%) showed evidence of use of feedback, while 31 (52%) showed no evidence. Approximately half of the students gave evidence that they used the previous searches to inform their subsequent search choices, indicating that students were learning while they used the independent time to search. A closer look at Table 6 and Table 7 reveals differing use of feedback.

Both tables show examples of two different research logs from the same class. The examples can be examined for of several types of searching: progression, use of feedback, demonstrating use of what was taught in class and complexity. In the class from which these research logs come, students were taught to use MeSH to search for neurological disorders. The student in Table 6 demonstrates progression through their use of the search terms: starting with the name of the disorder, checking it in MeSH, using the MeSH term as a major subject heading, and then continuing to add words and limits to the search until the student reaches what s/he determines to be a useful search. The student in Table 7 does not show progression. The second search used has more ideas, making it more complex; however, there is no indication of how the student arrived at the search or whether the student tried other searches that were more or less successful. The example in Table 6 demonstrates the use of feedback from previous results, making comments in the Notes to Self about how the search could be changed, which are then reflected in the searches and the resources found. The example in Table 7 demonstrates no use of changing based on previous results. The first example demonstrates what was taught in class – subject headings, subheadings, and the use of limits – while the second example indicates some use example has of the language discussed in class (e.g., etiology) but has used keywords in Medline and used no Boolean operators or phrase searching while in ScienceDirect. Also, the first example includes a complex search using many ideas and limits together, while the second example also has both a more and less complex search. The first recorded search is simple in query terms, though it might also include limits making it more complex, whereas the second search has more ideas included despite problems with the search construction.

Table 6

Example of Student Searching Demonstrating Progression in Searching

Resource Used	Search	Resources Found	Notes to Self
Medline	Guillain-Barre Syndrome	Overwhelming, 5,000+ articles	Use MeSH
Medline	(MM “Guillain-Barre Syndrome”)	1,800+ articles, but interesting subset headings	Advanced search “etiology”
Medline	(MM “Guillain-Barre Syndrome”) AND etiology	500+ articles	Advanced search check review articles
Medline	(MM “Guillain-Barre Syndrome”) and etiology + review articles	70 results. There are 7 solid articles I can use on first page	But there are NO COPIES!!??
ScienceDirect	Advanced → Review Articles, Title/Abstract/Keyword, Guillain-Barre Syndrome	72 results, 4-5 articles I can use	ScienceDirect actually has copies!

Table 7

Example of Student Searching not Demonstrating Progression in Searching

Resource Used

Search

Resources Found

Notes to Self

Medline

Narcolepsy and etiology – keywords

Many results found – reviewed, emailed

Link full text. Recent articles.

ScienceDirect

Narcolepsy sleep disorders REM sleep

e-mailed 8 articles for further review

Sleep disorders, etiology, sleep MRI, sleep EEG

The student whose search is represented in Table 6 demonstrates search progression, use of feedback, use of what was taught in class, as well as a complex search. The student whose search is represented in Table 7 partially demonstrates what was taught in class and a complex search. Without data triangulation through examining finished work it is not possible to determine if students’ recording of their searches was truly indicative of how they searched or of what they wanted to record for later use.

Discussion

Usefulness of ILI Sessions

The Web survey results indicate that students feel more prepared after the ILI session than before, and they rate the help from the librarian as the most useful part of the session. Counter to expectations, students rate the scaffolding they receive during the independent time to work much more highly than they rate the time they are given to work independently. Without further information about students’ perceptions and expectations, it is not possible to determine whether the ratings were due to perceived usefulness or prior expectations. Students may expect individualized help during ILI sessions. Overall, these results indicate that students do find ILI sessions useful in helping them to feel prepared to complete their research assignment. The data indicate that independent time to search is useful so far as it allows scaffolding to take place, as one-on-one help is viewed as more beneficial.

How Students Are Searching

Student searches were not very complex, with searches containing on average 3.7 words, and less than half of the research logs rated as having complex searches. Despite this, most students used Boolean operators and were able to use them appropriately. Students used truncation and phrase searching less frequently, and almost one-third of those using phrase searching could not use this technique appropriately. The majority of students are able to use, at least in part, what was taught in the ILI session. While this is encouraging, Cmor, Chan, and Kong (2010) found that while the majority of students could complete information literacy-related exercises in ILI sessions, few were able to demonstrate the ability to use new tools and search strategies or incorporate new knowledge into projects.

The researcher’s assumption that it is beneficial to give students hands-on time to search independently during the ILI session was partially substantiated. While students demonstrated that they could achieve many things during the time given to work independently, including formulating more complex and/or precise searches, using feedback to improve searches, and using what is taught in class, students most valued the individual help they received from the librarian. The research logs revealed several other aspects in which students could use instruction. For example, students’ searches were not very complex, and while not all search topics require complex searching, putting together a search strategy that increases both precision and recall is important. More instruction on search query formulation could be beneficial. In addition, more instruction about truncation and phrase searching could be beneficial; the number of incorrect uses indicates that students may not know what phrase searching will do.

Student Preparation to Search Independently

The majority of students who performed multiple searches showed evidence of search progression. This indicates that students are able, even within the course of an ILI session, to increase the complexity or precision of their searches. That fact that students are able to demonstrate this during class suggests that they may do the same in the searching that they engage in on their own time. Also encouraging was that almost half the student research logs contained multiple searches, evidence which suggests that students were using feedback to modify their searches, something not explicitly taught during ILI sessions. Additionally, the fact that students use feedback to make search modifications also lines up with the incremental changes seen in most research logs. Vocabulary and resources are the most commonly used ways to change searches, with little experimentation of search techniques. While changing terms is one of the best ways to modify a search query, students are not modifying search queries using search techniques, a topic that does not receives much attention during the ILI session.

Limitations

There were several limitations to this study, many of which stem from incorporating research into one-shot ILI sessions, which are particularly constrained. As the Web survey was completed at the end of the session, it had a lower response rate than the research logs, and the timing of the survey may also impact the results. This lower response rate may have been due to students rushing to finish class, which may also have impacted the survey results if students guessed at answers or chose answers which appeared first. In addition, the survey asked for usefulness of pedagogical tools in comparison with one another, making it more difficult to assess how these tools impacted student learning. The survey asked about the general preparedness of students, rather than preparedness related to specific tasks, which may have been less sensitive to differences in students’ levels of preparation and could have been affected by student interpretations of the question.

All data in the study were recorded by students. From in-class observations it was clear that some students were doing more searches than they were recording. From markings on the carbonless paper research logs it was clear that some students were also recording searches other places. Some students may have found recording their searches onerous. Some students might have difficulties performing the searches and making accurate recordings. This may have been the case particularly for students with less search experience, as recording searches adds another task and could increase the mental effort required to complete the work. While recording searches with research logs may have made the task more difficult, it has potential benefits as a pedagogical tool to help students think about their search process.

In addition to issues around students recording, the process of data analysis added limitations to the study. The researcher examined research logs together, by collapsing the classes into one group for comparison. Collapsing the classes has potential validity issues, though it was the most appropriate way to analyze the data collected. Since the research logs did not capture demographic data, other than class, and since the number of participants for each class was low, this means that group divisions were not meaningful. Further to issues of validity and reliability, the author categorized the research logs. In the future, to increase reliability and validity of the findings, more than one person should categorize the data.

Originally, the research project included an additional component to help triangulate the data, to gain further insight into student behaviours and to address whether the ILI had impact beyond the classroom. Those participants who completed the Web survey were asked if they would be interested in being contacted for a follow-up interview. Five participants indicated interest; however, only one participant took part in the semi-structured interview. Because of this, data collection was limited to the classroom and no follow-up information could be gathered. This lack of follow-up data means the results are limited to the quantitative results, and understanding reasons for participant behaviour is limited.

Implications for Future Practice and Research

From the progression seen in students’ searches produced during independent search time, it appears that students are making good use of that time. From the students’ point of view, that individual help from the librarian is the most useful part of the ILI session. Students find that independent time to search, when combined with scaffolding tailored to students’ individual needs, is beneficial or that it meets their expectations of an ILI session. Based on this evidence the author will continue to provide time for students to work independently and provide help on a one-to-one basis. The Web survey should be given to more students to determine how prevalent the view that the most important aspect of the session is one-on-one help with the librarian, in addition to what expectations students have of the session.

Recording searches can be challenging, particularly for students who are less familiar with searching and whose cognitive processing space is being used in doing the actual searches. To improve future research, as well as to help students’ keep track of their search process, new ways for students to accurately keep track of their search process should be explored. Database features such as search history, citation management tools, or instruments could be used for this purpose.

The majority of students demonstrated in their research logs what they learned in class. However, students are not demonstrating some of the important aspects of the research process. The author hoped for more evidence of using feedback to improve searches and metacognition about the search process. It is difficult to know whether students are not engaging in these activities, or whether the limitations of the situation (little time, computer lab environment, pressure to get work done, research log limitations) contribute to what students do or not do during class time. In addition, these types of higher-level thinking skills are not explicitly taught during class. Students are able to demonstrate activities taught during the ILI lesson during time given to search independently. It is possible that explicitly teaching skills such as thinking metacognitively and how to use feedback could also elicit those behaviours during independent search time. Future research will explore metacognitive aspects of the research process, both what aspects students engage in and how metacognitive thinking can be enhanced.

While students demonstrate that they can use what was taught in class, it is unknown if students can take that learning beyond the classroom and some research (e.g., Cmor et al., 2010) suggests it is doubtful. In addition, it is difficult to know whether students can extrapolate their learning in class to a greater understanding of research as a process. Librarians may want students to engage in metacognitive thinking and learn that research in a complex process, part of which involves library research. However, in one class it is unrealistic to expect students to become information literate. One-shot ILI sessions, while they may be effective, should be only one part of an overall library instruction program (Webster & Rielly, 2003). Because librarians often have only one class, working with instructors who have the semester and with programs that set four-year curricula becomes more important. ILI sessions need to go beyond teaching skills and into authentic student learning, requiring collaboration with classroom faculty (Wakimoto, 2010). Integrating ILI into what is done at the class and curricula level is necessary for students’ growth in information literacy.

Conclusions

Simply by examining what students do during a single ILI session, librarians can learn a lot about their own teaching, student information behaviour, and student learning. Students report that information literacy instruction sessions help them feel more prepared to do research. More than just time to work independently, students indicate scaffolding, the one-to-one instruction from the librarian, as being valuable. Within class, students are able to demonstrate generally correct use of skills taught during the ILI session, though their search queries are rather simple. While ILI sessions are only one part of a larger plan for developing IL skills, students report finding them useful. Also, demonstrating the usefulness of ILI sessions is the fact that many recorded student searches show increasingly complex searches or the use of feedback from previous searches to create more precise search queries. While skills are of immediate importance to students doing research assignments, these are important to students’ learning throughout their degrees. If students are not already demonstrating these understandings in their searches, they should be the focus of instruction. Again, this instruction cannot take place in a one-shot session as they are complex and take time to develop. Integrating more into courses and curriculum is important if we wish our students to attain these skills.

Acknowledgements

Support and funding for this research were provided by the Institute for the Scholarship of Teaching and Learning at Mount Royal University. A special thank you to Dr. Richard Gale, Dr. Karen Manarin, Dr. Deb Bennett, Dr. Miriam Carey, and the 2010 Scholar cohort. This research was first presented as a paper at the 6th International Evidence Based Library and Information Practice (EBLIP6) Conference, Manchester, UK, June 2011.

References

Baker, L. (2006). Library instruction in the rearview mirror: A reflective look at the evolution of a first-year library program using evidence-based practice. College & Undergraduate Libraries, 13(2), 1-20. doi: 10.1300/J106v13n02_01

Bates, M. J. (1989). The design of browsing and berrypicking techniques for the online search interface. Online Review, 13(5), 407-424.

Bren, B., Hillemann, B., & Topp, V. (1998). Effectiveness of hands-on instruction of electronic resources. Research Strategies, 16(1), 41-51. doi:10.1016/S0734-3310(98)90005-2

Buhay, D., Best, L. A., & McGuire, K. (2010). The effectiveness of library instruction: Do student response systems (clickers) enhance learning? The Canadian Journal for the Scholarship of Teaching and Learning, 1(1). Retrieved 4 Dec. 2012 from http://ir.lib.uwo.ca/cjsotl_rcacea/vol1/iss1/5/

Cmor, D., Chan, A., & Kong, T. (2010). Course-integrated learning outcomes for library database searching: Three assessment points on the path of evidence. Evidence Based Library and Information Practice, 5(1), 64-81. Retrieved 4 Dec. 2012 from http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/6509/6449

Dervin, B., & Reinhard, C. D. (2006). Researchers and practitioners talk about users and each other . Making user and audience studies matter – paper 1. Information Research, 12(1). Retrieved 7 Dec. 2012 from http://informationr.net/ir/12-1/paper286.html

Fain, M. (2011). Assessing information literacy skills development in first year students: A multi-year study. The Journal of Academic Librarianship, 37(2), 109-119.

Griffiths, J. R., & Brophy, P. (2005). Student searching behavior and the web: Use of academic resources and Google. Library Trends, 53(4), 539-554.

Johnson, C. M., Anelli, C. M., Galbraith, B. J., & Green, K. A. (2011). Information literacy instruction and assessment in an honors college science fundamentals course. College & Research Libraries, 72(6), 533-547.

Koufogiannakis, D., & Wiebe, N. (2006). Effective methods for teaching information literacy skills to undergraduate students: A systematic review and meta-analysis. Evidence Based Library and Information Practice, 1(3), 3-43. Retrieved 4 Dec. 2012 from http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/76/153

Kuhlthau, C. C. (2005). Kulhthau’s information search process. In K. E. Fisher, S. Erdelez, & L. E. F. McKechnie (Eds.), Theories of information behavior. (pp. 230-234). Medford, NJ: Published for the American Society for Information Science and Technology by Information Today, Inc.

Kuhlthau, C. C. (2004). Seeking meaning: A process approach to library and information services. Westport, CT: Libraries Unlimited.

Larkin, M. J. (2008). Scaffolding. In N. J. Salkind (Ed.), Encyclopedia of educational psychology. (pp. 863-864). Thousand Oaks, CA: Sage Publications.

Lau, E. P., & Goh, D. H. L. (2006). In search of query patterns: A case study of a university OPAC. Information Processing & Management, 42, 1316-1329. doi:10.1016/j.ipm.2006.02.003

Marcus, S., & Beck, S. (2003). A library adventure: Comparing a treasure hunt with a traditional freshman orientation tour. College & Research Libraries, 64(1), 23-44.

Markey, K. (2007a). Twenty-five years of end-user searching, part 1: Research findings. Journal of the American Society for Information Science and Technology, 58(8), 1071-1081. doi:10.1002/asi.20462

Markey, K. (2007b). Twenty-five years of end-user searching, part 2: Future research directions. Journal of the American Society for Information Science and Technology, 58(8), 1082-1099. doi:10.1002/asi.20601

Mount Royal University. (n.d.a). Fast facts. Retrieved 5 Dec. 2012 from http://mtroyal.ca/AboutMountRoyal/FastFacts/index.htm

Mount Royal University. (n.d.b). Inspiring learning: Mount Royal University academic plan 2012-2017. Retrieved 5 Dec. 2012 from http://www.mtroyal.ca/wcm/groups/public/documents/pdf/academic_plan.pdf

Novotny, E. (2004). I don’t think I click: A protocol analysis study of use of a library online catalog in the Internet age. College & Research Libraries, 65(6), 525-537.

Portmann, C. A., & Roush, A. J. (2004). Assessing the effects of library instruction. The Journal of Academic Librarianship, 30(6), 461-465. doi:10.1016/j.acalib.2004.07.004

Smith, T. G. (2001). Keeping track: Librarians, composition instructors, and student writers use the research journal. Research Strategies, 18, 21-28.

Tabatabai, D., & Shore, B. M. (2005). How experts and novices search the Web. Library & Information Science Research, 27(2), 222-248. doi:10.1016/j.lisr.2005.01.005

Urquhart, C., & Rowley, J. (2007). Understanding student information behavior in relation to electronic information services: Lessons from longitudinal monitoring and evaluation, part 2. Journal of the American Society for Information Science & Technology, 58(8), 1188-1197. doi:10.1002/asi.20562

Warner, D. A. (2003). Programmatic assessment: Turning process into practice by teaching for learning. The Journal of Academic Librarianship, 29(3), 169-176.

Wakimoto, D. (2010). Information literacy instruction assessment and improvement through evidence based practice: A mixed method study. Evidence Based Library and Information Practice, 5(1), 82-92. Retrieved 4 Dec. 2012 from http://ejournals.library.ualberta.ca/index.php/EBLIP/article/view/6456/6447

Wang, P., Berry, M.W., &Yang, Y. (2003). Mining longitudinal Web queries: Trends and patterns. Journal of the American Society for Information Science and Technology, 54(8), 743-758. doi:10.1002/asi.10262

Warner, D. A. (2003). Programmatic assessment: Turning process into practice by teaching for learning. The Journal of Academic Librarianship, 29(3), 169-176.

Webster, J., & Rielly, L. (2003). A library instruction case study: Measuring success from multiple perspectives. Research Strategies, 19, 16-32. doi:10.1016/j.resstr.2003.11.001

Willson, R., & Given, L. M. (2010). The effect of spelling and retrieval system familiarity on search behavior in online public access catalogs: A mixed methods study. Journal of the American Society for Information Science & Technology, 61(12), 2461-2476. doi:10.1002/asi.21433

Today’s Date	Resource Used (e.g. Academic Search Complete)	Keywords/Search (e.g. “global warming” and ocean* in keywords)	Resources Found (e.g. Good results, e-mailed Smith & Jones article)	Notes to Self (e.g. try synonyms for global warming)