Research Article
Assessing the Impact of an
Information Literacy Course on Students' Academic Achievement: A Mixed-Methods
Study
Wilma L. Jones
Professor and
Reference/Instruction Librarian
College of Staten Island
City University of New York
Staten Island, New York,
United States of America
Email: wilma.jones@csi.cuny.edu
Tara Mastrorilli
Director of Institutional
Research
College of Staten Island
City University of New York
Staten Island, New York,
United States of America
Email: tara.mastrorilli@csi.cuny.edu
Received: 14 Dec. 2021 Accepted: 14 Mar. 2022
2022 Jones and Mastrorilli. This
is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0
International (http://creativecommons.org/licenses/by-nc-sa/4.0/),
which permits unrestricted use, distribution, and reproduction in any medium,
provided the original work is properly attributed, not used for commercial
purposes, and, if transformed, the resulting work is redistributed under the
same or similar license to this one.
DOI: 10.18438/eblip30090
Abstract
Objective
–
The aim of this study is to demonstrate the impact of a stand-alone,
credit-bearing information literacy course on retention and GPA for students at
an open access urban college.
Methods
–
Researchers conducted a mixed-methods study with a two-part focus. The first
examined the impact of a credit-bearing course using propensity score matching
(PSM) techniques to compare academic outcomes for students who participated in
the course versus outcomes for similar students who did not enroll in the
course. Multiple regression analyses were conducted to measure impact on GPA
and performance in 100-level introductory English general education courses.
Logistic regression analysis was used to determine persistence one year after
enrolling in the course. The second part utilized a questionnaire to survey
students of this targeted group to determine impact of the course on their
information-seeking behaviour in subsequent academic courses and for non-academic
purposes.
Results
–
The quantitative analyses showed: (a) a higher GPA, though slight, for students
who have taken the course over the matched comparison group; (b) an increase in
persistence for students who have taken the course over the matched comparison
group after one year of taking the course; but (c) lower performance in
100-level introductory English courses by students who have taken the course in
contrast to the matched comparison group. Qualitative data provided through the
questionnaire revealed positive and substantive reflective statements that
support learning outcomes of the course.
Conclusion
–
The findings in this study underscored the importance of a stand-alone,
credit-bearing information literacy course for undergraduate students,
particularly for first-generation students attending an open access urban
institution. The findings also demonstrate the academic library’s contribution
to institutional retention efforts in support of students’ academic success.
Introduction
For the past two
decades, academic librarians have had to ensure that students, most of whom are
digital natives, acquire information literacy (IL) skills to be able to find
and critically sift through the appropriate materials for their research.
Moreover, the tenets of IL enable college students to distinguish between
scholarly, non-scholarly, or misinformation resources (e.g., fake news,
alternate facts). Academic librarians in the U.S. and around the world have
continued to revamp and develop innovative methods for delivering IL sessions
using current and emerging technologies (Burkhardt, 2016; Diep & Nahl, 2011; Inuwa & Abrizah, 2018; Mackey & Jacobson, 2010; McDevitt, 2011;
Polger & Sheidlower,
2017; Swanson & Jagman, 2015). Instruction
librarians at the College of Staten Island (CSI), an open access urban public
institution, have also experienced similar challenges in teaching students how
to access and retrieve the most appropriate information for their assignments
and research papers, and assessing how students have applied the knowledge
gained from library instruction sessions. The first-year students entering our
open access institution are primarily first-generation college students, many
of whom need to learn basic library skills to find research resources and how
to critically evaluate sources to use in their term papers or assignments. We,
librarians, continue to contemplate, grapple with, and develop effective
methods of instruction to engage our unique population of students in providing
them with the means to conduct academic-level research. These endeavors by academic libraries are rarely counted
as contributions towards the collective institutional retention efforts unless
quantitative evidence is provided.
Among the new
modes of instruction that CSI librarians have explored and implemented are:
one-shot sessions tailored to faculty requests, 50-minute workshops for co-curricular credits specifically designed for
first-year students, online tutorials with
interactive videos and mini quizzes attached, and course-embedded library instruction. However, none of these new
modes of instruction have been more satisfying to students than the one-credit
IL course, LIB102: Beyond Google: Research for College Success, which was
developed in 2014. It became apparent to us that long-term instruction, rather
than short-term instruction, provided our students with multiple opportunities
to become familiar, in depth, with library resources and IL concepts.
These multiple interactions enabled students to see how we, as librarians,
could really be more of a resource to them. Assessment of this course, after
one year of instruction, was done using a 5-item, post-evaluation questionnaire
in every section taught, and it revealed only positive comments from students.
Moreover, feedback from students who self-identified as having 60+ credits had
two common comments in the open-ended section: (i) they wished they had taken
this course when they were first-year students, and (ii) they suggested that
this course be made mandatory for all first-year students. The above feedback
and many other insightful comments were found to be a rewarding source that
validated the library’s worth in terms of contributing positively to students’
experiences at CSI.
These
preliminary observations were the impetus for this study, and the authors of
this study set out to obtain quantitative and qualitative evidence that
objectively supported the anecdotal feedback. The authors wanted to find out
whether there was a quantifiable association in persistence of students who had
enrolled in the stand-alone IL course. A secondary focus sought to examine the
information-seeking behaviour of these students to see whether knowledge gained
from LIB102: Beyond Google was ever utilized in other classes or for
non-academic purposes. While the library offers many opportunities for student
success, attempts to measure its impact on student learning, one that
ultimately contributes to institutional goals, are lacking in the library
literature as noted by Oakleaf (2015), Oliveira (2018), and Pierard
and Graves (2007). As such, the result of this study adds to the small but
growing literature of the academic library’s impact on student academic success
and institutional retention efforts.
Background about the LIB102 Course
The creation of
a one-credit course, LIB102: Beyond Google: Research for College Success (which
from here on will be referred to as LIB102), emerged as an opportunity to
develop a course to address critical thinking and IL skills for first-year
students. The course aimed to introduce students to the concept of strategic
exploration of a research topic, in particular learning how to match
information needs to appropriate sources. Course goals were to teach students
search techniques to efficiently retrieve reliable sources. The course
discussed research-based vs. fake news, citation styles, and how to avoid
plagiarism. In its design, LIB102 provided valuable research techniques to
enable students to move beyond the incessant use of Google to complete their assignments
and to become efficient in utilizing a variety of library research resources as
required by their instructors (e.g., peer-reviewed journals, reputable
newspapers, as well as some verifiable digital sources). Course topics included
distinguishing types of publications within the online discovery tool, basic
and advanced searching skills (i.e., Boolean operators, truncation, wildcards),
as well as effective Internet search strategies. Embedded in the course design
were online interactive tutorials, group work activities, discussion forums, a
pre- and post-test, and weekly self-assessment quizzes to evaluate learning
throughout the course.
Learning
outcomes of the course enabled students to have hands-on experiences in a
computer lab while learning about identifying, retrieving, and evaluating the
appropriate research materials, as prescribed at the time by the Information Literacy Competency Standards
for Higher Education (Association of College and Research Libraries [ACRL],
2000). These were fundamental skills that could be applied to any research
assignment in other classes. However, the Framework
for Information Literacy for Higher Education (ACRL, 2016) replaced the Standards, which was timely for us as we
were revising the final design of the course for adoption into the college
catalog. The learning outcomes for LIB102 were slightly modified to embrace the
threshold concepts of the new standards, the Framework, and included weekly assessment of the continuous process
of research. In addition to identifying, retrieving, and evaluating materials,
learning outcomes enabled students to develop and refine an academic research
topic or question, understand the concept of authority, and be able to think
critically about information and communication media. LIB102 was designed as a
seven-and-a-half week course, and fittingly, the end of the first
seven-and-a-half week course coincided with mid-term assessments, and the
second session coincided with final examinations.
This design, we
envisioned, would complement the gateway writing courses of ENG111 and ENG151
to enable first-year students who were enrolled in them to prepare for first
outlines/drafts of research papers due at midterms or to complete their
research papers due at finals. ENG111 and ENG151 are required courses in the
general education curriculum and are named Introduction to College Writing and
College Writing, respectively. In the undergraduate catalog, ENG151 includes in
its description “attention to reading, library skills, and research methods.”
LIB102 strives to reinforce all three items, including swapping out “attention
to reading” for “critical thinking.”
Institutional Setting
The CSI Library
is situated in an urban comprehensive college, one of 25 colleges that make up
the open access City University of New York. At present, CSI has an enrollment
of close to 13,000 students primarily consisting of first-time,
first-generation college students. Located on Staten Island, the college
carries the distinctive honor of being the only public institution among four
other institutions of higher learning in the borough. Seventy percent of
current students at CSI are residents of Staten Island (CSI, Office of
Institutional Research, 2020), which has a population of 474,893 (U.S. Census
Bureau, 2019). Since the establishment of a residential hall in 2014, the
diversity of the student body has broadened. In Fall 2020, data from the Office
of Institutional Research presented the profile of the students as: 0.1%
American Indian or Alaskan Native, 11% Asian, 13.5% Black or African American,
25% Hispanic, 45.3% White, and 5% Other. In 2020, first-time, first-generation
students were 47% of the student body. Also, in 2019, the college became a
Hispanic-Serving Institution (HSI) when self-reported data from Hispanic
students reached a minimum of 25%, full-time equivalent.
Consistent with
the mission of the college, the CSI Library strives to play its role in
assisting students, who are primarily first-generation students, in becoming
information competent, critical thinkers, and life-long learners through
traditional and virtual IL sessions.
Literature Review
Over the past 20
years, there has been a pointed focus on the academic library’s impact on
students’ academic success and retention as a result of accrediting bodies of
higher education institutions seeking such evidence, i.e., evidence
demonstrating the academic library’s contribution towards institutional
priorities of learning outcomes (Britto & Kinsley, 2018; Oakleaf, 2010). IL
instruction that is embedded in other courses have garnered much success, yet
only few studies in the library literature have captured empirical evidence of
its contribution towards student’s academic performance and success (Black
& Murphy, 2017; Hauck, 2017; Inuwa & Abrizah, 2018; Luetkenhaus et
al., 2015; Tumbleson et al., 2019). There are several
publications describing the positive relationship between students’ use of
library resources and services and their impact on higher GPA and retention
(Bowles-Terry, 2012; Cook, 2014; Haddow 2013; Pierard
& Graves, 2007; Rysavy et al., 2017; Soria et
al., 2013; Soria et al., 2014); however, the following are the very few that
discuss the impact of a credit-bearing course on students’ academic achievement
and retention.
Since our study
proposed the utilization of propensity score matching (PSM) techniques to
compare outcomes of students who had taken an IL course versus those who had
not, we searched for studies that had used PSM. We found two studies, one by
Soria et al. (2017) and another by Chiteng Kot and Jones (2015), that had utilized PSM to examine
outcomes of library resources and services as it related to student achievement
and retention. Soria et al. (2017) explored “the impact of library resources on
college students’ longitudinal outcomes as related to their graduation or
continued enrollment after four years of study” (p. 813). They used
quasi-experimental procedures (PSM techniques) to construct control
(non-library users) and treatment (library users) groups, they found that
“using the library at least one time in the first year of enrollment
significantly increased the odds that students would graduate in four years or
remain enrolled after four years as opposed to withdrawing from the
University'' (p. 819). Chiteng Kot
and Jones (2015) also used PSM to conduct a study on three cohorts of
first-time, full-time undergraduate students at a large, metropolitan, public
research university that provided evidence that the library had an impact on
academic performance. Their study revealed that first-term GPA, with regard to
students’ use of one of three library resources (workstations, study rooms, and
research clinics), was associated with a higher first-term GPA, though small.
Only a handful
of studies have reported on the contributions of a stand-alone IL course and
its impact on students’ academic performance that we found relevant to this
study. The study conducted by Daugherty and Russo (2011) found that “students
who took a library course understood the usefulness of what they had learned as
evidenced by their continued use of both the skills and resources taught in the
course” (p. 321). Heady et al. (2018) conducted a 20-year longitudinal study
examining records of students who had taken an IL course to determine the
impact of the course over time. Results of their study were mixed as
first-time, full-time students “who did not take the library course had higher
graduation rates” (p. 646), versus slightly higher GPA—“roughly
0.1 point”—for those who took the course (p. 647). The study by Mayer and
Bowles-Terry (2013) explored how students performed in an upper-level IL course
and found them to be more engaged when there was a class theme. The study by Lwoga (2014), though focused on the impact of an IL course,
was centered around undergraduate students in a Health Sciences program. Lwoga reported that not only did students continue “to use
online searching techniques for their academic assignments eleven months after
training” (p. 10), they utilized these search skills in their personal lives,
and “the IL module plays an important role in supporting lifelong learning” (p.
11).
Our study is
unique in the following two ways: 1) it compares retention and GPA outcomes of
students who have taken a library course with those who have not, utilizing
propensity score matching techniques; and 2) it explores how students have
applied the skills learned in the information literacy course to other research
activities, academic and non-academic, via a qualitative survey.
Aim
This study sought to obtain quantitative and
qualitative evidence to demonstrate the impact of the LIB102 course on
undergraduate student performance throughout their time in college. In
particular, the authors examined how this course contributes to student
academic success or retention, in support of institutional goals. The following
questions guided this study:
·
How does the cumulative grade point
average (GPA) of students who took LIB102 compare to those in a matched
comparison group?
·
How do students who took LIB102 perform
in mandatory 100-level English general education courses (i.e., ENG111 and
ENG151) compared to those in a matched comparison group?
·
How do one-year persistence rates of
students who took LIB102 compare to students in a matched comparison group?
·
How have students who took LIB102
applied the knowledge gained in subsequent academic courses or outside the
classroom, and how does this compare to a matched comparison group?
Methods
To determine the
impact of the LIB102 course, it was important to compare outcomes for students
who participated in the course to outcomes for students who did not enroll in
the course. Therefore, a match comparison group was devised using propensity
score matching (PSM). Initially introduced by Rosenbaum and Rubin (1983), PSM
is a statistical method used to match treated and control subjects using a set
of baseline characteristics. For our study, our treated subjects were students
who enrolled in LIB102. The control subjects were those who did not and will be
referred to as the “matched comparison group” from here on. Baseline
characteristics used for matching included: gender, race/ethnicity, Tuition Assistance
Program (TAP) or Pell Grant status, degree level, class level, age, and college
admissions average (CAA), i.e. CUNY’s equivalence for high school average.
The PSM routine
was run separately for 10 cohorts of students, from Fall 2014 to Spring 2019.
Descriptive statistics were calculated, including frequencies, means, and
standard deviations on the demographic, enrollment, and survey data. Multiple
regression analyses were conducted to measure the impact of the LIB102 course
on student outcomes, specifically cumulative grade point average (GPA) and
performance in introductory 100-level English gateway courses, ENG111 and
ENG151. Logistic regression measured the impact of the course on one-year
persistence. To reduce threats to the study’s internal validity, potential
confounding factors or covariates that have a relationship with the program or
outcome are included in each regression model. Covariates included gender,
race/ethnicity, TAP or Pell Grant status, age, degree sought, class status,
CAA, and cumulative GPA. Continuous covariates were grade mean centered.
A questionnaire
assessed learning outcomes from LIB102 as well as knowledge applied from LIB102
to subsequent academic courses or outside the classroom. The survey instrument
was adapted from two similar studies on information seeking behaviours of
undergraduate students (Daugherty & Russo, 2011; Lwoga,
2014). It consisted of 23 questions, including four open-ended questions to
allow for qualitative reflective statements. The first section asked
participants to provide basic demographic information such as class level,
major, and other background information (nine questions). The second section
consisted of six questions that examined how students applied IL skills in
classes other than LIB102. The third section included five questions that
sought to find out whether participants used IL skills outside of the academic
environment, with questions that spoke to applying lifelong learning skills.
The last section included three general questions: two open-ended questions
about the course, and the third, which was optional, asked participants to
provide contact information to enter a drawing for one of eight $50 Amazon.com
gift cards. The questionnaire is included in Appendix A.
Data Collection and Instrumentation
In compliance
with CSI’s Institutional Review Board (IRB) protocol, and in collaboration with
the Office of Institutional Research, the researchers accessed transcripts of
students who had participated in LIB102 and an equivalent number who had not
participated. Transcript information included student demographic and academic
performance data as well as student email addresses. Information on students’
prior achievement, as measured by their CAA was also retrieved. These data were
used as a resource for the PSM criteria as well as the outcome data around
performance. The academic performance data used for outcome analyses included
students’ cumulative GPA at the end of the semester in which the student took
LIB102 and 100-level English grades (if taken during or after LIB102).
Retention and graduation data were retrieved one year after participating in
the LIB102 course. The retention and graduation data were combined to calculate
a persistence rate.
The survey was administered using SurveyMonkey, an online tool, to survey
students enrolled in Fall 2019. The survey was emailed to those who had
participated in LIB102 and those targeted in the matched comparison group.
Interested participants were then directed to a web link where they anonymously
filled out the survey. It remained open for six weeks and reminders were sent
out four times after the initial deployment in an effort to receive a 15% confidence
level of response.
Demographics of Students Who Have Taken LIB102: Beyond Google
Across 10
semesters, Fall 2014 through Spring 2019, LIB102 had
enrolled a total of 675 students. More than half (59%) of the students were
female, 56% were students of color, and the average age of participating
students was 21. Although the course was designed for first-year students, 31%
of those enrolled were juniors or seniors. With a cap of 20 students per
section, on average, 14.4 students persisted in each section. At the time of
enrollment into the study, 57% of the participating students were pursuing a
bachelor’s degree and the remaining students were pursuing an associate’s
degree (43%). The average CAA score for this group of 675 students was 79.0.
Multiple
regression analyses were conducted using SPSS to measure the impact of the
LIB102 course on student outcomes, specifically cumulative GPA and performance
in ENG111 and ENG151. Table 1 presents the outcomes of students who took
LIB102. Of the 675 students who took the course, 147 of them who were enrolled
in ENG111 (Introduction to College Writing) at the same time or after taking
LIB102 were found to have a mean GPA of 2.63 (1.30 sd).
Table 1
LIB102: Beyond
Google Course Student Outcomes
|
|
N |
Value |
|
Mean
ENG111 GPA (SD) |
147 |
2.63 (1.30) |
|
Mean
ENG151 GPA (SD) |
162 |
2.84 (1.22) |
|
Overall
Mean GPA (SD) |
675 |
2.63 (0.83) |
|
One-Year
Retention (%) |
675 |
53.9 |
A slightly
higher mean GPA of 2.84 (1.22 sd) was found in 162 students who were taking ENG151
(College Writing) while being enrolled or after enrolling in LIB102. The
overall mean GPA of all 675 students who took LIB102 was 2.63 (0.83 sd). Logistic
regression was used to measure the impact of LIB102 on the retention of
students one year after enrolling in the course. Slightly more than half (54%)
of students who took LIB102 were retained or graduated after one year of taking
the course. The most recent CSI first-time, full-time freshmen retention rate
is 59.5%.
The Matched Comparison Sample
Students in the
LIB102 course from Fall 2014 semester to Spring 2019 were matched to the
students who did not participate in the course using PSM. Separate PSMs were
conducted for each LIB102 cohort. Table 2 follows with the demographic profile
of the analysis sample of 458 students in each sample group used for this
study. The difference between LIB102 enrollment and the LIB102 analysis sample
is the result of missing data in the matching criteria used as well as the
attempt to produce highly accurate matches in the PSM model. To determine if
the students in the match comparison groups are similar to students in the
LIB102 group, statistical significance testing was conducted by cohort of the
analysis sample.
Table 2
Demographic
Profile of Overall Analysis Sample for the Study by Group
|
|
LIB102 Analysis Sample (N=458) |
Matched Comparison Group (N=458) |
|
Female
(%) |
62.0 |
62.7 |
|
American
Indian or Alaska Native (%) |
0.0 |
0.0 |
|
Asian
or Pacific Islander (%) |
13.8 |
13.8 |
|
Black
(%) |
23.4 |
17.2 |
|
Hispanic
(%) |
19.4 |
19.4 |
|
White
(%) |
43.4 |
49.1 |
|
Received
TAP or Pell (%) |
58.3 |
59.0 |
|
Associate
(%) |
44.1 |
37.8 |
|
Bachelor
(%) |
55.9 |
58.5 |
|
Freshmen
(%) |
48.7 |
30.8 |
|
Sophomore
(%) |
24.2 |
31.0 |
|
Junior
(%) |
8.3 |
22.5 |
|
Senior
(%) |
18.8 |
15.7 |
|
Mean
Age |
20.8 |
24.9 |
|
Mean
College Admissions Average |
78.7 |
77.7 |
Table 3 presents
the outcomes of the overall analysis sample by group. Of the 458 students in
the analysis sample, only 105 of them were enrolled in ENG111 during or after
taking LIB102 and they were found to have a mean GPA of 2.62. Similarly, 67
students in the matched comparison group who had enrolled in ENG111 were found
to have a mean GPA of 2.57.
Table 3
Student Outcomes
of Overall Analysis Sample by Group
|
|
LIB102 Analysis Sample |
Matched Comparison Group |
||
|
|
N |
Value |
N |
Value |
|
Mean
ENG111 GPA (SD) |
105 |
2.62 (1.26) |
67 |
2.57 (1.38) |
|
Mean
ENG151 GPA (SD) |
124 |
2.85 (1.23) |
89 |
2.99 (0.96) |
|
Mean
GPA (SD) |
458 |
2.59 (0.83) |
458 |
2.63 (0.90) |
|
One-Year
Persistence (%) |
458 |
59.2 |
458 |
53.1 |
Of the 458
students in the analysis sample, there were only 124 students who had taken
ENG151 during or after enrolling in LIB102 and they had a mean GPA of 2.85.
Likewise, 89 students in the matched comparison group were found to have a mean
GPA score of 2.99. Overall, the LIB102 analysis sample had a mean GPA score of
2.59 versus 2.63 for the matched comparison group. The one-year persistence
rate for the LIB102 analysis sample was 59.2% versus the 53.1% for the matched
comparison group.
Although the PSM
resulted in highly similar matched comparison groups, there was no way to
identify whether students in the matched comparison groups received instruction
similar to that provided in the LIB102 course. As a result, there may be
students in the matched comparison groups that received similar instruction
(e.g., one-shot library instruction sessions in a 100-level psychology course,
or similar introductory instruction when students attended another college). We
are cautious that this could have potentially influenced the outcome results.
Demographics of Survey Participants
The survey was sent to 212 students who were in the
LIB102 analysis sample and 143 students in the match comparison group. Table 4
displays the results of students who participated in the survey. Of the LIB102
students who received the survey, 20.2% responded, and 12.6% of the match
comparison group students responded. Given these low response rates, we were
cautious when analyzing and interpreting the results.
The majority of respondents in the LIB102 group were
juniors and seniors, 60.5% of them having earned 61–124+ credits, and the
remainder was almost an even split with 20.9% having 60 or less credits, and
18.6% were graduate students. Likewise, the majority of respondents in the
matched comparison group were seniors and graduate students, 61.1% of them
having earned 91–124+ credits, while 27.8% had 61–90 credits, and 11.2% had 60
credits or less. Of the respondents in the LIB102 group, 97.6% were
non-transfer students vs. 44.4% in the matched comparison group. Of the LIB102
group, one-quarter of the respondents were male (23.8%), unlike the matched
comparison group where one-third were male (33.3%). With regard to ethnicity,
the two groups were very different; only 14.3% of the LIB102 group were White,
whereas more than half (55%) of the respondents in the matched comparison group
were White. Of note, 50% of the respondents in the LIB102 group checked the box
for being the “first in their family to attend college” versus 33.3% of the
matched comparison group. In both groups, Social Sciences majors held the
highest percentage—65.2% in the LIB102 group and 39.0% in the matched
comparison group. Lastly, 38.9% of the LIB102 students were also part of a
learning community program (i.e., Accelerated Studies in Associate Programs
(ASAP), Black Male Initiative (BMI), and the Perry Ellis Sutton SEEK Program
(SEEK)), whereas only 5.6% of the matched comparison group reported that they
had been part of a learning community.
Table 4
Profile of
Respondents Who Participated in the Survey
|
|
LIB102 Students N=42 |
Matched Comparison Group N=18 |
|
Female |
32 (76.2%) |
12 (66.7%) |
|
|
|
|
|
American
Indian or Alaska Native |
0 |
0 |
|
Asian
or Asian American |
9 (21.4%) |
1 (5.6%) |
|
Black
or African American |
9 (21.4%) |
3 (16.7%) |
|
Hispanic
or Latino |
13 (31.0%) |
1 (5.6%) |
|
White
or Caucasian |
6 (14.3%) |
10 (55.6%) |
|
Two or
More Races |
3 (7.1%) |
0 |
|
Prefer
not to disclose |
2 (4.8%) |
3 (16.7%) |
|
|
|
|
|
0–30
credits |
1 (2.3%) |
1 (5.6%) |
|
31–60
credits |
8 (18.6%) |
1 (5.6%) |
|
61–90
credits |
14 (32.6%) |
5 (27.8%) |
|
91–124+
credits |
12 (27.9%) |
9 (50.0%) |
|
Graduate
student |
8 (18.6%) |
2 (11.1%) |
|
|
|
|
|
Non-transfer
student |
41 (97.6%) |
8 (44.4%) |
|
|
|
|
|
Major: |
|
|
|
Humanities |
10 (21.8%) |
6 (33.3%) |
|
Science & Technology |
4 (8.7%) |
5 (27.8%) |
|
Social Sciences |
30 (65.2%) |
7 (38.9%) |
|
Undeclared |
2 (4.3%) |
0 |
|
|
|
|
|
First
generation to attend college |
21 (50.0%) |
6 (33.3%) |
|
|
|
|
|
Students
in a learning community |
16 (38.9%) |
1 (5.6%) |
Results
In this section,
researchers first report on the findings from the multiple regression and
logistic regression analyses to show evidence of the impact of LIB102 on
students’ academic achievements. Following this, researchers report on the
survey results from the same targeted group of students who were currently
enrolled at the time of this study.
Impact Findings Using PSM
Multiple
regression analyses were conducted using SPSS to measure the impact of the
LIB102 course on students’ cumulative grade point average (GPA) and performance
in 100-level English courses, after controlling for student demographics (i.e.,
gender, race/ethnicity, TAP or Pell status, age, degree sought, and class
level) and previous achievement (i.e., college admissions average). This was
done in pursuit of guiding question #1: How
does the cumulative GPA of students who took LIB102 compare to those in a
matched comparison group? Table 5 displays the results of the multiple
regression analysis on 916 students’ cumulative GPA. LIB102 students’
cumulative GPA represents their GPA at the end of the semester in which they
took the LIB102 course.
Table 5
Cumulative Grade
Point Average Multiple Regression Results (N=916)
|
|
B |
B SE |
|
Constant |
2.67** |
0.07 |
|
Female |
0.25** |
0.05 |
|
Blacka |
-0.19* |
0.07 |
|
Hispanica |
-0.21* |
0.07 |
|
Received
TAP or Pell |
0.09 |
0.05 |
|
Associateb |
-0.38** |
0.07 |
|
Freshmanc |
-0.15 |
0.08 |
|
Sophomorec |
-0.28 |
0.07 |
|
Age |
0.02** |
0.01 |
|
College
Admissions Average |
0.02** |
0.00 |
|
Group |
0.08 |
0.06 |
|
R2 |
0.21 |
|
|
F |
23.6** |
|
a The comparison group
included White or Asian.
b The comparison group included bachelor or
other degree-seeking students.
c The comparison group included juniors or
seniors.
* p < .05, ** p < .001.
The GPA for the matched comparison group represents
their GPA in the same semester of their matched LIB102 student. The results
indicate that, after controlling for differences in demographics and previous
achievement, the students who took the LIB102 course had a 0.08 higher
cumulative GPA than the matched comparison group. This difference, however, was
not statistically significant. Factors that did have a significant relationship
with cumulative GPA include: gender, race/ethnicity, degree level, age, and
CAA.
Table 6 displays the results of the multiple
regression analysis on 172 students’ performance in their ENG111 course and 213
students’ performance in their ENG151 course. We conducted this multiple
regression analysis in pursuit of guiding question #2: How do students who took LIB102 perform in mandatory 100-level English
general education courses (i.e., ENG111 and ENG151) as compared to those in a
matched comparison group? After controlling for differences in demographics
and previous achievement, the results indicate that the students who took the
LIB102 course had a 0.22 lower grade in ENG111 than the matched comparison
group and a 0.21 lower grade in ENG151 than the matched comparison group. These
differences, however, were not statistically significant. Factors that did have
a significant relationship with ENG111 performance include: gender,
race/ethnicity, and degree level. The CAA is the only factor that had a
significant relationship with ENG151 performance.
Table 6
Multiple
Regression Results for ENG111 and ENG151
|
|
ENG111 (N=172) |
ENG151 (N=213) |
||
|
|
B |
B SE |
B |
B SE |
|
Constant |
3.29* |
1.30 |
2.65** |
0.40 |
|
Female |
0.45* |
0.20 |
0.25 |
0.16 |
|
Blacka |
-0.06 |
0.23 |
-0.20 |
0.19 |
|
Hispanica |
-.071* |
0.25 |
-0.29 |
0.20 |
|
Received
TAP or Pell |
0.07 |
0.20 |
-0.11 |
0.16 |
|
Associateb |
-0.71* |
0.29 |
-0.16 |
0.19 |
|
Freshmanc |
-0.08 |
1.31 |
0.51 |
0.43 |
|
Sophomorec |
0.05 |
1.38 |
0.77 |
0.45 |
|
Age |
-0.01 |
0.02 |
0.00 |
0.02 |
|
College
Admissions Average |
0.01 |
0.01 |
0.02* |
0.00 |
|
Group |
-0.22 |
0.26 |
-0.21 |
0.21 |
|
R2 |
0.13 |
|
0.09 |
|
|
F |
2.46* |
|
1.87 |
|
a The comparison group
included White or Asian.
b The comparison group included bachelor or
other degree-seeking students.
c The comparison group included juniors or
seniors.
* p < .05, ** p < .001.
Table 7 displays the results of the logistic
regression analysis on 916 students’ performance in their one-year persistence
after enrollment in the LIB102 course semester, i.e., 458 from the LIB102 group
and 458 from the matched comparison group. We conducted this logistic
regression analysis in pursuit of guiding question #3: How do one-year persistence rates of students who took LIB102 compare
to students in a matched comparison group? The results indicate, after
controlling for differences in demographics and previous achievement, that the
students who took the LIB102 course were retained at higher rates (0.23) than
the matched comparison students.
When comparing students who took LIB102 to the matched
comparison group, LIB102 students were 1.26 times more likely to return to CSI
or graduate the next year. The difference, however, was not statistically
significant. Factors that did have a significant relationship on retention
include: gender, TAP or Pell status, class level, and CAA.
Table 7
One-Year Persistence Logistic Regression Results
(N=916)
|
|
B |
B SE |
Odds Ratio |
|
Constant |
0.11 |
0.18 |
1.12 |
|
Female |
0.29* |
0.14 |
1.34 |
|
Blacka |
-0.19 |
0.18 |
0.82 |
|
Hispanica |
-0.17 |
0.18 |
0.84 |
|
Received TAP or Pell |
0.44* |
0.14 |
1.55 |
|
Associateb |
-0.23 |
0.18 |
0.79 |
|
Freshmanc |
-0.42* |
0.22 |
0.66 |
|
Sophomorec |
-0.13 |
0.19 |
0.88 |
|
Age |
-0.01 |
0.02 |
0.99 |
|
College Admissions Average |
0.02* |
0.01 |
1.03 |
|
Group |
0.23 |
0.15 |
1.26 |
|
χ2 |
50.4** |
|
|
a The comparison group included White or Asian.
b The comparison group included bachelor or other
degree-seeking students.
c The comparison group included juniors or seniors.
* p <
.05, ** p < .001.
Survey Findings
Survey results
provided rich data that support knowledge gained in LIB102 and the application
of that knowledge in other academic courses or outside the classroom.
Application of IL Skills and Techniques in Courses Other Than LIB102
To explore how students have applied LIB102 skills in
other academic courses, the survey included questions that prompted the
following results (Table 8). Six databases rose to the top as the most commonly
used for academic purposes: OneSearch (21.2%), Academic Search Complete
(14.4%), The New York Times (13.6%),
JSTOR (10.6%), The Wall Street Journal
(10.6%), and Gale Virtual Reference Library (9.1%). Of the matched comparison
group, the most commonly used databases were the same, except in different
order and with a lower percentage of utilization: OneSearch (14.6%), The New York Times (14.6%), The Wall Street Journal (12.5%), Gale
Virtual Reference Library (10.4%), JSTOR (10.4%), and Academic Search Complete
(8.3%). It was rewarding to see that two of three databases regularly taught in
the course, OneSearch and Academic Search Complete, were heavily used by the LIB102 group to do research in other
academic classes. Given that over 78% of respondents in the control and
treatment group were juniors, seniors, and graduate students, we presumed that
the high use of The New York Times
and The Wall Street Journal was due
to the fact that they would have come across these resources in other one-shot
instructional sessions or encountered them by virtue of residing in the state
of New York, where both newspapers are published and widely available. Both
newspapers are also freely available via an app or online to students, staff,
and faculty, a welcome arrangement secured by our parent university.
Table 8
Use of Databases
for Academic Purposes
|
|
LIB102 Group |
Matched Comparison Group |
|
Most Used |
OneSearch
– 21.2% Academic
Search Complete – 14.4% The New York Times –
13.6% JSTOR –
10.6% The Wall Street Journal – 10.6% Gale
Virtual Reference Library – 9.1% |
OneSearch
– 14.6% The New York Times –
14.6% The Wall Street Journal – 12.5% Gale
Virtual Reference Library – 10.4% JSTOR –
10.4% Academic
Search Complete – 8.3% |
|
Least Used |
PsycINFO
– 6.8% Ebook Central – 3.8% MEDLINE
– 2.3% Kanopy – 1.5% Nexis
Uni – 1.5% CINAHL
– 0.8% Opposing
Viewpoints – 0.8% |
Ebook Central – 6.3% Kanopy – 6.3% CINAHL
– 4.2% MEDLINE
– 4.2% PsycINFO
– 4.2% Nexis
Uni – 2.1% Opposing
Viewpoints – 2.1% |
Of the 32
responses given by the LIB102 group as to whether they encountered any
difficulty using databases, 75% indicated no problems or difficulty. Difficulties that were expressed were
user–interface related, e.g., “some websites not user-friendly,” and “problems
specifying or narrowing search results.” However, of the 12 responses given by
the matched comparison group, seven (58%) were trouble-free interactions and
the remaining five responses were mainly about technical difficulties including
“unable to log in,” “problems logging on from off campus,” and “PDF not
available.”
Newspaper
articles, peer-reviewed articles from journals, and research reports topped the
list for both groups at 20%, respectively, when asked about types of materials
they were seeking. Magazine articles and statistical information ranked higher
for the matched comparison group at 14% and 10%, respectively, whereas the
LIB102 group sought these information sources at 10% and 9%, respectively. Of
note, there was one recorded response for a podcast search by a respondent in
the LIB102 group.
With regard to
skills used when searching library databases, keyword searching was the most
popular among both groups (21.2% in the LIB102 group vs. 26.5% in the matched
comparison group), followed by subject searching (19.2% vs. 22.4%; see Appendix
B). Regarding advanced searching, 16.4% of the LIB102 group indicated that they
used it, as did 10.2% of the matched comparison group respondents. Though
keyword searching remained the number one skill used by both groups, it was
heartening to see the high use of the advanced searching option by the LIB102 group,
which would indicate a good understanding of using it to yield the best
results. Similar proportions (between 8% and 12%) indicated having used Google
search operators, phrase searching, and use of synonyms in both groups. Nine
respondents in the LIB102 group selected Boolean operators and browsing the
index (corresponding to 6% of responses), contrary to our expectations of more
use of these skills. Not surprisingly, only 2 respondents (4%) in the matched
comparison group indicated that they had used Boolean operators and browsing
the index. Remarkably, in the LIB102 group, there were two responses for
truncation or wildcard, whereas there were no selections for these by the
matched comparison group.
To assess
whether students had any difficulty using search terms when utilizing databases
for their assignments, respondents were given the opportunity to indicate this
in an open-ended question. The majority of responses in both groups signaled
that they had no difficulty (73.3% of the LIB102 group and 63.6% of the matched
group). Sample statements of difficulty expressed by the LIB102 group include:
“Getting the right word can be hard to find the perfect source. It’s time
consuming,” “I wasn’t sure when to use the key term,” and “Using the right
keywords to get the most accurate results.”
Similarly, sample statements about difficulty using databases from the
matched comparison group were mainly expressed as follows: “Could not access Kanopy – a requirement for my French class,” “Google gives
more exact matches better,” “Not finding what I’m looking for,” and “Not
knowing about other techniques.”
Application of IL Skills and Techniques for Non-Academic Purposes
This section of
the survey sought to determine use of IL skills
outside the academic environment, for personal use. With regard to the LIB102
group, the most common databases used were: Google (30%), OneSearch (13.3%), The New York Times (11.1%), and
Wikipedia (10%; see Table 9). The top four databases selected for non-academic
purposes by the matched comparison group were Google (41.7%), Wikipedia
(20.8%), The New York Times (8.3%),
and MEDLINE (8.3%). Even though the use of Google still reigned supreme for
non-academic purposes, it was rewarding to see that the respondents in the
LIB102 group had also used library subscription resources for non-academic
purposes. This signaled that the LIB102 course, which was designed to enable
students to move beyond the incessant use of Google as a research tool for
their assignments was playing a large role in their information seeking behaviour for non-academic purposes.
Table 9
Use of Databases
for Non-Academic Purposes
|
|
LIB102 Group |
Matched Comparison Group |
|
Most Used |
Google
– 30% OneSearch
– 13.3% The New York Times –
11.1% Wikipedia
– 10% |
Google
– 41.7% Wikipedia
– 20.8% The New York Times – 8.3% MEDLINE
– 8.3% |
|
Least Used |
Academic
Search Complete – 6.7% The Wall Street Journal – 6.7% JSTOR –
4.4% Ebook Central – 2.2% Gale
Virtual Reference Library – 2.2% PsycINFO
– 2.2% Opposing
Viewpoints – 1.1% |
CINAHL
– 4.2% JSTOR –
4.2% Kanopy – 4.2% OneSearch
– 4.2% |
When asked in an
open-ended question about difficulty using any of the databases for
non-academic purposes, 86.2% of respondents in the LIB102 group reported no
difficulty, and likewise 90.9% of respondents in the matched comparison group
reported the same. Appendix C shows results for use of search skills when
seeking information for non-academic purposes, where keyword searching was the
most popular among both groups (21.6% in the LIB102 group vs. 27.5% in the
matched comparison group). Participants were also asked to identify types of
information they would search for non-academic purposes. Both groups identified
social/ entertainment as the top subject information sought, with 21.7% for the
LIB102 group and 18.9% for the matched comparison group (see Appendix D).
Thereafter, the groups differed slightly in subsequent subjects most searched.
For the LIB102 group, medical information was second (13.2%), political
information was third (12.4%), and nutrition/diet information was fourth
(10.9%). For the matched comparison group, it was technological (15.1%),
environmental (13.2%), and political and nutrition/diet information were a tie
for fourth (11.3%). When asked if they were ultimately successful in finding
the non-academic information needed, a yes/no question, 93% of the respondents
in both groups indicated success.
Benefits of Taking the Course and Additional Feedback
The last two
questions were open-ended, specifically targeted to the LIB102 group to provide
evaluative statements about the course. In the first question, respondents were
asked to share any benefits of having taken the course. Out of 43 respondents,
36 provided complimentary responses of how the course had helped them. Some of
the comments were as brief as: “It helped me write better papers”; “Being able
to use the databases efficiently”; “I learned how to find research material for
my other classes quickly and easily”; “Yes, it has helped me be able to
identify credible online resources to use for my research papers”; “It has
helped [me] develop skills to do research for classes and how to find valid
sources and spot bias”; and “Yes it showed me different ways to research and
how to identify legitimate sources of information.”
Other, lengthier,
comments were:
I learned how to access research materials, as well as the skills
necessary for writing citations, annotations, etc. I honestly feel this class
should be required core curriculum along with ENG111 & 151.
I took the class during my Freshman year. I found it helpful because I
was new to the college where expectations of writings were on a standard, and I
did not know that. Taking that class helped me shape my foundation for the expectations
of college writing.
It made me ahead of all other students in terms of knowing how to do
research papers. Many students don’t know that they have access to the library
from home.
Yes I have taken
LIB102, it helped me with developing new skills when searching for topics from
articles on the web by using techniques like typing in keywords and synonyms to
narrow down my search.
Yeah! I took LIB102 with [name redacted] which was great. She was
wonderful and while a lot of the stuff was intuitive, a lot of my friends are
actually very confused by it. It was helpful being walked through it.
The last
open-ended question asked respondents in the LIB102 group to share any
additional comments or feedback. Out of 30 responses, 26 noted “none” or
“nothing,” “n/a,” or “no.” The remaining four were enlightening enough to
document here:
G[r]e[a]t class I think it should be a gen ed requirement for all
incoming freshmen.
I think this class should be mandatory for all incoming freshman [sic].
CSI library should send out emails about any updates or specific changes
to the website. I also think that the library should send out brief emails with
information or instructions on how to do things on the library’s website just
so that taking a class or making a trip to see the librarian isn’t the only way
of learning how to use the CSI library website.
First, I want to thank you for conducting this study and consider the
interests of the CSI students. Second, I feel library resources and search
skills learned are not heavily promoted among students because my friends and I
sometime[s] would have questions or face problems regarding looking up an
article and did not know where to go. Again, thank you for your time.
Discussion
Results of the
multiple regression and logistical analyses on outcomes of students who took
LIB102 show evidence that the course has had a slight impact on their
persistence and GPA. Data from the regression analysis suggest that students
who have enrolled in LIB102 gain a slightly higher cumulative GPA than those
who have never taken the course. Though the magnitude of effects was small,
this study supports results of other studies that have shown that multiple
sessions of library instruction increased confidence of first year students in
conducting research and, in turn, have slightly impacted their overall GPA
(Bowles-Terry, 2012; Daugherty & Russo, 2011, Heady et. al, 2018; Soria et
al., 2017). The results of the logistic regression analysis on students’
performance in their persistence one year after enrollment in the LIB102 course
semester reveal that students were 1.26 times more likely to return to or
graduate than the matched comparison students. While the authors are cautioned
by the fact that the difference was not considered to be statistically
significant, this study does corroborate studies by Catalano and Phillips
(2016), Cook (2014), Haddow (2013), and Murray et al. (2016), who also found
that an IL credit-bearing course does have an impact on students’ academic
performance, and in particular pertaining to persistence.
Like Lwoga (2014) and Stonebraker and Fundator (2016), we discovered from the survey that
students who have enrolled in LIB102 continue to utilize online searching
skills gained in the course both in other academic classes and for non-academic
purposes. The LIB102 group differed from the matched comparison group in higher
use of academic databases frequently taught in the course, having less
difficulty using databases than the matched group, and having less difficulty using
search operators, including the use of wildcards and truncations. These
findings validate that taking the IL course may assist students in being better
at navigating familiar or new information resources. Moreover, in the answers
to the open-ended questions, the LIB102 group confirm findings in studies by
Daugherty and Russo (2011), Mery et al. (2012) and
Soria et al. (2017) that showed that an IL course has a lasting impact on
students as they progress through college. In comment after comment, each student
indicated how LIB102 had served them well in becoming better at conducting
research and writing papers. Their brief but concise testimonies spoke volumes
about the goals of the course. Several comments seemed to be directed at
librarians requesting that the course be made mandatory for freshmen or
first-year students.
Similar to
results in the studies conducted by Heady et al. (2018) and Squibb and
Mikkelsen (2016), a surprising outcome in this study was the results of the PSM
analysis, which revealed a lower performance of the LIB102 group in the
mandatory 100-level English courses than the matched comparison group. The
lower performance of students in these courses, even after taking LIB102, may
be due to several external factors beyond our control. Among them could be the
fact that the LIB102 course was taken at the same time as ENG111 and ENG151,
and that students were not able to make the association while taking the
course. A second possible factor could be that the intention of LIB102, which
is to teach students how to conduct research, was not in parity with the designs
of ENG111 (Introduction to College Writing) or ENG151 (College Writing), which
are designed to teach writing as their titles indicate. The lower performance
by the LIB102 group was surprising given their cumulative GPA was 0.08 points
higher than the matched comparison group. The rationale in the study by Heady
et al. (2018) was that most students “pass the course with a B or above, which
would modestly contribute to their GPA” (p. 647). The authors of this current
study agree that this could be a possible explanation, as the average grade in
LIB102 over 10 semesters has been a solid B. Either way, this is a revelation
that should be further examined and discussed by those teaching in the LIB102
program.
A disappointing
outcome was in the low utilization of Boolean operators reported by the LIB102
group. The use of Boolean operators is a skill that is always taught in the
course, in a dedicated class session. Librarians emphasize that these same
search skills can also be applied when searching in Google to retrieve the most
relevant sources. However, two recent students by Lowe et al. (2018) and Lowe
et al. (2020) confirm that teaching Boolean searching to first-year students is
a waste of class time, and that it is time well spent for upper-level students
who find it useful for their discipline-specific databases. Nevertheless, this
raises a second question regarding assessment of the ways search skills and
techniques are being taught, and whether some of these skills and techniques
are still relevant.
Limitations
There are a
couple of limitations to note in this study. First, as with any study involving
regression analysis, the findings should be interpreted with caution. In our
study, while the results show that the matched comparison group performed
better in ENG111 and ENG151 than students who had taken the LIB102 course, the
analysis reflects their course performance at the same time as or after taking
LIB102. It is unknown whether the negative coefficient is greatest when taking
ENG111 or ENG151 concurrently with LIB102 or after taking LIB102. Also,
although the analyses controlled for student demographics and previous
achievement, threats to internal validity still exist. Students in the matched
comparison group may have received instruction in IL in other courses that are
not controlled for in this study.
Second, while
the authors utilized several interventions prescribed by Dillman
et al. (2014) to obtain a high response rate and quality data when using
web-based surveys, the 30% response rate that is widely considered a robust
response in the social sciences was not achieved for this survey. However, the
thoughtful and lengthy responses to the open-ended questions were rich
contributions to the study.
Implications for Future Study
A strength of
this study, at this time of writing, is that this research is among a few
studies to utilize the PSM method to discover the impact of a stand-alone IL
credit-bearing course with regard to student academic achievements. As with any
study, the use of these results should inform an examination of the subject or
inform changes to be implemented. If this study were to be repeated, it is
advisable to conduct the multiple regression analyses separately for those
students who took 100-level English courses at the same time as LIB102 from
those who took the English course after LIB102. This might ensure an
unambiguous result. The negative coefficient results from the multiple
regression analyses suggest ENG111 and ENG151 may not have been the appropriate
courses to which LIB102 was linked. It may be worth an in-depth examination of
whether the intentions of LIB102 are in parity with the introductory 100-level
English courses. Perhaps LIB102 could be better paired with other courses in
which students conduct research.
Furthermore,
given the results of the logistic regression analysis regarding the one-year
persistence of students who have taken LIB102, it would be worth conducting the
analysis specifically on first-year students as compared with non-first-year
students. The course was actually designed for first-year students, and we are
remiss not to have included this analysis at the outset.
Conclusion
The findings in this study underscore the importance
of a stand-alone, credit-bearing IL course for undergraduate students who are
primarily first generation in an open access urban institution. While results
of the multiple regression and logistical analyses on outcomes of students who
took LIB102 show evidence that the course has had a slight impact on their
persistence and GPA, our survey findings yield valuable insights and validate
why taking this course may well assist students in becoming confident at
searching and retrieving information resources for their academic assignments.
This is illustrated by the LIB102 group attaining a slightly higher GPA than
the matched comparison group, as well as a persistence rate that showed that
students were 1.26 times more likely to return one year after taking LIB102.
Our findings also show that the knowledge gained in LIB102 is applied in
students’ personal lives for non-academic purposes, thus recognizing
information literacy as a lifelong learning skill. Moreover, the survey
revealed numerous positive testimonies of how helpful this course has been to
students for academic and non-academic purposes, further validating the value
of the course. In their own words, several students emphatically prescribed
that the course should be made mandatory for freshmen or included in the
general education curriculum. This, the researchers ascertained, comes from
students feeling empowered by the skills they learned in LIB102.
The researchers are confident that the quantitative
and qualitative evidence presented here not only support the anecdotal feedback
that served as an impetus for this study, but also demonstrate the library’s
commitment to supporting institutional outcomes that pertain to students’
development of learning. Indeed, this is a testament to one of the goals in the
CSI Department of the Library’s (n.d.) mission statement, which states that we
“cultivate scholarly inquiry and discovery through teaching information
literacy and critical thinking skills.” Moreover, this study makes a
substantial contribution to the gap in the library literature. In Oakleaf’s
(2010) comprehensive report for ACRL, she urged academic librarians to be more
systematic in collecting data that connect and demonstrate the library’s value
in institutional terms. This study answers that call, and the authors of this
paper aim to conduct follow-up studies.
Author Contributions
Wilma L. Jones: Conceptualization (lead),
Methodology, Formal analysis, Writing – original draft (lead), Writing – review
& editing (equal) Tara Mastrorilli: Conceptualization (supporting),
Methodology (lead), Formal analysis (lead), Writing – review & editing
(equal)
References
Association of College and Research Libraries. (2000). Information literacy competency standards
for higher education. http://hdl.handle.net/11213/7668
Association of College and Research Libraries. (2016). Framework for information literacy for
higher education. https://www.ala.org/acrl/standards/ilframework
Black, E. L., & Murphy, S. A. (2017). The out loud assignment:
Articulating library contributions to first-year student success. The Journal of Academic
Librarianship, 43(5), 409–416. https://doi.org/10.1016/j.acalib.2017.06.008
Bowles-Terry, M. (2012). Library instruction and academic success: A
mixed-methods assessment of a library instruction program. Evidence Based Library & Information Practice, 7(1),
82–95. https://doi.org/10.18438/B8PS4D
Britto, M., & Kinsley, K. (2018). Academic libraries and the academy: Strategies and approaches to
demonstrate your value, impact, and return on investment. Association of
College and Research Libraries.
Burkhardt, J. M. (2016). Teaching
information literacy reframed: 50+ framework-based exercises for creating
information-literate learners. ALA Neal-Schuman.
Catalano, A., & Phillips, S. R. (2016). Information literacy and
retention: A case study of the value of the library. Evidence Based Library & Information Practice, 11(4),
2–13. https://doi.org/10.18438/B82K7W
Chiteng Kot, F., & Jones, J. L. (2015). The impact of library
resource utilization on undergraduate students' academic performance: A
propensity score matching design. College
& Research Libraries, 76(5), 566–586. https://doi.org/10.5860/crl.76.5.566
College of Staten Island, Office of Institutional Research.
(2020). Semester enrollment: Student
demographic profile, Fall 2019. The City University of New York. https://applications.csi.cuny.edu/Institutional_Profile/SemesterEnroll_Profile.html
Cook, J. M. (2014). A library credit course and student success
rates: A longitudinal study. College
& Research Libraries, 75(3), 272–283. https://doi.org/10.5860/crl12-424
Daugherty, A. L., & Russo, M. F. (2011). An assessment of the
lasting effects of a stand-alone information literacy course: The students'
perspective. The Journal of Academic
Librarianship, 37(4), 319–326. https://doi.org/10.1016/j.acalib.2011.04.006
Department of the Library. (n.d.). The City University of New York, College of Staten Island. https://www.csi.cuny.edu/academics-and-research/department-library
Diep, K. C., & Nahl, D. (2011).
Information literacy instruction in four Vietnamese university libraries. International Information &
Library Review, 43(4), 198–206. https://doi.org/10.1016/j.iilr.2011.10.002
Dillman, D. A., Smyth,
J. D., & Christian, L. M. (2014). Internet,
phone, mail, and mixed-mode surveys: The tailored design method. John Wiley
& Sons.
Haddow, G. (2013). Academic library use and
student retention: A quantitative analysis. Library
& Information Science Research, 35(2), 127–136. https://doi.org/10.1016/j.lisr.2012.12.002
Hauck, J. (2017). From service to synergy: Embedding librarians in a
digital humanities project. College
& Undergraduate Libraries, 24(2–4), 434–451. https://doi.org/10.1080/10691316.2017.1341357
Heady, C., Morrison, M. M., & Vossler, J.
(2018). Ecological study of graduation rates and GPA in a library credit
course. The Journal of Academic
Librarianship, 44(5), 642–649. https://doi.org/10.1016/j.acalib.2018.07.010
Inuwa, S., & Abrizah, A. (2018). Embedded librarianship in research in
Nigerian universities: Practices and sources of practice knowledge. The Journal of Academic
Librarianship, 44(6), 738–746. https://doi.org/10.1016/j.acalib.2018.10.002
Luetkenhaus, H., Borrelli,
S., & Johnson, C. (2015). First year course programmatic assessment: Final
essay information literacy analysis. Reference
& User Services Quarterly, 55(1), 49–60. https://doi.org/10.5860/rusq.55n1.49
Lowe, M. S., Maxson, B. K., Stone, S. M., Miller, W., Snajdr, E., &
Hanna, K. (2018). The Boolean is dead, long live the Boolean! Natural language
versus Boolean searching in introductory undergraduate instruction. College & Research Libraries, 79(4),
517–534. https://doi.org/10.5860/crl.79.4.517
Lowe, M. S., Stone, S. M., Maxson, B. K., Snajdr, E., & Miller, W.
(2020). Boolean redux: Performance of advanced versus simple Boolean searches
and implications for upper-level instruction. The Journal of Academic Librarianship, 46(6), 102234. https://doi.org/10.1016/j.acalib.2020.102234
Lwoga, E. T. (2014).
Mapping information literacy outcomes and learning experiences of health sciences
undergraduate students. Partnership:
The Canadian Journal of Library and Information Practice and Research, 9(1). https://doi.org/10.21083/partnership.v9i1.2695
Mackey, T. P., & Jacobson, T. E. (2010). Collaborative information literacy assessments: Strategies for
evaluating teaching and learning. Neal-Schuman.
Mayer, J., & Bowles-Terry, M. (2013). Engagement and assessment in a
credit-bearing information literacy course. Reference
Services Review, 41(1), 62–79. https://doi.org/10.1108/00907321311300884
McDevitt, T. R. (2011). Let
the games begin! Engaging students with field-tested interactive information
literacy instruction. Neal-Schuman.
Mery, Y., Newby, J.,
& Peng, K. (2012). Why one-shot information literacy sessions are not the
future of instruction: A case for online credit courses. College & Research Libraries, 73(4), 366–377. https://doi.org/10.5860/crl-271
Murray, A., Ireland, A., & Hackathorn, J.
(2016). The value of academic libraries: Library services as a predictor of
student retention. College &
Research Libraries, 77(5), 631–642. https://doi.org/10.5860/crl.77.5.631
Oakleaf, M. (2010). The value
of academic libraries: A comprehensive research review and report.
Association of College and Research Libraries. http://www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/value/val_report.pdf
Oakleaf, M. (2015). The library’s contribution to student learning:
Inspirations and aspirations. College
& Research Libraries, 76(3), 353–358. https://doi.org/10.5860/crl.76.3.353
Oliveira, S. M. (2018). Retention matters: Academic libraries leading
the way. New Review of Academic Librarianship, 24(1),
35–47. https://doi.org/10.1080/13614533.2017.1365003
Pierard, C., &
Graves, K. (2007). Research on student retention and implications for library
involvement. In L. Hardesty (Ed.), The
role of the library in the first college year (pp. 155–168).
University of South Carolina, National Resource Center for the
First-Year Experience and Students in Transition.
Polger, M. A., & Sheidlower, S. (2017). Engaging diverse learners: Teaching strategies for academic librarians.
Libraries Unlimited.
Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the
propensity score in observational studies for causal effects. Biometrika, 70(1),
41–55. https://doi.org/10.2307/2335942
Rysavy, M. D. T.,
Michalak, R., & Wessel, A. (2017). 8 years of institutional assessment
feedback: Students’ satisfaction with library services. Reference Services Review, 45(4), 544–561. https://doi.org/10.1108/RSR-03-2017-0005
Soria, K. M., Fransen, J., & Nackerud, S. (2013). Library use and undergraduate student
outcomes: New evidence for students' retention and academic success. portal: Libraries and the
Academy, 13(2), 147–164. https://doi.org/10.1353/pla.2013.0010
Soria, K. M., Fransen, J., & Nackerud, S. (2014). Stacks, serials, search engines, and
students' success: First-year undergraduate students' library use, academic
achievement, and retention. The
Journal of Academic Librarianship, 40(1), 84–91. https://doi.org/10.1016/j.acalib.2013.12.002
Soria, K. M., Fransen, J., & Nackerud, S. (2017). The impact of academic library
resources on undergraduates' degree completion. College & Research Libraries, 78(6), 812–823. https://doi.org/10.5860/crl.78.6.812
Squibb, S. D., & Mikkelsen, S. (2016). Assessing the value of
course-embedded information literacy on student learning and achievement. College & Research
Libraries, 77(2), 164–183. https://doi.org/10.5860/crl.77.2.164
Stonebraker, I. R., & Fundator, R. (2016). Use it or lose it? A longitudinal
performance assessment of undergraduate business students' information
literacy. The Journal of Academic
Librarianship, 42(4), 438–444. https://doi.org/10.1016/j.acalib.2016.04.004
Swanson, T. A., & Jagman, H. (2015). Not just where to click: Teaching students
how to think about information. Association of College and Research
Libraries.
Tumbleson, B., Burke, J.,
& Long, J. (2019). Assessment, analytics, and analysis: Demonstrating the
impact of LMS embedded librarians on student learning. Journal of Library & Information Services in Distance
Learning, 13(1–2), 196–214. https://doi.org/10.1080/1533290X.2018.1499252
U.S. Census Bureau. (2019). Staten
Island borough, Richmond County, New York. https://data.census.gov/cedsci/profile?g=0600000US3608570915
Appendix A
Questionnaire
Impact of an Information Literacy Course
I. Demographics
1.
Please identify your current academic status. How many
credits will you have earned by the end of this semester?
a. ___ 0 – 30 credits
b. ___ 31 – 60 credits
c. ___ 61 – 90 credits
d. ___ 91 – 124+ credits
e. ___ Graduate Student
f. ___
Other
2.
Gender:
a. ___ Female
b. ___ Male
c. ___ Prefer not to disclose
d. ___ Other
3.
Ethnicity (check all that apply):
a. ___ American Indian or Alaska Native
b. ___ Asian or Asian American
c. ___ Black or African American
d. ___ Latino or Hispanic
e. ___ Native Hawaiian or Other Pacific
Islander
f. ___
White or Caucasian
g. ___ Prefer not to disclose
h. ___ Other __________
4.
What is your current major/intended major?
a. (drop down menu that includes all
majors, including “undeclared” major)
5.
Are you a Transfer student to CSI?
___
Yes ___ No
6.
Are you (or your siblings) the first in your family to
attend College?
___
Yes ___ No
7.
Are you part of any one of the learning community
programs below:
a. ___ ASAP – Accelerated Studies in
Associated Program
b. ___ BMI Program – Black Male
Initiative Program
c. ___ C-STEP – College Science &
Technology Entry Program
d. ___ Macaulay Honors College
e. ___ SEEK Program – Search for
Education, Elevation, & Knowledge Program
f. ___
Teacher Education Honors Academy
g. ___ Verrazano School
h. ___ Other
_____________________________
i. ___
No, I was never enrolled in a learning community
8.
Have you taken the course “Beyond Google: Research for
College Success” labeled as LIB102 or LIB501?
___
Yes ___ No
(if the answer
is NO, software skips to #10)
9.
What type of LIB102/LIB 501 did you take?
a. ___ Hybrid (where you had 3 or 4
classes held online)
b. ___ In Classroom (where you had
face-to-face interactions for all 7 class meetings)
II. Applications of IL skills and information
resources in classes other than Beyond Google: LIB102 / LIB 501
10.
Please identify the type(s) of class project(s) you
have taken at CSI (select all that apply):
a. 3-5 page research paper/project
b. 6-10 page research paper/project
c. 10-20 page research paper/project
d. Technical report for science lab
course
e. Undergraduate seminar project
f. Master’s
thesis
g. Capstone research project
h. Other __________________
11.
Please indicate which of any of the library databases
you have used for your class projects (select all that apply):
a. None
b. OneSearch
c. Academic Search Complete
d. CINAHL Complete
e. Ebook
Central
f. Gale
Virtual Reference Library
g. JSTOR
h. Kanopy
i. Lexis-Nexis
(now Nexis Uni)
j. MEDLINE
k. The
New York Times
l. Opposing
Viewpoints
m. PsycINFO
n. The
Wall Street Journal
o. Other(s) ______________
12.
Please explain any difficulty you encountered
when using any of the identified databases above.
____________________________________________________________
____________________________________________________________
____________________________________________________________
13.
What types of information were you searching for while
using the above identified databases? (Select all that apply:)
a. Newspaper articles
b. Magazine articles
c. Peer-reviewed articles
d. Research reports
e. Government documents
f. Statistical
information
g. Blogs or Websites
h. Podcasts
i. Films
or Video-clips
j. Other
_____________
14.
Please indicate which searching skills or techniques
you used (select all that apply):
a. Keyword searching
b. Subject searching
c. Boolean operators (AND, NOT, OR)
d. Advanced searching
e. Use of synonyms
f. Google
search operators (site, link, source, filetype, etc.)
g. Browsing the index
h. Phrase searching
i. Truncation
or wildcard (?, *)
j. Physical
navigation of the stacks
k. Other _____________
l. None
of the above
15.
Please explain any difficulty you encountered
when using those skills or techniques:
____________________________________________________________
____________________________________________________________
____________________________________________________________
III. Applications of IL skills and information
resources for non-academic purposes
16.
Please indicate which information resources you have
used for non-academic purposes. Examples of non-academic purposes: finding
information on future employer/corporation, researching a product you wish to
purchase (i.e., car, washer/dryer, etc.), or locating medical information on
treatment of a disease for a family member. (Select all that apply:)
a. None
b. OneSearch
c. Academic Search Complete
d. CINAHL Complete
e. Ebook
Central
f. Gale
Virtual Reference Library
g. Google
h. JSTOR
i. Kanopy
j. Lexis-Nexis
(now Nexis Uni)
k. MEDLINE
l. The New York Times
m. Opposing Viewpoints
n. PsycINFO
o. The
Wall Street Journal
p. Wikipedia
q. Other ___________
17.
Please indicate which searching skills or techniques
you have used (check all that apply):
a. Keyword searching
b. Subject searching
c. Boolean operators (AND, NOT, OR)
d. Advanced searching
e. Use of synonyms
f. Google
search operators (site, link, source, filetype, etc.)
g. Browsing the index
h. Phrase searching
i. Truncation
or wildcard (?, *)
j. Physical
navigation of the stacks
k. Other __________
l. None
of the above
18.
Please explain any difficulty you encountered
when using those skills or information resources?
____________________________________________________________
____________________________________________________________
19.
Were you ultimately successful in finding the
non-academic information you were looking for?
___ Yes ___ No
20.
What type(s) of information were you trying to find?
(Check all that apply:)
a. Social/Entertainment
b. Political
c. Legal
d. Medical
e. Nutrition/Diet
f. Financial/Economic
g. Environmental
h. Technological
i. Exercise
j. Other___________
21.
If you have taken LIB102/LIB 501 “Beyond Google,” has
it helped you in any way? Any benefits?
____________________________________________________________
____________________________________________________________
22.
Do you have any additional comments or feedback that
you would like to share?
____________________________________________________________
____________________________________________________________
23.
If you would like to participate in the drawing to win
one of eight (8) $50 Amazon.com gift cards, please provide your name and
email address (it does not have to be your CSI address) in the slot
below.
Thank you for your time and contribution in completing
this survey.
Appendix B
Search Skills Used for Academic Purposes
|
|
LIB102 group |
Matched comparison group |
|
Search skills |
Keyword searching – 21.2% Subject searching – 19.2% Advanced searching – 16.4% Google search operators – 9.6% Phrase searching – 8.2% Use of synonyms – 8.2% Boolean operators – 6.2% Browsing the index – 6.2% Physical navigation of the stacks – 2.1% Truncation or wildcard (?,
*) – 1.4% |
Keyword searching – 26.5% Subject searching – 22.4% Advanced searching – 10.2% Phrase searching – 10.2% Use of synonyms – 8.2% Boolean operators – 4.1% Browsing the index – 4.1% Physical navigation of the stacks – 2.0% |
Appendix C
Search Skills Used for Non-Academic Purposes
|
|
LIB102 group |
Matched comparison group |
|
Search skills used for non-academic purposes |
Keyword searching – 21.6% Google search operators – 18.9% Subject searching – 14.4% Advanced searching – 12.6% Use of synonyms – 10.8% Phrase searching – 7.2% Boolean operators (AND, NOT, OR) – 5.4% Browsing the index – 4.5% |
Keyword searching – 27.5% Subject searching – 22.5% Google search operators – 12.5% Advanced searching – 10% Use of synonyms – 10% Phrase searching – 7.5% Browsing the index – 5% Boolean operators (AND, NOT, OR) – 2.5% |
Appendix D
Types of Information Sought for Non-Academic Purposes
|
|
LIB102 group |
Matched comparison group |
|
Types of information sought for non-academic
purposes |
Social/Entertainment – 21.7% Medical – 13.2% Political – 12.4% Nutrition/Diet – 10.9% Environmental – 8.5% Exercise – 8.5% Technological – 8.5% Financial/Economic – 8.5% Legal – 6.2% Other – 1.6 % |
Social/Entertainment – 18.9% Technological – 15.1% Environmental – 13.2% Political – 11.3% Nutrition/Diet – 11.3% Medical – 9.4% Financial/Economic – 7.5% Exercise – 5.7% Legal – 3.8% Other – 1.9 % Personal – 1.9% |