Editorial

Avoiding Common Errors When Conducting Survey Research

Ann Medaille

Editor-in-Chief

Professor, Director of Research & Instructional Services

University of Nevada, Reno Libraries

Reno, Nevada, United States of America

2025 Medaille. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

DOI: 10.18438/eblip30862

At Evidence Based Library and Information Practice (EBLIP), our editors receive many research article submissions that use survey methodology. Survey studies can be very effective when done well, but there are some common errors in survey design and reporting that appear repeatedly in manuscripts. In this editorial, I will discuss a few common errors and offer some advice on how to avoid them.

Explaining the Purpose of the Survey

Surveys are used when a researcher wants to use quantitative approach to identify and describe some aspects of a particular population (Groves et al., 2004, p. 2; Orcher, 2007, p. 1). Survey methodology is a type of observational research in that the researcher is observing and measuring what exists rather than introducing an intervention that is subsequently measured. Surveys can be used to measure a variety of attributes, such as information behaviors, future needs, attitudes towards events, levels of satisfaction, degrees of knowledge, and responses to programs (Orcher, 2007, pp. 9–18). As such, they are a valuable tool in library and information science (LIS) research and evidence based practice.

Perhaps a researcher wants to measure the knowledge of engineering librarians, the behaviors of humanities researchers, or student satisfaction with library services. All of these are good reasons to employ survey methodology. Library and information science researchers often conduct surveys because they have a practical problem that they need to solve, and they want to identify characteristics that can inform the implementation of programs and services. Regardless of the reason, research articles that employ survey methodology should have a research question(s) that is appropriate for a quantitative approach.

Survey Sampling

Many surveys are conducted with the intention of discovering data that can be generalized to a larger population, and these types of surveys often require the use of rigorous sampling methods. However, generalization is not an essential requirement for survey research. Indeed, given the limited resources available for many LIS researchers, collecting unbiased samples of certain sizes means that generalization is not always an option.

For all survey studies—whether generalizable or not—researchers should pay substantial attention to how they sample their identified population. A population refers to everyone or everything within a particular category, and a sample refers to the subset of that population who are participating in the study. For example, a population might consist of biology researchers at large research universities, while a sample is a subset of those researchers. Library and information science researchers should have a clear understanding of who their population is (e.g., understanding their demographic characteristics), and should be able to describe how their sampling method ensures that participants reflect that population. For example, if a researcher uses purposive sampling to select participants, which is a type of non-probability sampling in which participants are selected based on specific characteristics, then they have to describe how using this sampling strategy results in a usable sample of the population.

When describing their sampling methods in a research article, researchers should adequately explain the efforts they made to collect data from a significant number of representative participants. This level of detail supports the credibility of the data. In addition, researchers need to identify any limitations that result from their sampling methods that might affect their ability to draw accurate conclusions from the data. For instance, perhaps participants who are likely to respond to a survey possess some characteristics that differ from the rest of the population. Researchers need to take these considerations into account. Finally, researchers should also report their survey’s response rate and consider how nonresponse may affect their conclusions.

Collecting Demographic Data

Collecting demographic data about participants is often an essential part of conducting a survey. However, researchers should be thoughtful about the kinds of demographic data they collect and the reasons for collecting it. There are two main reasons to collect demographic data: because the data are necessary to answer the research question(s) and because they are needed to adequately describe the participants for readers of a study (Orcher, 2007, p. 75). Whatever the reason, researchers should very carefully select which demographic data to ask about to avoid collecting unnecessary or potentially sensitive data.

When presenting demographic data in a research article, researchers may present it either in the methods or results sections, depending on where it best fits. If the demographic data are collected to answer the research question(s), then they are often presented in the results section; if collected to describe the participants, then they are often presented in the methods.

Writing Survey Questions and Response Choices

The phrasing of survey questions is critical for how participants interpret and respond to them, as slight variations in phrasing can have a significant impact on responses. Survey questions should be written in simple language without the use of jargon or acronyms. Questions should be clear, specific, and direct, and should be written in a neutral, non-emotional tone. Terms should be defined as needed. Researchers should avoid writing leading questions, which use subtle variations in language to influence participants to respond in certain ways (Nardi, 2018, pp. 71–113).

Researchers should also write response choices that make sense to their participants. They should be cautious about over-reliance on Likert scales, in which participants are forced to choose among a number of vague responses, such as strongly agree, agree, neutral, disagree, strongly disagree. It is often difficult for participants to know how to respond (e.g., What’s the difference between strongly agree and agree?) and for readers to know how to interpret the differences (Thalheimer, 2022, pp. 11-27). Depending on the purpose of the survey, researchers may want to provide answer choices that are specific and actionable.

Researchers can study other surveys and examine how their questions were phrased to understand how best to write survey questions. Researchers can even reuse questions from other surveys when permitted and appropriate, always making sure to credit the original authors. To ensure that participants will interpret questions as they are intended, researchers should test their survey with several people and collect feedback on how they interpreted the questions. Researchers can then use their feedback to revise the survey and test it again if needed. The process of survey testing should always be described in the methods section of a research article.

Accurate Reporting

When reporting results from surveys, researchers should ensure that they do not mislead readers about the results. Data should be presented in a straightforward manner, and researchers should avoid attributing reasons for responses that are not justified. If using percentages to describe survey results received from a small number of participants, researchers should also report the number of responses received. For example, instead of stating that “80% of graduate students agreed that…,” a researcher could report that “12 out of the 15 graduate students who participated in the student survey [80%] agreed that…” so as not to mislead readers about the size of this group. Similarly, care should be taken when reporting percentage increases or decreases for small numbers. If attendance at library events increased from five to 10 attendees, then the actual numbers should be reported; stating that attendance increased by 100% makes it seem as if the increase was actually much greater (Nardi, 2018, p. 229; Rea & Parker, 2005, p. 246).

Data presented in charts, graphs, and tables should receive some explanation in the text. Researchers should avoid reporting all of their data in the text but should instead highlight significant items and refer readers to tables, figures, or appendices for the complete responses. In addition, researchers should ensure that the types of visuals used are appropriate to the data presented. For example, bar charts are useful for comparing values across categories, stacked bar charts show how subcategories contribute to the whole, and line charts show changes over time. Researchers should be thoughtful in selecting which visuals to use to represent their data.

Final Thoughts

When conducting surveys, researchers should always demonstrate respect for their participants. Researchers should ensure that their survey instruments and data collection follow institutional ethics guidelines, including clear consent and protection of respondents’ privacy.

Finally, one of the best ways to learn about survey research design and reporting is to read some of the articles published in EBLIP over the years. Strong survey articles can provide good models for this type of research design.

References

Groves, R. M., Fowler, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2004). Survey methodology. Wiley.

Nardi, P. M. (2018). Doing survey research: A guide to quantitative methods (4th ed.). Routledge. https://doi.org/10.4324/9781315172231

Orcher, L. T. (2007). Conducting a survey: Techniques for a term project. Pyrczak Publishing.

Rea, L. M., & Parker, R. A. (2005). Designing and conducting survey research: A comprehensive guide (3rd ed.). Jossey-Bass.

Thalheimer, W. (2022). Performance-focused learner surveys: Using distinctive questioning to get actionable data and guide learning effectiveness (2^nd ed.). Work-Learning Press.