Pragmatic Human-Centred Design in Action: Assessing the Tone and Usability of the Behaviours in Dementia Toolkit Website
DOI:
https://doi.org/10.18438/eblip30754Abstract
Objective – The Behaviours in Dementia Toolkit is an online library to support dementia-related mood and behaviour changes. It is intended to be used by two key caregiving audiences: 1) formal, professional care partners, and 2) informal care partners. The purpose of this study was to assess the content tone and usability of the website to meet the needs of its two user groups.
Methods – The multi-method design included electronic surveys and interviews. Content tone was assessed using word reaction and highlighter tests. Task-specific usability was assessed with the single-ease question (SEQ), time on task, and completion rate. The System Usability Scale (SUS) measured overall usability before and after design changes based on initial results.
Results – Content tone was friendly, trustworthy, desirable, and clear, but informal care partners found medical jargon confusing. Task-specific usability was generally acceptable. Time one SUS score for formal care partners was assessed at a letter grade A, but for informal care partners was a C. Content tone and usability testing drove critical changes to the library. Time two SUS scores subsequently equalized for the user groups to an A- for formal care partners and a B for informal care partners. Divergent requirements for formal and informal care partners were identified and addressed using pragmatic human-centred design methods.
Conclusion – Jargon, information overload, and search flow require careful attention in an online health information library. Digital libraries and other projects facilitating access to health information seeking groups may find these usability methods useful to consider at the outset of their project planning given the actionable insights they can provide. These techniques are suitable for projects with a limited budget and tight timeframe.
Downloads
References
Albrecht, L., Ubels, N., Martinussen, B., Naglie, G., Rapoport, M., Hatch, S., Seitz, D., Checkland, C., & Conn, D. (2025). The Behaviours in Dementia Toolkit: A descriptive study on the reach and early impact of a digital health resource library about dementia-related mood and behaviour changes. Geriatrics (Basel, Switzerland), 10(3), 79. https://doi.org/10.3390/geriatrics10030079
Allison, R., Hayes, C., McNulty, C., & Young, V. A. (2019). Comprehensive framework to evaluate websites: Literature review and development of GoodWeb. JMIR Formative Research, 3(4). https://doi.org/10.2196/14372
Barbara, A. M., Dobbins, M., Haynes, R. B., Iorio, A., Lavis, J. N., Raina, P., & Levinson, A. J. (2016). The McMaster Optimal Aging Portal: Usability evaluation of a unique evidence-based health information website. JMIR Human Factors, 3(1), e4800. https://doi.org/10.2196/humanfactors.4800
Brett, K. R., Lierman, A., & Turner, C. (2016). Lessons learned: A Primo usability study. Information Technology and Libraries, 35(1), 7–25. https://doi.org/10.6017/ital.v35i1.8965
Brooke, J. (1986). SUS ‘quick and dirty’ usability scale. In P. W. Jordan, B. Thomas, B. A. Weerdmeester, & A. L. McClelland (Eds.), Usability evaluation in industry. Taylor and Francis.
Burgdorf, J. G., Reckrey, J., & Russell, D. (2023). “Care for me, too”: A novel framework for improved communication and support between dementia caregivers and the home health care team. The Gerontologist, 63(5), 874–886. https://doi.org/10.1093/geront/gnac165
Canadian Coalition for Seniors’ Mental Health (CCSMH). (2024, March). Canadian clinical practice guidelines for assessing and managing behavioural and psychological symptoms of dementia (BPSD). https://ccsmh.ca/wp-content/uploads/2024/03/CCSMH-BPSD-Clinical-Guidelines_FINAL_March-2024.pdf
Cerejeira, J., Lagarto, L., & Mukaetova-Ladinska, E. B. (2012). Behavioral and psychological symptoms of dementia. Frontiers in Neurology, 3, 73. https://doi.10.3389/fneur.2012.0007
Denton, A. H., Moody, D. A., & Bennett, J. C. (2016). Usability testing as a method to refine a health sciences library website. Medical Reference Services Quarterly, 35(1), 1–15. https://doi.org/10.1080/02763869.2016.1117280
Emde, J. Z., Morris, S. E., & Claassen-Wilson, M. (2009). Testing an academic library website for usability with faculty and graduate students. Evidence Based Library and Information Practice, 4(4), 24–36. https://doi.org/10.18438/B8TK7Q
Gale, P. (2014, September 2). A simple technique for evaluating content. User Research in Government UK. https://userresearch.blog.gov.uk/2014/09/02/a-simple-technique-for-evaluating-content/
Hampshire, N., Califano, G., & Spinks, D. (2022). Reaction card method. Mastering collaboration in a product team (pp. 94–95). Apress. https://doi.org/10.1007/978-1-4842-8254-0_47
Harte, R., Glynn, L., Rodríguez-Molinero, A., Baker, P., Scharf, T., Quinlan, L., & ÓLaighin, G. (2017). A human-centered design methodology to enhance the usability, human factors, and user experience of connected health systems: A three-phase methodology. JMIR Human Factors, 4(1). https://doi.org/10.2196/humanfactors.5443
Hassan, A. (2020). Challenges and recommendations for the deployment of information and communication technology solutions for informal caregivers: Scoping review. JMIR Aging, 3(2), e20310. https://aging.jmir.org/2020/2/e20310
Hill, J., Brown, J., Campbell, N., & Holden, R. (2021). Usability-in-place—Remote usability testing methods for homebound older adults: Rapid literature review. JMIR Formative Research, 5(11). https://doi.org/10.2196/26181
Hyzy, M., Bond, R., Mulvenna, M., Bai, L., Dix, A., Leigh, S., & Hunt, S. (2022). System Usability Scale benchmarking for digital health apps: Meta-analysis. JMIR Mhealth, 10(8). https://mhealth.jmir.org/2022/8/e37290
Isik, A. T., Soysal, P., Solmi, M., & Veronese, N. (2018). Bidirectional relationship between caregiver burden and neuropsychiatric symptoms in patients with Alzheimer’s disease: A narrative review. International Journal of Geriatric Psychiatry, 34(9), 1326–1334. https://doi.org/10.1002/gps.4965
ISO. (2018). ISO 9241-11:2018(en), Ergonomics of human-system interaction—Part 11: Usability: Definitions and concepts. https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en
ISO. (2019). ISO 9241-210:2019(en), Ergonomics of human-system interaction. Part 210: Human-centred design for interactive systems. https://www.iso.org/standard/77520.html
Jagoda, T., Dharmaratne, S., & Rathnayake, S. (2023). Informal carers’ information needs in managing behavioural and psychological symptoms of people with dementia and related mHealth applications: A systemic integrative review to inform the design of an mHealth application. BMJ Open, 13, e06978. https://doi.org/10.1136/bmjopen-2022-069378
Jilani, M. A., Sheikh, A., Shah, F., & Saqlain, S. M. (2025). Usability evaluation of academic library websites: A systematic literature review. Information Discovery and Delivery, 54(1), 50–70. https://doi.org/10.1108/IDD-09-2024-0144
Kales, H. C., Gitlin, L. N., & Lyketsos, C. G. (2015). Assessment and management of behavioural and psychological symptoms of dementia. British Medical Journal, 350, h369. https://doi.org/10.1136/bmj.h369
Kaplan, K. (2023, February 12). When to use empathy maps: 3 options. NNGroup. https://www.nngroup.com/articles/using-empathy-maps/
Kous, K., Pušnik, M., Heričko, M., & Polančič, G. (2018). Usability evaluation of a library website with different end user groups. Journal of Librarianship and Information Science, 52(1), 75–90. https://doi.org/10.1177/0961000618773133
Lewis, J., & Sauro, J. (2018). Item benchmarks for the System Usability Scale. Journal of User Experience, 13(5), 158–167. https://uxpajournal.org/item-benchmarks-system-usability-scale-sus/
Mutatina, B., Basaza, R., Sewankambo, N. K., & Lavis, J. N. (2019). Evaluating user experiences of a clearing house for health policy and systems. Health Information and Libraries Journal, 36(2), 168–178. https://doi.org/10.1111/hir.12257
Neilson, C., & Wilson, V. (2011). “We want it now and we want it easy”: Usability testing of an online health library for healthcare practitioners. Journal of the Canadian Health Libraries Association, 32, 51–59. https://doi.org/10.5596/c11-024
Nielsen, J. (2001, January 20). Usability metrics. NNGroup. https://www.nngroup.com/articles/usability-metrics
Nimmanterdwong, Z., Boonviriya, S., & Tangkijvanich, P. (2022). Human-centered design of mobile health apps for older adults: Systematic review and narrative synthesis. JMIR Mhealth Uhealth, 10(1). https://doi.org/10.2196/29512
Oakley-Girvan, I., Davis, S. W., Kurian, A., Rosas, L. G., Daniels, J., Palesh, O. G., Mesia, R. J., Kamal, A. H., Longmire, M., & Divi, V. (2021). Development of a mobile health app (TOGETHERCare) to reduce care partner burden: Product design study. JMIR Formative Research, 5(8). https://doi.org/10.2196/22608
Okhovati, M., Karami, F., & Khajouei, R. (2017). Exploring the usability of the central library websites of medical sciences universities. Journal of Librarianship and Information Science, 49(3), 246–255. https://doi.org/10.1177/0961000616650932
Polenick, C. A., Struble, L. M., Stanislawski, B., Broderick, B., Gitlin, L. N., & Kales, H. C. (2018). ‘The filter is kind of broken’: Family caregivers’ attributions about behavioural and psychological symptoms of dementia. The American Journal of Geriatric Psychiatry, 26(5), 548–556. https://doi.org/10.1016/j.jagp.2017.12.004
Poll, R. (2007). Evaluating the library website: Statistics and quality measures. Quality Issues in Libraries, 74. http://www.ifla.org/iv/ifla73/index.htm
Poncette, A. K., Mosch, L., Stablo, L., Spies, C., Schieler, M., Weber-Carstens, S., Feufel, M., & Balzer, F. (2021). Human-centered design with usability evaluation of a remote patient monitoring system for intensive care medicine: A mixed-methods study. JMIR Human Factors, 9(10). https://doi.org/10.2196/30655
Reeder, B., Le, T., Thompson, H. J., & Demiris, G. (2013). Comparing information needs of health care providers and older adults: Findings from a wellness study. Studies in Health Technology and Informatics, 192, 18–22. https://pubmed.ncbi.nlm.nih.gov/23920507
Rosala, M. (2022, August 17). How to analyze qualitative data from UX research: Thematic analysis. NNGroup. https://www.nngroup.com/articles/thematic-analysis/
Rutkowski, R. A., Ponnala, S., Younan, L., Weiler, D. T., Bykovskyi, A. G., & Werner, N. E. (2021). A process-based approach to exploring the information behavior of informal caregivers of people living with dementia. International Journal of Medical Informatics, 145, 10431. https://doi.org/10.1016/j.ijmedinf.2020.104341
Sauro, J. (2010, March 2). If you could only ask one question, use this one. MeasuringU. https://measuringu.com/single-question
Sauro, J. (2011). Measuring usability with the System Usability Scale (SUS). MeasuringU. https://measuringu.com/sus/
Schmidt, M., Earnshaw, Y., Tawfik, A. A., & Jahnke, I. (2020). Methods of user centered design and evaluation for learning designers. In M. Schmidt, A. A. Tawfik, I. Jahnke, & Y. Earnshaw (Eds.), Learner and user experience research: An introduction for the field of learning design and technology. EdTech Books.
Schnelli, A., Hirt, J., & Zeller, A. (2020). Persons with dementia as Internet users: What are their needs? A qualitative study. Journal of Clinical Nursing, 30(5-6), 849–860. https://doi.org/10.1111/jocn.15629
Silva, A., Caravau, H., Martins, A., Almeida, A., Silva, T., Ribeiro, Ó., Santinha, G., & Rocha, N. (2021). Procedures of user-centered usability assessment for digital solutions: Scoping review of reviews reporting on digital solutions relevant for older adults. JMIR Human Factors, 8(1). https://doi.org/10.2196/22774
Steinberg, M., Shao, H., Zandi, P., Lyketsos, C. G., Welsh-Bohmer, K. A., Norton, M. C., Breitner, J. C. S., Steffens, D. C., & Tschanz, J. T. (2008). Point and 5-year period prevalence of neuropsychiatric symptoms in dementia: The Cache County study. International Journal of Geriatric Psychiatry, 23(2), 170–177. https://doi.org/10.1002/gps.1858
Statistics Canada. (2021, December 3). How have Canadians been using the Internet during the COVID-19 pandemic? https://www.statcan.gc.ca/o1/en/plus/198-how-have-canadians-been-using-internet-during-covid-19-pandemic
Taylor, H. A., Sullivan, D., Mullen, C., & Johnson, C. M. (2011). Implementation of a user-centered framework in the development of a web-based health information database and call center. Journal of Biomedical Informatics, 44(5), 897–908. https://doi.org/10.1016/j.jbi.2011.03.001
Ubels, N., & Albrecht, L. (2024). Evidence-based principles to accelerate health information flow and uptake among older adults. Evidence Based Library and Information Practice, 19(2), 109–118. https://doi.org/10.18438/eblip30529
Ubels, N., Tan, L., Albrecht, L., & Long, A. (2024). Developing person-centered metadata: A case study of the Behaviours in Dementia Toolkit. Knowledge Organization, 51(7). https://doi.org/10.5771/0943-7444-2024-7-478
van den Haak, M., & van Hooijdonk, C. (2010). Evaluating consumer health information websites: The importance of collecting observational, user-driven data. 2010 IEEE International Professional Communication Conference, 333–338. https://doi.org/10.1109/IPCC.2010.5530031
van Velsen, L., Ludden, G., & Grünloh, C. (2022). The limitations of user-and human-centered design in an eHealth context and how to move beyond them. Journal of Medical Internet Research, 24(10). https://doi.org/10.2196/37341
Wilson, J., Heinsch, M., Betts, D., Booth, D., & Kay-Lambkin, F. (2021). Barriers and facilitators to the use of e-health by older adults: A scoping review. BMC Public Health, 21(1), 1556. https://doi.org/10.1186/s12889-021-11623-w
Wittenberg, E., Kerr, A. M., & Goldsmith, J. (2020). Exploring family caregiver communication difficulties and caregiver quality of life and anxiety. American Journal of Hospice and Palliative Medicine, 38(2), 147–153. https://doi.org/10.1177/1049909120935371
World Health Organization. (2023, March 31). Dementia factsheet. https://www.who.int/news-room/fact-sheets/detail/dementia
Ye, B., Chu, C. H., Bayat, S., Babineau, J., How, T. V., & Mihailidis, A. (2023). Researched apps used in dementia care for people living with dementia and their informal caregivers: Systematic review on app features, security, and usability. JMIR, 25, e46188. https://doi.org/10.2196/46188
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Nick Ubels, Lauren Albrecht, Woon Chee Koh, David Conn
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
The Creative Commons-Attribution-Noncommercial-Share Alike License 4.0 International applies to all works published by Evidence Based Library and Information Practice. Authors will retain copyright of the work.
