The usability assessment of PRiSMA and My.Pharma-C Web Application System
Abstract
Background: The Pharmaceutical Services Program (PSP) has not yet evaluated the usability of the web application system. The System Usability Scale (SUS) and Net Promoter Score (NPS) assessment tools give you a quick overview of your system's usability and help you improve it further. Objective: The goal of this study was to assess the usability of the PRiSMA and My.Pharma-C systems using SUS and NPS. Method: From April 2019 to July 2020, a survey was conducted using a validated self-administered questionnaire that was prompted after each successful log out from the PRiSMA and My.Pharma-C systems. The questionnaire was divided into three sections: demographic, device type, and SUS and NPS questionnaire.The mean SUS and NPS scores were computed. Results: The survey was completed by 3959 PRiSMA users and 811 My.Pharma-C users. The SUS score was interpreted as a grade, adjective, and range of acceptability. The PRiSMA mean SUS score was 70.2 (SD 15.44), indicating a 'Grade C' and 'Good' system. The mean SUS score for My.Pharma-C was 58.2 (SD 15.13), indicating a 'Grade D' and 'OK' system. Users accepted both systems on the basis of their usability. Conclusion: The SUS and NPS values serve as a baseline to highlight the areas that need to be improved in both systems. Therefore, both of the tools can be used to evaluate the usability of the web application systems that managed by the Pharmaceutical Services Program.
References
2. Bevan N, Carter J, Earthy J, Geis T, Harker S. New ISO standards for usability, usability reports and usability measures. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics). 2016;9731(July):268–78.
3. Paz F, Pow-Sang JA. A systematic mapping review of usability evaluation methods for software development process. Int J Softw Eng its Appl. 2016;10(1):165–78.
4. Lewis JR, Sauro J. The factor structure of the system usability scale. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics). 2009;5619 LNCS:94–103.
5. Sauro J, Lewis JR. When Designing Usability Questionnaires , Does It Hurt to Be Positive ? In: Proceedings of the 2011 annual conference on Human factors in computing systems - CHI ’11. 2011. p. 1–9.
6. Brooke J. SUS: A Retrospective. J Usability Stud. 2013;8(2):29–40.
7. Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Int J Hum Comput Interact. 2008;24(6):574–94.
8. Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: adding an adjective rating scale. Determ what Individ SUS scores mean adding an adjective Rat scale. 2009;4(3):114–23.
9. Tullis TS, Stetson JN. A Comparison of Questionnaires for Assessing Website Usability. Usability Prof Assoc Conf. 2004;(June 2006):1–12.
10. Sauro J. Measuring the Quality of the Website User Experience [Internet]. Vol. 1166, Electronic Theses and Dissertations. University of Denver Digital; 2016. Available from: https://digitalcommons.du.edu/etd/1166
11. Sauro J. DOES BETTER USABILITY INCREASE CUSTOMER LOYALTY? [Internet]. 2010 [cited 2021 May 3]. Available from: https://measuringu.com/usability-loyalty/
12. Hamilton DF, Lane J V., Gaston P, Patton JT, MacDonald DJ, Simpson AHRW, et al. Assessing treatment outcomes using a single question: The Net Promoter Score. Bone Jt J. 2014;
13. Hassan Basri N, Wan Adnan WA, Baharin H. System Usability Scale Evaluation of E-Participation in Malaysia [Internet]. Vol. 1032, Communications in Computer and Information Science. Springer International Publishing; 2019. 3–8 p. Available from: http://dx.doi.org/10.1007/978-3-030-23522-2_1
14. Pradini RS, Kriswibowo R, Ramdani F. Usability Evaluation on the SIPR Website Uses the System Usability Scale and Net Promoter Score. Proc 2019 4th Int Conf Sustain Inf Eng Technol SIET 2019. 2019;280–4.
15. M.Y. MH, A.S. SS, N. N. Pharmacist Registration Management System (PRiSMA): Is the system user friendly? In 2019.
16. Pharmaceutical Services programme. Laporan Tahunan Program Farmasi Pharmacy Programme Annual Report 2019. 2019.
17. Sauro J. 5 Ways To Interpret A SUS Score [Internet]. 19 September 2018. 2018 [cited 2019 Mar 10]. Available from: https://measuringu.com/interpret-sus-score/
18. Brooke J. SUS - A quick and dirty usability scale. In: Patrick W. Jordan, B. Thomas, Ian Lyall McClelland BW, editor. Usability Evaluation In Industry. CRC Press; 1996. p. 189–93.
19. Raosoft Sample Size Calculator [Internet]. [cited 2021 Jun 21]. Available from: http://www.raosoft.com/samplesize.html
20. IBM Corp. IBM SPSS Statistics for Windows. New York, NY: IBM Corp.; 2012.
21. Kortum P. How low can you go?: is the system usability scale range restricted? How low can you go? is Syst usability scale range restricted? 2013;9(1):14–24.
22. Lewis JR, Sauro J. Item Benchmarks for the System Usability Scale. J Usability Stud. 2018;13(January):158–67.
23. Ganapathy M, Shuib M, Azizan SN. Malaysian ESL students’ perceptions on the usability of a mobile application for grammar test: A case study of ESL undergraduates in universiti sains Malaysia. 3L Lang Linguist Lit. 2016;22(1):127–40.
24. Sasmito GW, Zulfiqar LOM, Nishom M. Usability Testing based on System Usability Scale and Net Promoter Score. 2019 2nd Int Semin Res Inf Technol Intell Syst ISRITI 2019. 2019;540–5.
25. Mujinga M, Eloff MM, Kroeze JH. System usability scale evaluation of online banking services: A South African study. S Afr J Sci. 2018;114(3–4).
26. Kortum P, Sorber M. Measuring the Usability of Mobile Applications for Phones and Tablets. Int J Hum Comput Interact. 2015;31(8):518–29.
27. Pal D, Vanijja V. Perceived usability evaluation of Microsoft Teams as an online learning platform during COVID-19 using system usability scale and technology acceptance model in India. Child Youth Serv Rev [Internet]. 2020;119(July):105535. Available from: https://doi.org/10.1016/j.childyouth.2020.105535
28. Kortum PT, Bangor A. Usability Ratings for Everyday Products Measured With the System Usability Scale. Int J Hum Comput
