The Correlation between Users’ Star Rating and Usability on Mobile Applications

Authors: Abdulmohsen A. AlBesher, Richard T. Stone

Abstract:

Star rating for mobile applications is a very useful way to differentiate between the best and worst rated applications. However, the question is whether the rating reflects the level of usability or not. The aim of this paper is to find out if the user’ star ratings on mobile apps correlate with the usability of those apps. Thus, we tested three mobile apps, which have different star ratings: low, medium, and high. Participating in the study, 15 mobile phone users were asked to do one single task for each of the three tested apps. After each task, the participant evaluated the app by answering a survey based on the System Usability Scale (SUS). The results found that there is no major correlation between the star rating and the usability. However, it was found that the task completion time and the numbers of errors that may happen while completing the task were significantly correlated to the usability.

Keywords: Mobile applications, SUS, Star rating, Usability.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1130171

References:

[1] Bangor, Aaron, Philip T. Kortum, and James T. Miller. "An empirical evaluation of the system usability scale." Intl. Journal of Human–Computer Interaction 24.6 (2008): 574-594.
[2] Bangor, Aaron, Philip Kortum, and James Miller. "Determining what individual SUS scores mean: Adding an adjective rating scale." Journal of usability studies 4.3 (2009): 114-123.
[3] Borsci, Simone, Stefano Federici, and Marco Lauriola. "On the dimensionality of the System Usability Scale: a test of alternative measurement models." Cognitive processing 10.3 (2009): 193-197.
[4] Lewis, J.R., and Sauro, J. The Factor Structure of the System Usability Scale. In Proc. HCII 2009, Springer-Verlag (2009), 94-103.
[5] Mendoza, V., & Novick, D. G. (2005, September). Usability over time. In Proceedings of the 23rd annual international conference on Design of communication: documenting & designing for pervasive information (pp. 151-158). ACM.
[6] Morris, M. G., & Dillon, A. (1997). The influence of user perceptions on software utilization: application and evaluation of a theoretical model of technology acceptance.
[7] Raptis, Dimitrios, et al. "Does size matter? Investigating the impact of mobile phone screen size on users' perceived usability, effectiveness and efficiency." Proceedings of the 15th international conference on Human-computer interaction with mobile devices and services. ACM, 2013.
[8] Turel, O., & Serenko, A. (2006). Satisfaction with mobile services in Canada: An empirical investigation. Telecommunications Policy, 30(5), 314-331.