Analyzing the Websites of Institutions Publishing Global Rankings of Universities: A Usability Study

Authors: Nuray Baltaci, Kursat Cagiltay

Abstract:

University rankings which can be seen as nouveau topic are at the center of focus and followed closely by different parties. Students are interested in university rankings in order to make informed decisions about the selection of their candidate future universities. University administrators and academicians can utilize them to see and evaluate their universities’ relative performance compared to other institutions in terms of including but not limited to academic, economic, and international outlook issues. Local institutions may use those ranking systems, as TUBITAK (The Scientific and Technological Research Council of Turkey) and YOK (Council of Higher Education) do in Turkey, to support students and give scholarships when they want to apply for undergraduate and graduate studies abroad. When it is considered that the ranking systems are concerned by this many different parties, the importance of having clear, easy to use and well-designed websites by ranking institutions will be apprehended. In this paper, a usability study for the websites of four different global university ranking institutions, namely Academic Ranking of World Universities (ARWU), Times Higher Education, QS and University Ranking by Academic Performance (URAP), was conducted. User-based approach was adopted and usability tests were conducted with 10 graduate students at Middle East Technical University in Ankara, Turkey. Before performing the formal usability tests, a pilot study had been completed to reflect the necessary changes to the settings of the study. Participants’ demographics, task completion times, paths traced to complete tasks, and their satisfaction levels on each task and website were collected. According to the analyses of the collected data, those ranking websites were compared in terms of efficiency, effectiveness and satisfaction dimensions of usability as pointed in ISO 9241-11. Results showed that none of the selected ranking websites is superior to other ones in terms of overall effectiveness and efficiency of the website. However the only remarkable result was that the highest average task completion times for two of the designed tasks belong to the Times Higher Education Rankings website. Evaluation of the user satisfaction on each task and each website produced slightly different but rather similar results. When the satisfaction levels of the participants on each task are examined, it was seen that the highest scores belong to ARWU and URAP websites. The overall satisfaction levels of the participants for each website showed that the URAP website has highest score followed by ARWU website. In addition, design problems and powerful design features of those websites reported by the participants are presented in the paper. Since the study mainly tackles about the design problems of the URAP website, the focus is on this website. Participants reported 3 main design problems about the website which are unaesthetic and unprofessional design style of the website, improper map location on ranking pages, and improper listing of the field names on field ranking page.

Keywords: university ranking, user-based approach, website usability, design