Towards a Broader Understanding of Journal Impact: Measuring Relationships between Journal Characteristics and Scholarly Impact

Authors: X. Gu, K. L. Blackmore

Abstract:

The impact factor was introduced to measure the quality of journals. Various impact measures exist from multiple bibliographic databases. In this research, we aim to provide a broader understanding of the relationship between scholarly impact and other characteristics of academic journals. Data used for this research were collected from Ulrich’s Periodicals Directory (Ulrichs), Cabell’s (Cabells), and SCImago Journal & Country Rank (SJR) from 1999 to 2015. A master journal dataset was consolidated via Journal Title and ISSN. We adopted a two-step analysis process to study the quantitative relationships between scholarly impact and other journal characteristics. Firstly, we conducted a correlation analysis over the data attributes, with results indicating that there are no correlations between any of the identified journal characteristics. Secondly, we examined the quantitative relationship between scholarly impact and other characteristics using quartile analysis. The results show interesting patterns, including some expected and others less anticipated. Results show that higher quartile journals publish more in both frequency and quantity, and charge more for subscription cost. Top quartile journals also have the lowest acceptance rates. Non-English journals are more likely to be categorized in lower quartiles, which are more likely to stop publishing than higher quartiles. Future work is suggested, which includes analysis of the relationship between scholars and their publications, based on the quartile ranking of journals in which they publish.

Keywords: Academic journal, acceptance rate, impact factor, journal characteristics.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1132304

References:

[1] Garfield E. The history and meaning of the journal impact factor. JAMA: the journal of the American Medical Association. 2006;295(1):90-3.
[2] Starbuck WH. How much better are the most-prestigious journals? The statistics of academic publication. Organization Science. 2005;16(2):180-200.
[3] Jacsó P. A deficiency in the algorithm for calculating the impact factor of scholarly journals: the journal impact factor. Cortex. 2001;37(4):590-4.
[4] Zuckerman H, Merton RK. Patterns of evaluation in science: Institutionalisation, structure and functions of the referee system. Minerva. 1971;9(1):66-100.
[5] Hargens LL. Scholarly consensus and journal rejection rates. American Sociological Review. 1988:139-51.
[6] Faria JR. The game academics play: Editors versus authors. Bulletin of Economic research. 2005;57(1):1-12.
[7] Björk B-C, Welling P, Laakso M, Majlender P, Hedlund T, Guðnason G. Open access to the scientific journal literature: situation 2009. PloS one. 2010;5(6):e11273.
[8] Gargouri Y, Hajjem C, Larivière V, Gingras Y, Carr L, Brody T, et al. Self-selected or mandated, open access increases citation impact for higher quality research. PloS one. 2010;5(10):e13636.
[9] Antelman K. Do open-access articles have a greater research impact? College & research libraries. 2004;65(5):372-82.
[10] Harnad S, Brody T. Comparing the impact of open access (OA) vs. non-OA articles in the same journals. D-lib Magazine. 2004;10(6).
[11] Björk B-C, Solomon D. Open access versus subscription journals: a comparison of scientific impact. BMC medicine. 2012;10(1):73.
[12] Gu X, Blackmore K. Characterisation of Academic Journals in the Digital Age. Scientometrics. forthcoming.
[13] Scopus. SCImago Journal & Country Rank - Journal Rankings. Available from: http://www.scimagojr.com/journalrank.php. Accessed on: 15 April 2016.
[14] Thomson Reuters. Cabell's International. Available from: http://cabells.com/index.aspx. Accessed on: 1 December 2015.
[15] Ulrich's Periodicals Directory Ulrich's has been a global source of periodicals information since 1932. Available from: http://www.ulrichsweb.com. Accessed on: 15 January 2016.
[16] Research Assessment. Journal Citation Ranking and Quartile Scores. Available from: https://researchassessment.fbk.eu/quartile_score. Accessed on: 26 June 2016.
[17] Maher J. The development of English as an international language of medicine. Applied linguistics. 1986;7(2):206-18.
[18] Saha S, Saint S, Christakis DA. Impact factor: a valid measure of journal quality? Journal of the Medical Library Association. 2003;91(1):42.
[19] Gu X, Blackmore KL. Recent trends in academic journal growth. Scientometrics. 2016:1-24.