Normalizing Scientometric Indicators of Individual Publications Using Local Cluster Detection Methods on Citation Networks

Authors: Levente Varga, Dávid Deritei, Mária Ercsey-Ravasz, Răzvan Florian, Zsolt I. Lázár, István Papp, Ferenc Járai-Szabó

Abstract:

One of the major shortcomings of widely used scientometric indicators is that different disciplines cannot be compared with each other. The issue of cross-disciplinary normalization has been long discussed, but even the classification of publications into scientific domains poses problems. Structural properties of citation networks offer new possibilities, however, the large size and constant growth of these networks asks for precaution. Here we present a new tool that in order to perform cross-field normalization of scientometric indicators of individual publications relays on the structural properties of citation networks. Due to the large size of the networks, a systematic procedure for identifying scientific domains based on a local community detection algorithm is proposed. The algorithm is tested with different benchmark and real-world networks. Then, by the use of this algorithm, the mechanism of the scientometric indicator normalization process is shown for a few indicators like the citation number, P-index and a local version of the PageRank indicator. The fat-tail trend of the article indicator distribution enables us to successfully perform the indicator normalization process.

Keywords: Citation networks, scientometric indicator, cross-field normalization, local cluster detection.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1474565

References:

[1] Garfield, E. (1998). The Impact Factor and Using It Correctly. Der Unfallchirurg, 101(6), 413–414.
[2] Bergstrom, C. T., West, J. D., Wiseman, M. A. (2008). The eigenfactor metrics. Journal of Neuroscience, 28(45), 11433–11434.
[3] Davis P. M. (2008). Eigenfactor: Does the principle of repeated improvement result in better estimates than raw citation counts. JASIST, 59(13), 2186–2188.
[4] Bollen, J., Rodriguez, M. A., Van de Sompel, H. (2006). Journal status. Scientometrics, 69(3), 669–687.
[5] Hirsch, J. E. (2005). An index to quantify an inidividual‘s scientific research output. PNAS, 102(46), 16569–16572.
[6] Schubert, A., Braun, T. (1996). Cross-field normalization of scientometric indicators. Scientometrics, 36(3), 311–324.
[7] Radicchi, F., Fortunato, S., Castellano, C. (2008). Universality of citation distributions: Toward an objective measure of scientific impact. PNAS, 105(45), 17268–17272.
[8] Waltman, L., van Eck, N. J. (2013). Source normalized indicators of citation impact: an overview of different approaches and an empirical comparison, Scientometrics, 96, 699, https://doi.org/10.1007/s11192-012-0913-4.
[9] Bouyssou, D., Marchant, T. (2016). Ranking authors using fractional counting of citations: An axiomatic approach. Journal of Informetrics, 10(1), 183–199, https://doi.org/10.1016/j.joi.2015.12.006.
[10] Zitt, M., Small, H. (2008) Modifying the journal impact factor by fractional citation weighting: The audience factor. J. Am. Soc. Inf. Sci., 59(11), 1856–1860, http://dx.doi.org/10.1002/asi.20880.
[11] Kostoff, R. N. (1997) Citation analysis cross-field normalization: a new paradigm. Scientometrics., 39(3), 225-230.
[12] Seglen, P. O. (1997). Why the impact factor of journals should not be used for evaluating research. BMJ, 314(7079), 498–502.
[13] Opthof, T. (1997). Sense and nonsense about the impact factor. Cardiovascular Research, 33(1), 1–7.
[14] Bornmann, L., Daniel, H.-D. (2008). What do citation counts measure? A review of studies on citing behavior. Journal of Documentation, 64(1), 45–80.
[15] Web of Science. http://www.webofknowledge.com, Accessed on 07/05/2018.
[16] Leydesdorff, L., Wagner, C. S., Bornmann, L. (2017). Betweenness and diversity in journal citation networks as measures of interdisciplinarity A tribute to Eugene Garfield. Scientometrics. https://doi.org/10.1007/s11192-017-2528-2.
[17] Page, L., Brin, S., Motwani, R., Winograd, T. (1999). The PageRank citation ranking: Bringing order to the Web, Technical Report 1999-66, Stanford InfoLab, November 1999. http://ilpubs.stanford.edu:8090/422/. Accesed 21 August 2013.
[18] Papp, I., Ercsey-Ravasz, M., Deritei, D., Sumi, R., J´arai-Szab´o, F., Florian, R. V., Cabuz, A. I., L´az´ar, Zs.I. (2013). The P-Index: Hirsch Index of Individual Publications. Proceedings of ISSI, 2013, 2086–2088.
[19] Waltman, L., van Eck, N. J. (2012), A new methodology for constructing a publicationlevel classification system of science, J. Am. Soc. Inf. Sci. Technol., 63, 2378–2392.
[20] Ruiz-Castilloa, J., Waltman, L. (2015), Field-normalized citation impact indicators using algorithmically constructed classification systems of science, Journal of Informetrics, 9, 102–117.
[21] ˘ Subelj, L., van Eck, N. J., Waltman, L. (2016), Clustering Scientific Publications Based on Citation Relations: A Systematic Comparison of Different Methods. PLOS ONE, 11, e0154404.
[22] van Eck, N. J., Waltman, L. (2017) Citation-based clustering of publications using CitNetExplorer and VOSviewer. Scientometrics, 111, 1053-1070.
[23] Newman, M. E. J. (2001). The structure of scientific collaboration networks. Proc. Natl. Acad. Sci. USA, 98(2), 404–409.
[24] Newman, M. E. J., Girvan, M. (2004). Finding and evaluating community structure in networks. Phys. Rev. E, 69, 026113.
[25] Castellano, C., Fortunato, S., Loreto, V. (2009). Statistical physics of social dynamics. Rev. Mod. Phys., 81, 591.
[26] Bagrow, J. P., Bollt, E. M. (2005). Local method for detecting communities. Phys. Rev. E, 72(4), 046108.
[27] Radicchi, F., Castellano, C., Cecconi, F., Loreto, V., Parisi, D. (2004). Defining and identifying communities in networks. PNAS, 101, 2658–2663.
[28] Deritei, D., Lazar, Zs.I., Papp I., Jarai-Szabo, F., Sumi, R., Varga, L., Regan, E., Ercsey-Ravasz, M. (2014). Community detection by graph Voronoi diagrams. New Journal of Physics, 16, 063007.
[29] Lancichinetti, A., Fortunato, S., Radicchi, F. (2008). Benchmark graphs for testing community detection algorithms. Phys. Rev. E, 78, 046110.
[30] Bastian, M., Heymann, S., Jacomy, M. (2009). Gephi: an open source software for exploring and manipulating networks. ICWSM, 8, 361–362.
[31] Adamic, L. A., Glance, N. (2005). The political blogosphere and the 2004 US Election. Proceedings of the WWW-2005 Workshop on the Weblogging Ecosystem.
[32] Durieux, V., Gevenois, P. A. (2010). Bibliometric indicators: quality measurements of scientific publication. Radiology, 255(2), 342–351.
[33] Hargens, L. L. (2000). Using the literature: reference networks, reference contexts, and the social structure of scholarship. Am Sociol Rev, 65, 846–865.
[34] Van Raan, A. F. J. (2006). Statistical Properties of Bibliometric Indicators: Research Group Indicator Distributions and Correlations. J. Am. Soc. Inf. Sci. Tec., 57, 408–430.
[35] Hutchins, B. I., Yuan, X., Anderson, J. M., Santangelo, G. M. (2016). Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level. PLOS Biology., 14, e1002541. https://doi.org/10.1371/journal.pbio.1002541.
[36] Gonzalez-Betancor, S. M., Dorta-Gonzalez, P. (2017). An indicator of the impact of journals based on the percentage of their highly cited publications. Online Information Review, 41, 398–411.
[37] Tsallis, C., De Albuquerque, M. P. (2000). Are citations of scientific papers a case of nonextensivity? Eur. Phys. J. B, 13, 777–780.
[38] Lehmann, S., Lautrup, B., Jackson, A. D. (2003). Citation networks in high energy physics. Phys. Rev. E, 68, 026113.
[39] Brzezinski, M. (2015). Power laws in citation distributions: evidence from Scopus. Scientometrics, 103, 213–228.
[40] Blondel, V. D., Guillaume, J.-L., Lambiotte, R., Lefebvre E. (2008). Fast unfolding of communities in large networks. J. Stat. Mech., P10008.