The Qualitative Methodology Exposure and Experiences of Journal Reviewers: A Qualitative Exploration

Authors: Salomé Elizabeth Scholtz

Abstract:

Reviewers are the gatekeepers of knowledge dissemination and promote the scientific validity of the research. However, authors often receive questionable feedback on qualitative manuscripts. Thus, this qualitative descriptive study sought to explore the qualitative knowledge and experiences of reviewers of psychology journals. A purposive and snowball sample (n = 27) of psychology journal reviewers completed an online questionnaire, and data were analysed using thematic analysis. Reviewers felt their postgraduate education, reading, and the process of reviewing qualitative articles equipped them to review qualitative manuscripts. Less than half of the reviewer’s published articles were qualitative and male reviewers published more than females. Despite not expecting authors to have the same level of research skills, reviewers still experienced authors as unskilled and biased, creating difficulty in accepting and reviewing qualitative articles. The applicability of the qualitative method and recommendations in preparing qualitative manuscripts for reviewing are reported.

Keywords: Journal reviewers, psychology, qualitative research, research method, research skills.

References:

[1] Ivankova, N. V., Creswell, J. W., & Plano Clark, V. L. (2016). Foundations and approaches to mixed methods research. In Maree, K. (Ed.), First steps in research (pp. 306-335). Van Schaick Publishers
[2] Dirette, D. P. (2020). How to be a good peer reviewer. Open Journal of Occupational Therapy, 8(2), 1-8. DOI: 10.15453/2168-6408.1720
[3] Bajwa, N., Langer, M., König, C. J., & Honecker, H. (2019). What might get published in management and applied psychology? Experimentally manipulating implicit expectations of reviewers regarding hedges. Scientometrics, 120, 1351-1371. https://doi.org/10.1007/s11192-019-03164-2
[4] Mouton, J., & Valentine, A. (2017). The extent of South African authored articles in predatory journals. South African Journal of Science, 113(7/8). http://dx.doi.org/10.17159/sajs.2017/20170010
[5] Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2-17. https://doi.org/10.1002/asi.22784
[6] Bol, L., & Hacker, D. J. (2014). Publishing in high quality journals: Perspectives from overworked and unpaid reviewers. Journal of Computing in Higher Education, 26(1), 39-53.
[7] Gergen, K. J., Josselson, R., & Freeman, M. (2015). The promises of qualitative inquiry. American Psychologist, 70, 1-9. http://dx.doi.org/10.1037/ a0038597
[8] Nieuwenhuis, J. (2016a). Introducing qualitative research. In K. Maree (Ed.), First steps in research (pp.50-69). Van Schaick Publishers.
[9] Levitt, H. M., Motulsky, S. L., Wertz, F. J., Morrow, S. L., & Ponterotto, J. G. (2017). Recommendations for designing and reviewing qualitative research in psychology: Promoting methodological integrity. Qualitative psychology, 4(1), 2.
[10] Gergen, K. J., Josselson, R., & Freeman, M. (2015). The promises of qualitative inquiry. American Psychologist, 70, 1-9. http://dx.doi.org/10.1037/ a0038597
[11] Lee, Y. A. (2014). Insight for writing a qualitative research paper. Family & Consumer Sciences Research Journal, 43(1), 94-97.
[12] DeVaney, S. A., Spangler, A., Lee, Y. A., & Delgadillo, L. (2018). Tips from the experts on conducting and reviewing qualitative research. Family and Consumer Sciences Research Journal, 46(4), 396-405. DOI: 10.1111/fcsr.12264
[13] Bluhm, D. J., Harman, W., Lee, T.W., &Mitchell, T. R. (2011). Qualitative research in management: A decade of progress. Journal of Management Studies, 48, 1866-1891.
[14] Zickar, M. J. (2016). Qualitative Researchers, Heal (and Help) Thyself Too. Industrial and Organisational Psychology, 9(4), 716-719.
[15] Herber, O. R., Bradbury-Jones, C., Böling, S., Combes, S., Hirt, J., Koop, Y., ... & Taylor, J. (2020). What feedback do reviewers give when reviewing qualitative manuscripts? A focused mapping review and synthesis. BMC medical research methodology, 20(1), 1-15.
[16] Lyons, H. Z., Bike, D. H., Johnson, A., & Bethea, A. (2012). Culturally competent qualitative research with people of African descent. Journal of Black Psychology, 38(2), 153-171.
[17] Tamminen, K. A., & Poucher, Z. A. (2018). Open science in sport and exercise psychology: Review of current approaches and considerations for qualitative inquiry. Psychology of Sport and Exercise, 36, 17-28.
[18] Coetzee, M., & Van Zyl, L. E. (2014). A review of a decade’s scholarly publications (2004–2013) in the South African Journal of Industrial Psychology. South African Journal of Industrial Psychology, 40, 1-16. http://dx.doi.org/10.4102/sajip.v40i1.1227
[19] Scholtz, S. E., de Klerk, W., & de Beer, L. T. (2020). The use of research methods in psychological research: A systematised review. Frontiers in research metrics and analytics, 5, 1.
[20] O'Neil, S., & Koekemoer, E. (2016). Two decades of qualitative research in Psychology, Industrial and Organisational Psychology and Human Resource Management within South Africa: A critical review. South African Journal of Industrial Psychology, 42(1), 1-16.
[21] Horbach, S. P. J. M., & Halffman, W. (2018). The changing forms and expectations of peer review. Res Integr Peer Rev., 3(8). doi: 10.1186/s41073-018-0051-5
[22] Maree, K., & Pietersen, J. (2016a). Surveys and the use of questionnaires. In K. Maree (Ed.), First steps in research (pp. 172-180). Van Schaik Publishers.
[23] Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). Sage
[24] Doyle, L., McCabe, C., Keogh, B., Brady, A., & McCann, M. (2020). An overview of the qualitative descriptive design within nursing research. Journal of Research in Nursing, 25(5), 443–455. https://doi.org/10.1177/1744987119880234
[25] Sandelowski, M. (2010). What's in a name? Qualitative description revisited. Research in nursing & health, 33(1), 77-84.
[26] Patton, M. Q. (2014). Qualitative research and evaluation methods: integrating theory and practice (4th ed.). Sage
[27] Kaushik, V., & Walsh, C. A. (2019). Pragmatism as a research paradigm and its implications for social work research. Social Sciences, 8(9), 255.
[28] Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77-101. doi:10.1191/1478088706qp063oa
[29] Gurung, R. A., & Stoa, R. (2020). A national survey of teaching and learning research methods: Important concepts and faculty and student perspectives. Teaching of Psychology, 47(2), 111-120. doi/pdf/10.1177/0098628320901374
[30] Marks, L. D. (2015). A pragmatic, step-by-step guide for qualitative methods: Capturing the disaster and long-term recovery stories of Katrina and Rita. Current Psychology 34, 3, 494–504.
[31] Marks, L. D., Kelley, H. H., & Galbraith, Q. (2022). Explosion or much ado about little?: a quantitative examination of qualitative publications from 1995-2017. Qualitative Research in Psychology, 19(3), 853-871. https://doi-org.nwulib.nwu.ac.za/10.1080/14780887.2021.1917740
[32] Reay, T. (2014). Publishing qualitative research. Family Business Review, 27(2), 95-102.
[33] Lincoln, Y. S., & Guba, E. (1985). Naturalistic enquiry. Sage.
[34] Jarzabkowski, P., Langley, A., & Nigam, A. (2021). Navigating the tensions of quality in qualitative research. Strategic Organization, 19(1), 70-80. https://journals.sagepub.com/doi/pdf/10.1177/1476127020985094
[35] Bledsoe, R. S., Richardson, D. S., & Kalle, A. (2021). Student Perceptions of Great Teaching: A Qualitative Analysis. Journal of the Scholarship of Teaching and Learning, 21(3), 21-32.
[36] McSweeney, B. (2021). Fooling ourselves and others: confirmation bias and the trustworthiness of qualitative research – Part 2 (cross-examining the dismissals). Journal of Organizational Change Management, 34(5), 841-859. https://doi-org.nwulib.nwu.ac.za/10.1108/JOCM-04-2021-0118
[37] King, E. B., Avery, D. R., Hebl, M. R., & Cortina, J. M. (2018). Systematic subjectivity: how subtle biases infect the scholarship review process. Journal of Management, 44, 843-853.
[38] Wiggins, B. J., & Christopherson, C. D. (2019). The replication crisis in psychology: An overview for theoretical and philosophical psychology. Journal of Theoretical and Philosophical Psychology, 39(4), 202.