Text Mining Algorithm for Large-Scale Social Media Data

Authors: Alexander A. Kharlamov, Maria Pilgun

Abstract:

This paper presents the validation results of a text mining algorithm applied to urban system development in the transportation sector, leveraging user-generated content from social media platforms. The study employed sentiment analysis, aggression detection, semantic network and core formation, associative search, associative network development, and word association analysis. Data collection was conducted using the Brand Analytics social media and media monitoring system. Data analysis and interpretation utilized TextAnalyst 2.32, GPT-3.5, GPT-4, and GPT-4o, while Tableau was used for interactive visualization and analytics. Social tension levels were assessed through calculated indices of social stress and well-being. Based on the findings, recommendations were proposed to improve project effectiveness by integrating residents' perspectives.

Keywords: Social media, text mining, neural network technologies, large-scale data.

References:

[1] U. Qamar, and M. S. Raza, Applied Text Mining. Cham: Springer Cham, 2024.
[2] A. Sharan, N. Malik, H. Imran, and I. Ghosh, Text Mining Approaches for Biomedical Data. Singapore: Springer Singapore, 2024.
[3] F. Groß, Eine Analyse der Anwendungspotenziale und Entwicklung eines Integrationskonzepts. Wiesbaden: Springer Gabler Wiesbaden, 2022.
[4] M. Lamba, M. Madhusudhan, Text Mining for Information Professionals. An Uncharted Territory. Cham: Springer Cham, 2022.
[5] K. Ning, “Methodologies of Multi-Omics Data Integration and Data Mining. Techniques and Applications,” in TRBIO, vol. 19. Singapore: Springer Singapore, 2024, pp. 119-144.
[6] Y. Tan, and Y. Shi, (Eds.) Data Mining and Big Data. 8th International Conference, DMBD 2023, Proceedings, Part I, II. Singapore: Springer Singapore, 2024.
[7] D.-N. Yang, Х. Xie, V. S. Tseng, J. Pei, J.-W. Huang, and C.-W. Lin (Eds.) Advances in Knowledge Discovery and Data Mining.28th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2024, Proceedings, Part II. LNCS, vol.14646. Singapore: Springer Singapore, 2024.
[8] D.L. Olson, and G. Lauhoff, Deskriptives Data-Mining. Springer Gabler. Cham: Springer Cham, 2023.
[9] X. Yang, H. Suhartanto, G. Wang, B. Wang, J. Jiang, B. Li, H. Zhu, and N. Cui (Eds.), Advanced Data Mining and Applications. 19th International Conference, ADMA 2023, Proceedings, Part I. LNCS, vol. 14176, Singapore: Springer Singapore, 2023.
[10] S. Godleski, A. Kothari, and D. Haraden, “The Impact of Social Media Use on Attitudes Toward Mental Healthcare: The Role of Emotional Connectedness and Routine Integration of Social Media”, J. technol. behav. sci., vol. 9, 2024, pp. 306–312.
[11] D. Liu, B. He, R. Feng, X. Huang, and L. Gaoqiang, “How social media sharing drives consumption intention: the role of social media envy and social comparison orientation”, BMC Psychol., vol. 12, pp. 157-165, 2024.
[12] B. Hemsley, F. Given, B. Almond, J. Cooper, R. Smith, K. Young, and E. Wright, “A Critical Review of Literature on Social Media and Developmental Communication Disability: Implications for Future Social Media and Generative AI,” Research. Curr Dev Disord Rep, vol. 11, 2024, pp. 11, 75–89.
[13] I.W. Metzger, M. Jernigan-Noesi, S.C.T. Jones, E.A. Turner, F. Saleem, J. Jackson, R. E. Anderson, L.A. Bartolomeo, P. Tineo, and I. Opara, “Measuring Social Media and Mass Media Impact for Promotion and Tenure in Social and Applied Sciences," High Educ Policy, February 2024.
[14] A.A. Kharlamov, A.N. Raskhodchikov, and M. Pilgun, “Smart City Data Sensing during COVID-19: Public Reaction to Accelerating Digital Transformation,” Sensors, vol. 21(12), 2021, pp. 3965-3981.
[15] A.A. Kharlamov, and M. Pilgun, “Perception of the Situation: Social Stress and Well-Being Indices,” Lecture Notes in Networks and Systems, vol. 544, 2023, pp. 778-790.