Post Pandemic Mobility Analysis through Indexing and Sharding in MongoDB: Performance Optimization and Insights

Authors: Karan Vishavjit, Aakash Lakra, Shafaq Khan

Abstract:

The COVID-19 pandemic has pushed healthcare professionals to use big data analytics as a vital tool for tracking and evaluating the effects of contagious viruses. To effectively analyse huge datasets, efficient NoSQL databases are needed. The analysis of post-COVID-19 health and well-being outcomes and the evaluation of the effectiveness of government efforts during the pandemic is made possible by this research’s integration of several datasets, which cuts down on query processing time and creates predictive visual artifacts. We recommend applying sharding and indexing technologies to improve query effectiveness and scalability as the dataset expands. Effective data retrieval and analysis are made possible by spreading the datasets into a sharded database and doing indexing on individual shards. Analysis of connections between governmental activities, poverty levels, and post-pandemic wellbeing is the key goal. We want to evaluate the effectiveness of governmental initiatives to improve health and lower poverty levels. We will do this by utilising advanced data analysis and visualisations. The findings provide relevant data that support the advancement of UN sustainable objectives, future pandemic preparation, and evidence-based decision-making. This study shows how Big Data and NoSQL databases may be used to address problems with global health.

Keywords: COVID-19, big data, data analysis, indexing, NoSQL, sharding, scalability, poverty.

References:

[1] Edouard Mathieu, Hannah Ritchie, Lucas Rode´s-Guirao, Cameron Appel, Charlie Giattino, Joe Hasell, Bobbie Macdonald, Saloni Dattani, Diana Beltekian, Esteban Ortiz-Ospina and Max Roser (2020) “Coronavirus Pandemic (COVID-19)”. Published online at OurWorldInData.org. Retrieved from: https://ourworldindata.org/coronavirus’ (Online Resource).
[2] Google. “COVID-19 Community Mobility Report.” COVID-19 Community Mobility Report, 2020, www.google.com/covid19/mobility/.
[3] “Database Sharding: Concepts & Examples.” MongoDB, 2022, www.mongodb.com/features/database-sharding-explained.
[4] R. Chopade and V. Pachghare, “MongoDB Indexing for Performance Improvement,” Advances in Intelligent Systems and Computing, pp. 529–539, 2020, doi: https://doi.org/10.1007/978-981-15-0936-0 56.
[5] A. Gomes et al., “An Empirical Performance Comparison between MySQL and MongoDB on Analytical Queries in the COMEX Database,” 2021 16th Iberian Conference on Information Systems and Technologies (CISTI), Jun. 2021, doi: https://doi.org/10.23919/cisti52073.2021.9476623.
[6] J. Antas, R. Rocha Silva, and J. Bernardino, “Assessment of SQL and NoSQL Systems to Store and Mine COVID-19 Data,” Computers, vol. 11, no. 2, p. 29, Feb. 2022, doi: https://doi.org/10.3390/computers11020029.
[7] “Login - University of Windsor,” brightspace.uwindsor.ca. https://brightspace.uwindsor.ca/content/enforced/139690-COMP8157-4- R-2023S/csfiles/home dir/courses/COMP81572-R-2022S/COMP8157-2- R-2022S/COMP8157-1-R-2022S/ADT Project Final Report%20(1).pdf? &d2lSessionVal=VPAtNyL0RTd65CzrniVjWrKCK&ou=139690 (accessed Jun. 25, 2023).
[8] “Deploy Sharded Cluster Using Ranged Sharding — MongoDB Manual.” Https://Github.com/Mongodb/Docs/Blob/V3.2/Source/Tutorial/ Deploy-Sharded-Cluster-Ranged-Sharding.txt, www.mongodb.com/docs/v3.2/tutorial/deploy-sharded-cluster-ranged- sharding/. Accessed 31 July 2023.
[9] Fan Zuo. Jingxing Wang, Jingqin Gao, Kaan Ozbay, Xuegang Jeff Ban, Yubin Shen, Hong Yang, Shri Iyer, “An Interactive Data Visualization and Analytics Tool to Evaluate Mobility and Sociability Trends During COVID-19.” https://arxiv.org/pdf/2006.14882.pdf
[10] Indexes — MongoDB Manual,” www.mongodb.com. https://www.mongodb.com/docs/manual/indexes/