The Role of Synthetic Data in Aerial Object Detection

Authors: Ava Dodd, Jonathan Adams

Abstract:

The purpose of this study is to explore the characteristics of developing a machine learning application using synthetic data. The study is structured to develop the application for the purpose of deploying the computer vision model. The findings discuss the realities of attempting to develop a computer vision model for practical purpose, and detail the processes, tools and techniques that were used to meet accuracy requirements. The research reveals that synthetic data represent another variable that can be adjusted to improve the performance of a computer vision model. Further, a suite of tools and tuning recommendations are provided.

Keywords: computer vision, machine learning, synthetic data, YOLOv4

References:

[1] J. Adams, E. Muphy, J. Sutor, A. Dodd. “Assessing the qualities of synthetic visual data production.” Proceedings of the ICIET 2019: 7th International Conference on Information and Education Technology Okayama, Japan March 27-29, 2021, Association for Computing Machinery New York NY United States, 2019.
[2] L. Zhang, A. Gonzalez-Garcia, V. J. Weijer, M. Danelljan, M., & F. S. Khan, “Synthetic data generation for end-to-end thermal infrared tracking.” IEEE Transactions on Image Processing, vol. 28 no. 4, pp. 1837-1850. 2019.
[3] A. R. Khadka, M. Oghaz, W. Matta, M. Cosentino, P. Remagnino, V. Argyriou, “Learning how to analyse crowd behaviour using synthetic data.” in CASA ‘19: Proc. of the 32nd Int. Conf. on Comp. Animation and Social Agents, Paris, France, July 2019, p 11-14
[4] N. Kalra, S. M. Paddock, “Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?” in Transportation Research Part A: Policy and Practice, vol. 94, RAND Corporation Santa Monica, CA. 2016, pp. 182-193,
[5] C. G. Northcutt, A. Athalye, J. Mueller, “Pervasive label errors in test sets destabilize machine learning benchmarks.” Preprint in review. Retrieved June 1, 2021 from https://arxiv.org/pdf/2103.14749.pdf.
[6] T. Xu, P. Zhang, Q. Huang, H. Zhang, Z. Gan, X. Huang, X. He, "AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks," IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA June 2018. pp. 1316-1324.
[7] Google Developers (2019, October 08). Overview of GAN structure | Generative Adversarial Networks. Retrieved Jan. 10, 2021, from https://developers.google.com/machine-learning/gan/gan_structure
[8] K. Lee, D. Moloney, “Evaluation of synthetic data for deep learning stereo depth algorithms on embedded platforms.” 4th International Conference on Systems and Informatics (ICSAI), Hangzhou, China, Nov. 2017.
[9] W. Qiu, F. Zhong, Y. Zhang, S. Qiao, Z. Xiao, T. S. Kim, Y. Wang, “UnrealCV: Virtual worlds for computer vision.” In: Hua G., Jégou H. (eds) Computer Vision – ECCV 2016 Workshops. European Conference on Computer Vision, Lecture Notes in Computer Science, vol. 9915.pp. 909–916.
[10] P. S. Rajpura, M. Goyal, H. Bojinov, R. S. Hegde, “Dataset Augmentation with synthetic images improves semantic segmentation.” In Rameshan R., Arora C., Dutta Roy S. (eds) Computer Vision, Pattern Recognition, Image Processing, and Graphics. NCVPRIPG 2017. Communications in Computer and Information Science, vol 841. Springer, Singapore.
[11] A. Komarraju, “Unfortunately, commercial AI is failing. Here’s why.” In Analytics Insight: Artificial Intelligence Latest News, Feb. 13, 2021. Retrieved June 5 2021, from https://www.analyticsinsight.net/unfortunately-commercial-ai-is-failing-heres-why/
[12] S. Yeong, L. King, S. Dol, “A review on marine search and rescue operations using unmanned aerial vehicles.” International Journal of Marine and Environmental Sciences, vol 9, no 2, pp 396 – 399, 2015.
[13] P. Nousi, I. Mademlis, I. Karakostas, A. Tefas and I. Pitas, "Embedded UAV real-time visual object detection and tracking," 2019 IEEE International Conference on Real-time Computing and Robotics (RCAR), pp. 708-713, 2019.
[14] C. Burke, P. R. McWhirter, J. Veitch-Michaelis, O. McAree, H. Pointon, S. Wich, S. Longmore, “Requirements and limitations of thermal drones for effective search and rescue in marine and coastal areas.” Drones. Vol. 3 no. 4, 2019.
[15] F. A. de Alcantara Andrade, A. Reinier Hovenburg, L. Netto de Lima, “Autonomous unmanned aerial vehicles in search and rescue missions using real-time cooperative model predictive control.” Sensors. 21 no. 4, Sep 20, 2019.
[16] A. B. Alexey, “Darknet.” Unpublished. Retrieved on June 12, 2021 from https://github.com/AlexeyAB/darknet/blob/master/README.md.
[17] M. Deserno, How to generate equidistributed points on the surface of a sphere. Unpublished. Sept. 28 2004, Retrieved on Jan. 18, 2021 from https://www.cmu.edu/biolphys/deserno/pdf/sphere_equi.pdf.
[18] F. Hutter, H. Hoos, K. Leyton-Brown, “An efficient approach for assessing hyperparameter importance.” In ICML, pages 754–762, 2014.
[19] Yu, Tong, and Hong Zhu. 2020. “Hyper-parameter optimization: A review of algorithms and applications.” http://search.ebscohost.com.proxy.lib.fsu.edu/login.aspx?direct=true&db=edsarx&AN=edsarx.2003.05689&site=eds-live&scope=site.
[20] Sutor, J. (2021, May 25). johnsutor/leopardi. GitHub. https://github.com/johnsutor/leopardi.
[21] P. Baldi, “Autoencoders, Unsupervised Learning, and Deep Architectures.” in Proceedings of Machine Learning Research. Proceedings of ICML Workshop on Unsupervised and Transfer Learning, vol. 27 pp. 37-49, 2012.
[22] K. Mason, S. Vejdan, S. Grijalva, “An ‘On the Fly’ Framework for Efficiently Generating Synthetic Big Data Sets” IEEE International Conference on Big Data (Big Data), pp. 379-338, 2019.
[23] S. Reitmann, L. Neumann, B. Jung, “BLAINDER-A Blender AI Add-On for Generation of Semantically Labeled Depth-Sensing Data.” Sensors. 21, no. 6, Mar 18, 2021.
[24] L. Li, “Why Does No One Use Advanced Hyperparameter Tuning?” Determined AI. Oct. 08, 2020, Retrieved from https://www.determined.ai/blog/why-does-no-one-use-advanced-hp-tuning