Attention-Based Spatio-Temporal Approach for Fire and Smoke Detection

Authors: A. Mirrashid, M. Khoshbin, A. Atghaei, H. Shahbazi

Abstract:

In various industries, smoke and fire are two of the most important threats in the workplace. One of the common methods for detecting smoke and fire is the use of infrared thermal and smoke sensors, which cannot be used in outdoor applications. Therefore, the use of vision-based methods seems necessary. The problem of smoke and fire detection is spatiotemporal and requires spatiotemporal solutions. This paper presents a method that uses spatial features along with temporal-based features to detect smoke and fire in the scene. It consists of three main parts; the task of each part is to reduce the error of the previous part so that the final model has a robust performance. This method also uses transformer modules to increase the accuracy of the model. The results of our model show the proper performance of the proposed approach in solving the problem of smoke and fire detection and can be used to increase workplace safety.

Keywords: Attention, fire detection, smoke detection, spatiotemporal.

References:

[1] Majid, S., Alenezi, F., Masood, S., Ahmad, M., Gündüz, E. S., & Polat, K. (2022). Attention based CNN model for fire detection and localization in real-world images. Expert Systems with Applications, 189, 116114.
[2] FireNet: a specialized lightweight fire & smoke detection model for real-time IoT applications. arXiv preprint arXiv:1905.11922.
[3] Atghaei, Ali, Ehsan Rahnama, Kiavash Azimi, and Hassan Shahbazi. "Industrial Scene Change Detection using Deep Convolutional Neural Networks." arXiv preprint arXiv:2212.14278 (2022).
[4] Atghaei, Ali, and Ehsan Rahnama. "Localizing the conceptual difference of two scenes using deep learning for house keeping usages." arXiv preprint arXiv:2208.04884 (2022).
[5] Töreyin, B. U., Dedeoğlu, Y., Güdükbay, U., & Cetin, A. E. (2006). Computer vision based method for real-time fire and flame detection. Pattern recognition letters, 27(1), 49-58.
[6] Chen, T. H., Wu, P. H., & Chiou, Y. C. (2004, October). An early fire-detection method based on image processing. In 2004 International Conference on Image Processing, 2004. ICIP'04. (Vol. 3, pp. 1707-1710). IEEE.
[7] Vipin, V. (2012). Image processing based forest fire detection. International Journal of Emerging Technology and Advanced Engineering, 2(2), 87-95.
[8] Çelik, T., Özkaramanlı, H., & Demirel, H. (2007, September). Fire and smoke detection without sensors: Image processing based approach. In 2007 15th European Signal Processing Conference (pp. 1794-1798). IEEE.
[9] Rafiee, A., Dianat, R., Jamshidi, M., Tavakoli, R., & Abbaspour, S. (2011, March). Fire and smoke detection using wavelet analysis and disorder characteristics. In 2011 3rd International conference on computer research and development (Vol. 3, pp. 262-265). IEEE.
[10] Xu, G., Zhang, Y., Zhang, Q., Lin, G., & Wang, J. (2017). Deep domain adaptation based video smoke detection using synthetic smoke images. Fire safety journal, 93, 53-59.
[11] Qiu, T., Yan, Y., & Lu, G. (2011). An autoadaptive edge-detection algorithm for flame and fire image processing. IEEE Transactions on instrumentation and measurement, 61(5), 1486-1493.
[12] Rinsurongkawong, S., Ekpanyapong, M., & Dailey, M. N. (2012, May). Fire detection for early fire alarm based on optical flow video processing. In 2012 9th International conference on electrical engineering/electronics, computer, telecommunications and information technology (pp. 1-4). IEEE.
[13] Mueller, M., Karasev, P., Kolesov, I., & Tannenbaum, A. (2013). Optical flow estimation for flame detection in videos. IEEE Transactions on image processing, 22(7), 2786-2797.
[14] Zhang, Q., Xu, J., Xu, L., & Guo, H. (2016, January). Deep convolutional neural networks for forest fire detection. In Proceedings of the 2016 international forum on management, education and information technology application. Atlantis Press.
[15] Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25.
[16] Xie, Y., Zhu, J., Cao, Y., Zhang, Y., Feng, D., Zhang, Y., & Chen, M. (2020). Efficient video fire detection exploiting motion-flicker-based dynamic features and deep static features. IEEE Access, 8, 81904-81917.
[17] Sharma, J., Granmo, O. C., Goodwin, M., & Fidje, J. T. (2017, August). Deep convolutional neural networks for fire detection in images. In International conference on engineering applications of neural networks (pp. 183-193). Springer, Cham.
[18] Muhammad, K., Ahmad, J., Mehmood, I., Rho, S., & Baik, S. W. (2018). Convolutional neural networks based fire detection in surveillance videos. IEEE Access, 6, 18174-18183.
[19] Luo, Y., Zhao, L., Liu, P., & Huang, D. (2018). Fire smoke detection algorithm based on motion characteristic and convolutional neural networks. Multimedia Tools and Applications, 77(12), 15075-15092.
[20] Zilu, Y., & Xieyan, F. (2008, October). Combining LBP and Adaboost for facial expression recognition. In 2008 9th International Conference on Signal Processing (pp. 1461-1464). IEEE.
[21] Tan, Mingxing, and Quoc Le. "Efficientnet: Rethinking model scaling for convolutional neural networks." International conference on machine learning. PMLR, 2019.
[22] Oh, S. H., Ghyme, S. W., Jung, S. K., & Kim, G. W. (2020, February). Early wildfire detection using convolutional neural network. In International Workshop on Frontiers of Computer Vision (pp. 18-30). Springer, Singapore.S.
[23] Maksymiv, O., Rak, T., & Peleshko, D. (2017, February). Real-time fire detection method combining AdaBoost, LBP and convolutional neural network in video sequence. In 2017 14th international conference the experience of designing and application of CAD Systems in microelectronics (CADSM) (pp. 351-353). IEEE.
[24] Yin, Z., Wan, B., Yuan, F., Xia, X., & Shi, J. (2017). A deep normalization and convolutional neural network for image smoke detection. Ieee Access, 5, 18429-18438.
[25] Li, T., Zhao, E., Zhang, J., & Hu, C. (2019). Detection of wildfire smoke images based on a densely dilated convolutional network. Electronics, 8(10), 1131.
[26] Muhammad, K., Ahmad, J., Lv, Z., Bellavista, P., Yang, P., & Baik, S. W. (2018). Efficient deep CNN-based fire detection and localization in video surveillance applications. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 49(7), 1419-1434.
[27] Dunnings, A. J., & Breckon, T. P. (2018, October). Experimentally defined convolutional neural network architecture variants for non-temporal real-time fire detection. In 2018 25th IEEE international conference on image processing (ICIP) (pp. 1558-1562). IEEE.
[28] Muhammad, K., Khan, S., Elhoseny, M., Ahmed, S. H., & Baik, S. W. (2019). Efficient fire detection for uncertain surveillance environment. IEEE Transactions on Industrial Informatics, 15(5), 3113-3122.
[29] Shen, D., Chen, X., Nguyen, M., & Yan, W. Q. (2018, April). Flame detection using deep learning. In 2018 4th International conference on control, automation and robotics (ICCAR) (pp. 416-420). IEEE.
[30] Kim, B., & Lee, J. (2019). A video-based fire detection using deep learning models. Applied Sciences, 9(14), 2862.
[31] Chen, T. H., Wu, P. H., & Chiou, Y. C. (2004, October). An early fire-detection method based on image processing. In 2004 International Conference on Image Processing, 2004. ICIP'04. (Vol. 3, pp. 1707-1710). IEEE.
[32] Yu, C., Mei, Z., & Zhang, X. (2013). A real-time video fire flame and smoke detection algorithm. Procedia Engineering, 62, 891-898.
[33] https://github.com/ultralytics/yolov5
[34] Bertasius, Gedas, Heng Wang, and Lorenzo Torresani. "Is space-time attention all you need for video understanding?." ICML. Vol. 2. No. 3. 2021.