Automatic Product Identification Based on Deep-Learning Theory in an Assembly Line

Authors: Fidel Lòpez Saca, Carlos Avilés-Cruz, Miguel Magos-Rivera, José Antonio Lara-Chávez

Abstract:

Automated object recognition and identification systems are widely used throughout the world, particularly in assembly lines, where they perform quality control and automatic part selection tasks. This article presents the design and implementation of an object recognition system in an assembly line. The proposed shapes-color recognition system is based on deep learning theory in a specially designed convolutional network architecture. The used methodology involve stages such as: image capturing, color filtering, location of object mass centers, horizontal and vertical object boundaries, and object clipping. Once the objects are cut out, they are sent to a convolutional neural network, which automatically identifies the type of figure. The identification system works in real-time. The implementation was done on a Raspberry Pi 3 system and on a Jetson-Nano device. The proposal is used in an assembly course of bachelor’s degree in industrial engineering. The results presented include studying the efficiency of the recognition and processing time.

Keywords: Deep-learning, image classification, image identification, industrial engineering.

References:

[1] J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, “How transferable are features in deep neural networks?” in Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2014, pp. 3320–3328.
[2] K. Schwab, The Fourth Industrial Revolution, C. Business., Ed. Crown Business., 2017, vol. 1, no. 1.
[3] S. F. Kurniawan, I. K. G. D. Putra, and A. A. K. O. Sudana, “Bone fracture detection using opencv,” Journal of Theoretical and Applied Information Technology, vol. 64, no. 1, pp. 249–254, June 2014.
[4] D. Jacobsen and P. Ott, “Cloud architecture for industrial image processing: Platform for realtime inline quality assurance,” in 2017 IEEE 15th International Conference on Industrial Informatics (INDIN), July 2017, pp. 72–74.
[5] S. Lee and C. Yang, “A real time object recognition and counting system for smart industrial camera sensor,” IEEE Sensors Journal, vol. 17, no. 8, pp. 2516–2523, April 2017.
[6] S. Jagtap, C. Bhatt, J. Thik, and S. Rahimifard, “Monitoring potato waste in food manufacturing using image processing and internet of things approach,” Sustainability (Switzerland), vol. 11, no. 11, 2019.
[7] A. Abdo, J. Siam, B. Salah, and M. Krid, “Multiple-sensor fault detection and isolation using video processing in production lines,” International Journal of Computer Integrated Manufacturing, 2019.
[8] N. Mowell, B. Sheumaker, T. Han, J. Chaung, S. Sanghavi, Y. Khopkar, F. Levitov, B. Bielec, D. Salvador, K. Naguib, and V. Nguyen, “Criticality of photo track monitoring for lithography defect control,” vol. 2019-May, 2019.
[9] I. Szabo, J. Sun, C. Selcuk, and T.-H. Gan, “A new automated in line quality control system based on non-destructive evaluation for additive manufacturing of net-shape parts from particulates,” World PM2016 Proceedings, 2016.
[10] F. Ozkan and B. Ulutas, “Use of an eye-tracker to assess workers in ceramic tile surface defect detection,” 2016 International Conference on Control, Decision and Information Technologies (CoDIT), St. Julian’s, 2016, pp. 088-091, doi: 10.1109/CoDIT.2016.7593540.
[11] A. Ghaitaranpour, A. Rastegar, F. Tabatabaei Yazdi, M. Mohebbi, and B. Alizadeh Behbahani, “Application of digital image processing in monitoring some physical properties of tarkhineh during drying,” Journal of Food Processing and Preservation, vol. 41, no. 2, 2017.
[12] G. Reddy, T. Jahnavi, D. Rushali, and B. Kumar, “Bliss bot for pharmaceutical inspection,” 2017 International Conference on Trends in Electronics and Informatics (ICEI), Tirunelveli, 2017, pp. 354-359, doi: 10.1109/ICOEI.2017.8300947.
[13] A. Thamna, P. Srisungsitthisunti, and S. Dechjarem, “Real-time visual inspection and rejection machine for bullet production,” 2018 2nd International Conference on Engineering Innovation (ICEI 2018), 2018, pp. 13–17.
[14] A. Su´arez, M. A. Alvarez-Feijoo, R. Fern´andez Gonz´alez, and E. Arce, “Teaching optimization of manufacturing problems via code components of a jupyter notebook,” Computer Applications in Engineering Education, vol. 26, no. 5, pp. 1102–1110, 2018.
[15] K. Fukushima and S. Miyake, “Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position,” Pattern Recognition, vol. 15, no. 6, pp. 455 – 469, 1982.
[Online]. Available: http://www.sciencedirect.com/science/article/ pii/0031320382900243
[16] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, Nov 1998.
[17] D. C. Cires¸an, U. Meier, J. Masci, L. M. Gambardella, and J. Schmidhuber, “Flexible, high performance convolutional neural networks for image classification,” in Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence - Volume Volume Two, ser. IJCAI’11. AAAI Press, 2011, pp. 1237–1242.
[18] J. Deng, W. Dong, R. Socher, L. Li, Kai Li, and Li Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition, June 2009, pp. 248–255.
[19] M. Inoue, S. Inoue, and T. Nishida, “Deep recurrent neural network for mobile human activity recognition with high throughput,” Artif. Life Robot., vol. 23, no. 2, pp. 173–185, Jun. 2018.
[20] “An analysis of convolutional long short-term memory recurrent neural networks for gesture recognition,” Neurocomputing, vol. 268, pp. 76 – 86, 2017, advances in artificial neural networks, machine learning and computational intelligence.
[21] F. J. Ord´o˜nez and D. Roggen, “Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition,” Sensors, vol. 16, no. 1, 2016.
[22] M. Edel and E. K¨oppe, “Binarized-blstm-rnn based human activity recognition,” in 2016 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Oct 2016, pp. 1–7.
[23] C. Avil´es-Cruz, A. Ferreyra-Ram´ırez, A. Z´u˜niga-L´opez, and J. Villegas-Cort´ez, “Coarse-fine convolutional deep-learning strategy for human activity recognition,” Sensors, vol. 19, no. 7, 2019.
[24] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” CoRR, vol. abs/1412.6980, 2014.