A 2D-3D Hybrid Vision System for Robotic Manipulation of Randomly Oriented Objects

Authors: Moulay A. Akhloufi

Abstract:

This paper presents an new vision technique for robotic manipulation of randomly oriented objects in industrial applications. The proposed approach uses 2D and 3D vision for efficiently extracting the 3D pose of an object in the presence of multiple randomly positioned objects. 2D vision permits to quickly select the objects of interest for 3D processing with a new modified ICP algorithm (FaR-ICP), thus reducing significantly the processing time. The extracted 3D pose is then sent to the robot manipulator for picking. The tests show that the proposed system achieves high performances

Keywords: 3D vision, Hand-Eye calibration, robot visual servoing, random bin picking.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1073042

References:

[1] Wilson, W.J., Hulls, C.C.W., Bell, G.S.: Relative End-Effector Control Using Cartesian Position Based Visual Servoing. IEEE Trans. Robotics and Automation. Vol. 12, No. 5, pp. 684ÔÇö696 (1996)
[2] Grosso, E., Metta, G., Oddera, A., Sandini, G.: Robust Visual Servoing in 3-D Reaching Tasks. IEEE Trans. Robotics and Automation RA. Vol. 12, No. 5, pp. 732ÔÇö742 (1996)
[3] Wijesoma, S.W., Wolfe, D.F.H., Richards, R.J.: Eye-to-Hand Coordination for Vision-Guided Robot Control Applications. International Journal of Robotics Research. Vol. 12, pp. 65--78 (1993)
[4] Sanchez, A.J., Martinez, J.M.: Robot-arm Pick and Place Behavior Programming System Using Visual Perception. In: 15th International Conference on Pattern Recognition, Vol. IV, pp. 507ÔÇö510, Barcelona, Spain (2000)
[5] Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press (2000)
[6] Lowe, David G. (1999). "Object recognition from local scale-invariant features". Proceedings of the International Conference on Computer Vision. 2. pp. 1150-1157.
[7] Herbert Bay, Andreas Ess, Tinne Tuytelaars, Luc Van Gool "SURF: Speeded Up Robust Features", Computer Vision and Image Understanding (CVIU), Vol. 110, No. 3, pp. 346--359, 2008
[8] Krystian Mikolajczyk and Cordelia Schmid "A performance evaluation of local descriptors", IEEE Transactions on Pattern Analysis and Machine Intelligence, 10, 27, pp 1615--1630, 2005.
[9] Sarfraz, S., Hellwich, O.:"Head Pose Estimation in Face Recognition across Pose Scenarios", Proceedings of VISAPP 2008, Int. conference on Computer Vision Theory and Applications, Madeira, Portugal, pp. 235-242, January 2008
[10] Tarsha-Kurdi, F., Landes, T., Grussenmeyer, P., 2007a. Houghtransform and extended RANSAC algorithms for automatic detection of 3d building roof planes from Lidar data. ISPRS Proceedings. Workshop Laser scanning. Espoo, Finland, September 12-14, 2007
[11] D.H. Ballard, "Generalizing the Hough Transform to Detect Arbitrary Shapes", Pattern Recognition, Vol.13, No.2, p.111-122, 1981
[12] P. Besl and N. McKay. A method for Registration of 3-D Shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 14(2):239 - 256, February 1992.
[13] Andreas N├╝chter, Kai Lingemann, and Joachim Hertzberg. Cached k-d tree search for ICP algorithms. In Proceedings of the 6th IEEE International Conference on Recent Advances in 3D Digital Imaging and Modeling (3DIM '07), IEEE Computer Society Press, pp. 419-426, Montreal, Canada, August 2007,
[14] K. S. Arun, T. S. Huang, and S. D. Blostein. Least square fitting of two 3-d point sets. IEEE Transactions on Pattern Analysis and Machine Intelligence, 9(5):698 - 700, 1987.
[15] B. K. P. Horn. Closed-form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America A, 4(4):629 - 642, April 1987.
[16] B. K. P. Horn, H. M. Hilden, and Sh. Negahdaripour. Closed-form solution of absolute orientation using orthonormal matrices. Journal of the Optical Society of America A, 5(7):1127 - 1135, July 1988.
[17] M. W. Walker, L. Shao, and R. A. Volz. Estimating 3-d location parameters using dual number quaternions. CVGIP: Image Understanding, 54:358 - 367, November 1991.
[18] A. Lorusso, D. Eggert, and R. Fisher. A Comparison of Four Algorithms for Estimating 3-D Rigid Transformations. In Proceedings of the 4th British Machine Vision Conference (BMVC '95), pages 237 - 246, Birmingham, England, September 1995.
[19] Y. Chen, G. G. Medioni. Object modelling by registration of multiple range images. Image and Vision Comp. 10(3), pp 145-155, 1992.
[20] Z. Y. Zhang. Iterative point matching for registration of free-form curves and surfaces. Int. J. of Computer Vision, 13(2), pp 119-15, Oct. 1994.
[21] T. Masuda, N. Yokoya. A robust method for registration and segmentation of multiple range images. Comp. Vision and Image Under. 61(3), pp 295-307, May 1995.
[22] C. Dorai, J. Weng, A. K. Jain. Optimal registration of object views using range data. IEEE Trans. Pat. Anal. and Mach. Intel. 19(10), pp 1131- 1138, Oct 1997.
[23] G. C. Sharp, S. W. Lee, D. K. Wehe. Invariant features and the registration of rigid bodies. Proc. IEEE Int. Conf. on Robotics and Autom., pp 932-937, 1999.
[24] Martin A. Fischler and Robert C. Bolles (June 1981). "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography". Comm. of the ACM 24: 381- 395
[25] Daniel B. Russakoff, Torsten Rohlfing, Calvin R. Maurer, Jr., "Fast Intensity-based 2D-3D Image Registration of Clinical Data Using Light Fields," iccv, vol. 1, pp.416, Ninth IEEE International Conference on Computer Vision (ICCV'03) - Volume 1, 2003
[26] A. Kubias, F. Deinzer, T. Feldmann, D. Paulus, B. Schreiber, Th Brunner. 2D/3D image registration on the GPU. Pattern Recognition and Image Analysis, Vol. 18, No. 3. (2008), pp. 381-389
[27] M.A. Akhloufi,. Real-time 3D visual servoing of an industrial cutting robot manipulator. Proc. International Conference on Control, Automation, Robotics and Vision (ICCARV 2009), WASET Vol. 60, pp. 432-436, ISSN: 2070-3724, Bangkok, Thailand, December 2009.
[28] Fanuc Robotics, http://www.fanucrobotics.com/
[29] KUKA Robotics, http://www.kuka-robotics.com/
[30] ABB Robotics, http://abb.com