Deep Learning Based 6D Pose Estimation for Bin-Picking Using 3D Point Clouds

Authors: Hesheng Wang, Haoyu Wang, Chungang Zhuang

Abstract:

Estimating the 6D pose of objects is a core step for robot bin-picking tasks. The problem is that various objects are usually randomly stacked with heavy occlusion in real applications. In this work, we propose a method to regress 6D poses by predicting three points for each object in the 3D point cloud through deep learning. To solve the ambiguity of symmetric pose, we propose a labeling method to help the network converge better. Based on the predicted pose, an iterative method is employed for pose optimization. In real-world experiments, our method outperforms the classical approach in both precision and recall.

Keywords: Pose estimation, deep learning, point cloud, bin-picking, 3D computer vision.

References:

[1] B. Drost, M. Ulrich, N. Navab, and S. Ilic, “Model globally, match locally: Efficient and robust 3D object recognition,” in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010, pp. 998–1005.
[2] P. J. Besl and N. D. McKay, “A method for registration of 3-D shapes,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239–256, 1992.
[3] C. Papazov and D. Burschka, “An Efficient RANSAC for 3D Object Recognition in Noisy and Occluded Scenes,” in Computer Vision-ACCV 2010, PT I, Heidelberger Platz 3, D-14197 Berlin, Germany, 2011, vol. 6492, no. I, pp. 135–148.
[4] W. Abbeloos and T. Goedemé, “Point Pair Feature Based Object Detection for Random Bin Picking,” in 2016 13th Conference on Computer and Robot Vision (CRV), 2016, pp. 432–439.
[5] Vidal, C. Lin, and R. Martí, “6D pose estimation using an improved method based on point pair features,” in 2018 4th International Conference on Control, Automation and Robotics (ICCAR), 2018, pp. 405–409.
[6] Y. Xiang, T. Schmidt, V. Narayanan, and D. Fox, “PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes.” 2018.
[7] C. Wang et al., “DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 3338–3347.
[8] Y. He, W. Sun, H. Huang, J. Liu, H. Fan, and J. Sun, “PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 11629–11638.
[9] Z. Dong et al., “PPR-Net:Point-wise Pose Regression Network for Instance Segmentation and 6D Pose Estimation in Bin-picking Scenarios,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019, pp. 1773–1780.
[10] K. Kleeberger, C. Landgraf, and M. F. Huber, “Large-scale 6D Object Pose Estimation Dataset for Industrial Bin-Picking,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019, pp. 2573–2578.
[11] Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017, pp. 23–30.
[12] C. R. Qi, O. Litany, K. He, and L. Guibas, “Deep Hough Voting for 3D Object Detection in Point Clouds,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 9276–9285.
[13] L. Jiang, H. Zhao, S. Shi, S. Liu, C. -W. Fu, and J. Jia, “PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 4866–4875.
[14] O. Cicek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger, “3D U-Net: learning dense volumetric segmentation from sparse annotation,” in Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016. 19th International Conference. Proceedings: LNCS 9901, 2016, pp. 424–32.
[15] K. K. Liang, “Efficient conversion from rotating matrix to rotation axis and angle by extending Rodrigues’ formula.” 2018.