Feature Point Reduction for Video Stabilization

Authors: Theerawat Songyot, Tham Manjing, Bunyarit Uyyanonvara, Chanjira Sinthanayothin

Abstract:

Corner detection and optical flow are common techniques for feature-based video stabilization. However, these algorithms are computationally expensive and should be performed at a reasonable rate. This paper presents an algorithm for discarding irrelevant feature points and maintaining them for future use so as to improve the computational cost. The algorithm starts by initializing a maintained set. The feature points in the maintained set are examined against its accuracy for modeling. Corner detection is required only when the feature points are insufficiently accurate for future modeling. Then, optical flows are computed from the maintained feature points toward the consecutive frame. After that, a motion model is estimated based on the simplified affine motion model and least square method, with outliers belonging to moving objects presented. Studentized residuals are used to eliminate such outliers. The model estimation and elimination processes repeat until no more outliers are identified. Finally, the entire algorithm repeats along the video sequence with the points remaining from the previous iteration used as the maintained set. As a practical application, an efficient video stabilization can be achieved by exploiting the computed motion models. Our study shows that the number of times corner detection needs to perform is greatly reduced, thus significantly improving the computational cost. Moreover, optical flow vectors are computed for only the maintained feature points, not for outliers, thus also reducing the computational cost. In addition, the feature points after reduction can sufficiently be used for background objects tracking as demonstrated in the simple video stabilizer based on our proposed algorithm.

Keywords: background object tracking, feature point reduction, low cost tracking, video stabilization.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1063360

References:

[1] J. Shi and C. Tomasi, "Good Features to Track", IEEE Conference on Computer Vision and Pattern Recognition, June 1993.
[2] C. Harris and M. Stephens, "A Combined Corner and Edge Detection", Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 287-293, May 2002.
[3] D. P. Kottke and Y. Sun, "Motion Estimation Via Cluster Matching", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 16, no. 11, pp. 1128-1132, November 1994.
[4] H-C. Changl, S-H. Lai, and K-R. Lu, "A Robust and Efficient Video Stabilization Algorithm", IEEE International Conference on Multimedia and Expo (ICME), 2004.
[5] C. Morimoto and R. Chellappa, "Automatic digital image stabilization", IEEE Intern. Conf on Pattern Recognition, pp. 660-665, 1997.
[6] T. Chen, "Video Stabilization Algorithm Using a Block-Based Parametric Motion Model", EE392I Project Report, winter 2000.
[7] B. Lucas and T. Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision", Proceedings of Imaging Understanding Workshop, pp. 121-130, 1981.
[8] K. G. Derpanis, "The Harris Corner Detector", October 2004.
[9] G. Bradski and A. Kaehler, Learning OpenCV• Computer Vision with the OpenCV Library, Sebastopol: O'Rielly Media, Inc.
[10] D. C. Montgomery, G. C. Runger and N. F. Hubele, Engineering Statistics, Hoboken: John Wiley & Sons, Inc.