A Reliable FPGA-based Real-time Optical-flow Estimation

Authors: M. M. Abutaleb, A. Hamdy, M. E. Abuelwafa, E. M. Saad

Abstract:

Optical flow is a research topic of interest for many years. It has, until recently, been largely inapplicable to real-time applications due to its computationally expensive nature. This paper presents a new reliable flow technique which is combined with a motion detection algorithm, from stationary camera image streams, to allow flow-based analyses of moving entities, such as rigidity, in real-time. The combination of the optical flow analysis with motion detection technique greatly reduces the expensive computation of flow vectors as compared with standard approaches, rendering the method to be applicable in real-time implementation. This paper describes also the hardware implementation of a proposed pipelined system to estimate the flow vectors from image sequences in real time. This design can process 768 x 576 images at a very high frame rate that reaches to 156 fps in a single low cost FPGA chip, which is adequate for most real-time vision applications.

Keywords: Optical flow, motion detection, real-time systems, FPGA.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1077797

References:

[1] C. Anderson, P. Burt, and G. van der Wal, "Change detection and tracking using pyramid transformation techniques", in Proceedings of SPIE. Intelligent Robots and Computer Vision, vol. 579, pp. 72-78, 1985.
[2] J. Barron, D. Fleet, and S. Beauchemin, "Performance of optical flow techniques", International Journal of Computer Vision, vol. 12, pp. 42- 77, 1994.
[3] A. Kasinski and A. Hamdy, "Efficient Separation of mobile objects on the scene from the sequence taken with an overhead camera". Proc. Int. Conf. on Computer Vision and Graphics, Zakopane, vol. 1, pp. 425-430, Sept. 2002.
[4] C. Ridder, O. Munkelt, and H. Kirchner, "Adaptive Background Estimation and Foreground Detection Using Kalman-Filtering". Proc. Int. l Conf. Recent Advances in Mechatronics, ICRAM .95, pp. 193-199, 1995.
[5] Y. Ivanov, A. Bobick, and J. Liu, "Fast Lighting Independent Background Subtraction". Technical Report no. 437, MIT Media Laboratory, 1997.
[6] G. Halevy and D. Weinshall, "Motion of disturbances: detection and tracking of multi-body non-rigid motion", Machine Vision and Applications, vol. 11, Issue 3, pp. 122-137, 1999.
[7] E.M. Saad, A. Hamdy, and M.M. Abutaleb, "FPGA-based Implementation of a Low Cost and Area Real-time Motion Detection", 15th IEEE Conference in Mixed Design of Integrated Circuits and Systems, Poznan, Poland, pp. 249-254, June 19-21, 2008.
[8] P. Anandan, "A Computational Framework and an Algorithm for the Measurement of Visual Motion", International Journal of Computer Vision, vol. 2, 1989.
[9] M. Okutomi and T. Kanade, "A Locally Adaptive Window for Signal Matching", International Journal of Computer Vision, vol. 7, no. 2, 1994.
[10] A. J. Lipton, "Local Application of Optic Flow to Analyze Rigid versus Non-Rigid Motion", ICCV Workshop on Frame-Rate Vision, 1999.
[11] K. N. Ngan, T. Meier, and D. Chai, "Advanced Video Coding: Principles and Techniques", Elsevier, 1999.
[12] B. Horn, B. Schunck, "Determining optical flow", Artificial Intelligence, vol. 17, pp. 185-203, 1981.
[13] J. L. Martín, A. Zuloaga, C. Cuadrado, J. Lázaro, and U. Bidarte, "Hardware implementation of optical flow constraint equation using FPGAs", Computer Vision and Image Understanding, vol. 98, pp. 462- 490, 2005.
[14] B. D. Lucas, T. Kanade, "An iterative image registration technique with an application to stereo vision", Proc. DARPA Image understanding Workshop, pp. 121-130, 1984.
[15] J. Díaz, E. Ros, F. Pelayo, E. M. Ortigosa, and S. Mota, "FPGA-based real-time optical-flow system", IEEE Trans. Circuits and Systems for Video Technology, vol. 16, no. 2, pp. 274-279, Feb 2006.
[16] Z.Y. Wei, D.J. Lee, B.E. Nelson, and M.A. Martineau, "A fast and accurate tensor-based optical flow algorithm implemented in FPGA", IEEE WACV, Austin, Texas, USA, p. 18 (6 pages), Feb 21-22, 2007.
[17] M.M. Abutaleb, A. Hamdy, and E.M. Saad, "FPGA-Based Real-Time Video-Object Segmentation with Optimization Schemes", International Journal of Circuits, Systems, and Signal Processing, vol. 2, issue 2, pp. 78-86, 2008.