Enhancement of Stereo Video Pairs Using SDNs To Aid In 3D Reconstruction

Authors: Lewis E. Hibell, Honghai Liu, David J. Brown

Abstract:

This paper presents the results of enhancing images from a left and right stereo pair in order to increase the resolution of a 3D representation of a scene generated from that same pair. A new neural network structure known as a Self Delaying Dynamic Network (SDN) has been used to perform the enhancement. The advantage of SDNs over existing techniques such as bicubic interpolation is their ability to cope with motion and noise effects. SDNs are used to generate two high resolution images, one based on frames taken from the left view of the subject, and one based on the frames from the right. This new high resolution stereo pair is then processed by a disparity map generator. The disparity map generated is compared to two other disparity maps generated from the same scene. The first is a map generated from an original high resolution stereo pair and the second is a map generated using a stereo pair which has been enhanced using bicubic interpolation. The maps generated using the SDN enhanced pairs match more closely the target maps. The addition of extra noise into the input images is less problematic for the SDN system which is still able to out perform bicubic interpolation.

Keywords: Genetic Evolution, Image Enhancement, Neuron Networks, Stereo Vision

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1073445

References:

[1] L. E. Hibell, H. Liu, and D. J. Brown, "Combining multi-frame images for enhancement using self-delaying dynamic networks," in IEEE World Congress on Computational Intelligence, Canada, 2006.
[2] "Eye movements," 2004. (Online). Available: www.cis.rit.edu/vpl/eye movements.html
[3] J. Cooper, "All about strabismus," 2006.
[Online]. Available: http://www.strabismus.org/ all about strabismus.html
[4] J. P. C. Southall, Physiological Optics. New York, NY: Dover, 1937.
[5] R. D. Henkel, "Fast stereovision by coherence detection," in Computer Analysis of Images and Patterns, 1997, pp. 297-304.
[6] ÔÇöÔÇö, "A simple and fast neural network approach to stereo vision," in NIPS-97 in Denver. Cambridge: MIT Press, 1997, pp. 808-814.
[7] R. Tsai and T. Huang, Multiframe Image Restoration and Registration, R. Tsai and T. Huang, Eds. JAI Press, 1984, vol. 1.
[8] M. Irani and S. Peleg, "Super resolution from image sequences," 10th ICPR, vol. 2, pp. 115-120, 1990.
[9] ÔÇöÔÇö, "Improving resolution by image registration," CVGIP: Graphical Models and Image Processing, vol. 53, no. 3, pp. 231-239, May 1991.
[10] R. Schultz and R. Stevenson, "Extraction of high resolution frames from video sequences," IEEE Transactions on Image Processing, vol. 5, no. 6, pp. 996-1011, 1996.
[11] S. Kim, N. Bose, and H. Valenzuela, "Recursive reconstruction of high resolution image from noisy undersampled multiframes," IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 38, no. 6, 1990.
[12] S. Borman and R. Stevenson, "Spatial resolution enhancement of low resolution image sequences. A comprehensive review with directions for future research," University of Notre Dame, Tech. Rep., 1998.
[13] E. Salari and S. Zhang, "Integrated recurrent neural network for image resolution enhancement from multiple image frames," in IEE Proceedings on Vision, Image and Signal Processing, vol. 150, no. 5, 2003.
[14] C. Miravet and F. B. Rodr'─▒guez, "A hybrid MLP-PNN architecture for fast image superresolution." in ICANN, 2003, pp. 417-424.
[15] ÔÇöÔÇö, "Accurate and robust image superresolution by neural processing of local image representations." in ICANN (1), 2005, pp. 499-505.
[16] M. Negnevitsky, Artificial Intelligence, 2nd ed. Harlow, UK: Pearson, 2005.
[17] A. S. Ogale and Y. Aloimonos, "A roadmap to the integration of early visual modules," International Journal of Computer Vision: Special Issue Of Early Cognitive Vision, In Press.
[18] "Microsoft Research, Cambridge. i2i: 3D visual communication," March 2006. (Online). Available: http://research.microsoft.com/vision/cambridge/i2i/default.htm