Video Super-Resolution Using Classification ANN

Authors: Ming-Hui Cheng, Jyh-Horng Jeng

Abstract:

In this study, a classification-based video super-resolution method using artificial neural network (ANN) is proposed to enhance low-resolution (LR) to high-resolution (HR) frames. The proposed method consists of four main steps: classification, motion-trace volume collection, temporal adjustment, and ANN prediction. A classifier is designed based on the edge properties of a pixel in the LR frame to identify the spatial information. To exploit the spatio-temporal information, a motion-trace volume is collected using motion estimation, which can eliminate unfathomable object motion in the LR frames. In addition, temporal lateral process is employed for volume adjustment to reduce unnecessary temporal features. Finally, ANN is applied to each class to learn the complicated spatio-temporal relationship between LR and HR frames. Simulation results show that the proposed method successfully improves both peak signal-to-noise ratio and perceptual quality.

Keywords: Super-resolution, classification, spatio-temporal information, artificial neural network.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1087428

References:

[1] G. Caner, A. M. Tekalp, W. Heinzelman, Super resolution recovery for multi-camera surveillance imaging, in Int. Conf. on Multimedia and Expo, Baltimore, USA, 2003, pp. 109-112
[2] J. A. Kennedy, O. Israel, A. Frenkel, R. BarShalom, H. Azhari, Super-resolution in PET imaging, IEEE Trans. on Medical Imaging 25 (2) (2006) 137-147.
[3] F. Li, X. Jia, D. Fraser, A. Lambert, Super resolution forremole sensing images based on a universal hidden Markov tree model, IEEE Trans. on Geoscience and Remote Sensing 48 (3) (2010) 1270-1278
[4] K. S. Ni, T. Q. Nguyen, Image super resolution using support vector regression, IEEE Trans. on Image Processing 16 (6) (2007) 1596-1610.
[5] H. Takeda, P. Milanfar, M. Protter, M. Elad, Super-resolution without explicit sub-pixel motion estimation, IEEE Trans. on Image Processing 18 (9) (2009) 1958-1975.
[6] S. Haykin, Neural Networks and Learning Machines, 3rd ed., Prentice Hall, Ontario, 2008.
[7] M. Elad, On the origin of the bilateral filter and ways to improve it, IEEE Trans. on Image Processing 11 (10) (2002) 1141