Video Quality Assessment Methods: A Bird’s-Eye View

Authors: P. M. Arun Kumar, S. Chandramathi

Abstract:

The proliferation of multimedia technology and services in today’s world provide ample research scope in the frontiers of visual signal processing. Wide spread usage of video based applications in heterogeneous environment needs viable methods of Video Quality Assessment (VQA). The evaluation of video quality not only depends on high QoS requirements but also emphasis the need of novel term ‘QoE’ (Quality of Experience) that perceive video quality as user centric. This paper discusses two vital video quality assessment methods namely, subjective and objective assessment methods. The evolution of various video quality metrics, their classification models and applications are reviewed in this work. The Mean Opinion Score (MOS) based subjective measurements and algorithm based objective metrics are discussed and their challenges are outlined. Further, this paper explores the recent progress of VQA in emerging technologies such as mobile video and 3D video.

Keywords: 3D-Video, no reference metric, quality of experience, video quality assessment, video quality metrics.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1092540

References:

[1] Venkataraman.M, Chatterjee. M. and Siddhartha Chattopadhyay, "Evaluating Quality of Experience for streaming video in real time”, IEEE Global Telecommunications Conference, 2009, GLOBECOM 2009.
[2] ITU: New definitions for inclusion in Recommendation ITU-T P.10/G.100, ITU-T Rec. P.10/G.100, International Telecommunication Union, 2008
[3] Khalil ur Rehman Laghari, Omneya Issa, Filippo Speranza, and Tiago H. Falk, "Quality-of-Experience Perception for Video Streaming Services: Preliminary Subjective and Objective Results”, IEEE Signal & Information Processing Association Annual Summit and Conference (APSIPA ASC), 2012 .
[4] Andrew Perkis. (5 March 2013), Quality of experience (QoE) in multimedia applications (Online). Available: http://spie.org/x92222.xml
[5] S. Winkler and P. Mohandas, "The evolution of video quality measurements: from PSNR to hybrid metrics”, IEEE Transactions on Broad- casting 54, pp. 660–668, 2008
[6] ITU-R Recommendation BT.500-11, "Methodology for the subjective assessment of the quality of television pictures,” International Telecommunication Union, Geneva, Switzerland, 2002
[7] ITU-T Recommendation P.910, "Subjective video quality assessment methods for multimedia applications,” International Telecommunication Union, Geneva, Switzerland, 1999.
[8] Mario Vranjes, Snjezana Rimac-Drlje and Kresimir Grgic, "Review of objective video quality metrics and performance comparison using different databases”, Signal processing: Image Communication 28, pp.1-19, 2013.
[9] H.R. Wu and K.R. Rao, "Digital Video Image Quality and Perceptual Coding”, Taylor & Francis Group, London, 2006.
[10] J. L. Mannos and D. J. Sakrison, "The effects of a visual fidelity criterion on the encoding of images,” IEEE Transactions on Information Theory, vol. 20, no. 4, pp. 525–536, July 1974.
[11] F. X. J. Lukas and Z. L. Budrikis, "Picture quality prediction based on a visual model,” IEEE Transactions on Communications, vol. 30, no. 7, pp. 1679–1692, July 1982.
[12] S. Daly, "The visible differences predictor: An algorithm for the assessment of image fidelity,” in Digital Images and Human Vision, A. B. Watson, Ed. MIT Press, pp. 179–206, 1993
[13] C. J. Van den Branden Lambrecht and O. Verscheure, "Perceptual quality measure using a spatio-temporal model of the human visual system,” in Proc. SPIE Digital Video Compression: Algorithms and Technologies,vol. 2668, San Jose, CA, January 28–February 2, pp. 450–461, 1996
[14] J. Lubin and D. Fibush, "Sarnoff JND vision model,” T1A1.5 Working Group Document #97-612, ANSI T1 Standards Committee, 1997.
[15] S. Winkler, "A perceptual distortion metric for digital color video,” in Proc. SPIE Human Vision and Electronic Imaging, vol. 3644, San Jose, CA, January 23–29, pp. 175–184, 1999
[16] D.M. Chandler and S.S. Hemami, "VSNR: a wavelet-based visual signal- to-noise-ratio for natural images”, IEEE Transactions on Image Processing 16, pp. 2284–2297, 2007
[17] S.Rimac-Drlje, M.Vranjes and D. Zagar, "Foveated mean squared error—a novel video quality metric”, Multimedia Tools and Applications, vol. 49, pp.425–445, 2010
[18] K. Seshadrinathan and A.C. Bovik, "Motion tuned spatio-temporal quality assessment of natural videos”, IEEE Transactions on Image Processing, vol. 2, pp.335–350, 2010
[19] Y. Zhao, L. Yu, Z. Chen and C. Zhu, "Video quality assessment based on measuring perceptual noise from spatial and temporal perspectives”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 21, pp.1890–1902, 2011
[20] S. Susstrunk and S. Winkler, "Color image quality on the Internet,” in Proc. SPIE Internet Imaging, vol. 5304, San Jose, CA, January 19–22, pp. 118–131, invited paper, 2004
[21] Z. Wang, L. Lu and A. C. Bovik, "Video quality assessment based on structural distortion measurement,” Signal Processing: Image Communication, vol. 19, no. 2, pp. 121–132, February 2004.
[22] M.H. Pinson and S. Wolf, "A new standardized method for objectively measuring video quality”, IEEE Transactions on Broadcasting, vol. 50, pp.312–322, 2004
[23] H. R. Sheikh and A.C. Bovik, "A visual information fidelity approach to video quality assessment” (Invited paper), in: The First International Workshop on Video Processing and Quality Metrics for Consumer Electronics, USA, 2005.
[24] Z.Wang and Q.Li, "Video quality assessment using a statistical model of human visual speed perception”, Journal of the Optical Society of America A—Optics, Image Science and Vision, vol. 24, pp.B61–B69, 2007
[25] Z. Wang, E.P. Simoncelli and A.C. Bovik, "Multi-scale structural similarity for image quality assessment” (invited paper), in: Proceedings of the IEEE Asilomar Conference on Signals, Systems and Computers, USA, 2003.
[26] S. Chikkerur, V. Sundaram, M. Reisslein and L.J. Karam, "Objective video quality assessment methods: a classification, review, and perfor- mance comparison”, IEEE Transactions on Broadcasting, vol. 57, pp.165–182, 2011
[27] K. Seshadrinathan, R. Soundararajan, A.C. Bovik and L.K. Cormack, "Study of subjective and objective quality assessment of video”, IEEE Transactions on Image Processing, vol. 19, 1427–1441, 2010
[28] A. Takahashi, D. Hands, and V. Barriac, "Standardization activities in the ITU for a QoE assessment of IPTV,” IEEE Commun. Mag., vol. 46, no. 2, pp. 78–84, Feb. 2008.
[29] Ma Lin, Deng Chenwei, King Ngi Ngan and Lin Weisi, "Recent advances and challenges of visual signal quality assessment”, IEEE Communications, China, vol.10, pp. 62-78, May 2013.
[30] Tao Liu and Bloom. J, "On quality assessment in the delivery of mobile video”, IEEE International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS), pp.1-4, 2010
[31] Anush Krishna Moorthy, Lark Kwon Choi and Alan Conrad Bovik, "Video Quality Assessment on Mobile Devices: Subjective, Behavioral and Objective Studies”, IEEE Journal of Selected Topics in Signal Processing, Vol. 6, No. 6, pp. 652 – 671, October 2012
[32] Tao Liu, Cash, G, Wen Chen and Chunhua Chen, "Real time video quality monitoring for mobile devices” IEEE 44th Annual Conference on Information Sciences and Systems (CISS), pp.1-6, 2010
[33] Chen Chen, Li Song, Xiangwen Wang and Meng Guo, "No-reference video quality assessment on mobile devices”, IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), pp.1-6, 2013
[34] Quality of Experience for mobile video users, (Online).Available: http:// www.dialogic.com /~/media/ products/docs/whitepapers/11681-qoe-mobile-video-wp.pdf
[35] Touradj. Ebrahimi, "Towards 3D visual quality assessment for future multimedia” (Online). Available: http://www.slideshare.net/ touradj _ebrahimi/towards-3d-visual-quality-assessment-for-future-multimedia
[36] Quan Huynh-Thu, Le Callet and P. Barkowsky. M , "Video quality assessment: From 2D to 3D – Challenges and future trends”, IEEE 17th International Conference on Image Processing (ICIP), pp.4025-4028, 2010.
[37] NTT Network Technology Laboratories, (Online). Available: http://www.ntt.co.jp/qos/qoe/eng/technology/3d/01_3.html
[38] Chunting Yang, Yang Liu and Jing Yu, "Research on video quality assessment”, Journal of Multimedia, Vol 7, No 1, pp. 49-56, Feb 2012.
[39] O. Verseheure, P. Frossard and M. Hamde, "User-oreinted QoS analysis in MPEG-2 delivery”, Real-Tine Imaging, vol. 5, no. 5, pp. 305-314, 1999.
[40] S. Kanumuri, P. C. Cosman, A. R. Reibman and V. A. Vaishampayan, "Modeling Packet Losss Visiblility in MPEG-2 Video”, IEEE Tranasctions on Multimedia, vol. 8, no. 2, pp. 341-355, 2006.
[41] NTT Network Technology Laboratories, (Online). Available: http://www.ntt.co.jp/qos/qoe/eng/technology/visual/02_3.html