Assessing objective video quality in multi-screen video delivery

In today’s online video delivery systems, videos are streamed and displayed on various devices with different screen sizes, from large-screen UHD and HDTVs to smaller-screen devices such as mobile phones and tablets. A video will be perceived differently depending on the device’s screen size, pixel density, and viewing distance when viewed on different devices. Quality models which can estimate the relative differences in perceptual quality of a video on different devices can be used to understand the end-user QoE, design optimal encoding ladders for a multi-screen delivery environment, and better rate-adaptation algorithms. We previously presented a BC-KU Multi-Screen dataset1 consisting of subjective scores for different contents encoded in different resolution-bitrate pairs when viewed on three different devices. This paper presents several contributions extending the earlier dataset, which is of interest to the multimedia quality of experience (QoE) community. We first present an in-depth statistical data analysis on the previously unpublished individual subjective ratings of the Multi-Screen dataset. To better understand the relative differences in MOS scores, we present and analyze various demographic information about the test participants. We then evaluate the performance of twelve quality metrics based on five different performance measures. Individual subjective ratings, analysis scripts, and results are available as an open-source dataset. We believe the newly contributed results, files, and scripts will help analyze and design improved, low-complexity parametric models for multi-screen video delivery systems.