An evaluation of video quality assessment metrics for passive gaming video streaming

Nabajeet Barman; Steven Schmidt; Saman Zadtootaghaj; Maria G. Martini; Sebastian M├Âller

An evaluation of video quality assessment metrics for passive gaming video streaming

Nabajeet Barman, Steven Schmidt, Saman Zadtootaghaj, Maria G. Martini, Sebastian M├Âller

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Video quality assessment is imperative to estimate the user experience to provide a reasonable Quality of Experience in video streaming applications to the end-user. Recent years have seen a tremendous advancement in the field of video quality assessment (VQA) metrics, with the development of models that can predict the quality of the videos streamed over the Internet. However, no work so far has attempted to study the performance of such quality assessment metrics on gaming videos which are artificial and synthetic and have different streaming requirements than traditionally streamed videos. Towards this end, we present in this paper an objective and subjective quality assessment study of gaming videos considering passive streaming applications. For subjective quality assessment, we consider six different gaming video sequences and 15 resolution-bitrate pairs. Objective quality assessment considering eight widely used VQA metrics is performed on a dataset of 24 reference videos and 576 compressed sequences obtained by encoding them at 24 different resolution-bitrate pairs. We present an evaluation of the performance behavior of the VQA metrics. Our results indicate that VMAF predicts subjective video quality ratings the best, while NIQE turns out to be a very promising alternative as a no-reference metric.

Original language	English
Title of host publication	This paper was published in PV '18 Proceedings of the 23rd Packet Video Workshop (2018), pages 7-12. ISBN: 9781450357739 Organising Body: Association for Computing Machinery Special Interest Group on Multimedia Organising Body: Association for Computing Machinery Special Interest Group on Multimedia
Publication status	E-pub ahead of print - 12 Jun 2018

Bibliographical note

Note: This paper was published in PV '18 Proceedings of the 23rd Packet Video Workshop (2018), pages 7-12. ISBN: 9781450357739

Organising Body: Association for Computing Machinery Special Interest Group on Multimedia

Organising Body: Association for Computing Machinery Special Interest Group on Multimedia

Keywords

Computer science and informatics

Access to Document

http://dx.doi.org/10.1145/3210424.3210434

Cite this

Barman, N., Schmidt, S., Zadtootaghaj, S., Martini, M. G., & M├Âller, S. (2018). An evaluation of video quality assessment metrics for passive gaming video streaming. In This paper was published in PV '18 Proceedings of the 23rd Packet Video Workshop (2018), pages 7-12. ISBN: 9781450357739 Organising Body: Association for Computing Machinery Special Interest Group on Multimedia Organising Body: Association for Computing Machinery Special Interest Group on Multimedia Advance online publication. http://dx.doi.org/10.1145/3210424.3210434

Barman, Nabajeet ; Schmidt, Steven ; Zadtootaghaj, Saman et al. / An evaluation of video quality assessment metrics for passive gaming video streaming. This paper was published in PV '18 Proceedings of the 23rd Packet Video Workshop (2018), pages 7-12. ISBN: 9781450357739 Organising Body: Association for Computing Machinery Special Interest Group on Multimedia Organising Body: Association for Computing Machinery Special Interest Group on Multimedia. 2018.

@inproceedings{06f4e243defc44f4bd11f51edda04a66,

title = "An evaluation of video quality assessment metrics for passive gaming video streaming",

abstract = "Video quality assessment is imperative to estimate the user experience to provide a reasonable Quality of Experience in video streaming applications to the end-user. Recent years have seen a tremendous advancement in the field of video quality assessment (VQA) metrics, with the development of models that can predict the quality of the videos streamed over the Internet. However, no work so far has attempted to study the performance of such quality assessment metrics on gaming videos which are artificial and synthetic and have different streaming requirements than traditionally streamed videos. Towards this end, we present in this paper an objective and subjective quality assessment study of gaming videos considering passive streaming applications. For subjective quality assessment, we consider six different gaming video sequences and 15 resolution-bitrate pairs. Objective quality assessment considering eight widely used VQA metrics is performed on a dataset of 24 reference videos and 576 compressed sequences obtained by encoding them at 24 different resolution-bitrate pairs. We present an evaluation of the performance behavior of the VQA metrics. Our results indicate that VMAF predicts subjective video quality ratings the best, while NIQE turns out to be a very promising alternative as a no-reference metric.",

keywords = "Computer science and informatics",

author = "Nabajeet Barman and Steven Schmidt and Saman Zadtootaghaj and Martini, \{Maria G.\} and Sebastian M├{\^A}ller",

note = "Note: This paper was published in PV '18 Proceedings of the 23rd Packet Video Workshop (2018), pages 7-12. ISBN: 9781450357739 Organising Body: Association for Computing Machinery Special Interest Group on Multimedia Organising Body: Association for Computing Machinery Special Interest Group on Multimedia",

year = "2018",

month = jun,

day = "12",

language = "English",

booktitle = "This paper was published in PV '18 Proceedings of the 23rd Packet Video Workshop (2018), pages 7-12. ISBN: 9781450357739 Organising Body: Association for Computing Machinery Special Interest Group on Multimedia Organising Body: Association for Computing Machinery Special Interest Group on Multimedia",

}

Barman, N, Schmidt, S, Zadtootaghaj, S, Martini, MG & M├Âller, S 2018, An evaluation of video quality assessment metrics for passive gaming video streaming. in This paper was published in PV '18 Proceedings of the 23rd Packet Video Workshop (2018), pages 7-12. ISBN: 9781450357739 Organising Body: Association for Computing Machinery Special Interest Group on Multimedia Organising Body: Association for Computing Machinery Special Interest Group on Multimedia. <http://dx.doi.org/10.1145/3210424.3210434>

An evaluation of video quality assessment metrics for passive gaming video streaming. / Barman, Nabajeet; Schmidt, Steven; Zadtootaghaj, Saman et al.
This paper was published in PV '18 Proceedings of the 23rd Packet Video Workshop (2018), pages 7-12. ISBN: 9781450357739 Organising Body: Association for Computing Machinery Special Interest Group on Multimedia Organising Body: Association for Computing Machinery Special Interest Group on Multimedia. 2018.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - An evaluation of video quality assessment metrics for passive gaming video streaming

AU - Barman, Nabajeet

AU - Schmidt, Steven

AU - Zadtootaghaj, Saman

AU - Martini, Maria G.

AU - M├Âller, Sebastian

N1 - Note: This paper was published in PV '18 Proceedings of the 23rd Packet Video Workshop (2018), pages 7-12. ISBN: 9781450357739 Organising Body: Association for Computing Machinery Special Interest Group on Multimedia Organising Body: Association for Computing Machinery Special Interest Group on Multimedia

PY - 2018/6/12

Y1 - 2018/6/12

N2 - Video quality assessment is imperative to estimate the user experience to provide a reasonable Quality of Experience in video streaming applications to the end-user. Recent years have seen a tremendous advancement in the field of video quality assessment (VQA) metrics, with the development of models that can predict the quality of the videos streamed over the Internet. However, no work so far has attempted to study the performance of such quality assessment metrics on gaming videos which are artificial and synthetic and have different streaming requirements than traditionally streamed videos. Towards this end, we present in this paper an objective and subjective quality assessment study of gaming videos considering passive streaming applications. For subjective quality assessment, we consider six different gaming video sequences and 15 resolution-bitrate pairs. Objective quality assessment considering eight widely used VQA metrics is performed on a dataset of 24 reference videos and 576 compressed sequences obtained by encoding them at 24 different resolution-bitrate pairs. We present an evaluation of the performance behavior of the VQA metrics. Our results indicate that VMAF predicts subjective video quality ratings the best, while NIQE turns out to be a very promising alternative as a no-reference metric.

AB - Video quality assessment is imperative to estimate the user experience to provide a reasonable Quality of Experience in video streaming applications to the end-user. Recent years have seen a tremendous advancement in the field of video quality assessment (VQA) metrics, with the development of models that can predict the quality of the videos streamed over the Internet. However, no work so far has attempted to study the performance of such quality assessment metrics on gaming videos which are artificial and synthetic and have different streaming requirements than traditionally streamed videos. Towards this end, we present in this paper an objective and subjective quality assessment study of gaming videos considering passive streaming applications. For subjective quality assessment, we consider six different gaming video sequences and 15 resolution-bitrate pairs. Objective quality assessment considering eight widely used VQA metrics is performed on a dataset of 24 reference videos and 576 compressed sequences obtained by encoding them at 24 different resolution-bitrate pairs. We present an evaluation of the performance behavior of the VQA metrics. Our results indicate that VMAF predicts subjective video quality ratings the best, while NIQE turns out to be a very promising alternative as a no-reference metric.

KW - Computer science and informatics

UR - https://2018.packet.video/

M3 - Conference contribution

BT - This paper was published in PV '18 Proceedings of the 23rd Packet Video Workshop (2018), pages 7-12. ISBN: 9781450357739 Organising Body: Association for Computing Machinery Special Interest Group on Multimedia Organising Body: Association for Computing Machinery Special Interest Group on Multimedia

ER -

Barman N, Schmidt S, Zadtootaghaj S, Martini MG, M├Âller S. An evaluation of video quality assessment metrics for passive gaming video streaming. In This paper was published in PV '18 Proceedings of the 23rd Packet Video Workshop (2018), pages 7-12. ISBN: 9781450357739 Organising Body: Association for Computing Machinery Special Interest Group on Multimedia Organising Body: Association for Computing Machinery Special Interest Group on Multimedia. 2018 Epub 2018 Jun 12.

An evaluation of video quality assessment metrics for passive gaming video streaming

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

An evaluation of video quality assessment metrics for passive gaming video streaming

Cite this