Quality enhancement of gaming content using generative adversarial networks

Nasim Jamshidi Avanaki; Saman Zadtootaghaj; Nabajeet Barman; Steven Schmidt; Maria G. Martini; Sebastian M├Âller

doi:10.1109/QoMEX48832.2020.9123074

Quality enhancement of gaming content using generative adversarial networks

Nasim Jamshidi Avanaki, Saman Zadtootaghaj, Nabajeet Barman, Steven Schmidt, Maria G. Martini, Sebastian M├Âller

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Recently, streaming of gameplay scenes has gained much attention, as evident with the rise of platforms such as Twitch.tv and Facebook Gaming. These streaming services have to deal with many challenges due to the low quality of source materials caused by client devices, network limitations such as bandwidth and packet loss, as well as low delay requirements. Spatial video artifact such as blockiness and blurriness as a result of as video compression or up-scaling algorithms can significantly impact the Quality of Experience of end-users of passive gaming video streaming applications. In this paper, we investigate solutions to enhance the video quality of compressed gaming content. Recently, several super-resolution enhancement techniques using Generative Adversarial Network (e.g., SRGAN) have been proposed, which are shown to work with high accuracy on non-gaming content. Towards this end, we improved the SRGAN by adding a modified loss function as well as changing the generator network such as layer levels and skip connections to improve the flow of information in the network, which is shown to improve the perceived quality significantly. In addition, we present a performance evaluation of improved SRGAN for the enhancement of frame quality caused by compression and rescaling artifacts for gaming content encoded in multiple resolution-bitrate pairs.

Original language	English
Title of host publication	Published in 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), ISSN 2372-7179, ISBN 9781728159669 Organising Body: Institute of Electrical and Electronics Engineers Organising Body: Institute of Electrical and Electronics Engineers
DOIs	https://doi.org/10.1109/QoMEX48832.2020.9123074
Publication status	Published - May 2020

Bibliographical note

Note: Published in 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), ISSN 2372-7179, ISBN 9781728159669

Organising Body: Institute of Electrical and Electronics Engineers

Organising Body: Institute of Electrical and Electronics Engineers

Keywords

Computer science and informatics

Access to Document

10.1109/QoMEX48832.2020.9123074

Barman-N-45172-AAMAccepted author manuscript, 0.98 MBLicence: Unspecified

https://doi.org/10.1109/QoMEX48832.2020.9123074

1 Paper

Quality enhancement of gaming content using generative adversarial networks
Avanaki, N. J., Zadtootaghaj, S., Barman, N., Schmidt, S., Martini, M. G. & M├Âller, S., May 2020.
Research output: Contribution to conference › Paper › peer-review

Open Access
File

Cite this

Avanaki, N. J., Zadtootaghaj, S., Barman, N., Schmidt, S., Martini, M. G., & M├Âller, S. (2020). Quality enhancement of gaming content using generative adversarial networks. In Published in 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), ISSN 2372-7179, ISBN 9781728159669 Organising Body: Institute of Electrical and Electronics Engineers Organising Body: Institute of Electrical and Electronics Engineers https://doi.org/10.1109/QoMEX48832.2020.9123074

Avanaki, Nasim Jamshidi ; Zadtootaghaj, Saman ; Barman, Nabajeet et al. / Quality enhancement of gaming content using generative adversarial networks. Published in 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), ISSN 2372-7179, ISBN 9781728159669 Organising Body: Institute of Electrical and Electronics Engineers Organising Body: Institute of Electrical and Electronics Engineers. 2020.

@inproceedings{abf774ddb65b493daeeffa676215ae83,

title = "Quality enhancement of gaming content using generative adversarial networks",

abstract = "Recently, streaming of gameplay scenes has gained much attention, as evident with the rise of platforms such as Twitch.tv and Facebook Gaming. These streaming services have to deal with many challenges due to the low quality of source materials caused by client devices, network limitations such as bandwidth and packet loss, as well as low delay requirements. Spatial video artifact such as blockiness and blurriness as a result of as video compression or up-scaling algorithms can significantly impact the Quality of Experience of end-users of passive gaming video streaming applications. In this paper, we investigate solutions to enhance the video quality of compressed gaming content. Recently, several super-resolution enhancement techniques using Generative Adversarial Network (e.g., SRGAN) have been proposed, which are shown to work with high accuracy on non-gaming content. Towards this end, we improved the SRGAN by adding a modified loss function as well as changing the generator network such as layer levels and skip connections to improve the flow of information in the network, which is shown to improve the perceived quality significantly. In addition, we present a performance evaluation of improved SRGAN for the enhancement of frame quality caused by compression and rescaling artifacts for gaming content encoded in multiple resolution-bitrate pairs.",

keywords = "Computer science and informatics",

author = "Avanaki, \{Nasim Jamshidi\} and Saman Zadtootaghaj and Nabajeet Barman and Steven Schmidt and Martini, \{Maria G.\} and Sebastian M├{\^A}ller",

note = "Note: Published in 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), ISSN 2372-7179, ISBN 9781728159669 Organising Body: Institute of Electrical and Electronics Engineers Organising Body: Institute of Electrical and Electronics Engineers",

year = "2020",

month = may,

doi = "10.1109/QoMEX48832.2020.9123074",

language = "English",

booktitle = "Published in 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), ISSN 2372-7179, ISBN 9781728159669 Organising Body: Institute of Electrical and Electronics Engineers Organising Body: Institute of Electrical and Electronics Engineers",

}

Avanaki, NJ, Zadtootaghaj, S, Barman, N, Schmidt, S, Martini, MG & M├Âller, S 2020, Quality enhancement of gaming content using generative adversarial networks. in Published in 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), ISSN 2372-7179, ISBN 9781728159669 Organising Body: Institute of Electrical and Electronics Engineers Organising Body: Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/QoMEX48832.2020.9123074

Quality enhancement of gaming content using generative adversarial networks. / Avanaki, Nasim Jamshidi; Zadtootaghaj, Saman; Barman, Nabajeet et al.
Published in 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), ISSN 2372-7179, ISBN 9781728159669 Organising Body: Institute of Electrical and Electronics Engineers Organising Body: Institute of Electrical and Electronics Engineers. 2020.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Quality enhancement of gaming content using generative adversarial networks

AU - Avanaki, Nasim Jamshidi

AU - Zadtootaghaj, Saman

AU - Barman, Nabajeet

AU - Schmidt, Steven

AU - Martini, Maria G.

AU - M├Âller, Sebastian

N1 - Note: Published in 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), ISSN 2372-7179, ISBN 9781728159669 Organising Body: Institute of Electrical and Electronics Engineers Organising Body: Institute of Electrical and Electronics Engineers

PY - 2020/5

Y1 - 2020/5

N2 - Recently, streaming of gameplay scenes has gained much attention, as evident with the rise of platforms such as Twitch.tv and Facebook Gaming. These streaming services have to deal with many challenges due to the low quality of source materials caused by client devices, network limitations such as bandwidth and packet loss, as well as low delay requirements. Spatial video artifact such as blockiness and blurriness as a result of as video compression or up-scaling algorithms can significantly impact the Quality of Experience of end-users of passive gaming video streaming applications. In this paper, we investigate solutions to enhance the video quality of compressed gaming content. Recently, several super-resolution enhancement techniques using Generative Adversarial Network (e.g., SRGAN) have been proposed, which are shown to work with high accuracy on non-gaming content. Towards this end, we improved the SRGAN by adding a modified loss function as well as changing the generator network such as layer levels and skip connections to improve the flow of information in the network, which is shown to improve the perceived quality significantly. In addition, we present a performance evaluation of improved SRGAN for the enhancement of frame quality caused by compression and rescaling artifacts for gaming content encoded in multiple resolution-bitrate pairs.

AB - Recently, streaming of gameplay scenes has gained much attention, as evident with the rise of platforms such as Twitch.tv and Facebook Gaming. These streaming services have to deal with many challenges due to the low quality of source materials caused by client devices, network limitations such as bandwidth and packet loss, as well as low delay requirements. Spatial video artifact such as blockiness and blurriness as a result of as video compression or up-scaling algorithms can significantly impact the Quality of Experience of end-users of passive gaming video streaming applications. In this paper, we investigate solutions to enhance the video quality of compressed gaming content. Recently, several super-resolution enhancement techniques using Generative Adversarial Network (e.g., SRGAN) have been proposed, which are shown to work with high accuracy on non-gaming content. Towards this end, we improved the SRGAN by adding a modified loss function as well as changing the generator network such as layer levels and skip connections to improve the flow of information in the network, which is shown to improve the perceived quality significantly. In addition, we present a performance evaluation of improved SRGAN for the enhancement of frame quality caused by compression and rescaling artifacts for gaming content encoded in multiple resolution-bitrate pairs.

KW - Computer science and informatics

U2 - 10.1109/QoMEX48832.2020.9123074

DO - 10.1109/QoMEX48832.2020.9123074

M3 - Conference contribution

BT - Published in 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), ISSN 2372-7179, ISBN 9781728159669 Organising Body: Institute of Electrical and Electronics Engineers Organising Body: Institute of Electrical and Electronics Engineers

ER -

Avanaki NJ, Zadtootaghaj S, Barman N, Schmidt S, Martini MG, M├Âller S. Quality enhancement of gaming content using generative adversarial networks. In Published in 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), ISSN 2372-7179, ISBN 9781728159669 Organising Body: Institute of Electrical and Electronics Engineers Organising Body: Institute of Electrical and Electronics Engineers. 2020 doi: 10.1109/QoMEX48832.2020.9123074

Quality enhancement of gaming content using generative adversarial networks

Abstract

Bibliographical note

Keywords

Access to Document

Fingerprint

Research output

Quality enhancement of gaming content using generative adversarial networks

Cite this