CVEGAN: A Perceptually-inspired GAN for Compressed Video Enhancement

About

In this work, we propose a new Generative Adversarial Network for Compressed Video quality Enhancement (CVEGAN). The CVEGAN generator benefits from the use of a novel Mul2Res block (with multiple levels of residual learning branches), an enhanced residual non-local block (ERNB) and an enhanced convolutional block attention module (ECBAM). The ERNB has also been employed in the discriminator to improve the representational capability. The training strategy has also been re-designed specifically for video compression applications, to employ a relativistic sphere GAN (ReSphereGAN) training methodology together with new perceptual loss functions.

Performance

The proposed network has been fully evaluated in the context of two typical video compression enhancement tools: post-processing (PP) and spatial resolution adaptation (SRA). CVEGAN has been fully integrated into the MPEG HEVC video coding test model (HM16.20) and experimental results demonstrate significant coding gains (up to 28% for PP and 38% for SRA compared to the anchor) over existing state-of-the-art architectures for both coding tools across multiple datasets.

Subjective Comparison

[Download] subjective comparison examples.

Source code

[Download] source code from github.

Citation

@article{ma2020cvegan,
  title={CVEGAN: A Perceptually-inspired GAN for Compressed Video Enhancement},
  author={Ma, Di and Zhang, Fan and Bull, David R},
  journal={arXiv preprint arXiv:2011.09190},
  year={2020}
}[paper]