Optimal Presentation Duration for Video Quality Assessment

About

Subjective quality assessment is an essential component of modern image and video processing for both the validation of objective metrics and the comparison of coding methods. However, the standard procedures used to collect data can be prohibitively time consuming. One way of increasing the efficiency of data collection is to reduce the duration of test sequences from a 10s length currently used in most subjective video quality assessment (VQA) experiments. Here, we explore the impact of reducing sequence length upon perceptual accuracy when identifying compression artifacts. A group of four reference sequences, together with five levels of distortion, are used to compare the subjective ratings of viewers watching videos between 1.5 and 10s long. We identify a smooth function indicating that accuracy increases linearly as the length of the sequences increases from 1.5 to 7s. The accuracy of observers viewing 1.5s sequences was significantly inferior to those viewing sequences of 5, 7, and 10s. We argue that sequences between 5 and 10s produce satisfactory levels of accuracy but the practical benefits of acquiring more data lead us to recommend the use of 5s sequences for future VQA studies that use the single and double stimulus continuous quality scale methodologies.

Citation

@article{moss2015optimal,
  title={On the optimal presentation duration for subjective video quality assessment},
  author={Moss, Felix Mercer and Wang, Ke and Zhang, Fan and Baddeley, Roland and Bull, David R},
  journal={IEEE Transactions on Circuits and Systems for Video Technology},
  volume={26},
  number={11},
  pages={1977--1987},
  year={2015},
  publisher={IEEE}
}[paper]

@article{moss2016support,
  title={Support for reduced presentation durations in subjective video quality assessment},
  author={Moss, Felix Mercer and Yeh, Chun-Ting and Zhang, Fan and Baddeley, Roland and Bull, David R},
  journal={Signal Processing: Image Communication},
  volume={48},
  pages={38--49},
  year={2016},
  publisher={Elsevier}
}[paper]