SJTU 4K Video Subjective Quality Dataset for Content Adaptive Bit Rate Estimation without Encoding

Transcription

1 SJTU 4K Video Subjective Quality Dataset for Content Adaptive Bit Rate Estimation without Encoding Yutong Zhu, Li Song, Rong Xie, Wenjun Zhang Shanghai Jiao Tong University

2 Outline Motivation Subjective VQA on 4K sequences Test setup Test result Content adaptive bit rate estimation Parametric model Clustering analysis Performance Conclusions - 2-

3 Motivation 4K video sequences bring better quality of experience. More bit rate and bandwidth is required for 4K videos sequences compression. Which bit rate is appropriate when given certain video quality for 4K sequences? The availability of open 4K video sequences with MOS values is very limited

4 Subjective VQA on 4K sequences Sequences selection Spatial and Temporal Complexity * Spatial Information SI = max time {std space [Sobel(F n )]} (1) Temporal Information TI = max time {std space [F n (i,j) F n-1 (i,j)]} (2) where F n is the video frame at time n. Sequences span a wide rage of SI and TI * ITU-T, Subjective video quality assessment methods for multimedia applications, Recommendation ITU-R P 910, Sep

5 Subjective VQA on 4K sequences Source sequences 10 Source Sequences 8 bits color depth YUV 4:2:0 30fps Mobile Runners Rush Hour Traffic Flow Tall Building Campfire Party Coastguard Marathon Fountains Wood - 5 -

6 Subjective VQA on 4K sequences Encoding algorithm Configuration/Codec HM 11.0 RA Profile Main(6.2) Reference Frames 4 R/D Optimization On Motion Estimation TZ search Search Range 64 Group of Pictures 4 Deblock Filter On Intra Period 8 Sample adaptive offset On Coding Unit size/depth 64/4 Transform Unit size min/max 4/32-5 -

7 Subjective VQA on 4K sequences Test methodology Double Stimulus Impairment Scale (DSIS) Variant II Reference sequence Test sequence Reference sequence Test sequence Imperceptible Perceptual but not annoying Slightly annoying Annoying Very annoying

8 Subjective VQA on 4K sequences Test environment Test subjects 55 inches UHD monitor (SONY KD-55X9000A) 1.5H distance 2 subjects rate sequences seated in front of the monitor 42 subjects (18 female) age ranges from 20 to 31 non-experts - 8 -

9 Subjective VQA on 4K sequences Data processing Normal distribution test (β 2 test) * Mean score u jkr = 1 u N jkr i=1 where u jkr : the score of observer i for test condition j, sequence k, repetition r N : the number of observers Standard deviation S jkr = N i=1 N (u jkr u ijkr ) 2 N 1 Kurtosis coefficient β 2jkr = m 4 (m 2 ) 2 m x = N i=1(u ijkr u ijkr ) x N * ITU-R BT , Methodology for the subjective assessment of the quality of television pictures, International Telecommunication Union,

10 Subjective VQA on 4K sequences Data processing Normal distribution test (β 2 test) * If 2 β 2jkr 4, then: else: If where if u ijkr u jkr + 2S jkr then P i = P i + 1 if u ijkr u jkr 2S jkr then Q i = Q i + 1 if u ijkr u jkr + 20S jkr then P i = P i + 1 if u ijkr u jkr 20S jkr then Q i = Q i + 1 P i + Q i J K R > 0.05 and P i Q i P i + Q i < 0.3 then reject observer i J: number of test conditions K: number of test sequences R: number of repetitions 1 subject is rejected. * ITU-R BT , Methodology for the subjective assessment of the quality of television pictures, International Telecommunication Union,

11 Subjective test performance Test result Full dataset is released on the website:

12 Subjective test performance Test result MOS spans the most range of quality levels. Video quality appears quite variable trends for different sequences Content adaptive bit rate estimation is required

13 Outline Motivation Subjective VQA on 4K sequences Test setup Test result Content adaptive bit rate estimation Parametric model Clustering analysis Performance Conclusions - 2-

14 Content adaptive bit rate encoding ABR optimized encoding is not only for different network or screen like MPEG-DASH, but also for different contents, like recent Per-Title Encode Optimization from Netflix Fig is from

15 Content adaptive bit rate estimation Parametric Mode MOS = f(v c, BR, f r, r s, e c ) where Benchmark * : where c 1 to c 6 are model coefficients. f r = 30fps r s 4K e c HEVC BR p v c, MOS = ( β ) l n10 c 6 α v α c c 3 v c = TI SI α v c = c 1 + c 2 log (v c ) MOS β v c, MOS = γ(γ MOS) γ v c = c 4 + c 5 log (v c ) BR p = g v c, MOS g = f 1 v c : video content; BR: bit rate; f r : frame rate; r s : resolution; e c : encoding type * M. Takagi, H. Fujii and A. Shimizu, Optimized spatial and temporal resolution based on subjective quality estimation without encoding, IEEE International Conference on Visual Communications and Image Processing, pp , 2014.

16 Content adaptive bit rate estimation Clustering analysis Results without clustering analysis Between BR p and real bit rate Category PCC SCC RMSE All sequences K-means algorithm Silhouette value * Evaluate the performance K ca is the number of categories. Compact matching to its own category, poor matching to others 1-1 * A. Khan, L. Sun and E. Ifeachor, Content clustering based video quality prediction model for mpeg4 video streaming over wireless networks, IEEE International Conference on Communications, pp. 1-5,

17 Content adaptive bit rate estimation Clustering analysis Results of clustering analysis K ca =2, poor performance K ca =3, lowest Silh min (0.1383) K ca =5, too much categories (Silh max =1) K ca PCC SCC RMSE Silhouette value Kca = 2 K ca = 3 K ca = 4 Kca = 5 Silh min Silh max Silh mean Silh dev

18 Content adaptive bit rate estimation Clustering analysis K ca is set as 4. Highest Silh min High Silh max Acceptable Silh mean Least Silh dev Category Category A Category B Category C Category D Source sequences Mobile, Runners Campfire Party, Wood Fountains, Traffic Flow, Tall Buildings Coastguard, Rush Hour, Marathon Silhouette value Kca = 2 K ca = 3 K ca = 4 Kca = 5 Silh min Silh max Silh mean Silh dev

19 Content adaptive bit rate estimation Performance analysis The highest PCC is After clustering, PCC increases by 28.76%, and RMSE reduces by 68.98%. Different categories have diverse average MOS. Complex relationship exists between video quality and content. Category PCC SCC RMSE MOS Category A Category B Category C Category D All sequences

20 Conclusions Conclusions A open 4K dataset with subjective assessment results is contributed. Methodology of clustering first is key to success of content adaptive prediction. A parametric model is proposed as a benchmark. Further directions Verify the results with more test sequences Improve the model with more features