MS-UNIQUE: Multi-model and Sharpness-weighted Unsupervised Image Quality Estimation

Transcription

1 MS-UNIQUE: Multi-model and Sharpness-weighted Unsupervised Image Quality Estimation M. Prabhushankar, D.Temel, and G. AlRegib Center for Signal and Information Processing School of Electrical and Computer Engineering Georgia Institute of Technology Atlanta, GA 1

2 Outline I. Introduction II. Literature Review III. UNIQUE: Unsupervised Image Quality Estimation Overview of UNIQUE Unsupervised Learning Mechanism Preprocessing IV. MS-UNIQUE: Multi-model and Sharpness-weighted UNIQUE Multi-model Sharpness-weighted Multi-model V. Validation VI. Conclusion 2

3 I. Introduction Image Quality Assessment : Why? Application Average daily shared photos 390 Million [1] Remote Assistance 700 Million 70 Million 760 Million Smart Capturing [2] [3] [1] Adweek, Jun 2015 [2]LG, Ultra Clarity, Ultra Scale, [3] PetPixel, July 8,

4 I. Introduction Image Quality Assessment : In Practice Test setup Reference images [1] Distorted images [1] Subjective Scores 1 Bad very annoying 2 Poor annoying Fair Good Very Good slightly annoying distortion but not annoying no perceived distortion [1] Sheikh, H.R., Wang, Z., Cormack, L. and Bovik, A.C., "LIVE Image Quality Assessment Database Release 2", Mean opinion scores 4

5 Outline I. Introduction II. Literature Review III. UNIQUE: Unsupervised Image Quality Estimation Overview of UNIQUE Unsupervised Learning Mechanism Preprocessing IV. MS-UNIQUE: Multi-model and Sharpness-weighted UNIQUE Multi-model Sharpness-weighted Multi-model V. Validation VI. Conclusion 5

6 II. Literature Review Data-driven Image Quality Estimators YEAR QUALITY ESTIMATORS LBIQ DIIVINE CORNIA BRISQUE MLIQM CB/SF QAC SPARQ Tang QAF Kang IQA-CNN++ Li DLIQA Gao CNN-SVR UNIQUE MS-UNIQUE Visual system Color Distortion specific data in the training Do not require Labels in the training Handcrafting features Multiple layers/models without handcrafting 6

7 Outline I. Introduction II. Literature Review III. UNIQUE: Unsupervised Image Quality Estimation Overview of UNIQUE Unsupervised Learning Mechanism Preprocessing IV. MS-UNIQUE: Multi-model and Sharpness-weighted UNIQUE Multi-model Sharpness-weighted Multi-model V. Validation VI. Conclusion 7

8 III. UNIQUE: Unsupervised Image Quality Estimation Overview of UNIQUE D. Temel, M. Prabhushankar, and G. AlRegib, UNIQUE: Unsupervised Image Quality Estimation, the IEEE Signal Processing Letters, vol.23, no.10, pp

9 III. UNIQUE: Unsupervised Image Quality Estimation Preprocessing D. Temel, M. Prabhushankar, and G. AlRegib, UNIQUE: Unsupervised Image Quality Estimation, the IEEE Signal Processing Letters, vol.23, no.10, pp

10 III. UNIQUE: Unsupervised Image Quality Estimation Unsupervised Learning Mechanism D. Temel, M. Prabhushankar, and G. AlRegib, UNIQUE: Unsupervised Image Quality Estimation, the IEEE Signal Processing Letters, vol.23, no.10, pp

11 III. UNIQUE: Unsupervised Image Quality Estimation Visualization D. Temel, M. Prabhushankar, and G. AlRegib, UNIQUE: Unsupervised Image Quality Estimation, the IEEE Signal Processing Letters, vol.23, no.10, pp

12 Outline I. Introduction II. Literature Review III. UNIQUE: Unsupervised Image Quality Estimation Overview of UNIQUE Unsupervised Learning Mechanism Preprocessing IV. MS-UNIQUE: Multi-model and Sharpness-weighted UNIQUE Multi-model Sharpness-weighted Multi-model V. Validation VI. Conclusion 12

13 IV. MS-UNIQUE: Multi-model and Sharpness-weighted UNIQUE Multi-model Varying the number of neurons to learn global and local features

14 IV. MS-UNIQUE: Multi-model and Sharpness-weighted UNIQUE Sharpness-weighted Multi-model Sharpness Threshold 14

15 IV. MS-UNIQUE: Multi-model and Sharpness-weighted UNIQUE Visualization Images UNIQUE MS-UNIQUE 15

16 Outline I. Introduction II. Literature Review III. UNIQUE: Unsupervised Image Quality Estimation Overview of UNIQUE Unsupervised Learning Mechanism Preprocessing IV. MS-UNIQUE: Multi-model and Sharpness-weighted UNIQUE Multi-model Sharpness-weighted Multi-model V. Validation VI. Conclusion 16

17 V. Image Quality Estimators Validation LIVE TID Total Compression Image Noise Databases Communication Blur Color Global LIVE database Local TID 2013 database Performance Metrics Root mean square error (RMSE) Accuracy EE XX YY 2 Outlier Ratio (OR) Consistency NN oooooooooooooooo NN tttttttttt Pearson Linear Correlation Coefficient (PLCC) Linearity EE[(XX μμ XX )(YY μμ YY )] σσ XX σσ YY Spearman Rank Correlation Coefficient (SRCC) Ranking XX ii, YY ii xx ii, yy ii 1 6 ii=1 NN 2 xx ii yy ii NN(NN 2 1) 17

18 V. Image Quality Estimators Validation LIVE TID13 RMSE OR RMSE TID

19 V. Image Quality Estimators Validation PLCC LIVE SRCC LIVE 0.80 PLCC TID SRCC TID13 19

20 Outline I. Introduction II. Literature Review III. UNIQUE: Unsupervised Image Quality Estimation Overview of UNIQUE Unsupervised Learning Mechanism Preprocessing IV. MS-UNIQUE: Multi-model and Sharpness-weighted UNIQUE Multi-model Sharpness-weighted Multi-model V. Validation VI. Conclusion 20

21 VI. Conclusion Contributions and Observations YEAR QUALITY ESTIMATORS LBIQ DIIVINE CORNIA BRISQUE MLIQM CB/SF QAC SPARQ Tang QAF Kang IQA-CNN++ Li DLIQA Gao CNN-SVR UNIQUE MS-UNIQUE Do not require Visual system Color Distortion specific data in the training Labels in the training Handcrafting Multiple layers/models without handcrafting To measure perceived quality Hand-crafting is not sufficient, we should also learn from the data. Labels are not easy to find, we need to focus more on unsupervised approaches. Color perception must be included in a comprehensive visual system model. The best example is our visual system, we should model it as much as we can. 21