PNN-Based QoE Measuring Model for Video Applications over LTE System

Transcription

1 2012 7th International ICST Conference on Communications and Networking in China (CHINACOM) PNN-Based QoE Measuring Model for Video Applications over LTE System Yuan He, Chao Wang, Hang Long, Kan Zheng Wireless Signal Processing and Network Lab Key Laboratory of Universal Wireless Communication, Ministry of Education Beijing University of Posts & Telecommunications Beijing, , China Abstract Users quality of experience (QoE) is a key factor in the success of video applications over the Long Term Evolution (LTE) networks. Thus, evaluating the QoE of video applications is of tremendous importance in design and optimization of wireless video processing and transmission systems. In this paper, we propose a QoE measuring model for the quality of video applications by using probabilistic neural network (PNN). We conduct a subjective test, in which OPNET modeler is employed to build a system level simulation platform for the wireless network and distortion is added into original video sequences when transmitting on the platform. A subject pool is utilized to evaluate the distorted videos. Based on the subjective test, we create a distorted-video database. PNN is used to train the mapping function between the corresponding parameters and QoE. The results demonstrate the effectiveness of the proposed model and show that it can provide a high correlation rate with human perception. I. INTRODUCTION The provision of high speed access to Internet and IP-based services is one of the main goals of the long term evolution (LTE) system. It aims at providing broadband wireless access that will enhance the user experience of data services provided. For that purpose, LTE has aimed at increasing peak data rates, improving spectral efficiency, increasing cell capacity, supporting scalable bandwidth or reducing radio access network latency [1]. User s quality of experience (QoE) is defined in [2] as the overall acceptability of an application or service, as perceived subjectively by users. It is therefore important to choose and adapt proper approach to measure QoE of users for video applications over LTE networks, since the everlasting endeavor by network and multimedia researchers is to satisfy the users growing needs. There are works on video quality evaluation. The analysis of the video can be done using either objective tests or subjective ones. Subjective tests are based on evaluations made by human-beings under well defined and controlled conditions [4]. Obviously, subjective tests have limitations in application because they require stringent environment, and they are time consuming and very costly, making it hard 1 This research was supported by the National Science Foundation for Postdoctoral Scientists of China under grant and the Program for New Century Excellent Talents in University under grant NCET to use in real-time quality assessment. Objective ones are usually explicit functions of measurable parameters related to the encoder or to the network. Some widely-used existing objective methods are mean square error (MSE) or peak signalto-noise ratio (PSNR) [6], which measure the quality by simple pixel-to-pixel comparisons. There are other more complicate methods, such as the Structural Similarity Index Metric [7] and the Video Quality Metric [8]. Also, objective tests still have disadvantages, such as: it is difficult to build a model that take into account the effect of many quality-affecting parameters and they do not correlate well with human visual perception [5]. The most widely-used metric of quantifying QoE is the mean opinion score (MOS) [3]. In the MOS test for a particular application, human-beings are asked to assessing the perceived quality by using score ranging from 5 to 1, which means excellent to bad. The MOS for the application is then the arithmetic mean of all individual scores. Videos are classified into 5 categories including excellent, good, fair, poor and bad when observers using MOS to evaluate them. In fact, most users may care more about the category of the video service than the exact number of MOS. Alternatively, MOS can be obtained from objective methods like PSNR to avoid the personnel cost and the trouble to set up a standard testing environment. In PSNR metric, the MOS values are obtained by PSNR to MOS conversion from [6]. Neural network is one of the artificial intelligence techniques. The concept of neural network is based on the human neural network, which learns to react in an appropriate way to certain input stimuli. A neural network can be considered as a black box that performs a mathematical function on all the inputs, to produce an output, consisting of one or more scalar values. An important advantage is the fact that the neural network can be learned appropriate behavior by applying sample training patterns. Considering advantages and drawbacks of the previous approach, to achieve the perceptual quality evaluation, there is research by using the neural network to build QoE evaluation model [9]-[11]. There is no need to make explicit correlations between QoE-affecting parameters and users perceived qulaity when applying the neural network in the QoE evaluation model. This avoids the need of constructing difficult and sound training rules. An admission /12/$ IEEE

2 Fig. 1: The framework structure of the subjective study control mechanism based on QoE is proposed in [10], using a tool called Pseudo Subjective Quality Assessment, which is based on statistic learning using Random Neural Network. Also, back-propagation neural network is used to analyze the connection of QoE and network circumstance [11]. Probabilistic neural network (PNN) is a kind of supervised neural network that are widely-used in the area of pattern recognition, nonlinear mapping, and estimation of probability of class membership and likelihood ratios [12] [13]. One of the benefits of PNN is training speed advantage over the standard back-propagation paradigm with approximately the same generalization capability. Another benefit is that new training patterns can be incorporated into a previously trained classifier quite easily, this might be important for a particular on-line application. In this paper, we proposed an approach that can evaluate the combined effect of an arbitrary number of parameters on the quality of video sequences. Based on the classification theory, we try classification technique when we are building the QoE evaluation model and establish efficient systems to solve video quality classification problems by using PNN. We adopt PNN that takes as input the values of a set of qualityaffecting parameters and correspondingly quantifies the video quality using MOS. Based on the MOS metric, it presents high accuracy and a robust way the function mapping the various involved parameters to the quality metric. The remainder of this paper is organized as follows. Section II describes the details of the subjective study, including creating test video sequences and subjective evaluation. PNN mapping is presented in Section III. In Section IV, we analyze the performance of the QoE evaluation model in terms of its correlation with human perception. Conclusions of the paper are presented in Section V. II. DETAILS OF THE SUBJECTIVE STUDY The framework structure of the subjective study is shown in Fig. 1. The whole system consists of two parts. Part I Fig. 2: Steps for creating distorted-video database is creating the distorted video sequences through the system level simulation of wireless networks. The result of this part is objective QoE-affecting parameters, such as frame loss, jitter and delay. Part II is subjective evaluation which will result in users perceived quality. Then the PNN-based QoE measuring model will combine the objective parameters and user perceived qulaity. The details of the subjective study will be given in following sections. A. Creating Test Sequences The steps of creating test sequences is shown in Fig. 2. For the test we select a video sequence of qcif resolution (176x144 pixels/frame, 30 frames/sec, 400 frames) as original video and encoded in MPEG4 format with an open source ffmpeg encoder/decoder [6]. Each frame will be fragmented into 1000 bytes for transmission (Maximun packet length will be 1028 bytes, including IP header (20bytes) and UDP header (8bytes)). During the encoding and packing of the original video sequences, we store the traces of the video sequences with the time-stamps. The video traffic is modeled based on the video trace file, and is transmitted in the wireless network simulation platform. In this study, we employ OPNET modeler to build a system 59

3 TABLE I: User experience quantification MOS Quality Impairment 5 Excellent Imperceptible 4 Good Perceptible but not annoying 3 Fair Slightly annoyingb 2 Poor Annoying 1 Bad Very annoying level simulation platform of the wireless network for video sequences transmission. As shown in Fig. 1, we used OPNET to model the video traffic and simulate the path loss and fast fading according to the 3GPP TR [15]. During the transmission, the received video trace file is stored which records the the packet loss and delay introduced by errors in wireless transmission. This simulation platform supports different error patterns and the packet loss rates can range from 0% to 100%. After the simulation of the wireless network transmission, the reconstruction of the received video is implemented by comparing the transmitted and the received trace files. With this reconstruction, distortions are added into original video, and the following objective metrics are stored and calculated: End-to-end delay Delay jitter Packet loss rate Throughput The distorted video sequences are obtained after inserting loss patterns and distort process. Since every distorted video sequence is based on some parameters conditions, we select a set of quality affecting factors including LRs (packet-loss Rate), jitters, delays and MLBSs (mean loss burst size) with their ranges of values, which is called a configuration [16]. We obtain 70 distorted videos in all, then we generate a distorted video database varying with representative configuration and it can be expressed as: D i = {x 1,x 2,x 3,x 4 }, i =1, 2,..., 70 where x 1,x 2,x 3 and x 4 represent LR, Jitter, Delay and MLBS, respectively. B. Subjective Evaluation In the second stage, we evaluate the chosen configurations in the previous stage by means of a subjective campaign. For that, a questionnaire experiment is conducted in the Wireless Signal Processing and Network (WSPN) Lab. The subject pool consisted of postgraduate students from WSPN, which is a mix of males and females. The average subject age is between 20 and 28 years and the subjects were inexperienced with video quality assessments. No monetary compensation for participating in the study was offered. Observers are asked to evaluate perceived quality of the test video sequences using MOS, the quantification is given in Table I. We compute MOS using average score obtained by all observers, each sequence thus receives a MOS. Based on the classification theory, we round the MOSs. We denote the MOS of sequence D i by Q i, we now create a database based on the Fig. 3: PNN architecture subjective study and all further discussion can be studied based on the distorted-video database. III. PNN TRAINING We now identify configuration D i with quality value Q i, and look for a real function f( ) of four variables associated with the selected parameters, such that for any set of values in the set. The function returns a number close to the associated MOS value. As we mentioned before, the characteristics of PNN make it particularly suitable for this classification study, such as its training speed advantage and the new training pattern. Then we train PNN to learn the mapping of configurations and subjective MOSs as defined in the distorted-video database. Once the tool has been trained, we have a function f( ) that can map any possible value of parameters into MOS. PNN is composed of three layers: the input layer, the pattern layer and the summation layer, as depicted in Fig. 3, which is a modified version of traditional PNN [14]. This version of PNN needs only one training step, thus its training is very fast. The modifiable weights including w i of class C i connecting from the input to pattern units will be trained. For each pattern x passed by the input layer to a neuron in the pattern layer, it computes the output for C i. The computation is performed as follows, F k,i (x) = 1 ( x T ) 2πσ exp w ki 1 σ 2 (1) where x is the configuration input vector, which is the pattern characteristics. The w ki is the weight for the k th neuron of class C i, k N i, whereas N i is the number of neurons of C i. In addition, x was normalized so that x T x =1and w T ki w ki = 1. The parameter σ is the Gaussian standard deviation, which determines the receptive field of the Gaussian curve. p i (x) = Ni F k,j (x) k=1 (2) k =1, 2,..., N i ; i =1, 2,..., C The next step is the summation layer. In this layer, all weight vectors are summed according to Equation 2, in each cluster C i producing values p i (x), where C is the total number of classes, that is, the number of categories. 60

4 TABLE II: PSNR to MOS conversion PSNR,dB MOS TABLE III: Correlation rate PNN-based QoE evaluation model PSNR metric R 92.86% 48.57% Finally, for the selection of the classes, we consider the most likely classes pointed out by the summation layer based on a chosen threshold. In this study, PNN is trained with the distorted-video database. The training process refers to establishing the mapping relation between the network input D i and target output Q i, based on seventy groups of input-target sample data. The maximum training times is 100, without limit to the maximum training duration. Fig. 4: Training result of PNN IV. TRAINING RESULT ANALYSIS In our study, we also conducted PSNR metric [6]. PSNR calculates the MSE between the original o and the distorted d versions of a video sequence, as follows: MSE = 1 M N M i=1 j=1 2 I max PSNR =10 log 10 MSE N (o i,j d i,j ) 2 (3) where each frame has M N pixels, and o i,j and d i,j are the luminance pixels in positions (i, j) of a frame of the sequence. I max is the maximum possible pixel value. The mapping between MOS and PSNR is given in Table II. The precisions of the mapping function can be measured with different metrics. In this study, the correlation rate of subjective MOS and training result is calculated as: R = N γ i { i N,γ 1,Mi = M i = i 0,M i M i where M i is the subjective MOS of the ith video, M i is the output of trained mapping model, and N is the number of video sequences. Fig. 4 shows the scatter plots of PNN model. When the subjective MOS and the PNN output overlaps, it means that the PNN output correlate with the subjective MOS. From Fig. 4 it can be seen that the variation trends of the result of both training output and target values are basically accordant. On the other hand, we calculate all the PSNRs based on the distorted videos and original video. Then we get the MOSs (4) (5) Fig. 5: Result of PSNR metric corresponding to the PSNRs according to Table II. As shown in Fig. 5, the outputs do not correlate well with subjective MOSs. As for the correlation rate, PNN model gets a correlation rate of 92.86% as shown in Table III, while PSNR metric gets one of 48.57%, which means good fitting effect is achieved in PNN-base QoE evaluation model and the PNN training can provide significant improvements in the classification rate when compared with the PSNR metric. In our test, the built PNN-base QoE evaluation model can basically reflect the relationship between QoS and QoE, and the prediction can be conducted with the trained network. Our QoE evaluation model can evaluate the quality of video applications in a manner that is close to real-human observations. V. CONCLUSION In this paper, we have proposed a PNN-based QoE measuring model for video applications over LTE system. With the help of the equipment of video quality evaluation model, based on PNN, we established the relationship between the network parameters such as Jitter, Delay, LR, MLBS, etc. and 61

5 MOS. This model does not rely on the interaction from real humans once PNN is trained and weaken the human labor. By using PNN, there is no need to study the complex relationship of QoE-affecting parameters and users perceived quality and it can automatically be learned appropriate classification rules by applying sample training patterns. It also outperforms the PSNR metric and provids a high correlation rate with the human perception. Furthermore, based on this PNN, we can also adjust the input network parameters to get the ideal output to satisfy the users need. REFERENCES [1] 3GPP, UTRA-UTRAN Long Term Evolution (LTE) and 3GPP System Architecture Evolution. Available from: [2] ITU-T Recommendation P.10/G.100 (2006) Amendment 1 (01/07), P.10: New Appendix I C Definition of Quality of Experience (QoE), January [3] ITU-R Recommendation P.800, Methods for subjective determination of transmission quality, [4] ITU-T Recommendation P.910, Subjective video quality assessment methods for multimedia applications, International Telecommunication Union, Geneva, Switzerland, [5] S. Mohamed and G. Rubino, A study of real-time packet video quality using random neural networks, IEEE Trans. Circuits Syst. Video Technol., vol. 12, no. 12, pp , [6] J. Klaue, B. Rathke, and A. Wolisz, EvalVid. A framework for video transmission and quality evaluation, in Proceedings of the International Conference on Modelling Techniques and Tools for Computer Performance Evaluation, 2003, pp [7] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13:600C612, [8] M.H. Pinson and S. Wolf. A new standardized method for objectively measuring video quality. IEEE Transactions on broadcasting, 50(3):312C322, [9] P. Simoens, S. Latre, B. De Vleeschauwer, W. Van de Meerssche, F. De Turck, B. Dhoedt, P. Demeester, S. Van Den Berghe, and E. Gilon, Design of an autonomic QoE reasoner for improving access network performance, in The Fourth International Conference on Autonomic and Autonomous Systems (ICAS), [10] G. Rubino, M. Varela, and J.-M. Bonnin, Controlling multimedia QoS in the future home network using the PSQA metric, Computer Journal, vol. 49, no. 2, pp , [11] H. Du, C. Guo, Y. Liu, and Y. Liu, Research on relationship between QoE and QoS based on BP neural network, Network Infrastructure and Digital Content, IC-NIDC IEEE International Conference on, vol., no., pp , 6-8 Nov [12] Donald E Specht, Probabilistic neural networks, Neural Networks, 3, (1990). [13] J. Tang, C.-Y. Zhang, and B. Luo, A graph and PNN-based approach to image classification, in Proceedings of International Conference on Machine Learning and Cybernetics (ICMLC 05), vol. 8, pp , Guangzhou, China, August [14] P. M. Ciarelli, R. A. Krohling, and E. Oliveira, Particle swarm optimization applied to parameters learning of probabilistic neural networks for classification of economic activities, Particle Swarm Optimization, ISBN: , pp [15] 3GPP TR v1.4.1, Further Advancements for E-UTRA Physical Layer Aspects, Sep [16] K. Piamrat, A. Ksentini, C. Viho, and J.-M. Bonnin, QoE-based network selection for multimedia users in IEEE wireless networks, LCN 08, pp , Oct