Proceedings of Meetings on Acoustics

Proceedings of Meetings on Acoustics Volume 19, 213 http://acousticalsociety.org/ ICA 213 Montreal Montreal, Canada 2-7 June 213 Engineering Acoustics Session 2pEAb: Controlling Sound Quality 2pEAb1. Subjective and objective evaluation of sound quality of radio programs transmitted via Digital Audio Broadcst(DAB+) System Andrzej B. Dobrucki* and Maurycy J. Kin *Corresponding author's address: Faculty of Electronics, Wroclaw University of Technology, Wybrzeze Wyspianskiego 27, Wroclaw, 57, Lower Silesia, Poland, andrzej.dobrucki@pwr.wroc.pl The work presents results of research on the sound quality of different radio programs transmitted via Digital Audio Broadcasting (DAB+). This assessment has been provided with a use of psychoacoustic model as well as standard listening tests, using an ACR (Absolute Category Rating) method of scaling, and CCR (Comparison Category Rating) method. Results have shown that sound quality gets worse when bitstream is of the lowest values (48 kbit/s or 24 kbit/s). Application of the Spectral Band Replication processor significantly improves the perceived quality, which is satisfying for bit streams higher than 64 kbit/s, particularly for jazz and popular music. The assessment with ACR method (recommended for broadcast by International Telecommunication Union) showed better notes than CCR one. It means that recommended method is less critical. Also results obtained with psychoacoustic model are more similar to obtained with CCR method. The attributes of spatial impression change in different ways. The greatest distortion has been observed for the perspective and spaciousness of sound image, while the sound color as well as localization stability and accuracy of phantom sources remained almost the same. Published by the Acoustical Society of America through the American Institute of Physics 213 Acoustical Society of America [DOI: 1.1121/1.479917] Received 15 Jan 213; published 2 Jun 213 Proceedings of Meetings on Acoustics, Vol. 19, 351 (213) Page 1

INTRODUCTION The quality of digital audio signals is defined by different parameters than in analogue case. Such objective parameters as signal-to-noise ratio, linearity, distortion and the bandwidth frequency [7] are not in use because of the nature of this kind of transmission. The resulting quality of audio signals is mainly determined by the parameters of the coding algorithms and it is mostly independent from the parameters of transmission path in traditional analog meaning, based not only on electroacoustic devices but also on some aspects of propagation of radio waves. There could be found many aspects of objective sound quality evaluation of so called digital sound, and they concentrate for example on the high-order formats of audio. Another interest of quality defining and measurement comes from data reduction of the transmitted signals and it leads to answer the question: how much information could be lost or changed and not affecting a subjective quality of the sound? The impairments introduced by a coding system include linear distortion, quantization noise, pre-echoes, frequency bandwidth limitations, and changes in timbre and stereo image. Thus, it is necessary to find a measurement method which is suitable to that assessment. One of them is the CCR (Comparison Category Rating) method, based on the subjective comparison between two signals: the original and the compressed one. Another one is the ACR (Absolute Category Rating) method - the subjective assessment which is recommended for the broadcasted signal [6]. These methods of scaling differs in performance, i.e. subjects have to give their opinion in absolute scale without comparing to the original signal (ACR) in a 5-degree scale (from very annoying to imperceptible distortion), while in the CCR listeners had to answer what was the perceived difference between the second interval in comparison to the first in particular pair, and their opinions were according to recommended grade: from to +3: the lowest note () meant that the second sound sample had its quality not acceptable while the highest (+3) note was given when the second sample sounded much better than the first one. The ACR method of evaluation reflects the real impression of listeners while listening to the radio programs and by the CCR method one can evaluate an overall impression as the assessed quality of two signals. The main aim of the research is to compare both methods mentioned above and to find the minimal value of bit-rate which guarantees almost the same hearing impression than obtained by an original uncompressed signal. Moreover, the results of subjective tests were compared to the results of so called objective measurement of sound quality based on the psychoacoustic model. The concept of psychoacoustic modeling is based on the comparison between two signals: the original and the compressed one. This comparison based on the human hearing properties allows to evaluate which artifacts appearing at the coding process can be detected by human hearing system what influences the general sound quality. Psychoacoustic model it is a mathematical model defining which components or information of sound could be perceived, or not, by human ear in very particular situations. An output estimation of perceived components of difference signal takes into account the complex of hearing process, thus, the final results of measurement can be regarded as inner representation of human hearing which refers to psychophysical representation in impression domain. This procedure as well as the model had been tested previously as a tool for sound quality of vacuum-tube amplifiers [5] as well as deformations of signals done by some MP3 codecs [4]. In those cases, the high correlation between these so called objective results and subjective assessment by a large group of experienced listeners has been obtained for both subjects of research. METHODS OF RESEARCH Objective Assessment Thirteen sound samples of radio programs recorded at the analog output of consumer set DAB Clint AUDIO 1 passed by transmission channel: multiplex DAB transmitter receiver have been tested. The recording machine was DAT Tascam DA 3, with sampling frequency of 44,1 khz. These samples contained all kinds of musical styles (from to heavy metal) as well as male/female speech samples. The samples were transmitted with six different bit-rates (136 kbit/s, 128 kbit/s, 96 kbit/s, 8 kbit/s, 64 kbit/s and 48 kbit/s) and sampling frequency 48 khz. The lowest bit-rate equal to 24 kbit/s was not taken into account because of the worse quality which is independent on further operations so the results have been omitted. For the all bit-rate values, a Spectral Band Replication processor as well as Parametric Stereo processor for the lowest bitstream, have been on (together or separately). As a reference signals, Proceedings of Meetings on Acoustics, Vol. 19, 351 (213) Page 2

CD recordings with the identical samples to broadcasted ones, have been used. In addition, the same samples have been tested after passed by a simulation of typical FM channel. Perceptual Measurement Quality System (PQMS) with Perceptual Audio Quality Measure Algorithm (PAQM) protocol [1], [2] were applied to objective research. It should be noted that PAQM protocol was based on a bark scale in this case. [1, 3]. The output results from the model are given as a value of similarity between two samples: when these signals are exactly the same, the output value is of zero, and when two different signals are compared, the model output gives a value of. Subjective Tests Subjective evaluation of sound quality of DAB+ radio programs has been provided with ten-pieces group of listeners. The paradigm of stimuli was based on both: ACR and CCR procedures: subjects have to give their opinion in absolute scale without comparing to the original signal (ACR) in a 5-degree scale, or two samples, the reference and tested, are presented in pairs with random order (CCR) and listeners had to answer what was the perceived difference between the second interval in comparison to the first in particular pair, in a 7-grade scale. The evaluated samples were exactly the same as used in objective evaluation with the psychoacoustic model. The all listeners were experienced in listening tests and their hearing thresholds had no more than 5dB difference with the reference to normal hearing, what had been confirmed previously with an audiometric tests. RESULTS AND ANALYSIS Results of both types of research taking into account the overall quality, are presented in Fig. 1. Because of statistical treatment by the means of t-student testing has shown that all but two kinds of musical material (an opera and ) could be averaged for specific bit-rates, and the values obtained with the psychoacoustic model have been averaged over the 11 signals (t < t α = 2.23 at α =.5). In the case of subjective assessment, the results show the similar way of change, and they have been divided into two groups: music and pop&jazz and they have been averaged over the six signals for each group. Moreover, the variances of results obtained for particular sound material as well as for all bit-rates have been found as homogeneous what have been shown in statistical Bartlett test (χ 2 < χ α 2 = 5.99 at α =.5). Thus, it allowed to average the results over all the ten listeners and their three repetition for the 6 types of program. It can be seen that the results of objective assessment of musical samples are similar to the ones obtained by subjective test with CCR method. In order to confirm that both results: objective and subjective, are similar, the sum of ranks testing with the asymptotic distribution χ 2 has been used. It turned out that for the lower bit-rate value (48 kbit/s) the results of subjective assessment are in a good agreement with the objective measurements (χ 2 < χ α 2 = 3.841, α =.5) but when the SBR processor is on these results are different from statistical point of view (χ 2 > χ α 2 = 3.841, α =.5), and the main conclusion resulting from this part is that listeners opinion gets worse than the measurement results with the use of psychoacoustical model. It means that for quick and simple evaluation of signals the objective method may be used for non-critical purposes. Proceedings of Meetings on Acoustics, Vol. 19, 351 (213) Page 3

-,5 quality assessment,5,5 objective assessment subjective assessment (CCR) FIGURE 1. Results for research of sound quality for different bit-rates for 11 kinds of musical programs. Results have shown that sound quality of tested material strongly depends on bit-rate values (t = 8.97 > t α = 2.77, α =.5) and the use of Spectral Band Replication processor makes the sound quality significantly better (t = 12.7 > t α = 2.77, α =.5) for lower values of bit-rate, but there is no statistical difference in results when Parametric Stereo processor is on (t < t α = 2.8, α =.5). The results for the distinguished kinds of music can differ from 5% to 8% in comparison to the averaged values what could be seen in Fig. 2. Moreover, the greatest differences have been observed for the higher bit-rates for lower bitstreams these differences are smaller. It clearly means that signals with complex spectrum and the time structure can be taken into account as specific material and some tools useful for so called simple signals cannot be simply applied to more complex musical structures and this complexity is reduced when compression of data is relatively high. The obtained results also show that traditional frequency modulation (FM) achieves the sound quality nearly the same as the bitstrem of 96 kbit/s without SBR, or 64 kbit/s with SBR do. It should be noted that the listeners opinion was that some signals have been assessed as good ones but most of subjects pointed out that the samples sounded different as they were under various mix process. It is a main feature of this method of subjective assessment which allows to compare two signals directly when presented one by one. When the pattern is a CD sample which provides a highest sound quality, the method applied in the described research is more critical than the ACR method recommended by the ITU [6]. The results of subjective testing with the ACR method are presented in Fig. 3. Comparing the results from Fig. 2 and 3, it can be noted that the CCR method is more accurate, especially for higher bitstreams, what allows listeners to distinguish little differences between two audio signals. Proceedings of Meetings on Acoustics, Vol. 19, 351 (213) Page 4

-,5 136 128 96 8 64 48 FM CCR results,5,5,5 FIGURE 2. Results of sound quality assessment obtained by the CCR method for different bit-rates and different musical programs. 6 5 ACR results 4 3 2 1 136 128 96 8 64 48 FM FIGURE 3. Results of sound quality assessment obtained by the ACR method for different bit-rates and different musical programs. On the base of obtained results for various kinds of sounds it could be noted that the most degradation of sound quality took place for signals with complex time/frequency structure, i.e. opera arias as well as a heavy metal piece, but it should be noted that the dependence of the bit-rate on the objective and subjective evaluation is not linear. For the rest kinds of the sounds this dependence is more or less linear, with the exception of SBR processor being on. Figure 4 presents the results of CCR testing but the listeners task was to evaluate the spaciousness of musical recording. Proceedings of Meetings on Acoustics, Vol. 19, 351 (213) Page 5

CCR of spaciousness,5 -,5,5,5,5 FIGURE 4. The results of CCR testing of spaciousness of musical recordings.,5 CCR of sound color -,5,5,5,5 FIGURE 5. The results of CCR testing of sound color of musical recordings. It can be seen that the spatial attributes of sound (as represented by the spaciousness impression) rapidly get worse for bit-rates lower than 96 kbit/s (the similar way of distortion has been observed for the perspective of musical piece), while the sound color (as well as localization stability and accuracy of phantom sources) remained almost the same for wide range of bit-rate values (from 64 kbit/s with SBR, up to 136 kbit/s). In authors opinion, this fact allows to give some directions of sound material preparation when it is designed for digital broadcasting with a data compression without any significant degradation of quality. Proceedings of Meetings on Acoustics, Vol. 19, 351 (213) Page 6

CONCLUSIONS If one assumed that the sound quality of tested programs has been assessed as only a little bit degraded in comparison to the original CD quality (with average rating of -.5) the satisfactory sound quality can be obtained with the bit-rate values of 136 kbit/s, 1,28 kbit/s, 96 kbit/s, 8 kbit/s and 64 kbit/s but with SBR processor being on at the last case. The other thing which could be considered is the fact that listening test were performed in a recording studio with good acoustical conditions, without any disturbing outer sounds. The two facts mentioned above are far away from typical listening to the radio. Moreover, today s young generation which lives digitally does not care about CD, DVD or FM Radio because they discover the music from YouTube and other internet sources, very often with data-compression. In authors opinion, the sufficient sound quality for the radio consumers can be guaranteed by the bit-rate of 64 kbit/s with Spectral Band Replication processor being on. Only for two kinds of musical styles (from total amount of 15) the significant degradation of sound quality has been observed. It simply suggests that for some specific groups of listeners and some radio program profiles, DAB+ system should be set with special parameters. REFERENCES 1. J. G. Beerends, J. A. Stemerdink, Perceptual Audio Quality Measure Based on a Psychoacoustic Sound Representation (PAQM), J. Audio Eng. Soc., vol. 4, no 12 (1992). 2. J. G. Beerends, J. A. Stemerdink, A Perceptual Speech Quality Measure Based on a Psychoacoustic Sound Representation (PAQM), J. Audio Eng. Soc., vol. 42, no 3 (1994). 3. W. Hoeg, T. Lauterbach, Digital Audio Broadcasting, Principles and Applications of Digital Audio, Wiley, 2 nd edition (23). 4. P. Z. Kozłowski, A. B. Dobrucki, The comparison between subjective and objective, perceptual based evaluation of compressed speech and audio signals, Proc. of the 18 th ICA, Kyoto, Japan, 1811814, (24). 5. A. B.Dobrucki, S. J. Maleczek, M. J. Kin, Subjective and objective evaluation of the vacuum-tube amplifiers, 126 th AES Convention, Munich, Germany, paper 786, (29). 6. ITU-R: Recommendation 1284. Methods for the subjective assessment of sound quality general requirements. Geneva, Switzerland, (1997). 7. IEC Rep., Publication 2683: 1985, Listening test on Loudspeaker. Bureau Central de la Commision Electrotechnique Internationale. Geneva, Switzerland, (1985). Proceedings of Meetings on Acoustics, Vol. 19, 351 (213) Page 7