Assessing Objective and Subjective Quality of Audio/Video for. Internet based Telemedicine Applications

Transcription

1 Assessing Objective and Subjective Quality of Audio/Video for Internet based Telemedicine Applications Dissertation Proposal Submitted to School of Information Science Claremont Graduate University Bengisu Tulu November 8, 2004

2 Table of Contents Chapter 1: Introduction... 2 Chapter 2: Problem Statement... 7 Chapter 3: Literature Review Telemedicine, Telehealth, and e-health Audio/Speech Quality Measures Objective Measures Subjective Measures Video Quality Measures Objective Measures Subjective Measures Quality Measurement in Telemedicine applications Session Initiation Protocol (SIP) for Internet-based Videoconferencing Chapter 4: Proposed Study Stage 1 Development of Telemedicine Taxonomy Definition of Taxonomy Dimensions Interaction of Proposed Dimensions Stage 2 Assessment of Objective and Subjective Quality Measures for Telemedicine Technological Factors Affecting Quality in Telemedicine over IP Networks Experimental Test-bed Experimental Procedures Stage 3 Development of SIP-based Videoconferencing Tool with Real-time Telemedicine Capability Index Research Methodology Contributions and Potential Implications Timeline References of 60

3 Chapter 1: Introduction Health expenditures as a share of Gross Domestic Product (GDP) have been rising in the United States and other member countries of the Organization of Economic Cooperation and Development (OECD) since 1960s. A study, conducted to examine the reasons for this increase, concluded that Information technology (IT) can play an important role to reduce increasing costs [1]. However, experts around the world believe that new demands in providing healthcare will require fundamental changes in the structure of the industry. Besides the failure to disseminate medical knowledge quickly enough or use it in a methodical manner, there is another shortfall: medical practitioners with scarce, specialized knowledge cannot bring it to bear beyond their geographical confines [2-4]. Telemedicine s effort to bridge this gap has been reported repeatedly [2, 3]. Telemedicine, and associated technologies, are touted as critical to solve the above-mentioned problems. As a result of growing interest in telemedicine during the last decade [5], many telemedicine applications have been developed and deployed during recent years [6]. Telemedicine and related healthcare technologies aim to provide efficient healthcare to improve the well being of patients and bring medical expertise at a lower cost to the right people at the right time. The Internet is moving towards becoming the most widely used communication medium around the world. Given its objective, telemedicine has been quite slow in utilizing Internet technologies to provide medical expertise for a larger audience. One of the reasons for this slow adoption is the quality that can be provided through varying Internet connections. 2 of 60

4 In telemedicine, quality of the data obtained at the receiving end of the connection is critical for the medical decisions. The final outcome of the medical session and hence the success of the whole process, depends on the amount and quality of the data received. Lack of necessary information may increase the frustration and dissatisfaction for parties involved and may lead to erroneous decisions, which can severely affect the overall outcome of a telemedicine event. To prevent this frustration and the negative effects of an unsuccessful telemedicine experience, current telemedicine applications are conducted with high quality equipment over high-speed connections that are not IP based. Hence, the spread of telemedicine to areas which are underdeveloped and which do not have a good telecommunications infrastructure to support such expensive connection lines and equipment have been limited. The Internet is more or less accessible from many locations in the world for a very low cost compared to the current alternatives used in telemedicine. Figure 1.1 illustrates an end-to-end Internet-based telemedicine network implementing several telemedicine scenarios. For each scenario, user requirements and available technical infrastructure may vary and this variation affects the quality of medical decisions made in a session. IP-based telemedicine systems are vulnerable to various impairments that can occur at the physical, network, and application levels. Unlike circuit switched networks, the Internet is a packet switched network where packet loss, delay, and delay variation can occur easily. In addition, available bandwidth can vary from one location to another. On the application level, different coding and compression techniques have been developed to enable the transmission of audio/video data, which can consume large amounts of bandwidth. On the network level, to provide a more reliable and stable connection, many service providers are offering Quality of Service (QoS) features that provide some guarantee of performance such as traffic delivery 3 of 60

5 priority, speed, delay, or delay variation by prioritizing and guaranteeing bandwidth for selected applications to achieve optimal service performance. However, implementation of these features is still very limited. Figure 1.1 End-to-End IP-based Telemedicine Networks Even though information quality is a critical factor in medical decision making, some decisions may not require the highest quality of information possible. Certain decisions can be made with 4 of 60

6 limited quality. An examination of current knowledge reveals that there is no study focusing on answering the following question: For any given communication channel and a given telemedicine purpose, what is the right amount of information to transfer given the limitations of communication technology and the devices in use for specific scenarios? Knowing the boundary for the minimum information required can help us utilize limited channels and prevent unnecessary information load on larger channels. Today, there is an increased awareness of the errors that may occur in medicine and the lack of decision support systems and tools that can help physicians. The problem is particularly acute when it comes to telemedicine. In order to provide support for physicians that are involved in telemedicine events, this study proposes to develop and utilize objective and subjective audio/video quality measures to calculate a real-time quality index given a specific telemedicine setting for IP-based networks. This index will be utilized by study participants, while making decisions, regarding whether a telemedicine setting is capable of providing the required quality to complete a specific task in a given application domain. Early evaluation of the existing setting can prevent frustrations and time loss while enhancing the response time and satisfaction in telemedicine events. This study will first, investigate the factors that affect the quality of information required in a telemedicine event that will lead to a taxonomy of telemedicine applications. Second, considering that audio and video are the two data formats that are most affected by the factors affecting quality, objective and subjective quality measures for providing real time feedback to the participants of a telemedicine event will be collected in an experimental testbed. Finally, an a videoconferencing tool that can predict and present perceived quality of a telemedicine session based on the objective values collected in real time will be developed. 5 of 60

7 Contributions of this study are threefold. First, it will provide a telemedicine taxonomy for classifying different telemedicine events based on factors affecting quality and outcome. Second it will develop a subjective measure database for telemedicine in two application domains and investigate a heuristic measure to predict perceived quality of a telemedicine event in real time. Third, it will implement this heuristic method in a new artifact, a videoconferencing tool with a telemedicine capability indicator, capable of measuring objective quality and providing subjective quality feedback to users in real time. One limitation of this study is that the subjective measures will be developed only for two specific telemedicine events (listening to heart beats and viewing an eye image). The generalizability of these measures to other areas of telemedicine is unknown; hence future research will be needed to test and expand these quality measures to other application domains and telemedicine events. Another limitation may arise from the number of subjects that will be recruited for subjective tests. To minimize the effects of this limitation, ITU s minimum requirements for subjective tests will be the criteria for recruiting subjects. 6 of 60

8 Chapter 2: Problem Statement Since the introduction of the term telemedicine, various studies have outlined how one can utilize this new technology and reap its benefits. However, telemedicine remained a black box for the public as a result of the fact that even the authorities have not yet reached a consensus on a clear and precise definition of telemedicine content and boundaries [7]. What is telemedicine and how does it differ from traditional medicine? What are the necessary new laws and regulations to bring this technology across the globe? Despite the fact that the government has supported it and there have been continued reductions in the equipment and transmission costs, there have not been enormous numbers of actual implementations of telemedicine. Among the many barriers listed in the literature, the lack of information about the effect of telemedicine on cost, quality, and access has been a significant one [8]. It is important to analyze and predict the success of the future applications to make better decisions. In telemedicine, the quality of the data obtained at the receiving end of the connection is critical for the medical decision. The final outcome of the telemedicine session and hence the success of the entire process depends on the amount and quality of the data received. If the information necessary to make a decision cannot be retrieved during the telemedicine event, both parties involved in the process will feel frustration and dissatisfaction, which can severely affect the results of telemedicine visit. To prevent this frustration and the negative effects of an unsuccessful telemedicine experience, current telemedicine applications are conducted over high-speed connections that are not IP based. One drawback of this approach is the barriers it introduces regarding the spread of telemedicine to areas which are 7 of 60

9 underdeveloped and which do not have a good telecommunications infrastructure to support such expensive connection lines. On the other hand, the Internet is more or less accessible from many locations worldwide for a very low cost compared to the current alternatives used in telemedicine. Use of the Internet for telemedicine has been studied for the last ten years, and some of these studies have demonstrated that even on Internet and IP-based connections it is feasible to conduct telemedicine sessions, but no study has been able to provide convincing evidence to the telemedicine community for its widespread adoption. The unreliable connection properties of IPbased systems prevent the spread of these applications. However, if one can measure the predicted quality that can be obtained for a specific telemedicine setting and compare the predicted quality with the requirements outlined by the parties involved, then a feasibility and capability value can be presented to the parties involved and a decision to either continue with the telemedicine event or switch to an alternative method can be made. Early evaluation of the existing setting can prevent frustration and time loss while enhancing the response time and satisfaction. An evaluation method, which is described here, introduces the existing gap in the literature for predicting quality of telemedicine event settings. Existing studies [9] indicate the importance of studying quality of the existing or future applications. Use of quality improvement process not only results in improved output quality but it also makes the production process sensitive to changes in input, output and the environment. [10] Quality is important; however, the channel used to deliver this quality information is limited. Thus, there is a need to understand the requirements of each application in its own domain and 8 of 60

10 define the amount and quality of information required to provide telemedicine services. It is important to classify telemedicine applications based on their potential use by taking the medical domains they serve into consideration. Then one can identify the IT infrastructure needs and requirements for each of these applications in order to provide a satisfactory telemedicine experience to end users. There are a variety of applications, devices, and communication technologies that are used in telemedicine. The reasons for this variety are: (1) the diversity of telemedicine locations and physical limitations of each location; (2) the application areas that utilize the telemedicine applications; and (3) the purpose for the use of telemedicine. Communication infrastructure technologies, such as telephone lines or leased lines, also have a critical impact on the applications utilized in telemedicine, and hence, on the outcome. A telediagnosis case in the psychiatry domain or a teleconsultation case in telecardiology domain are expected to have different requirements since the information that is necessary to make a clinical decision differs based on the application domain. Therefore, it is not reasonable to expect similar results from the same technology when it is being used in different domains and/or for different purposes. With the goal of solving some of the problems introduced above, this proposed study will try to answer the following research questions: RQ1. What are the factors that affect the quality of information required in a telemedicine event? RQ2. Given the current set of objective and subjective audio and video quality measurements, which ones if any provide the appropriate quality assessment for IP-based telemedicine systems that can aid decision makers? 9 of 60

11 RQ3. Using the appropriate objective and subjective quality assessment measures for telemedicine, is it possible to integrate them as a capability index in a SIP-based desktop videoconferencing tool to provide real time feedback for decision makers regarding quality? 10 of 60

12 Chapter 3: Literature Review In this chapter, an overview of literature in telemedicine as a concept, voice and video quality measurement techniques in general, and quality measurement in telemedicine is presented. The first section introduces a brief summary of telemedicine literature including various definitions of telemedicine and related terminology, as well as how the technological changes affected the evolution of telemedicine applications. Next two sections provide a summary of objective and subjective quality measures developed and utilized in prior research and practice for voice and video respectively. The forth section summarizes the quality measurement literature in telemedicine and provides a list of factors used in these quality measurement studies. The last section introduces the Session Initiation Protocol, which will be utilized in this study to develop a videoconferencing application for telemedicine. 3.1 Telemedicine, Telehealth, and e-health Telemedicine has various potential uses such as clinical, educational and administrative. The promising potential of bringing high quality service to under-served areas via telemedicine is an example of how IT can reduce the quality-adjusted cost. Bashshur [7] notes that telemedicine provides a solution to the problems such as access to care for large segments in the population, continuing healthcare cost inflation, and uneven geographic distribution of quality by: (1) enhancing accessibility to care for underserved populations, (2) containing cost inflation as a result of providing appropriate care to remote patients in their home communities, and (3) improving quality as a result of providing coordinated and continuous care for patients, targeted 11 of 60

13 and highly effective continuous education for providers, and highly effective tools for decision support. The evolution and growth of telemedicine is highly correlated with the developments in communication technology and IT software development. This dependency is evident if we quickly browse through the history of telemedicine technologies, which was categorized into three eras [11]. All the definitions during the first era of telemedicine focused on medical care as the only function of telemedicine. The first era can be named as telecommunications era of the 1970s [11]. Telemedicine programs during the first era ended as the government terminated funding before these programs matured. It is important to note that telemedicine is a product of the information age, just as the assembly line was the product of the industrial age. [11]. The application in this era was dependent on broadcast and television technologies where telemedicine application was not integrated with any other clinical data. The second era of telemedicine, the dedicated era, started during the late 1980s as a result of digitization in telecommunications and it grew during 1990s [11]. The transmission of data was supported by various communication means ranging from telephone lines to Integrated Service Digital Network (ISDN) lines. The high costs attached to the communications infrastructure that can provide higher bandwidth became an important bottleneck for telemedicine. The dedicated era turned into the Internet era where more complex and ubiquitous networks are supporting telemedicine. The third era of telemedicine is supported by the technology that is cheaper and accessible to an increasing user population [11]. The enhanced speed and quality offered by Internet2 is providing new opportunities in telemedicine as well. In this new era of telemedicine, the research strategies should include an understanding of the functional 12 of 60

14 relationships between telemedicine technology and the outcomes of cost, quality, and access beyond the assessment of technical sufficiency [11]. During the evolution of telemedicine, new terminologies were developed as the applications and delivery options increased in variety, and the application areas expanded to almost all the fields medicine can cover. This resulted in confusion and misidentification of what could be termed telemedicine and what could be termed telehealth or e-health. This became even more complicated as these fields advanced. Cybermedicine is yet another term introduced lately into the literature. Since the first formal definition of telemedicine by Bird in 1971, many researchers tried to define this term in order to clarify the boundaries of telemedicine and its use. Even though the core of these definitions is the same, telemedicine, and hence its definition, evolved dramatically as a result of the tremendous changes experienced in the telecommunication and information technologies. These changes were so significant that new terminologies like telehealth, e-health, and others were introduced, and explaining the difference between telemedicine and these new terms became important. Studies defined telehealth as a big umbrella that encompasses more applications than the definition of telemedicine can cover [12, 13]. Table 3.1 presents a selected list of definitions proposed in the literature for telemedicine, telehealth, and e-health. This list of definitions gives an indication of the competing terminologies; more terminologies may be introduced in the future as further technological advances are achieved. Therefore, it is important to understand that the purpose of research in this field is to support the ultimate quest which is to cure disease, prevent it if possible, reduce infirmity, and enhance quality of life, as stated by Bashshur [7]. 13 of 60

15 Some may question whether this is telemedicine, telehealth, e-health, health informatics, or biohealth informatics. It does not really matter what we call it or where we draw boundaries. collective and collaborative efforts from various fields of science, including what we call now telemedicine is necessary. [7] Table 3.1. Definition of terms Definition Telemedicine is the practice of medicine without the usual physician-patient confrontation via an interactive audio-video communications system. Telemedicine is a system of care composed of six elements: (1) geographic separation between provider and recipient of information, (2) use of information technology as a substitute for personal or face-to-face interaction, (3) staffing to perform necessary functions (including physicians, assistants, and technicians), (4) an organizational structure suitable for system or network development and implementation, (5) clinical protocols for treating and triaging patients, and (6) normative standards of behavior in terms of physician and administrator regard for quality of care, confidentiality, and the like. Telemedicine is the use of electronic information and communications technologies to provide and support healthcare when distance separates the participants. Telemedicine is the delivery of health services when there is geographical separation between healthcare provider and patient, or between healthcare professionals. Telemedicine is the provision of healthcare services, clinical information, and education over a distance using telecommunication technology. Telehealth is the removal of time and distance barriers for the delivery of healthcare services or related healthcare activities. (In this study, telemedicine is a subset of telehealth) E-health refers to all forms of electronic healthcare delivered over the Internet, ranging from informational, educational, and commercial products to direct services offered by professionals, non-professionals, businesses or consumer themselves. Ref Bird [11] 1975 Bashshur [11] 1996 Committee on Evaluating Clinical Applications of Telemedicine [14] 2001 Miller [8] 2001 Maheu [13] 2001 American Nurses' Association [12] 2001 Maheu [13] 14 of 60

16 It is important to note that the ultimate goal of any telemedicine effort is to improve the well being of patients. However, since the first definition of the term, uncertainty on the meaning of telemedicine became evident over time. This uncertainty is hindering efforts in developing a clear definition and a classification method for telemedicine. Previous attempts [14] to classify telemedicine were motivated by the demand for evidence of its effectiveness and therefore, were focused on developing a strategy to evaluate the telemedicine applications and their effects on quality, accessibility or cost of healthcare. In 1996, Committee on Evaluating Clinical Applications of Telemedicine published a report [14] that classified clinical application of telemedicine under six categories (p.30): (1) initial urgent evaluation, (2) supervision of primary care, (3) provision of specialty care, (4) consultation, (5) monitoring, and (6) use of remote information and decision analysis resources to support or guide care for specific patients. The broad classification that will be developed for this study, which is more focused on identifying different dimensions of telemedicine and telehealth, and which can then be used to identify user requirements for different categories in an organized manner, is expected to have a positive impact on the use and development of current and future applications. 3.2 Audio/Speech Quality Measures Measuring speech quality has been researched for many years and its results were utilized in public-switched telephone networks (PSTN). The goal of many studies was to find an objective measure that can be used to predict the perceived subjective quality of a human subject. Many objective and subjective speech (audio) quality measures were developed. However, most of these measures, if not all, were originally developed for PSTN networks, which are circuitswitch networks, and recent research indicates that these measures may not work well for packet switched networks, such as Internet telephony [15]. In his work, Hall [15] compared three 15 of 60

17 different measures, which are: (1) perceptually weighted distortion measures such as enhanced modified Bark spectral distance (EMBSD) and measuring normalizing blocks (MNB), (2) worderror rates of continuous speech recognizers, and (3) the ITU E-model, under conditions of a typical VoIP system. The results of his study indicate that E-model provides the highest correlation with Mean Opinion Score (MOS) for VoIP systems. This section summarizes the most popular quality measures with a brief description of each one Objective Measures The most widely adopted objective speech quality measure is the Signal-to-Noise Ratio (SNR), which compares original and processed speech signals sample by sample. The SNR is the simplest measure possible as it measures the distortion of the waveform coders that reproduce the input waveform [16]. The SNR is also defined as ratio of the energy of the original target source to the energy of the difference between original and reconstruction that is, the energy of a signal which, when linearly added to the original, would give the reconstruction [17]. A modified version of the SNR is called segmental Signal-to-Noise Ratio (SNRseg), which decomposes the entire signal into segments and calculates the average SNR of these short segments [16]. These measures are easy to compute; however, their disadvantages limit their use in various scenarios. First of all, SNR measures require access to the original signal, which eliminates them for use in real time measurements. Other drawbacks of these time-domain measures are reported in [16, 17]. When speech quality must satisfy human listeners, there is no better way then performing subjective tests. However, due to the cost of such evaluations, researchers often utilize algorithms that can estimate the outcomes of these tests. These algorithms can be grouped under perceptual models whose measures are based on human auditory perception models[16]. One 16 of 60

18 example for these perceptual models is Bark Spectral Distortion (BSD) [18], which is based on the assumption that speech quality is directly related to speech loudness (the magnitude of auditory sensation). It works well when the distortion in voiced regions represents the overall distortion, and hence identifying the voiced regions is required [16]. An Enhanced Modified BSD, which consists of a perceptual transform followed by a distance measure that incorporates cognition model, was also proposed. Based on the test in [16], its correlation with subjective results is relatively good for encoding impairments but poor on network impairments. Another example of perceptual models for estimating subjective quality is the ITU Recommendation P.861, Perceptual Speech Quality Measure (PSQM). The PSQM algorithm measures the distortion experienced by a speech signal in an internal psychoacoustic domain when transmitting through various codecs and transmission media. The transformation of physical domain to loudness domain is used to mimic the sound perception of human subjects in real-life situations. An extension of PSQM, named PSQM+, improves the performance of its predecessor by adopting a simple algorithm in the cognition module. This improves the poor performance of PSQM for temporal clipping distortions but the performance of PSQM+ for other types of distortion is questionable [16]. There are other examples of perceptual models in the literature such as Measuring Normalizing Block (MNB) [19]. However, each of these measurements has its limitations on certain impairment types. The most commonly used objective speech quality measure is the ITU s E-model, which was originally developed to evaluate the speech quality for PSTN. It takes into account multiple variables such as encoding distortion, delay, jitter, echo, etc. As mentioned above, the E-model provides the closest correlation to MOS results among other measures discussed in this section [15]. One important advantage of this model is that it does not require access to the original 17 of 60

19 speech signal and hence it can be used for real-time quality assessments [16]. The E-model generates the rating R and the formula for its computation is provided below: R = R 0 I s I d I e + A R 0, the highest possible rating for this system with no distortion [15, 16], is the basic signal-tonoise ratio based on send, receive loudness, electrical, and background noise [20]. I s is the impairment of the speech signal itself [16] and captures impairments that happen simultaneously with the voice signal, such as sidetone and PCM quantizing distortion [20]. These two values do not depend on the transmission over the network. I d is the impairment level caused by delay, jitter, and echo. I e, also known as the equipment factor [20], is the level of impairments caused by encoding and hence captures the degradation in quality due to compression and loss during transmission. A stands for the advantage factor that captures the willingness of users to accept some degradation of quality in return for the other benefits the system may provide such as mobility in the case of cellular phones. E-model values can be directly matched with MOS values by using a simple table provided in the standard Subjective Measures Measuring subjective quality of speech has been an important issue since the transmission of audio over telephone networks began. Over the years, standards emerged based on the results of various studies carried out in various laboratories. Today, ITU recommendations are the most widely used standards utilized by researchers while working on quality assessment methods. The ITU-T P.800 provides numerous methods for the subjective assessment of transmission quality. Scales for these methods are provided in Table 3.2. Results from 5-point category scales are averaged across participant responses to provide a Mean Opinion Score (MOS). 18 of 60

20 Table 3.2. Speech Quality Measurement Scales provided by ITU-T recommendations Listening Quality Scale Conversation Difficulty Scale Quality of the speech/connection Score Did you or your partner have any Score Excellent 5 difficulty in talking or hearing over the Yes 1 Good 4 connection? No 2 Fair 3 Poor 2 Bad 1 Listening Effort Scale Loudness Preference Scale Effort required to understand the meaning of the Score Loudness Preference Score sentences Complete relaxation possible; no effort required 5 Much louder than preferred 5 Attention necessary; no appreciable effort required 4 Louder than preferred 4 Moderate effort required 3 Preferred 3 Considerable effort required 2 Quieter than preferred 2 No meaning understood with any feasible effort 1 Much quieter than preferred 1 Comparison Category Rating Scale Degradation Opinion Scale The quality of the second compared to the first is Score Degradation is inaudible 5 Much better 3 Degradation is audible but not annoying 4 Better 2 Degradation is slightly annoying 3 Slightly better 1 Degradation is annoying 2 About the same 0 Degradation is very annoying 1 Slightly worse -1 Worse -2 Much worse -3 Watson and Sasse [21] criticized these recommended scales, with respect to speech in real time multimedia communications, in three main areas: (1) vocabulary of the scale labels, (2) length of the recommended test material, and (3) conversation difficulty scale. They note that transmission of speech in real time over IP-based networks may be carried on low bandwidth connections and is subject to various network impairments. Hence the reason for their first criticism regarding scale labels claims that even with training, it is likely that responses will be skewed towards the lower end of the scale. Regarding their second criticism, they note that the recommended test length of 10 seconds is too short in duration to understand the rapid and unpredictable changes that can occur in speech quality due to changes in network conditions. And finally, they criticize the binary scale by arguing that even a small amount of packet loss is likely to cause difficulty in hearing or talking, even if it is short-lived. In one study [22] they proposed a new subjective 19 of 60

21 quality scale termed polar continuous quality scale, which was shown to be a reliable means of measuring perceived quality. During their experiments, users were consistent in their use of it and the rating trend followed the same slope obtained with MOS. One other important finding of their study was that the perceived quality of speech is not affected with network impairments as much as it is affected by factors such as volume discrepancies, poor quality microphones, or echo. 3.3 Video Quality Measures Video quality has been an important issue, first for television broadcasting applications. Various measures have been developed for analog video systems to evaluate the effects of transmission on the original video signal. However, today, digital video systems are replacing these analog systems and are becoming an essential part of the U.S. and world economy [23]. Wolf and Pinson [23] states that, To be accurate, digital video quality measurements must be based on the perceived quality of the actual video being received by the users of the digital video system rather than the measured quality of traditional video test signals (e.g., color bar). Hence, new measurement techniques for measuring quality of digital video signals are being developed by various researchers and organizations. This section presents a survey of existing objective and subjective video quality measurement techniques utilized in the literature Objective Measures Peak Signal-to-Noise Ratio (PSNR) is the most commonly used metric for measuring video and image quality. It measures how close a sequence is compared to the original one [16]. The calculation of the PSNR for a video sequence of K frames each having NxM pixels with m-bit 20 of 60

22 depth is explained below [16]. First, the Root Mean Square Error is calculated according to the following formula: RMSE = 1 N. M. K K N M k= 1 n= 1 m= 1 [ x( i, j, k) x( i, j, k)] 2 where x ( i, j, k) and x ( i, j, k) are the pixel luminance value in the i, j location in the k frame for the original and distorted sequences respectively. Once the RMSE is calculated, the PSNR can be calculated using the following formula: 2 m PSNR = 10.log RMSE The PSNR is usually reported in decibels (db) [24]. An image with a PSNR of 25 db or below is usually unacceptable. Between 25 db and 30 db, perceived quality usually improves and above 30 db, images are often perceived as good as the original image. Markopoulou [25] notes that the PSNR is exclusively used as a quality measure, partly because of its mathematical tractability and partly because of the lack of better alternatives. It is has also been noted [16] that the PSNR does not always correlate well with subjective measures. One other commonly used metric is the Video Quality Metric (VQM) [23], which was developed by the Institute for Telecommunication Sciences (ITS). It requires the extraction and classification of features from both the original and processed video sequences similar to the other measurement techniques. Once these features are extracted, the distance between the original and processed video sequences are computed based on these features, and this distance is mapped to a subjective score [23]. Compared to the PSNR, this metric offers different models for various transmission types, such as videoconferencing or TV models. It is also possible to identify the nature of an impairment using the VQM, which the PSNR does not provide [25] of 60

23 There are other standard and proprietary measurement techniques that have been developed and reported in the literature that are not mentioned here. One commonality between these objective measures, however, is that they require access to both original and processed video sequences. One recent study [16] proposed a new measure, which does not require access to the original video sequence, similar to the idea of E-model for voice quality. In this new method, artificial neural networks (ANN) are used to predict perceived voice and video quality using a trained engine based on previous objective and subjective tests. This type of measurement techniques enables real-time measurement of video quality and is an open area for research Subjective Measures The ITU-R 500 is the standard for subjective assessment of image quality and has evolved over the years to include measures for digital video transmissions as well. This standard provides scales for single and double stimulus methods. The Absolute Category Rating (ACR) is a single stimulus method where test sequences are presented one at a time and are rated on a category scale after they are viewed. Usually a 5-point category scale is used as illustrated in Table 3.3. The Single Stimulus Continuous Quality Evaluation (SSCQE) is different from the ACR in terms of the scale it uses and the assessment process. The scale used in the SSCQE is a continuous quality scale, illustrated in Figure 3.1, and assessment takes place in a continuous manner during the presentation of the video sequence. Table 3.3 ITU Video Quality Assessment Scales 5-point Quality Scale 5-point Impairment Scale Estimated Quality Score Estimated Impairment Level Score Excellent 5 Imperceptible 5 Good 4 Perceptible 4 Fair 3 Slightly Annoying 3 Poor 2 Annoying 2 Bad 1 Very Annoying 1 22 of 60

24 Figure 3.1 Continuous 5-point Quality Scale Figure 3.2 Continuous 5-point Quality Scale for DSCQS Among the double stimulus methods, the Double Stimulus Impairment Scale (DSIS) - also known as the Degradation Category Rating (DCR) - presents pairs of original and impaired video sequences during the test respectively. In this case, subjects are asked to rate the impairment of the second stimulus with respect to the reference (first stimulus) using the 5-point impairment scale illustrated in Table 3.3. In the Double Stimulus Continuous Quality Scale (DSCQS) method, the sequences are presented in pairs like in the DSIS and subjects are asked to evaluate the quality of both sequences. The original sequence is included for reference; however, the observers are not told which one is the reference sequence and the order of appearance changes for each test. The scale used in this method is illustrated in Figure 3.2. There are other test methods where the two sequences are shown simultaneously and the observers are asked to make a comparison of the two based on stimulus comparison scale. 3.4 Quality Measurement in Telemedicine applications Quality in telemedicine has been studied from different perspectives in the literature. As a common way of assessing quality of a telemedicine event, user satisfaction was used in a large 23 of 60

25 number of articles. Another approach common in literature is to study the quality of the transmitted media (image, audio, etc.). These studies have been usually limited to the compression techniques and their effects on the perceived quality of the users. For example, Eikelboom [26] investigated image compression of digital retinal images and the effect of various levels of compression on the quality of the images. They compared JPEG and Wavelet image compression techniques and concluded that; for situations where digital image transmission time and costs should be minimized, Wavelet image compression to 15 KB is recommended, although there is a slight cost of computational time. Where computational time should be minimized, and to remain compatible with other imaging systems, the use of JPEG compression to 29 KB is an excellent alternative. To answer the question of which compression technique is better in a generic way, some studies focused on quality measures. Cosman et al. [27] studied an interesting question, How does one decide if an image is good enough for a specific application, such as diagnosis, recall archival, or educational use?, and compared and contrasted three approaches to the measurement of medical image quality: the signal-to-noise ratio (SNR), a subjective rating, and diagnostic accuracy. They concluded that there is a need for computable measures of image quality that can accurately predict the outcomes of image quality evaluation studies. Another recent article on image quality by Przelaskowski [28] states that, A numerical measure, which is able to predict diagnostic accuracy rather than subjective quality, is required for compressed medical image assessment.. A new vector measure for image quality, reflecting diagnostic accuracy was developed in this recent study. A recent study by Rosenthal [24] focused on understanding the impact of certain variables affecting the transmission of video over IP networks. This study is one of the few studies that 24 of 60

26 investigated the effects of network impairments and the codec bit rate on the quality of video on IP networks for telemedicine purposes. This study used the PSNR and a proprietary objective measurement technique, the Picture Quality Rating (PQR). His findings suggests that an increase in codec bit rate and network bandwidth have positive effects on the PQR and the PSNR levels for sequences subjected to delay and jitter impairments, but not for those in which periodic packet drops were introduced. He concludes that with or without the existence of selected packet-specific impairments, increases in bandwidth and codec bit rate improve the objective quality of video transmitted over IP networks. Another study by Dev et al. [29] presented a method to obtain an end-to-end characterization of the performance of an application over a network by taking into account network impairments and application constraints. The applications selected for testing were two medical education tools: (1) an image serving application that delivers a sequence of linked images based on user movement of the mouse cursor and, (2) an application intended to train students remotely in various surgical procedures. They were tested on four different types of networks. They propose that the subjective evaluations used in their study can be utilized to predict the conditions under which the application will be running based on predefined requirements. 3.5 Session Initiation Protocol (SIP) for Internet-based Videoconferencing A recent Voice over IP signaling standard approved by IETF called Session Initiation Protocol (SIP) is attracting telemedicine application developers due to its ability to handle voice, video, as well as multimedia communications over IP-based networks and with a native security mechanism built-in. Until the introduction of SIP, the only standard available for videoconferencing applications was the H.323 family of ITU standards. However, the H.323 standard does not lend itself to integration with web and messaging, and does not have a native 25 of 60

27 security mechanism. With the increasing importance of security in the medical field, additional effort and integration with other security mechanisms is necessary to provide authentication and authorization. A brief technical summary of SIP is provided in this section. Session Initiation Protocol (SIP) is the Internet Engineering Task Force (IETF) standard for IP Telephony [30]. It is an application layer control protocol that can create, modify, and terminate multimedia sessions. Different types of entities are defined in SIP: user agents, proxy servers, redirect servers, and registrar servers. Figure 3.3 shows a typical SIP session including these entities. Figure 3.3 Entities in Session Initiation Protocol In a typical SIP session, user agents first register to a registrar and forward their request to a SIP proxy, which is responsible from discovering the location of the requested destination so that two user agents can negotiate their session description [31]. Figure 3.4 illustrates a simple call flow with single proxy server. 26 of 60

28 Using SIP for telemedicine is relatively new compared to the previous H.323 standard that has been the dominant protocol since the early 1990s. SIP was first introduced in 1999 and in 2002 a revised version of the protocol was published. Since then, it became the commonly used protocol for Internet telephony and videoconferencing applications. However, use of the SIP for telemedicine has been slow. A small number of studies [32, 33] have mentioned how the SIP can be used in telemedicine applications. Figure 3.4 SIP Call Flow 27 of 60

29 Chapter 4: Proposed Study The proposed study consists of three stages. During the first stage of this study, a telemedicine taxonomy will be developed to classify telemedicine applications based on their potential use and by taking the medical domains they serve into consideration. The goal is to identify the IT infrastructure needs and requirements for each of these applications in order to provide a satisfactory telemedicine experience to end-users. The second stage of this study will first review the available objective and subjective audio/video quality measurements in the literature and select appropriate measures for telemedicine environments keeping the proposed telemedicine taxonomy dimensions in mind which, as has been discussed, can play an important role in the decision making process during telemedicine events. Experiments will then be conducted with physicians to identify the proper subjective audio/video quality required to make a decision in a telemedicine event for specific application area and purpose for IP-based telemedicine networks. The findings from these experiments will be used to define indices for telemedicine event capability. The last stage will be the development and testing of a videoconferencing tool in a telemedicine environment incorporating the developed index. The next three sections explain each stage in more detail. Later, the research methodology, potential implications, and a timeline for this study are presented. 4.1 Stage 1 Development of Telemedicine Taxonomy This study proposes five dimensions that will help to categorize different telemedicine efforts. These dimensions were derived from a survey of literature and reflect a combination of various 28 of 60

30 classification schemes proposed in early studies. The first subsection will provide a description of these five dimensions: Application Purpose, Application Area (Domain), Environmental Setting, Communication Infrastructure, and Delivery Options. The next subsection will explain how these dimensions are related in the taxonomy Definition of Taxonomy Dimensions Application Purpose refers to the purpose of communication and is categorized under two main groups: Clinical and Non-clinical [34]. In addition to the six categories proposed in [14], it is stated that clinical purpose covers diagnostic and treatment (surgical and non-surgical) components of patient care as well. Telemedicine not only provides a tool that can be utilized by professional medical technicians, but it is slowly moving in the direction where a patient can be treated through electronic channels without the intervention of a local professional. Hence Table 4.1 extends the previous classification and presents a list of clinical telemedicine application purposes. Non-clinical purpose includes medical education, administrative meetings, and does not involve decisions about care for particular patients. Table 4.2 shows non-clinical purposes that will be utilized in this taxonomy. This study will not focus on the non-clinical applications of telemedicine. Table 4.1. Clinical application purpose Triage Diagnostic Non-Surgical Treatment Surgical Treatment Consultation Monitoring Provision of specialty care Supervision of primary care Clinical Table 4.2. Non-Clinical application purpose Professional Medical Education Patient Education Non- Clini 29 of 60

31 Research Public Health Administrative Application Area refers to the domains in the medical field. The domains listed in Table 4.3 represent a high-level example list of medical domains and can be expanded as necessary. The reason for including medical domains as a dimension in this taxonomy is to point out the domain specific differences that affect the information required and gathered through communication channels. For example, the information required to make a diagnostic decision may differ significantly in the cardiology domain compared to the psychiatry domain. Information can be in various formats, such as text, audio, and video, and the application purpose and application area defines the amount and type of information required to make a clinical decision. Based on a review of the current literature, no studies have identified the application domain as a classification criterion for telemedicine efforts. Table 4.3. An example list of application areas Microbiology and Neurology Home Care Immunology Cardiology Ophthalmology Mental Health Pathology Dermatology Otolaryngology Radiology Rheumatology Emergency Room Pediatrics Surgery Obstetrics and Gynecology Environmental Setting refers to the type of physical environment that the physician or the patient will be using during the telemedicine event. These settings can be dramatically different and can range from a patient at a primary care hospital to a mobile patient, or a professional at a 30 of 60

32 fully equipped hospital to a professional being reached at home. Considering the physical environment attributes of medical videoconferencing identified in [35], a difference in the quality of the information transferred between two ends is inevitable regardless of the communication channel, as long as the two sites involved are not identical in terms of environmental setting. These physical attributes are usually related to the characteristics of the physical location. Therefore, environmental setting was included in this taxonomy as the third dimension. Table 4.4 illustrates some possible telemedicine settings that can be encountered during a telemedicine event. Table 4.4. Environmental settings Location 1 Location 2 Large Hospital Small Hospital Outreach Clinic Health Center Home Mobile Large Hospital Small Hospital Outreach Clinic Health Center Home Mobile LeRouge et al. [35] has provided a list of physical environment attributes for videoconferencing. These attributes are facilitating décor, quite/soundproof environment, privacy of the exam room, space and room size, and room lighting. Some of these attributes are very specific to videoconferencing. However, some of them can be generalized to various delivery options. The main idea is to be able to provide a meaningful description of the physical setting and environmental values with regards to the telemedicine event. The personal preferences and skills of patients and physicians should also be taken into account in order to assess the feasibility of a telemedicine system use by the parties involved. Some patients may be capable of performing related tasks only through the help of others as noted by Kaufman et al.[36]. 31 of 60

33 Therefore, setting attributes should also include the presence of assistive personnel and their relevant skills. Communication Infrastructure refers to the channels that are available for the transmission, emission, or reception of data or information in any format. The communications infrastructure can be based on wired networks, radio waves, fiber optic lines, and many other forms of telecommunication technologies. Each of these technologies comes with their own limitations and advantages. These need to be considered carefully before a telemedicine event occurs in order to understand the possible limitations, available resources, and how these various factors can affect the event. Table 4.5 illustrates communication infrastructure possibilities as a function of the telecommunication technologies that can be used in a telemedicine event and the bandwidth they provide. Table 4.5. Telecommunication technologies and their bandwidth capabilities Technology Bandwidth Dial-up 33.6kbps DSL 64kbps 1.544Mbps up 128kbps-1.544Mbps down Cable Modem 200kbps 2Mbps High Speed 10/100Mbps to 1Gbps b 11Mbps g 54Mbps a 70Mbps 3G 144kbps-1Mbps 2G >128kbps Wired Wireless Delivery Options is the final dimension of the taxonomy and it refers to the applications provided to conduct a telemedicine event by fully complying with the requirements generated based on the other dimensions explained above, as well as the requirements posed by the professionals and patients. Even though various delivery options exists in today s world of advanced technological innovations, delivery options in telemedicine can be categorized under 32 of 60

34 two main groups [13, 37]: (1) synchronous and (2) asynchronous. Synchronous and asynchronous communication refers to information transactions that occur among two or more number of participants simultaneously and at different points in time respectively [38]. Table 4.6 presents some examples of these delivery options based on these two main categories. The chosen delivery options can have an important affect on the final quality of the telemedicine event and the outcome. Table 4.6. Delivery options Synchronous Asynchronous Audio Telephone, Voic Audioconferencing Video Videoconferencing Video/Audiostreaming Data Instant Messaging, Shared Electronic white boards Paging, Fax, , Web Pages, Store and Forward, Web Forums Interaction of Proposed Dimensions These five dimensions can be grouped under two main themes. The first two are dimensions strictly related to the medical field. Therefore, they are in the medical dimensions group. The rest form a group of various dimensions (environmental setting, infrastructure, delivery options), which are related to the way healthcare is delivered. This group in termed delivery dimensions since all the dimensions have a common goal, that is, to support the medical dimensions needs in order to deliver health services. A simple picture of the taxonomy is presented in Figure 4.1. As Figure 4.1 illustrates, there is an additional group termed organizational dimension in the taxonomy that is pervasive to all healthcare organizations and their activities. This group consists of various aspects of the organization such as human resources and IT management. These issues will not be addressed in this study since the main focus is on the higher levels of the taxonomy. 33 of 60

35 However, future studies will be conducted to more fully understand the effects of the organizational dimension on the final outcome of the telemedicine event. Figure 4.1. Telemedicine taxonomy Two other important dimensions that were excluded from this study, but have significant importance for future telemedicine efforts, are the cost dimension and the legal issues dimension, which we grouped under the organizational dimension. The taxonomy excluded these two dimensions so as to concentrate on the core dimensions of telemedicine and to provide a simple way of identifying varying efforts. These core dimensions will eventually affect the cost and legality of the telemedicine applications. Legal issues and cost have been discussed and are very important in the healthcare industry. One study [39] reported how laws regarding telemedicine are being enacted by different states 34 of 60

36 and how the cost of telemedicine applications is affecting the decision making process. Further studies are needed to understand how the core dimensions can make a difference on the decisionmaking processes of lawmakers and payers. 4.2 Stage 2 Assessment of Objective and Subjective Quality Measures for Telemedicine Based on the five core dimensions identified in the previous stage as factors affecting telemedicine events, this phase of the study proposes to integrate specific technical factors into a measurement technique, which can then be used to predict telemedicine capability within a specific setting, potentially real-time. While doing so, further dimensions will be treated as constants in laboratory experiments to identify the effects of the selected technical factors on the telemedicine capability of a setting. This study will focus on two application areas (ophthalmology and cardiology) and one application purpose within each application area. The delivery mechanisms will be videoconferencing and audioconferencing over IP networks using SIP. The next subsection provides a brief overview regarding the technical factors that will be included in this study. The experimental test-bed details are presented next. This section concludes with a discussion on objective and subjective tests that will be conducted Technological Factors Affecting Quality in Telemedicine over IP Networks The Internet Protocol (IP) is a packet-based network protocol that enables the transmission of data packets, from one end system to another based on address information carried in the data message. It can be used with two different transport layer protocols: Transmission Control Protocol (TCP) and User Datagram Protocol (UDP). TCP is a connection oriented, reliable transport protocol designed for data transmission. However, it is not suitable for real-time applications because the retransmission of packets may cause high delay and increase delay 35 of 60

37 variation, which can significantly affect the quality of real-time applications. There are other problems associated with using TCP for real time applications, which are not mentioned here. Hence, real-time applications use UDP, a connectionless transport layer protocol that does not guarantee the arrival of a packet. Real-time multimedia applications use two protocols that run over UDP: the real-time transport protocol (RTP) and the RTP control protocol (RTCP) [40]. RTP is designed to carry data that has real-time properties. RTCP is designed to monitor the quality of service and to convey information about the participants in an on-going session. Even though RTP is the commonly used protocol for real-time applications; RTP, by design, does not provide any mechanism to ensure timely delivery or provide other quality-of-service guarantees, but relies on lower-layer services to do so. Therefore, real-time multimedia applications are vulnerable against any impairment that can happen in the lower layers of the network. The next subsection presents these impairments followed by two additional subsections that provide an overview of audio and video codecs available for multimedia applications Network Impairments Since the Internet was not designed for real-time applications and only provides best effort service, carrying real-time applications over the Internet presents a number of challenges. These include lack of guarantee in terms of bandwidth, packet loss, delay, and jitter, all of which affect the quality of voice and video over the Internet as reported in various studies [20, 24, 41]. Packet loss Unlike circuit-switched networks, in packet switched networks no physical endto-end circuit is established [41]. Packets are transmitted from the source to the destination over the Internet by the help of routers. Arriving packets to a router are first queued and then transmitted one-by-one, usually with the first in first out (FIFO) policy. However, if the queue 36 of 60

38 (buffer) of a router is already full when a packet arrives, then this packet is dropped and consequently, is not transmitted to its destination. Network congestion occurs when routers start dropping packets. The effects of packet loss on real-time multimedia applications are critical. During a voice conversation, human cognition can handle only a certain amount of packet loss. If too many packets are lost, the voice becomes incomprehensible. For video the effect of extensive packet loss is more acute. If packet loss happens, some parts of the video cannot be decoded and displayed. It is easy to understand the effects of packet loss on the perceived quality of voice and video applications. Researchers have developed various techniques to overcome, or at least ease, the effects of packet loss on applications; some of these techniques are discussed in [16, 41]. Packet Delay End-to-end packet delay is typically caused by a number of components [41]: (1) codec delay is the time it takes to convert analog voice to digital data and vice versa, (2) serialization delay is the time it takes to place a packet on the transmission line, and is determined by the speed of the line, (3) queuing delay occurs at the various switching and transmission points of the network, such as routers and gateways, where voice packets wait behind other packets waiting to be transmitted over the same outgoing link, and (4) propagation delay is the time required by signals to travel from one point to another, which is fixed as determined by the speed of light. The effects of large packet delay become even more severe for voice communications, as timing is an important characteristic of voice. This is especially true when an interactive conversation is being transmitted on the network; delay effects can turn the conversation into a half-duplex mode where one speaks and other listens and pauses to make sure the other is done. Echo is another unwanted effect of packet delay. Various techniques were also developed to overcome these problems over packet-based networks since in current circuitswitched networks the primary source of delay is propagation delay. 37 of 60

39 Packet Delay Variation (Jitter) Packet delay variation refers to the variation or gaps between packet arrival times at the receiving buffer. This occurs due to the variability in queuing and propagation delays. To eliminate the effects of this variation, usually a playout buffer is used. The receiver holds the first packet in the buffer for a specific amount of time before playing it out. Therefore, a small jitter is tolerable but large fluctuation causes difficulty in decoding and playback, and cause quality degradation. The effects of delay variation are similar to the effects of packet loss. Large variation in delay will result in some packets arriving long after the playout time scheduled for them based on the buffer size. The receiver will discard these packets since they are out of order Audio Codecs Audio data does not contain as much redundant data as video data and hence, it is harder to compress. Speech coding techniques can be categorized in three groups: (1) waveform coding, (2) source coding, and (3) hybrid coding. They are used at high, low, and moderate bit rates respectfully. Waveform encoding is almost a lossless coding scheme since the resultant signal is very close to the original one. The simplest form of this coding is Pulse Code Modulation (PCM). Many codecs try to predict the value of the next sample from the previous ones and an error signal is computed from the original and predicted signals. Another method that utilizes this error signal for encoding is called Differential Pulse Code Modulation (DPCM). Other examples of waveform coding are sub-band coding (SBC) and discrete cosign transformation (DCT). Source codecs implement the idea of understanding how the speech signal is produced and sending certain parameters of the signal to the decoder. Hybrid coding is a mix of these two techniques. Analysis-by-Synthesis (AbS) coding is the most famous type of hybrid coding. Using these 38 of 60

40 coding techniques, a large number of audio codecs have been developed over time and below is an overview of some of these codecs. The ITU-T G.711 (PCM at 64Kbits/s) codec, also known as µ law, is a variant of PCM codec, which is commonly used in North America and Japan for digital telephony. It does not require much CPU power and it provides good quality with simplicity. However, sometimes the resulting bit rate may be higher compared to other codecs. Two other public standards by the ITU-T for compressing voice data are G.721 (ADPCM at 32 Kbits/s) and G.723 (ADPCM at 24 and 40 Kbits/s). They both use Adaptive DPCM (ADPCM), which utilizes an adaptive prediction and quantization scheme to increase the performance of DPCM coding. Another application of ADPCM is the DVI codec, a recommendation from the Interactive Multimedia Association (IMA) Digital Audio Technical Working Group. It compresses 16-bit linear PCM samples into 4-bit samples, yielding a compression rate of 4:1. Finally, GSM stands for Global System for Mobile Communications and is a variant of LPC called RPE-LPC (Regular Pulse Excited - Linear Predictive Coder). It is a European standard originally for use in encoding speech for satellite distribution to mobile phones. Its use results in very good compression with good quality output but is very costly in terms of performance Video Codecs Video streaming is a resource and bandwidth intensive application type [24] that requires the video to be compressed before transmission to utilize the existing resources efficiently without saturating them. The goal of video compression is to remove the redundancy in the original source signal, which will eventually reduce the amount of bandwidth required for transmission [16]. There are three types of video compression coding, they are: (1) lossless coding, (2) lossy coding, and (3) hybrid coding. 39 of 60

41 Lossless coding (e.g. Huffman coding) is a reversible process with the perfect recovery of original data. Therefore no quality degradation exists due to lossless coding. Lossy (e.g. Source coding) coding is an irreversible process in which the recovered data is degraded. Hybrid (e.g. JPEG) coding is the one used by most multimedia systems and it combines both lossy and lossless coding. H.261, H.263, MPEG-1, MPEG-2, and MPEG-4 are the most popular video codec standards. In this study, H.261 and H.263 will be used as video codecs during the experiments. H.261 is an ITU video-coding standard originally designed for ISDN lines. Its output bit rates are multiples of 64Kbits/s. It is a constant-bit-rate codec with no constant quality and variablebit-rate encoding meaning that the encoding algorithm trades the picture quality against motion. Therefore, to obtain higher quality, it is suitable to use this codec for scenes having a small amount of motion. It supports only two resolutions: (1) Common Interchange Format (CIF), which is 352x288 pixels and, (2) Quarter CIF (QCIF), which is 176x144 pixels. H.263 is also an ITU video-coding standard originally designed for low bit rate communications (less than 64Kbits/s this limitation has now been removed). It uses a similar coding algorithm with H.261 with some changes to improve the performance and error recovery. As a result of these improvements, H.263 output stream is more resilient to packet loss, which makes it very attractive for real-time communications over the Internet. It supports five resolutions. In addition to CIF and QCIF, it provides resolution at SQCIF (128x96 pixels), 4CIF (704x576 pixels), and 16CIF(1408x1152 pixels) Working Around Impairments: Application and Network Level Quality of Service Previous subsections summarized how certain network impairments can affect real-time applications on IP-based networks. Currently two approaches exist to provide Quality of Service 40 of 60

42 (QoS) for real-time applications: (1) QoS at the application level and, (2) QoS at the network level. Application-level QoS provides quality improvements without requiring changes of the network infrastructure. In initial implementations of real-time applications, incoming data was played out either immediately upon arrival or after a fixed delay. Since both methods lead to significant signal degradation under high delay variance conditions, adaptive playout techniques were introduced to make real-time applications more tolerant of delays and delay jitter and to dynamically adjust the playback point [25]. Researchers have also studied reconstruction methods at the receiver to compensate for packet loss in real-time applications. Various error concealment methods for audio are summarized in Table 4.7. Table 4.7 Error Concealment Techniques for Audio [41] Name Silence Substitution Noise Substitution Packet Repetition Packet Interpolation Frame Interleaving Technique Substitutes lost packet with silence. Causes voice clipping. Deteriorates voice quality when packet size is large and loss rate is high. Substitutes lost packet with background noise. Better than silence substitution. Relies on the ability of human brain to repair the received message if there is background noise. Substitutes the lost packet with the replays of the last correctly received packet. Substitutes lost packet with a replacement packet produced based on the characteristics of the packets in the neighborhood of the lost one (a.k.a. waveform substitution). Reduces the effect of packet loss by interleaving voice frames across different packets. Error concealment techniques for video try to recover the corrupted data by exploiting the spatial and temporal redundancies of the video data [43]. The spatial-domain error concealment algorithms interpolate the lost area using spatially neighboring image data and since these algorithms recover an isolated lost macroblock (MB), which is made by the coded modification, and provide good performance. On the other hand, temporal-domain error concealment schemes utilize the previously decoded image data to recover the lost MBs where they estimate motion 41 of 60

43 vectors (MVs) for the lost MBs, and compensate for the lost MBs with the estimated MVs. Some error concealment techniques are provided in Table 4.8. Table 4.8 Error Concealment Techniques for Video [16] Name Block Replacement Linear Interpolation Motion Vector Hybrid Technique Technique Replaces the lost areas with the corresponding areas of the previous frame or field. Works quite well in still parts of the picture but fails in areas where there is a lot of motion. Replaces the lost areas with the linearly interpolated values calculated from the neighboring areas of the same frame. Assumes that surrounding areas are correctly received and works well in a uniform surface. Replaces the lost areas with pixel blocks of the previous frame shifted by the average motion vector of the neighboring blocks. Performance drops when the blocks have different motion vectors. Uses both spatial and temporal redundancies to predict the lost MBs. Development of network Quality of Service (QoS) features was partially motivated by the fact that real-time traffic (as well as other applications) may sometimes require priority treatment to achieve good performance on the Internet [44]. QoS can be achieved by managing router queues and by routing traffic around congested parts of the network. The IETF proposed two models to provide Internet QoS: Integrated Services (Int-Serv) [45] and Differentiated Services (Diff-Serv) [46]. In IntServ, resources are reserved for each flow through the network using the Resource ReSerVation Protocol (RSVP) [47]. When an application requests a specific QoS for its data stream, the RSVP can be used to deliver the request to each router along the path and to maintain router state to provide the requested service [44]. Current implementations of IntServ allow a choice of Guaranteed Service [48] or Controlled-Load Service [49]. In Guaranteed Service agreements, peak traffic is limited by a certain rate and packet size is restricted to be in a specific range at all times. Based on these limitations and restrictions, a bandwidth requirement is declared, and sufficient bandwidth is reserved on each hop to satisfy all the requirements of the flow. If each node and hop can accept the service request, the flow should be lossless. 42 of 60

44 Controlled-Load Services [49] on the other hand uses only traffic specifications and does not define any service request specifications. Hence, flows using this service should experience the same performance as they would in a lightly loaded best-effort network. Several reasons, including scalability problems, were reported for not using IntServ for IPbased real time applications in [44]. To overcome these problems, a simpler framework and architecture to support DiffServ was developed [46]. The primary goal of differentiated services is to allow different levels of service to be provided for traffic streams on a common network infrastructure [44]. In the Diff-Serv model, the QoS information is carried in a band within the packet in the Type of Service (TOS) field in the IPv4 header or the Differentiated Service (DS) field in IPv6 [50]. The TOS or the DS field is used to indicate the need for low-delay, highthroughput, or low-loss-rate service. Backbone routers provide per-hop differential treatments to different service classes as defined by Per Hop Behavior (PHB) that describes the forwarding behavior a packet receives at a given network node. Despite the fact that DiffServ is a simpler mechanism that provides performance improvements compared to best effort IP networks, it has some shortcomings; it relies on ample network capacity for expedited forwarding traffic and makes use of standard routing protocols that make no attempt to use the network efficiently [44]. One other type of network level QoS technique is provided by the Multiprotocol Label Switching (MPLS) architecture offering IP networks the capability to provide traffic engineering as well as a differentiated services approach to voice quality [44]. In IP networks, as packets travel from one router to another, each router independently chooses a next hop for the packet, based on its analysis of the packet's header and the results of running the routing algorithm [51]. Analysis of the packet header identifies the forwarding equivalence class (FEC) of a packet and routing algorithm maps this FEC to a next hop. This is repeated at each hop until the packets 43 of 60

45 reach their destination. Notice that no distinction can be made between the packets with the same FEC value in conventional IP networks. In MPLS, the assignment of a particular packet to a particular FEC is done just once, as the packet enters the network [51] and hence the MPLS separates routing from forwarding [44]. This FEC value is encoded as a short fixed length value known as a "label" and when a packet is forwarded to its next hop, the label is sent along as well. DiffServ and the MPLS can be combined to provide better QoS for real-time applications. Regardless of the techniques developed for network QoS, the implementation of these techniques is limited and available to only a small group of users. The reasons for this slow adoption of network QoS techniques is discussed in [52] extensively and the conclusions of this study suggests that the QoS community and researchers need to reach out and include business, systems control, and marketing expertise in their efforts to get IP QoS meaningfully deployed and used Experimental Test-bed Previous sections identified the important factors that play a role on the perceived and measured quality of voice and video over the Internet. This study will setup a test-bed where these factors will be individually defined as variables within the experiments. Objective and subjective evaluations will measure the effects of variance in these variables on the decisions made for the selected telemedicine purpose. Figure 4.2 illustrates a summary of the application and network components and how the identified factors can be positioned between these components. 44 of 60

46 Figure 4.2 Application and Network Components Figure 4.3 illustrates the simple test-bed setup for the experiments. There will be two computers with video cameras to capture voice and video for an ophthalmology scenario and a stethoscope for the cardiology scenario. Several software tools will be utilized in this test-bed to capture audio and video, to transmit the captured information on the network, and to manipulate and measure network impairments during the transmission. Figure 4.3 Experimental Testbed Hardware and equipments available for experiments of this study are listed in Table 4.9. Table 10 presents a list of tools that will be utilized during the experiments for manipulating network 45 of 60

47 parameters and for capturing and storing video and audio sequences. Network monitoring tools presented in this table are selected from [53] where a comprehensive list of products can be found. Table 4.9 List of Available Hardware for the Testbed Network Equipments Make Model Hub 1 SMC EtherEZ Hub 3605T 10Mbps Hub 2 D-Link Hub DSH-5 100Mbps Router 1 D-Link 2.4Ghz Wireless Router DI-614+ Router 2 Linksys EtherFast Cable/DSL Router BEFSR41 ver.3 Computers, Laptops, and Servers Operating System CPU RAM Laptop 1 Windows XP Home Edition 2.0 GHz 256 MB PC 1 Windows XP Professional 1.8 GHz 256 MB PC 2 Windows 2000 Professional 1.8 GHz 256 MB PC 3 Windows 2000 Professional 1.8 GHz 256 MB Proxy Server Linux Red Hat GHz 256 MB Name Type Description JMStudio Ethereal Distributed Internet Traffic Generator (D-ITG) Netperf Bing Java based Media Player Packet Capture Tool Network Monitoring Tool Thruput Tool Table 4.10 Software List for the Testbed The Java Media Framework API (JMF) enables audio, video and other timebased media to be added to applications and applets built on Java technology. JMStudio is an application developed based on JMF which can capture, play, record audio and video files. It can also receive and play RTP Media Streams. It is a free network protocol analyzer for Unix and Windows that provides features to examine data from live network or from a capture file on disk. D-ITG is a platform capable to produce traffic (network, transport and application layer) and accurately replicate appropriate stochastic processes for both IDT (Inter Departure Time) and PS (Packet Size) random variables (exponential, uniform, cauchy, normal, pareto, etc.). It provides general measures of performance of a network such as latency between request and response of generic transactions across a TCP/IP network. It is maintained by HP. Bing is a point-to-point bandwidth measurement tool (hence the 'b'), based on ping. Pathrate measures end-to-end capacity. Pipechar is a tool for reporting dynamic network characteristics in particular the bottleneck bandwidth. Bandwidth Pathrate Estimation Tool Pipechar Traceping Ping It measures the packet loss to nodes along a route. 46 of 60

48 4.2.3 Experimental Procedures The initial step for the experiments is to identify the telemedicine application area and purpose under consideration. The two telemedicine application areas selected for this study are ophthalmology and cardiology. The application purpose is currently restricted to diagnosis. The next step is to obtain sample exam sequences for the selected application area and purpose. For the experiments of diagnosis in ophthalmology application area, video sequences of an eye examination session will be necessary. There are two possible ways of obtaining this video sequence. One way is to request a readily available video sequence from the National Library of Medicine (NLM) and feed this video sequence into the experimental test-bed for objective and subjective measurement collection. Another way is to record a live session using the listed devices in the previous section and use this self-obtained video sequence for testing purposes. For the experiments of diagnosis in cardiology application area, audio sequences of heart beats will be required. The first possible way of obtaining this audio sequence is to request it from a source like NLM. Another possible way is to use electronic stethoscopes to capture the audio sounds of heartbeats and directly feed this to the computer as audio input. Once the audio and video files are captured and ready for use in the test-bed, the next step will be to feed these files into the test-bed while manipulating the factors identified to have an effect on the quality of degradation for voice and video over IP-based networks. Factors that will be manipulated during the experiments are audio/video codecs, packet loss, packet delay, packet delay variation, and bandwidth. As a result of these experiments, a set of distorted signals will be collected and stored for future use in subjective tests. During the transmission of the original signal over the test-bed, data for objective measurements will be collected. Ethereal will be used as the main tool to monitor network traffic and to capture traffic on the network for further 47 of 60

49 packet and traffic analysis. In this test-bed, the sender (patient-end) will control the selection of the codec; the router will control the loss rate, loss pattern, delay, and delay variation; and the receiver (physician-end) will store the received signals, decode them, and use concealments methods selected by the application in use to recover lost packets. At this point, all values necessary to evaluate objective measures will be collected and stored. The next step is to measure objective quality. Among the several objective quality measures introduced in Section 3.2.1, the ITU-Emodel will be used for measuring audio quality for the test sequences. This measure was chosen based on evidence that it is the only available measure that does not require the original signal for calculations and it correlates well with the MOS values. As mentioned in section 3.3.1, there are no objective measurements available other than the ones proposed in [16] that can measure objective quality of a video sequences in the absence of the original sequence. However, the VQM explained in section can be used for this study to measure the video quality of the distorted signals since it will be available as the original signal. These will not affect the results of the final real-time tool for quality measurement because the new tool will rely on the previously collected values in the database for assessment. The objective measurement values calculated in this step will be stored in a database with the values of the impairment factors. Table 4.11 illustrates the predicted fields of the quality database for audio and video quality excluding objective measurement value field and MOS field. Table 4.11 Fields of the proposed Quality Database For Audio Quality For Video Quality Packet Loss Rate Packet Loss Rate Consecutive Lost Packets Consecutive Lost Packets Max. Jitter Max. Jitter Max. Packet Delay Max. Packet Delay Available Bandwidth Available Bandwidth Video Codec Audio Codec Bit Rate Frame Rate 48 of 60

50 Subjective measurements will follow the objective measurements. Selection of test subjects will be based on availability. First, invitations to physicians familiar with telemedicine applications in Loma Linda University Medical School will be sent. Based on the response rate, if further recruitment of subjects is required, final year medical students will be recruited for subjective tests. The ITU recommends that 4 to 40 test subjects be used for completing subjective quality tests. Subjective tests will involve at least the minimum required number of subjects. Since the use of subjective measurements for telemedicine related to voice and video still is an immature area of research, this study will utilize different subjective measurement techniques discussed in and Test subjects will be asked to view the recorded sessions and provide their opinion for the questions asked in the standard. MOS scores of the subjective test results will also be calculated and added as a new field to the quality database illustrated in Table The last step in this stage is to find a correspondence between objective and subjective measures in the database. The quality database will be the final outcome of the second stage in this study. A summary of the second stage is illustrated in Figure 4.4 below. Figure 4.4 Process flow for the second stage of the study 49 of 60

51 4.3 Stage 3 Development of SIP-based Videoconferencing Tool with Real-time Telemedicine Capability Index In this last stage, an existing SIP videoconferencing client, the CGUsipClient, will be enhanced with a simple quality indicator based on the results obtained in the previous stage of this study. The CGUsipClient was developed by the Network Convergence Laboratory (NCL) to provide low-cost, low-bandwidth videoconferencing. It is a java-based client that utilized the Java Media Framework (JMF) Sun libraries for voice and video handling. The video codecs supported by this client are H.261 and H.263, the latter being the default codec for video communications. The audio codecs supported are G.723, DVI, GSM, and G.711 (µ-law); the user can change the default audio codec. Detailed information regarding the CGUsipClient architecture can be found at [54]. Another study [32] reported the many useful features of this client for use in telemedicine and how it can add value in the telemedicine setting. The CGUsipClient will feature new user interface windows that will provide real-time quality information, derived from the objective measures that will be collected in real-time during a telemedicine session and the calculation of their correspondence to subjective measures using the quality database. In order to achieve this goal, several improvements are required on the client. First, a real-time objective measure collection module will be incorporated with the existing client. This module will collect packet loss, delay, bit rate, and frames per second information from the network. Second, a new module for calculating a correspondence to these objective measures in terms of a subjective MOS value will be developed and incorporated into the CGUsipClient. Finally, two graphical user interfaces (GUI) will be developed. The Session information GUI will collect information regarding application area, purpose, and delivery option (only audio, audio and video) before the session begins as part of objective measures. 50 of 60

52 Based on this information, relevant quality database will be used for calculations. The Telemedicine Capability Index GUI will provide the outcomes of the correspondence calculations in real-time to the user. A snapshot of the predicted Telemedicine Capability Index GUI is provided in Figure 4.5. One final improvement can be to add a module to obtain instant evaluations from the users and add these values to the relevant session database for future use. Figure 4.5 GUI for Telemedicine Capability Index Indicator 4.4 Research Methodology This study focuses on three research objectives. First, provide a telemedicine taxonomy as a method to classify different telemedicine events while defining them based on five dimensions. Second, evaluate the quality of information necessary to make medical decisions under fluctuating conditions of network and application parameters. Third, develop an artifact that provides a real-time quality and capability index for users based on evaluation results. To meet these research objectives, a hybrid research methodology is utilized. This study will first define a new taxonomy based on the exiting definitions and theories for telemedicine after an extensive literature review. Later, an evaluation study will be conducted to complete the second stage. There are two types of evaluation formative and summative. As described in [55] (p.208) Formative evaluation is intended to help in the development of the 51 of 60