The Association of System Performance Professionals

Transcription

1 The Aociation of Sytem Performance Profeional The Computer Meaurement Group, commonly called CMG, i a not for profit, worldwide organization of data proceing profeional committed to the meaurement and management of computer ytem. CMG member are primarily concerned with performance evaluation of exiting ytem to maximize performance (eg. repone time, throughput, etc.) and with capacity management where planned enhancement to exiting ytem or the deign of new ytem are evaluated to find the neceary reource required to provide adequate performance at a reaonable cot. Thi paper wa originally publihed in the Proceeding of the Computer Meaurement Group 003 International Conference. For more information on CMG pleae viit Copyright 003 by The Computer Meaurement Group, Inc. All Right Reerved

2 Performance Implication of Hyper-Threading Yiping Ding, Ethan Bolker, Arjun Kumar BMC Software, Inc Intel recently introduced Hyper-Threading Technology (HTT) make a ingle phyical proceor appear a two logical proceor. Operating ytem and application can chedule procee or thread on thoe logical proceor. The performance impact of HTT varie, depending on the nature of the application running on the proceor and on how the hardware i configured. Thi paper illutrate that variation by preenting a et of controlled tudie and offer a imple queueing model to interpret the obervation. We alo examine the iue involved in collecting performance metric for HTT ytem and their implication for modeling. 0. Introduction Computer technology i continually evolving to improve application performance. Each innovation mut be evaluated in the field o that practitioner can undertand the performance implication for their particular application, ince the actual impact may differ ubtly from what wa intended. Intel Hyper-Threading Technology (HTT) i one of thoe recent innovation. One Intel tudy how that HTT ometime improve performance by 6% to 8% [Marr, et. al.]. But there are time when HTT may degrade application performance. Since all new highend Intel proceor come with HTT it there on your new machine whether you know it or not. So it important to undertand it and to know what it doing for you (or againt you).. The Architecture of Hyper-Threading (HTT v. Dual Proceor) One can improve application performance by optimizing the oftware (at the application or interpreter/compiler level) or by peeding up the computation in hardware. But a hardware modification may boot performance le than expected if there i a bottleneck omewhere ele in the ytem. For example, increaing proceor clock peed won t do much if the CPU wait becaue memory acce for intruction fetch i low. So another route to performance improvement i to remove bottleneck that caue reource to be idle. Chip architect have been doing thi for year, introducing cache, pipelining and intruction prefetch in order to keep the CPU a buy a poible doing ueful work. HTT i another tep in that direction. A hyper-threaded proceor duplicate ome of the hardware (adding le than 5% to the chip ize [Marr, et. al.]) on the path from memory to CPU, but not the CPU itelf. Ethan Bolker i alo Profeor of Computer Science at the Univerity of Maachuett, Boton.

3 That architecture allow for ome parallel proceing except at the actual intruction execution phae. That keep the CPU itelf buier, increaing the actual intruction execution rate. You will find technical decription in [Marr, et. al.]. HTT make the chip (IA-3 architecture) appear to the operating ytem a if it contained two complete phyical proceor. That mean neither the operating ytem nor your application need be rewritten to run on a ytem with HTT, a long a the operating ytem follow Intel traightforward intruction for chooing a (logical) proceor on which to chedule work when the ytem ha multiple phyical proceor. Of coure in order to take advantage of HTT, a i with any multiproceor ytem, multiple thread (in one or everal procee) mut imultaneouly want acce to a proceor [Guptha]. For information on how Microoft ha addreed that iue (and for Window licening conequence), ee [Borozan]. For Linux HTT upport ee [Vianney]. Fortunately, you do not need to undertand the chip architecture in order to undertand how HTT affect the performance of the application you care about. You can view and analyze HTT a a black box to feel it performance impact. In ection 6, we will preent a imple queueing model to predict HTT repone time baed on ome baeline meaurement. We proceed to that analyi now, tarting with the meaurement.. The Meaning of Meaurement Multi-proceor preent a reporting dilemma to the meaurement community. For example, when both proceor on a dual proceor machine are active during an interval i the utilization 00% or 00%? To eliminate the ambiguity, we have to pecify the number of proceor for which the Some pecialized application like compiler and video game might benefit from HTT aware coding. utilization i reported. A performance meaurement tool mut ue one convention conitently and make clear how it output i to be interpreted. The problem i compounded for a hyper-threaded proceor, which i a ingle phyical CPU deigned to appear to the operating ytem and it meaurement tool a two eparate proceor. Since hyper-threading i in principle tranparent to the operating ytem and application, exiting meaurement tool need no modification to run in a hyperthreaded environment. But what will they report? The anwer varie. We experimented with four program: Window Tak Manager, BMC PATROL Perceive, and two off-the-helf tool that make ytem call to provide CPU utilization information to a running Java program. The firt i Vladimir Roubtov package [Roubtov] for querying to dicover how many econd of CPU the Java Virtual Machine (JVM) conume when running an application. The econd i part of the Java Virtual Machine Profiler Interface (JVMPI) [JVMPI] provided by Sun. It collect JVM CPU conumption information on a per thread bai. The table below how the output for a compute intenive batch workload running on a hyper-threaded machine with one phyical proceor (two virtual proceor) with hyper-threading both on and off. 3. A you can ee, the meaurement are contradictory unle properly interpreted. Hyper-threading off on Number of active thread Meaurement tool Tak manager PATROL Perceive JVMPI Roubtov Table. Reported % CPU Conumption 3 Since hyper-threading i et in the BIOS you mut reboot to enable or diable it.

4 The two profiling tool do not report utilization directly. We derive that metric uing the formula (reported CPU econd conumed)/(elaped time). For batch job tream in at mot two thread the only problem we ee i the factor of two ambiguity illutrated above. But when the work to be done i multithreaded and intermittent more confuing problem occur. We have een intance in which both tool report utilization larger than 00%. We have not looked at the code to undertand why, but upect that each proce (or thread) i thought by the tool to be conuming CPU when it i running on one of the logical proceor. But when both logical proceor are buy (in that ene) only one of them can be executing an intruction, ince they hare the ingle phyical proceor. Thu both will be charged for the ame real CPU time, leading to over counting that will be clearly viible if the total CPU econd the tool think the proce conumed exceed the elaped time. Even when it not, the problem perit. The buier the ytem the greater the probability that the tool will report that it take more econd of (purportedly real) CPU to do the ame amount of work. In order for a meaurement tool to report reliable and ueful information it hould know the map between logical and phyical proceor. Intel provide an aembly level API [HTT-API] for acceing that information. For.Net / Window 003, Microoft will provide thi information in it performance regitry a well. 3. Batch Proceing The application we ran in order to invetigate what the meaurement tool report doe more than exercie thoe tool. It i deigned and intrumented to tudy proceor performance under a variety of configuration. The firt tet we performed wa to et up a number of computation intenive batch job tream, each running in a eparate (Java) thread, and meaure ytem performance in thi cae the tatitic of interet are the number of thread, the throughput, and the CPU utilization. We expect the lat of thee to be 00% of all requeted proceor, uitably interpreted. Table report the throughput for four configuration: ingle and dual (phyical) proceor with hyper-threading both enabled ( on ) and diabled ( off ). In each cae the utilization (which we have not included) how that all the virtual proceor for which there i a ready thread are contantly buy. Phyical Proceor Proceor Proceor 400 MHz 000 MHz each Hyper-threading Thread off on off on Table. Batch throughput throughput number of thread one CPU, HTT off two CPU, HTT off one CPU, HTT on two CPU, HTT on Figure. Batch throughput curve

5 We can draw ome intereting concluion from thee data. With hyper-threading on or off the ingle proceor ytem behave jut a we expect it to. The batch throughput i eentially independent of the number of thread. It i lightly larger with hyperthreading on even for ingle threaded work, probably becaue the other background work on the ytem (like data collection tool) can ue the econd logical proceor and thu interfere le with the batch job tream. The two-proceor ytem (four virtual proceor) can run up to four thread imultaneouly. With hyper-threading off the throughput per proceor i eentially proportional to proceor clock peed when compared to the ingle proceor ytem. When the batch job tream i ingle threaded it keep jut one of the real proceor buy. The econd i free to do ytem work, whether or not hyper-threading i enabled, o batch throughput i the ame in both cae. With two batch thread the ytem i mart enough to chedule work on both phyical proceor and the throughput nearly double. It doen t quite double, probably becaue the ytem mut till ue reource to run itelf. A the number of thread increae to three and then to four the throughput continue to increae when hyper-threading i enabled. Here we ee performance improvement in the batch job tream themelve, due to hyper-threading, which introduce ome low level parallelim in firt one and then both proceor. 4. Tranaction Proceing probability that there will be many job (and hence many thread) active imultaneouly. We tudied how repone time varie with increaing load. We et out to validate our benchmark oftware by tudying how meaured repone time for a ingle proceor (with hyper-threading off) matched thoe predicted by tandard queueing theory: when proceor utilization i U (0 < U < ) it hould take /(-U) econd of real (wall clock) time (on average) to accumulate one econd of proceor time. Thu the repone time for an average econd job hould be /(-U) econd. Table 3 and Figure compare meaured and predicted repone time. In the table utilization and throughput are nearly the ame ince the average job ize i cloe to one econd. Throughput Repone time Utilization (job/ec) meaured predicted Table 3. Repone time for one-econd job (ingle proceor, hyper-threading off, imple M/M/ model for prediction) In our next et of experiment we configured our benchmark driver to imulate the computation part of a tranaction proceing or web erver workload. At random time requet for a random amount of CPU ervice arrive at the ytem. Each requet i aigned a thread. Thu the number of active thread varie over time. The larger the arrival rate and average ervice time the greater the utilization and the higher the

6 repone time utilization meaured predicted meaured/predicted Figure Repone time curve for oneecond job (ingle proceor, hyperthreading off, imple M/M/ model for prediction) We found meaured repone time ytematically maller than predicted repone time. The dicrepancy increae with increaing utilization. We eem to ee le randomne than the exponential ditribution the driver i programmed to produce. We meaured the actual coefficient of variation for thee tatitic and dicovered that the interarrival time are in fact more regular than they hould be, i.e., the coefficient of variation i le than. But the anomaly perit even after we correct for that phenomenon by modifying the naïve /(-U) prediction to take into account the actual coefficient of variation of the job interarrival time. That ugget that a different queueing model may be needed to reflect the true execution tructure of a proceor with hyper-threading technology even when hyper-threading i diabled. We will preent uch a model in Section Hyper-threading On v. Hyperthreading Off We do not need a queueing theory model jut to compare ytem performance with hyper-threading on and off. We need only run our benchmark with hyper-threading enabled in order to ee what happen. Figure 3 illutrate the reult. (The repone time data when hyper-threading i on appear in a later table). repone time (ratio) throughput htt on htt off on/off Figure 3. Repone time curve for oneecond job (hyper-threading on and off) A expected, hyper-threading improve repone time for given throughput, and the improvement increae a the throughput increae. The fact that repone time for given throughput are ignificantly maller when hyper-threading i on indicate ignificant benefit for tranaction proceing workload. The maximum throughput poible while till meeting repone time ervice level objective i larger when hyper-threading i on. It important to ue throughput and not utilization a the independent variable here, ince we ve een that utilization meaurement are upect when hyperthreading i enabled. Fortunately, the important metric in a tranaction proceing ytem in t the utilization, it the repone time that correpond to a given throughput. You need not care how buy your ytem i what really matter i how repone time

7 varie with ytem load a defined by ueful work done that i, by throughput. 6. A Queueing Theory Model for a Hyper- Threaded Proceor In order to undertand the repone time when hyper-threading i enabled we developed a queueing theory model for the hyper-threaded architecture decribed in Section when there i jut one phyical proceor. From the application point of view each job viit one of the two logical proceor offered to it by the operating ytem. On the chip, each intruction execution goe through two phae. Firt one intance of the duplicated part of the hardware doe preparatory work, looking up data and intruction in the cache or getting them from memory. Then the ingle real CPU execute the intruction. We model thi ituation with the three queue hown in Figure 4. Tak arrive at the ytem at the rate of λ per econd and go to one of the two preparatory queue with probabilitie p and p, repectively. Since each job goe omeplace, p + p =. We will aume that load balancing work a it hould, o that p = p = /. p λ p Figure 4. A imple queueing model for hyper-threading. The olid circle repreent the ingle phyical proceor; the open circle the place where work i done prior to intruction execution. Suppoe the ervice time at the preparatory queue i econd/job and the ervice time at the real CPU i econd/job and that the job interarrival time and ervice time λ are exponentially ditributed. We then etimate the mean repone time uing an open model: R p p = + +, pλ pλ λ λ λ ince we have aumed p = p = /. = +, () In order to ue thi model to predict the repone time we meaured in our experiment, we need value for λ, and. The firt of thee we know. It an input parameter. We can meaure the average total ervice time for each job, een from the outide, but that doe not tell u the internal ervice time. All we can be ure of (when the ytem i not aturated) i p λ <, and p λ <, and λ <. In order to tet the model and to undertand hyper-threading architecture, we found the value of and that bet fit the meaured data. Thoe value turned out to be = 0.3, = 0.8. That ugget that about 5% of the work occur in the parallel preparatory phae of the computation, while intruction execution conume about 65%. Table 4 and Figure 5 how the reult of our experiment and analyi. The fact that we can fit the meaured data o well with reaonable parameter value ugget that the model i ound.

8 Utilization (nominal) Throughput Repone time (job/ec) meaured predicted Table 4. Repone time for one-econd job (ingle proceor, hyper-threading on, prediction from model in Figure 4) match. We need a better model. Fortunately, the one we developed for the ytem when hyper-threading i enabled work here too. When hyper-threading i diabled we aume that job till viit the chip hardware that doe preparatory work before moving to the execution phae. The difference i that there i jut one queue and one erver in the preparatory phae. That i the model in Figure 4 with p = and p = 0. We redraw it in Figure 6. λ λ repone time throughput meaured predicted meaured/predicted Figure 5. Repone time curve for oneecond job (ingle proceor, hyperthreading on, prediction from model in Figure 4) 7. A Tandem Queue Model for a Proceor with Hyper-Threading off In Section 4 we ued a imple M/M/ model to try to predict the repone time of job running on a hyper-threaded proceor for which hyper-threading wa diabled. The theoretical and meaured reult did not Figure 6. A tandem queue model for hyper-threading diabled. The olid circle repreent the ingle phyical proceor; the open circle the place preparatory work i done prior to intruction execution. The repone time, under the ame tatitical aumption, i R = +, () λ λ where i the mean ervice time for the preparatory work, which i non-zero even when the hyper-threading i diabled, and i the mean ervice time in the execution phae. A in the previou ection, we found the value of and that bet predict the meaured data: = 0.045, = Note that when hyper-threading i diabled the nominal ervice time for the preparation work i much maller than that when hyperthreading i on.

9 Table 5 and Figure 7 how the meaured and predicted repone time uing the tandem queueing model. Thee match remarkably well, uggeting that thi i in fact a good model for the chip architecture. Utilization Throughput Repone time (job/ec) meaured predicted Table 5. Repone time for one-econd job (ingle proceor, hyper-threading off, prediction from tandem queue model in Figure 6) repone time throughput meaured predicted meaured/predicted Figure 7. Repone time curve for oneecond job (ingle proceor, hyperthreading off, prediction from tandem queue model in Figure 6) 8. Future Work Our work o far how that it i poible to meaure and model application performance in a hyper-threaded environment. Our plan now are to Explore how the model parameter and depend on the nature of the intruction being executed. Find way to compute or etimate and from firt principle rather than by finding the bet fit to meaured data. Tet our general queueing model on ytem with multiple phyical proceor each of which i hyperthreaded. 9. Summary Hyper-threading i complex, intereting and ubiquitou. Until meaurement tool have caught up to it, you mut treat what they ay with ome caution and inight. In general, Intel claim for the performance benefit are believable, but you mut be a little careful. Single threaded application may run better with hyper-threading diabled. Meaure and act accordingly. It i poible to contruct relatively imple queueing theory model for hyper-threaded proceor. 0. Reference [Borozan] John Borozan, Microoft Window- Baed Server and Intel Hyper-Threading Technology, Microoft Window Technical Article, April 00, /hyperthreading.doc

10 [Friedman] Mark Friedman. Hyper-threading - Two for the price of one?, Meaure IT, March, m.html [Guptha] Soumya Guptha, Multithreaded Programming in a Microoft Win3 Environment, [HTT-API] t_dt_v/tutorial/index.htm [JVMPI] Java Virtual Machine Profiler Interface (JVMPI) de/jvmpi/jvmpi.html [Marr, et. al.] Deborah Marr, Frank Binn, David Hill, Glenn Hinton, David Koufaty, J. Miller, Michael Upton, Hyper-Threading Technology Architecture and Micro architecture, Intel Technology Journal (Hyper-Threading Technology), Volume 06 Iue 0, February 4, me06iue0/vol6i_hyper_threading_tech nology.pdf [Roubtov] Vladimir Roubtov, Profiling CPU uage from within a Java application, 00-/0-qa-08-cpu.html [Vianney] Duc Vianney, Hyper-Threading peed Linux, IBM DeveloperWork, 06.ibm.com/developerwork/linux/library/lhtl/ ytem on both machine i Window 000, running tandard utilitie (Tak Manager, BMC Collect, viru can) and Java.4. Benchmark oftware. We wrote and intrumented a multithreaded Java application configurable to generate variou kind of compute intenive workload. In our experiment the computation conited of multiple invocation of method.log in the Java Math cla; we varied the way in which the ytem wa called upon to compute thoe logarithm. Batch, with a pecifiable number of job tream each running in it own thread. Each tream preent the ytem with a equence of identical job. Each job take about 00 econd on the ingle proceor machine with hyper-threading diabled. Open, with pecified arrival rate and ervice time, each job running in it own thread. In thee experiment the job are horter and their actual ize are determined randomly. On average, each call for one econd of computation on the ingle proceor machine. Interarrival time and ervice demand are exponentially ditributed. Cloed, with a pecifiable number of job tream, pecified think time and job ervice time. Job are a in the open experiment. Think time and ervice demand are exponentially ditributed. Thi paper doe not deal with the reult for cloed job tream. Appendix. Experimental Setup Sytem configuration. The ingle proceor machine run a 400 MHz Intel PowerEdge CPU. The dual proceor machine run two imilar 000 MHz CPU. The operating