Getting the System Sizing and Performance Test Right

Transcription

1 Getting the System Sizing and Pefomance Test Right Steve Hazeltine - Sema Gou Systems Ltd. (now Oacle UK Ltd.) ( shazelti@uk.oacle.co - Telehone ) Andew Johnston - Indeendent Consultant and Sema Gou Associate ( andewj@comuseve.com - Telehone ) 1. Intoduction This document descibes how we secified the Sun seve equied fo the oted Rental Systems at Livingston UK. Livingston is the Euoean leade in oviding sevices to uses of electonic equiment and comutes. A imay business activity is the ental of electonic equiment and comutes, much of which is conducted ove the telehone, so esonse times ae key to the business. We theefoe had to be sue that the new system would efom at least as well as the old one without wasting money though ove-secified hadwae. The execise was a comlete success. We wee able to educe ou oiginal hadwae estimates by ove 200,000 fom the manufactue s ecommendations, and fom day one the system was faste than its elacement. The obseved CPU and memoy usage figues demonstate that we ocued exactly the ight secification fo the client. Ou aoach had a numbe of additional benefits to Livingston, including being able to accuately edict, and thus educe, the numbe of Oacle licenses equied. The key to the success was: undestanding the imotance of efomance to the client, and constucting an aoach which met thei needs, taking measuements fom the old system to geneate the eal use equiements and tansaction volumes, exeienced staff using this data to select the equied latfom, using a combination of a simulato and live uses to ovide a ealistic efomance test, to validate ou aoach befoe oceeding to live unning. Because it elies on having an existing system in lace, this aoach is not alicable to evey situation. Today, howeve, the elacement of an old system is obably moe common than the ceation of a comletely fesh alication, so we hoe ou exeiences will be of some use to othes. 2. Aoach to Secifying the Hadwae Thee ae a numbe of aoaches to hadwae secification, including: aoximate sizing based on the numbe of uses, and the ecommendations of hadwae manufactues, tansaction ocessing and othe benchmak based sizing, Getting the System Sizing Steve Hazeltine and Andew Johnston Page 1

2 comaison with simila existing systems, ototye based benchmaking. The choice of aoach should be based on: undestanding the vaious otions, the cost of estimating the size of the latfom, the isks to the sulie and client of mis-sizing the latfom. Cost is imotant. A ototying execise is not chea, and fo a small numbe of uses it may be bette to make an aoximate sizing calculation and ove-secify the hadwae. Uses aely comlain about a system unning too quickly. Fo a lage system, the isks need to be undestood, and thei cost evaluated. If an extenal sulie is involved, both aties need to be clea on who will bea the cost if the isks mateialise. These could include: the cost of a hadwae ugade, aticulaly significant if the selected model is at the to of a ange, cutting back to the oiginal system, and eeating the system imlementation, consequential loss of business and client s image, loss of faith in the oject by uses, who may not be willing to give the IT deatment o sulie a second chance. We had the following situation: efomance was citical to sales uses, cutting back to the old system would have caused immense disution to the business, with otential loss of sales, evious exeiences had been vey ainful, both fo individuals and the business, we had conflicting signals fom ou initial assessment of the tansaction ate (which suggested quite a small system), and the advice of the hadwae manufactues based on the numbe of uses, which suggested a vey much lage system. We had to get it ight fist time, and it was clea we could not ely on evious exeience and manufactues data alone, so a ototye based aoach was selected. The ocess we adoted is summaised in the figue below: Measue Load and Pefomance of Existing System Measue efomance and esouce usage of key tansactions in ototye Calculate equied CPU, memoy and disk fo live latfom Select Live Platfom Validate choice with eeated measuements and fomal efomance test Getting the System Sizing Steve Hazeltine and Andew Johnston Page 2

3 3. Measuing the Tansaction Load 3.1 Identifying the Key Tansactions We stated the sizing execise with vey little eliable data: The numbe of concuent uses was high (otentially > 100), which suggested a lage system, The numbe of enquiies and odes ocessed e day was elatively low, suggesting a much smalle system. We theefoe ealised we had to find some key indicatos which would allow us to secify the efomance much moe accuately, with the following chaacteistics: The old and new system achitectues wee entiely diffeent (moving fom a oietay database on Data Geneal hadwae unning AOS/VS, to an Oacle7 database unning unde Unix). Theefoe low-level figues elated to hysical system attibutes (e.g. %CPU loading) could not easily be comaed between the achitectues. Vey high-level business tansactions (such as the numbe of new odes e day) would be useful in comaing total volumes of business, but not much use fo measuing subjective efomance which is elated to individual ocessing actions. The key tansaction timings would have to be meaningful (to allow them to be elated to the uses equiements), and accuately and easily measuable (fo an exactly comaable ocess) on both the existing and oted system. We would not have a comlete oted system until quite late in the oject, so we would have to size the system using a elatively small subset of the tansactions. The tansactions would have to be tyical of live usage. In most systems, a small gou of functions ae used most of the time, and these should fom the basis of the ototye. Futhemoe, you need to simulate a eesentative mix of ead and wite accesses, acoss a eesentative section of the database and disk. Fotunately, we identified a small gou of tansactions (a cycle though fom a sales enquiy to ode confimation) which wee suoted by the ealy ilot of the oting execise and which satisfied these citeia vey well. The cycle contained key tansactions fo which the uses had secified taget esonse times: oduct queies, custome queies, and the time taken to finish ocessing a confimed ode. The same enquiy sceens could geneate quey-only tansactions, allowing the mix of quey and udate accesses to the database to be contolled. 3.2 Measuing the Existing Usage We stated by veifying the numbe of active uses on the system. An initial manual assessment (walking aound at a busy time with a cliboad) suggested that the numbe of uses actively loading the system was a vey small ecentage of those logged in: Getting the System Sizing Steve Hazeltine and Andew Johnston Page 3

4 Activity No. In Sceen & Active 6 Sceen (Passive) 11 Sceen (Inactive) 15 At Menu 20 Oeatos/testes 9 Total 61 We tied to distinguish between those uses obviously woking fom the data on the sceen (assive) and those whee the system had been left sitting in a sceen but thee was not obviously anyone using the data, o thee was no data on sceen (inactive). Allowing fo eos, the numbe of active (and assive) sceen uses was aound 17 ± 4 (out of 60+ logged on). This could not only account fo the conflict between the initial estimates, but would also suggest much lowe licensing and memoy equiements. Howeve, the figues wee ehas too low to be believed, and needed to be confimed moe scientifically. We also needed to measue the numbes of each key tansaction and the actual efomance deliveed by the old system. A shot attemt at using manual measuements oved only that the tansaction timings vaied consideably, and the mix of tansactions in the system was much boade than we had been lead to believe, so we could not just geneate enquiies and ode confimations and hoe this would mimic live use. Fotunately the Data Geneal system ovided a athe elegant solution to ou oblems, in the fom of an otional log of all sceen activity fo each use session duing a time eiod. We could then elay these logs and obseve both what the uses had been doing, and the times taken fo key tansactions. Simila mechanisms exist in most oeating systems, o can be simulated using standad test tools. The only emaining oblem was the volume of data this would geneate. In the end we used ou test tool (see the next chate) to automate elaying the logs and to assess which function, on which sceen, each use was efoming. All such combinations wee classified as a database ead, database udate, o ignoed action (such as navigation fom sceen back to menu ). As we elayed the logs the time stams wee used to both time key tansactions and assess when use sessions wee left inactive fo a eiod of time. The figues fom these measuements boadly suoted the ealie estimates, but with a few inteesting asects: Thee was a wide vaiation in the aeas and functions used, with no obvious ule, The numbe of active uses was slightly highe than the manual measuements had suggested, but still less than half those logged on at any time, Database quey actions outstied those which would geneate an udate by a facto of aound 2.5:1, Most key tansactions on the old system wee comleted within the taget times, but with quite a wide vaiation. 3.3 Establishing the Real Tagets Having established an accuate analysis of the numbe of tansactions of vaious tyes thoughout the day (on a few chosen days), we then had to set the tagets fo the new system. Thee wee a numbe of adjustment factos which each had to be taken into account: Getting the System Sizing Steve Hazeltine and Andew Johnston Page 4

5 Gowth - we had to size the system to allow fo a few yeas in a gowing business, Othe uses - we had to conside the inceased system loading which might come fom the intoduction of moe flexible eoting tools, bette intefaces with the OA softwae, ossible access fom othe Euoean sites, netwok management and system monitoing and a numbe of othe such changes, Oeato and develoe activity - command line sessions wee not catued by the logging tool, Business volume - the measuements wee taken duing a quiet month in business tems (a nomal seasonal vaiation due to summe holidays). We couldn t wait fo business to ick u, but we could comae the numbe of odes being ocessed on the measuement days with the eak values fom a few months ealie, Geate sead of functions - we could only simulate a subset of tansactions, and a geate sead in eality would undoubtedly decease efomance due to, fo examle, educed data cacheing, Losses in the measuement ocess - fo examle, we had to beak the logging down into one hou sessions, and lost about seven minutes esetting the system befoe continuing with the next. This inevitably distubed the analysis of things like inactive sessions and the daily totals of tansactions, The wost-case numbe of active uses was calculated as follows: Peak fom Automated Measuements 21 Add maximum eo on measuements 4 Add 60% fo business volume adjustment 15 Add 2 fo oeato/develoe access 2 Maximum Concuent Uses 42 We also decided that the new seve would have to suot a load equivalent to oughly 3700 quey tansactions e hou, deived as follows: Maximum load obseved in tests 1276 Allow 10% fo measuement eos 128 Add 20% fo oeato/develoe access 255 Subtotal 1659 Business adjustment (atio of measued business to ast eak) of 50% gives 2488 Allow gowth facto of 50% to give 3732 Taget quey tansaction ate 3732 Fom the obseved slit between quey and udate tansactions, we noted that the test load would have to include at least 1500 udate tansactions e hou. The equiement was that at this level, 90% of oduct and custome queies and 90% of ode confimations should comlete within the defined tagets. Ou measuements also confimed that these would be a small imovement comaed with the existing system. 4. Building and Measuing the Pototye In ode to un a ototying/benchmaking execise of this sot, you have to do seveal things: Getting the System Sizing Steve Hazeltine and Andew Johnston Page 5

6 1. Choose, souce and set u the hadwae, 2. Choose and develo / set u the test tools, 3. Develo o ot key ats of the taget system, 4. Measue the efomance of the ototye, 5. Secify the eal latfom, 6. Plan and un the efomance tests. You may be constained by decisions which have aleady been made. In ou case, these constaints existed but actually made the job slightly easie. It had aleady been decided that the taget system would be a Sun seve, unning Solais and Oacle7, because of Sema Gou s evious good exeience of this envionment and Livingston s business elationshi with Sun. Futhemoe, the oting execise was aleady unde way, so we had at of the system, albeit incomlete and atially tested, to wok with fom a vey ealy stage. 4.1 Choosing the Pototye Envionment You have to have access to adequate hadwae and softwae to make the execise meaningful. Thee ae vaious obvious souces: develoment equiment, sale o etun ageements, and loans fom the manufactues may be feasible, but uchase is obably the last otion. Given the natue of ou client, we went fo the othe obvious solution - ental. This is a vey good solution and allows the exloation of a numbe of otions, within limits. Unless you test envionment is exactly what you lan fo live use, you then have to decide what is easonable to extaolate fom these tests, and what should not be extaolated. In ou case, we knew that we would be using the same CPU achitectue and we wee confident that we could use the manufactue s CPU efomance figues to comae one model with anothe. On the othe hand, we had decided fo esilience easons that the live system would use a RAID disk aay, but none was available fo us to test. Since we wee confident the system would not be disk bound we decided to isk extaolating figues fom a diffeent disk achitectue, but this might not have been wise in a disk-bound system. You also need to ovide a sufficient latfom to un the test tool(s) themselves. As well as a Sun SacStation unning the taget alication, we needed 4 Pentium PCs unning Windows 95 dedicated to contolling the tests, simulating the load and stoing the esults. As a geneal ule you should ty to avoid test tools which un on the taget machine itself unless you ae confident you can assess thei own imact on efomance. 4.2 Choosing / Develoing the Test Tools The choice of test tools vaies with the alication. We identified a numbe of key equiements elated to ou situation: The alication was esented though a chaacte-mode teminal emulato unde Windows, so ou test tool had to wok with that envionment, To allow vaiations in the tests deending on the esonse of the taget system, the test tool had to include a sciting language of some sot, The tool had to be able to inteet the sceen of the teminal emulato as an 80*25 aay, not just a text contol with a sting in it, Given the lage volume of data, the tool had to be able to wite esults away to stuctued Getting the System Sizing Steve Hazeltine and Andew Johnston Page 6

7 stoage, ideally a PC database, We would have to be able to un seveal test sessions simultaneously fom a single PC, to make unning a sufficient numbe of simulated sessions acticable. Suisingly, we could not find a commecial test tool which met these equiements. The main oblem seemed to be the ability to inteet text within a teminal emulato window, but the ability to wite esults to a database was also lacking in many otions we looked at. Thus we decided to wite ou own! While this should not be undetaken lightly, in ou case it tuned out to be a vey effective solution. We used Visual Basic, which could use DDE to communicate intelligently with the emulato, could wite its esults diectly to an Access database, and used Windows messages to synchonise between diffeent test ocesses. The initial scits we wote to inteet the logs fom the Data Geneal ovided most of the coe functions to decide what the alication was doing, and it was then quite simle to join these togethe with new code diving the alication itself. We also develoed a elated tool to catue keystokes and commentay text, so that exets on the alication could develo the famewok of each test without needing any exetise with the test tool itself. We found that a test cycle coveing about 10 sceens could be ecoded and develoed into a woking test outine in about 1-2 days. Since (with the level of automation) it was easy to do so, we eeated the efomance tests on seveal occasions: when the chosen live latfom became available, with vaious secification client PCs, and when the oted system was functionally comlete. This built u confidence in ou decisions, and ointed the way to some efomance imovements in the alication, and the coect client PC secification. 4.3 Measuing the Pefomance of the Pototye The efomance load simulation was a outine which dove the oted alication in a eeating cycle of enquiy, ode confimation, some geneal enquiies, deleting the eviously ceated ode, and some moe geneal enquiies. The cycle, vaious oduct and custome seaches, and the ode confimation ocess wee each timed and esults witten to the esults database. Analysing this outine in the same way as the existing system showed that each cycle geneated 54 quey tansactions and 23 udates. The test PC could dive u to eight theads - sessions on the taget system. Thee wee fou aametes which affected the actual load geneated: Numbe of test theads, The Shot Delay - a delay of a few seconds inseted between each gou of a few keystokes to simulate natual oeato enty seed (and allow the system to esond), The Long Delay - a maximum delay (in seconds) between eetitions of the cycle, Numbe of eetitions - this mainly affected the duation of a test to make sue that the esults wee statistically viable. By vaying these factos we could simulate widely vaying test loads. The behaviou of the ototye system was then assessed using thee factos: The numbe of quey tansactions e hou (deived fom the aveage ode cycle time and the numbe of theads unning), Getting the System Sizing Steve Hazeltine and Andew Johnston Page 7

8 Whethe the timed tansactions comleted in a satisfactoy time, Whethe the taget Unix host s efomance monitoing tools eoted satuation of CPU and/o disk I/O. The following table summaises the esults of ou ototye test against ou develoment SacStation (a 110MHz Sac 5): Test Run Numbe of Theads Shot Delay Long Delay Cycle Time Quey Tx e Hou Poduct Seach Custome Seach Ode Confim CPU Usage :02: % :03: % :05: % :07: % :06: % :07: % The disk I/O aely exceeded 15% of maximum thoughout all the tests, but the 3 and 4 thead figues showed CPU usage aoaching satuation, and the 4 thead figues showed seach times stating to exceed the taget. Fom this we concluded the following: efomance should not be disk bound, but we wee obably getting unealistic data cacheing and would need a bette sead of functions in the final efomance test, The SacStation 5 could suot u to about 1750 tansactions e hou (keeing machine usage and tansaction times within tagets), so the CPU of the live machine would have to be about 2.5 times this oweful. 5. Secifying the Live Platfom The aim is to oduce a latfom secification in tems of model numbe, numbe and seed of CPUs, memoy size, disk achitectue and caacity. The inuts to this ocess ae: the CPU, memoy and disk esouces consumed by the ototye, the tansaction load and esonse times suoted by the ototye, the equied esonse time and tansaction load, esilience equiements (which may dictate exta CPU o memoy caacity, o a cetain disk achitectue), memoy and disk sizing guidelines fom hadwae and / o database sulies. The following sections ovide some guidelines, esecially fo ocesso sizing, based on ou exeiences. Getting the System Sizing Steve Hazeltine and Andew Johnston Page 8

9 5.1 Identifying the Bottlenecks You must eview you ototye to ensue that thee ae no excetional cicumstances, such as the database being vey ooly tuned, o the test hadwae being so shot on memoy that swaing and aging occued at vey low loads. If the test hadwae can comfotably suot the exected live load, you may be able to educe the secification. Othewise, you will have to identify the bottlenecks in efomance, and hence decide what needs to change: With a lightly loaded machine unning only a few tansaction tyes (whee most data will be cached) the esonse time eesents the time fo one ocesso and the disks to comlete one tansaction. You should see little disk ead activity, and CPU activity oughly ootional to the tansaction load. Inceasing the numbe of ocessos will not imove the individual esonse times, but inceasing individual ocesso owe/ seed will. In actice, available ocesso seeds tyically vay by less than a facto of 2:1 at any one time. Theefoe, if the esonse time of a lightly loaded ototye is inadequate, the cue is to modify the softwae design, not to exect geat imovements fom faste hadwae. If as the load inceases, the disk activity emains accetable, esonse times will incease as the obability of any one ocess having to wait fo othe active ocesses inceases. If the ocesso load gets too high (geneally above 75-80% on most Unix systems) excessive swaing will cause the esonse to wosen damatically. The onset of this can be delayed by faste o additional ocessos. If disk efomance is the bottleneck, this will show itself eithe as an inability to fully load the CPUs when unning a highe tansaction load, o a damatically wosening esonse as the mix of tansactions is inceased to educe cacheing. You may need to look at altenative disk achitectues, vey much moe memoy (to extend the effects of data cacheing), o softwae otimisation. If unde the same cicumstances aging activity becomes vey high then the system is memoy bound. You will have to secify a lage amount of memoy fo the eal system, and extaolate fom moe lightly loaded figues. Altenatively, memoy is one esouce which you should be able to ent o boow to check the effects of a lage quantity. It is usually a false economy to unde-secify the memoy on a machine whee the CPU and disk can suot bette efomance. In ou case, we found that disk and memoy activity wee accetable u to the limit of the tansaction load ou test machine s CPU could suot. We wee theefoe able to secify the live machine as a ootional incease in CPU and memoy caacity, with a disk achitectue deived fom the esilience equiements. 5.2 Selecting the Pocesso(s) The following shows how to: select a ocesso seed that ensues that a lightly loaded machine will give an adequate esonse time, calculate how many ocessos you need to suot the exected eak tansaction load. We use the following tems: c CPU ating (e.g. clock seed) of the ototye s ocesso CPU ating (e.g. clock seed) of the live ocesso c Getting the System Sizing Steve Hazeltine and Andew Johnston Page 9

10 f c P t t Adjustment fo the effects of a eal load comaed with the highe cacheing of a tyical simulated load Numbe of ocessos in the seve measued ototye esonse time of the most citical tansaction (e.g. with the slowest execution seed) equied esonse time of the same tansaction in 90% of cases tansaction load suoted (at <80% CPU Loading) by the ototye eak tansaction load edicted fo the live system, including any gowth o adjustment factos You can make an initial estimate of the equied ocesso owe/seed as: c c >= 11. The facto of 1.1 allows fo an exected incease in aveage esonse time when the ocesso has a easonable (40%) loading. If a ocesso of this seed/owe is not available, the emedy lies in changing the softwae, not in hadwae. The equied numbe of ocessos is then given by the atio of the eak tansaction ate to the tansaction ate suoted (at a easonable CPU loading) by the ototye: P >= fct c ct This gives the (factional) numbe of CPUs that would be fully loaded by the system. f c is a facto to allow fo non-key tansactions not included in the ototye, and othe asects which will make the eal tansaction mix moe heteogeneous than simulated in the ototye. In ou case, we found that a small numbe of tansactions accounted fo aound 60% of the load, and teated the emainde as being equivalent in total to the main sales enquiy tansaction. Thus we estimated a value of 1.5 fo f c. The next ste is to comae otential ocesso configuations fom the manufactue s liteatue that will meet the efomance goal. Fo each otential configuation, you should use the above equations to check that the esonse time will be accetable and the suoted eak tansaction ate adequate: 11. c c <= and ct P fc c >= t Given a ange of accetable altenatives, you final choice will then be based on: ice, elative exected esonse times, likely tansaction load gowth ove the oject lifetime. Getting the System Sizing Steve Hazeltine and Andew Johnston Page 10

11 Ou Poject Exeience In ou oject, we had aleady decided on Sun Micosystems as the hadwae sulie. The measuements on the existing system showed two eak loading times: aound noon when sales activity was at a eak, and ealy evening when desatch was ocessing the day s odes (but sales activity was educed). We calculated the load fo both cases, and concluded that the geatest CPU usage was in the moning. The key tansaction fo esonse time had been identified as a oduct enquiy. The ototye measuements and above calculations showed that: the minimum individual CPU owe could not be less than 75% that of a 110Mhz Sac 5, the total CPU owe should be at least 2.5 times that of the 110Mhz Sac 5. This immediately allowed us to conside the less exensive Sac 20 ange athe than the moe exensive Sac 1000 (caable of taking significantly moe ocessos). Without the ototye, we would not have been able to make this economy with confidence. Thee wee thee main otions: 1. two 50MHz ocessos 2. fou 50MHz ocessos 3. two 70MHz ocessos Of these, the loading fo the 2x50MHz configuation was too high fo comfot. A 4x50MHz unit would have been able to suot inceased thoughut, and theefoe suot geate shottem oganisational gowth, but the 2x70MHz otion offeed bette esonse times, with bette exansion otential in the medium to long tem. The decision was theefoe taken to ocue that configuation, as a fast esonse time in the shot to medium tem was moe imotant than shot-tem gowth. This would suot the business bette, even though it had a lowe secification in simle Secmaks. 5.3 Memoy Sizing The guidelines fo memoy sizing ae usually elated to the numbe of uses. As we noted above, many of the uses on the old Livingston system wee logged in fo long eiods but efomed elatively few tansactions. We could theefoe use as a basis the measued numbe of active, athe than concuent uses. Amed with this numbe we stated with the standad guidelines fo a tyical Unix/Oacle installation: Souce Memoy (Mb) Unix 32 Live database: 40 active 3Mb each 120 Test database 16 CQCS 4GL: Mb Each 20 TOTAL 188 Fom exeience, we wee awae that memoy size is citical in Oacle efomance, and any exta memoy would be useful. We also had to allow fo gowth and othe ocesses (such as monitoing softwae). We theefoe decided to buy 256Mb of RAM, safe in the knowledge that if this was substantially ove-secified any exta could be deloyed on anothe installation. Getting the System Sizing Steve Hazeltine and Andew Johnston Page 11

12 Whee a decision like this is bodeline, but good efomance is essential fom the outset, the ental otion is vey actical. Exta memoy can be ented, and the system should go live with this additional memoy in lace. Use the efomance measuing tools to decide if the memoy is being used. If not, etun the excess to the ental comany. If it is needed, extend the ental until new memoy can be bought. This guaantees efomance at a much lowe cost than outight uchase. 5.4 Secifying the Disks Sizing the Disks Disk sizing is coveed in standad texts and manuals (e.g. the DBA guide fo Oacle), and will not be coveed hee, excet fo a coule of salutay tales: Ou fist estimate of disk sizing was based on two good numbes: an estimate based on the known numbes of ows in each table in the old system, with a facto of thee fo indexing; and some knowledge of the disk sizes in the old Data Geneal system. We wee theefoe athe suised to find that the fist-cut imot into Oacle occuied ove thee times this volume of disk, o almost ten times the size of the aw data. This was due to a numbe of factos: A vey denomalised data stuctue, Lage achive tables than we had been led to believe, A data stuctue enfoced by the 4GL in which most fields in the database wee of fixed length and NOT NULL, even if usually emty. Most of ou new database was emty sace! This tye of oblem is athe insidious: it can haen by default as the esult of a low-level decision outside you diect contol, and it will have a majo cumulative effect. In ou case, if affects not only database sizing, but also efomance, size and seed of backus, and so on. To limit any isk to the oting execise, we decided to live with the stuctue in the shot-tem, but ae now ogessively making manual changes to educe this effect. Othe factos to conside in disk sizing ae the following: Use files and alication softwae will gow in size, no matte how good you intentions to contol them. This is aticulaly tue in the PC aena. You will lose moe to fomatting and the oeating system than you edict. In ou case, we had decided on a RAID aay fo the main disks of the live system, and found that we lost 10-20% to aw disk fomatting, and anothe 20% to the RAID oeations, so that the available size is only about 65% of the total. Othe oeating systems have diffeent featues, but with simila effects. The message is simle: be geneous in sizing you disks, o make sue you have an achitectue in which you can easily at least double you disk caacity if equied. Estimating the Numbe and Seed of Disks To ensue efomance, it is imotant to have enough disks of sufficient seed available. While unning the ototye, you should un a disk monitoing utility (sa o efmete ae adequate unde Unix). You should ensue that you ototye database is sufficiently lage and choice of data sufficiently andom to make the data cacheing as ealistic as ossible. Getting the System Sizing Steve Hazeltine and Andew Johnston Page 12

13 You will need the following data fom the ototye: disk usage (tyically a ecentage) duing a fixed eiod, numbe of hysical and logical eads and wites ove the same eiod. Look at the atio of logical to hysical eads and wites. It is easonable fo small tables and indexes to be cached, and you ae tying to tune fo the best atio ossible. Howeve, if you get a atio in excess of 4:1 you ae obably getting moe cacheing than you will see in eal use. Fom the manufactues statistics you should be able to calculate the aoximate time taken to ead o wite a block of data, D: D= (mean seek time) + 0.5*(otation time) + (block size)/(eak tansfe ate) As a confidence check, you may be able to see a limit to the numbe of hysical eads o wites e second when the system is heavily loaded: this will be oughly 1/D. Given the ocessing times D and D fo the ototye and oosed live disk systems esectively, the stes ae vey simila to those fo ocesso secification. 1. Review the degee of caching in the ototye comaed to the live system. To do this, conside which tables and indexes should be cached, and efe to the code in you ototye. Conside also how much sae memoy you may have. 2. Ensue that the live system s disk ocessing time satisfies (D /D ) < ( / ), othewise you may have a oblem with individual tansaction times even if the oveall tansaction load can be handled. 3. Calculate the numbe of disks N equied to handle the eak tansaction load in a fashion analogous to the calculation of numbe of ocessos: fcdt N >= Dt Note that because disk ae mechanical devices, they ae slow, and that additional disk delays can have a vey maked effect on efomance. Most manufactues ecommend that disk loading in oeational systems is ket unde 40%. To achieve this, we would ecommend at this stage aiming fo a loading of unde 25% - so take a value of t whee this is tue. 4. In you database design and when setting u the system, stie o distibute the live database acoss at least N sindles, so that each is equally used, and so that none of those disks fom a bottleneck. 5. Remembe to conside the eliability asects. Note the manufactue s guidelines and you standads fo location of oeating system softwae. In aticula, you may want to ensue that tansaction logging uses a seaate disk to ensue that the database can be ecoveed in case of a disk cash. You may need to allow fo seaate disks fo this, and you may want to stie it acoss seveal disks to ensue adequate thoughut to the logs. Howeve, in ou exeience with Oacle one logging disk is obably sufficient unless you have a vey heavy tansaction ocessing system. In ou oject, we found that disk loading was usually vey low. Fo eliability uoses we went fo a RAID aay in which the disk seek times wee adequate, and thoughut fom the RAID aay was much highe than we equied due to the stiing fom the RAID 5 configuation emloyed. Note that because ou system is dominated by quey tansactions, the eduction in efomance fom a RAID 5 configuation is not a oblem. In cases whee a Getting the System Sizing Steve Hazeltine and Andew Johnston Page 13

14 highe ootion of tansactions ae udates, othe disk configuations might be moe aoiate. 6. Doing the Real Pefomance Test Befoe live use, we caied out a efomance test which would be moe eesentative of live oeation. This was based on the automated test, but with the following changes: the oiginal test cycle was sulemented by two moe to execise othe ats of the system and educe the effects of data cacheing, a lage numbe of sessions wee simulated in which vey few tansactions wee geneated, but the system was navigated fom sceen to sceen at andom to simulate the exected inactive and assive uses, a few uses wee invited efomed a few tansactions while the system was unde load, to veify that the obseved efomance was in fact accetable. The automated test suite was used to geneate a load oughly equivalent to the edicted eak (including an allowance of 50% fo eos and gowth), and fou eal uses joined in. The conclusion was that the system was noticeably slowe than when unloaded, but vey definitely bette than the Data Geneal unde heavy load, and always accetable. The cut-ove to live oeation went well, without any significant efomance oblems, and was also judged a success. In live use thee have been a few oblems elated to a small numbe of athe extensive and inefficient eots, which ae being addessed by making these moe efficient. In geneal, howeve, the system has efomed well, desite eventually being a much lowe secification than the hadwae manufactues would have ecommended fom the simle numbes of uses. 7. Conclusions We have leaned a numbe of lessons fom this execise: You need to choose you aoach caefully. This ototye-based execise cost aound 20k. Offset against this wee notional savings of aound 200k against the oiginal oosals fo hadwae and licensing, and a vey eal (but less quantifiable) value in educing the isk to the business and the oject by getting it ight fist time. Thus in ou case the costs wee well justified. If you don t ototye, you have two eal otions: ovesecify the hadwae (which may not cost too much in some smalle systems), o take the best guess fom the available documentation, and manage the isk. Know what you need to measue, and get good fist-hand measuements of all significant asects of any existing system and you ototye. Do not tust data which you haven t gatheed o checked diectly - as with ou database sizing unfoeseen factos can have damatic effects, aticulaly when comaing vey diffeent achitectues. Concentate on the factos which ae likely to be a bottleneck in you system. In ou case, the disks wee likely to be good enough, but we needed to get the ocessos ight. In a disk-bound system this would have to be evesed. Do make sue you ototye coves a ange of uses. In hindsight, we should have built eoting oeations into ous, and we would then have identified ealy some asects which have caused oblems, as well as making the cacheing and memoy usage moe ealistic. Getting the System Sizing Steve Hazeltine and Andew Johnston Page 14

15 Be eaed to invest in good test tools. Ou aoach, witing ou own, should not be aoiate o necessay in evey case. Howeve, you should define the equiements fo these tools caefully, and be eaed to be inventive to make the tools ovide you with the data you actually need. Do get the uses involved in the efomance tests. Howeve, don t ely on lage amounts of manual inut - it s difficult to contol, and difficult to eeat. An automated test suite will ove to have a numbe of uses duing the develoment and imlementation. The method we have descibed woked well in ou case. It is not a substitute fo a documented analysis of usage, ojected gowth and equied efomance but it is a way of deiving them whee a evious system exists. Neithe does this method emove the equiement fo skilled system design, but it will educe some of the isks if oely used. A efomance ototye like ous could be even moe useful whee the achitectue of a system is moe comlicated. Ove the yeas, we have both seen systems fail because the sizing and efomance testing wee not ight. In this case, we did get it ight, and hoefully this ae will hel you to size you system coectly. Getting the System Sizing Steve Hazeltine and Andew Johnston Page 15