A Perfect Recipe to maintain SAS programs Shiva Srinivasan, PJM Interconnection LLC, Norristown, PA Shailaja P Ramesh, Supreme Tech Solutions LLC, VA

Transcription

1 A Perfet Reipe to mintin SAS progrms Shiv Srinivsn, PJM Interonnetion LLC, Norristown, PA Shilj P Rmesh, Supreme Teh Solutions LLC, VA ABSTRACT The gol of this pper is to identify the essentil ingredients in writing, mintining nd deugging SAS ode. The stndrds nd guidelines presented herein will ssist softwre developers nd progrmmers to develop ler, effiient nd esily mintinle SAS ode regrdless of the environments. INTRODUCTION The SAS progrmming lnguge while roust provides flexiility. It llows progrmmers to write vlid SAS progrms in severl different wys nd yet hieve the sme result. It hs een oserved tht SAS progrms written refully dhering to ertin priniples nd guidelines re muh esier to mintin. The guidelines mentioned in this pper do not pply to SAS ode generted vi pplition tools suh s SAS Enterprise Guide or ny other third prty tool. SOFTWARE DEVELOPMENT LIFE CYCLE It is reommended to dpt the Softwre Development Life Cyle to fully utilize the potentil of implementing SAS stndrds.. Requirements. Anlysis. Design d. Coding e. Testing f. Integrtion g. Aeptne h. Implementtion i. Mintenne The steps re s follows. Colleting User requirements: This is to identify the prolem definition. This is sttement of need from the user s perspetive. These requirements re usully n Verl desription of wht the progrm should do (lerly explining the lultions nd nlysis involved in dt olletion terminology). Anlysis nd Plnning: This involves generting the high level progrm speifitions nd doumenttion of sttements ontining the funtionl speifitions, fesiility of the projet, proposed timeline, the resoures required nd onsidertion of lterntive solutions.. Design: The Developer retes progrm speifitions tht tke the requirements nd trnsltes them into dtse nd progrmming terminology. This phse determines how

2 the system or progrm will get the work done. A development work pln nd vlidtion protool re doumented during this phse. d. Coding: Progrmming tivities re now performed using the requirements, the speifitions, nd the vlidtion protool s referenes. Progrms re developed using forml progrmming guidelines whih re gered towrds produing redle nd mintinle ode. e. Testing: Progrms re then tested for ll possile ugs y the developer while ensuring miniml reline on the vlidtor to identify errors. Regression testing is lso performed to ensure tht the funtionlity of the progrm hs not een ffeted due to modifitions. f. Integrtion: This phse involves integrting the progrm to new/existing systems/progrms nd verifition of overll integrity. g. Aeptne: The solution must e presented to the user for his eptne nd sign-off. h. Implementtion: Any hnges to vlidted progrm must follow well-defined Chnge Mngement proess. In this phse, doumenttion regrding why hnge is needed (is it ug fix or progrm enhnement) is performed. One the hnge is mde, the progrm must e revlidted through extensive testing efore it is put k in prodution. i. Mintenne: A hnge in the usiness logi or n enhnement to n existing one would lmost for ertin need progrm modifitions or new ones need to e developed. All mintenne tivities should gin follow the steps g of the Softwre development life yle. The ove mentioned steps n e ompressed to only those relted to speifi tsk or projet. Regrdless of the numer of steps involved in projet, doumenttion nd testing re indispensle stges required to ensure desired outomes.. STANDARDS AND GUIDELINES Essentils 1. Clrity 2. Consisteny 3. Mintinility 4. Reusility 5. Trnsfer Cpility (i.e. ross environments) 6. Trnsfer Cpility (i.e. ross vrious memers) 7. Not just redle ut understndle Ctegories 1. Nming Conventions 2. Comptiility 3. Doumenttion 4. Apperne 5. Effiieny 6. Mintinility 2

3 1. Nming Conventions Use pproprite, meningful nd unmiguous nmes for lirries, progrm files, dtsets, mros, dtsets, vriles, formts nd mro vriles. Following re some of the stndrds pertining to nming. Nme Convention Exmples XYZ1.ss, XYZ2.ss Cn e nmed sequentilly 1 Progrms dev_rep.ss, dev_formt.ss Cn e nmed logilly Should e nmed logilly mroli 2 Lirries Should inlude some kind of li_mro identifier 3 Dtsets Depend on the pplition nd fility stndrd Should e logil Vriles Should e speified for speifi pplition needs or deprtment guidelines 4 Trunte long nmes rther thn revite Input i_vrnme Vriles Output o_vrnme Vriles Arry Vriles _rrynme _rrynme IN = Nmes 5 Formt Should inlude some kind of identifier Cse 1: When pplied to one vrile MERGE X(IN=INX) Y(IN=INY) FORMAT NAMETYPE NAMEFMT FORMAT NAMETYPE FMT_NAME Cse 2: When pplied to tegory FORMAT DATETYPE DATEFMT FORMAT DATETYPE DATEFMT FORMAT DATETYPE FMT_DATE 6 Mro Use self doumenting nmes for mro nmes nd mro prmeters %mro totl_qty(dsn=,strtdte=); 3

4 7 Nmes to void Nmes tht too losely resemle one nother Amiguous nmes suh s FLAG1, COUNT Insted use IS_FOUND, nonnormlurveflg, errount, iter_ount 8 Generl Nming Nmes should e short, desriptive nd unique In se longer nmes re needed, use undersores to seprte words Sequentil progrms should hve the sme nme-prefix ut lso hve numeril suffixes to indite their ple in the sequene pro_not_exeed, seson_prmeter, log_of_sles 2. Comptiility Comptiility dels with the flexiility of SAS ode to e trnsported to other pltforms. Following stndrd for omptiility helps hieve the trnsfer with reltive ese. Comptiility generlly involves the following. Filittes trnsfer of progrms to other pltforms. Generlize the ode suh tht speil fetures of one system n e esily trnslted to work on nother system. Follow onsisteny so tht progrms n e shred ross memers of group d. Progrms must e written keeping in mind tht it would e used severl times or reused severl times under different irumstnes Some of the stndrds pertining to omptiility re s follows Environment Vriles: The following sttements n e stored seprtely so tht they n e %INCLUDE-ed in ll pplition progrms %let sspth = %sysget(appl_sas); %let mropth = %sysget(appl_macrolib); %let formtpth = %sysget(appl_formatlib); %let dtpth = %sysget(appl_data); %let logpth = %sysget(appl_log); %let outpth = %sysget(appl_output); Log nd Output Rediret log nd output files (lst files) to seprte diretories or lirries using stndrd pth/lirry nmes 4

5 PROC PRINTTO LOG = &logpth/xyz1.log; PROC PRINTTO LOG = &outpth/xyz1.lst; Options Options relting to printer suh s LINESIZE nd PAGESIZE should not e hrdoded 3. Doumenttion For mintenne nd deugging purposes dd numerous explntory omments with the SAS progrm explining detils out:. Module Desription. Conditionl nd Susetting Logi. Explntion of Clultions, dtse queries nd logi pertining to the module Some of the stndrds pertining to Doumenttion re s follows Nme Desription Exmple A progrm heder should See Prologue Setion Progrm e the strt of ny Heder Dtset Lels Vrile Lels progrm A lel should e used when sving permnent SAS dtset Vrile lels must e inluded while sving SAS permnent dtsets DATA MYLIB.Y123(Lel= "File 123 of Customer Dt for Yer 2004"); DATA MYLIB.Y123(Lel= "File 123 of Customer Dt for Yer 2004"); SET Y; LABEL V_NAME1 = 'Vrile 1' V_NAME2 = 'Vrile 2'; 4 Comments d DATA step nd PROC step sttements must lwys e ommented Identify the tion eing performed (explin omplex lultions, explin non stndrd logi) Use Inline omments to more fully identify nmes whose required revity osures their mening Identify the funtionlity of omments DATA AREA; SET RADIUS; AREA = 3.14 * (RADIUS**2); /* Cl Are */ /*smll point*/ /***More Importnt***/ /********** Minor Brek **********/ /***********************************/ /*** Signifint Susetion ***/ /***********************************/ 5

6 5 Output Alwys speify titles when generting outputs PROC PRINT DATA = orig_dt; TITLE 'ORIGINAL DATA'; A prologue desries wht the progrm ws intended to omplish, its uthor, nd the resons for its development (suh s, wht, why, for whom, y whom nd when); the prologue should lso inlude hnge mngement ontrol log, whih desries the susequent modifitions mde to the progrm. A stndrd prologue should ontin the following ut my lso ontin ny other informtion pertining to the projet requirements. Prologue * ************ ******************** ************ ************ * * PROGRAM NAME : * * DESCRIPTION : * * * * DEPENDENCIES * * CALLED BY : * * CALLS TO : * * * * PROGRAMMER : * * DATE ORIGINAL : * * ************ ******************** ************ ************ * * INPUT FILES/LIBRARIES : * * OUTPUT FILES/LIBRARIES : * * ************ ******************** ************ ************ * * MODIFICATIONS * * DATE : * * CHANGE # : * * PROGRAMMER : * * DESCRIPTION : * * ************ ******************** ************ ************ * 4. Apperne nd Redility Progrms should e written in mnner whih is redle nd esy to mintin.. Progrms should look net nd orderly. The visul pperne of the progrm should mirror its logil flow Apperne is very losely relted to style nd vries etween users, ut the reommendtion is to dhere to n pproprite style nd mintin it ross SAS progrms. Some of the guidelines for writing SAS progrms tht re esy to mintin nd understndle re s follows 6

7 Nme Desription Exmple 1 # of sttements Use one sttement per line 2 Blnk Lines Skip line efore every DATA nd PROC sttements Seprte loks of ode to mke progrms redle DATA X; sttements.; run; PROC pronme; sttements.; run; 3 Alignment nd Indenttion d Left justify DATA, PROC, OPTIONS sttements. Indent ll sttements within. Indent sttements within DO loop. Align END with DO Indent sttements within n IF lok. Align END with IF nd ELSE Align mro prmeters in seprte lines to improve redility e Align relted elements The inverted 'C' look is the est wy to follow logi. It lerly indites tht there is mthing END for eh DO nd IF sttements %mro new ( vr1=, vr2=, vr3= ) ity street zipode zip. f Line up the ses in SELECT sttement selet (ity); when (phl) do; odes; end; when (hl) do; odes; end; otherwise do; odes; end; end; 7

8 4 Plement of Sttements Inside Progrm 1. OPTIONS sttement first 2. %LET sttements 3. FILENAME, LIBNAME sttements 4. PROC FORMAT 5. Mro Defn 6. Input Steps 7. Clultions/ Opertions/Ativities 8. Output 5 Pge Option 6 Upperse Inside DATA step Use the ompiler PAGE option t the eginning of eh routine. This ensures pge ejet when ompiling listing in doumenttion inder ensuring fst ess to the routine you re looking for Use upperse for seleted elements to enhne redility 1. All non exeutle sttements first(length, KEEP) 2. All exeutle sttements next pge; ll vriles Constnts sttement lels omments tle nmes prmeters dt set nmes mro keywords ****new routine**** 7 Reports 8 Column # 9 Complexity 10 Titles 11 Prentheses Mke sure tht titles nd footnotes used in printed reports urtely reflet the informtion eing presented Try not to extend ny sttement eyond olumn 72 Brek omplex sttements into numer of simpler sttements If title sttements help explin wht the progrm does, put them ner the eginning of the progrm so they'll e esy to find. Put title sttements for step t the eginning of the step If prentheses would lrify the order of evlution of n expression, go hed nd use them even if the syntx rules don't require them 8

9 5. Effiieny Adpt liner nd modulr style of progrmming. This redues the deugging time, updting or modifying time for the progrmmer. Some of the guidelines pertining to effiieny re s follows 1 Nme Desription Exmple When extrting dt from soures suh s flt file, input only the vriles needed Use KEEP=/DROP= nd DATA test(keep= Vr1 Vr2); KEEP/ DROP Input sttements when inputting Vriles SAS dtsets. Reommend to use KEEP insted of DROP whih improves redility. It is lso reommended to use the KEEP=/DROP= dtset option rther thn KEEP/DROP euse in this wy SAS retes spe only for the vriles used in the PDV (Progrm dt vetor) rther thn loding the entire vrile set nd then dropping or keeping vriles. DROP intermedite vriles used in lultions DATA my_loop; DROP j k temp; DO j=1 to 3; DO k=1 to 5; temp=j; new=temp*k; END; END; 2 Where ond Use WHERE rther thn IF while susetting SAS dtsets, sine WHERE susets the dt efore entering into the Progrm Dt Vetor, while IF susets the dt fter inputting the entire dtset DATA SCHOOL_YEAR; set sh_yr(where=(yr gt 85); insted of DATA SCHOOL_YEAR; SET sh_yr; IF YR gt 85; 3 IF/ELSE onstruts Chek the most prole ondition first when stepping through IF onstrut Use IF/ELSE for mutully exlusive onditions Use IF/THEN/ELSE insted of multiple IF sttements if ode = 1 then stte = NY ; else if ode = 2 then stte = MO ; 9

10 else if ode = 3 then stte = MI ; 4 Sorting When sorting permnent dtsets sort it into nother dtset, sine sorting into permnent dtset tkes more I/O, nd sorting lrge files requires more spe PROC SORT DATA=TEST.NEW(KEEP A); OUT=NEWSORT; Use CLASS sttement to void unneessry sorts PROC MEANS NOPRINT NWAY DATA=TEST; CLASS YEAR; VAR COST; OUTPUT OUT=MEANS MEAN=; 5 Ctegoril Vriles 6 DATA steps Use hrter vlues for tegoril vriles or flgs insted of numeri vlues This tion sves spe sine hrter '1' uses one yte wheres numeri 1 uses 8 ytes. However there needs to e hrter to numeri onversion if Boolen logi needs to e used whih outputs NOTE nd lso onsumes more CPU time. Use DATA _NULL_ when you need to perform lultions on dtsets or generte reports Use MY_FLAG='1'; insted of MY_FLAG=1; DATA _NULL_; FILE PRINT; SET OLD; 7 LENGTH Ple most of the nonexeutle sttements efore the exeutle sttements with exeption of DROP nd KEEP sttements Use LENGTH sttement to ensure the length of the entered vrile DATA Short; LENGTH height weight 4; INPUT ID $ ($1.) 10

11 Sves spe nd lso progrms re muh more height weight 3.; DATALINES; ; 8 PROC DATASETS Used to opy, renme nd delete SAS dtsets. Used to renme vriles, hnge or dd formt nd lels, nd ppend dt sets to other dt sets. LIBNAME liref 'MY_DATA_LIBRARY'; PROC DATASETS LIBRARY=liref; MODIFY OLD; RENAME X=NEWX Y=NEWY; LABEL HT='Height of Sujet'; FORMAT DOB MMDDYY8.; 9 RETAIN Use RETAIN to initilize onstnts 10 Ending End every DATA nd PROC sttements with End every Pro SQL sttement with QUIT; RETAIN tolerne 0.1 ftor 20; DATA TEMP; ; PROC SORT DATA = COUNTRY; BY STATE; PROC SQL; SELECT * FROM SASHELP.AIR; QUIT; Use 11 Redue numer of times dt is red Minimize the numer of psses through the dt Minimize the numer of DATA steps Red nd store only the dt tht re needed Sort the dt only when it is solutely neessry. Pro Sort Dt = Cities Out = new_ities By Stte; run; Insted of Pro Sort Dt = Cities; y Stte; run; Dt sorted_ities; Set Cities; Run; 12 Reding externl files Use pointer ontrols, informts, INPUT sttement, to red only the dt needed 11

12 13 DELETE 14 Conversion 15 INDEX 16 Arrys Minimize workspe usge y using DELETE sttement in PROC DATASETS step, to eliminte temporry dtsets tht re no longer needed y the progrm While onversions re very useful to onvert numeri to hrters nd vie vers, extensive onversions in lrge dt set n ost time. If lrge sorted dtset is eing used multiple times then it is worth reting n index Whenever there re group of vriles to e proessed in the dt step, it is enefiil to onsider rrys to omplish the sme tsk. Arrys help redue redundny in ode, lookups Dt Without_Arry; Set ontents; if x1 = 999 then x1 =.; if x2 = 999 then x2 =.; if x100 = 999 then x100 =.; if A = 999 then A =.; if B = 999 then B =.;... if E = 999 then E =.; run; Dt With_Arry(drop=i); Set ontents; Arry _ontents[105] x1-x100 A B C D E; Do i = 1 to 105; If _ontents[i] = 999 then _ontents[i] =.; End; run; 6. Mintinility Write progrms keeping in mind tht they n e reused sometime in the future. A generl pproh is very useful. Some of the guidelines for mintenne re s follows Nme Desription Exmple 1 Constnts Use %LET sttements to define onstnts rther thn hrd oding them into progrms 2 Brnhing Do not use GOTO/RETURN sttements (These mke the progrms diffiult to tre) %LET WK_START=1 %LET WK_STOP=20 DO I = &WK_START to &WK_STOP; 12

13 3 Nested Clls Do not rnh to nother progrm from within progrm. Use the min progrm to ontrol ll progrms Do not use more thn 2-3 nested mro lls 4 Output Formulte n effiient method to tegorize output dtsets suh tht the progrm does not rete permnent dtsets whih re overwritten while testing. 5 Notes/ Wrnings 6 Mros Autoll 7 Progrm Flow 8 Dependenies Avoid unneessry NOTES/WARNING messges in the log. Use mros if the routine is lled more thn one The routine depends upon the vlue of vrile The routine requires progrmming logi whih nnot e inluded in dtstep Dt=permli.x; set x; run; The routine is used y more thn one progrm The routine hnges often Progrms should flow linerly ProgA=>ProgB=>ProgC d e If there re mny progrms in system then hve min progrm ll eh one rther thn hving progrm to ll the next Listing the dependenies inreses the mintinility of the progrm Some of the dependenies inlude Reord input files Vrile nmes, types nd other required hrteristis of the SAS input dtset Limittions on the vlues of input dt Memory requirements when neessry Informts nd Formts stored in formt lirry Use n OPTIONS sttement t the eginning of the progrm to expliitly set ny required system options Use LENGTH sttement to delre nd set the length of hrter vrile 13

14 CONCLUSION Consistent use of good progrmming stndrds helps improve redility, effiieny nd redues mintenne. Adpting new stndrd definitely inreses development time, ut not dpting good stndrds osts more in terms of deugging, mintenne nd in most ses results in re-oding. Hene spending some extr time initilly helps sve onsiderle time lter. REFERENCES Lor D. Delwihe., Susn J. Slughter, (1996) The Little SAS Book, A Primer, Seond Edition Pul Gill, (1997) The Next Step Integrting the Softwre Life Cyle with SAS progrmming Rih Aster, (2002) Professionl SAS Progrmming Shortuts, Alie M. Cheng., Pfizer In., (1999), Roust progrmming tehniques in the SAS System, NESUG, Cly Mrtin., Lori Mrtin., Mrtin Consulting, (2000), Clen-up, Comments nd Code Mking it Mintinle, NESUG Vness Hyden, Poliy Anlysis In., (2000), Bulletproofing Your SAS Results, NESUG ACKNOWLEDGEMENTS I would lso like to thnk Hether Forsythe, Fin Roinson nd Roert Jones of PJM Interonnetion LLC for their ontinuous support nd motivtion throughout the entire ourse of the pper. Shilj nd I would like to express our sinere thnks to fmily nd friends for their onstnt enourgement nd well wishes. CONTACT INFORMATION Your omments nd questions re vlued nd enourged. Contt the uthor t: Author Nme: Shiv Srinivsn Compny: PJM Interonnetion LLC Address: 955 Jefferson Ave City Stte ZIP: Norristown, PA Work Phone: Fx: Emil: srinis@pjm.om, shiv_wvu@yhoo.om Seondry Author Nme: Shilj Rmesh Compny: Supreme Teh Solutions LLC Address: 125 Brookview Dr City Stte ZIP: Pottstown, PA Work Phone: Emil: shilj_shiv@yhoo.om SAS nd ll other SAS Institute In. produt or servie nmes re registered trdemrks of SAS Institute In. in the USA nd other ountries. indites USA registrtion. Other rnd nd produt nmes re trdemrks of their respetive ompnies. 14