Preserving your SAS Environment in a Non-Persistent World A Detailed Guide to PROC PRESENV Steven Gross, Wells Fargo, Irving, TX ABSTRACT For Enterprise Guide users, one of the challenges often faced is doing large development in a non-persistent environment. A non-persistent environment is one in which the connection to the SAS server on the grid is disconnected after an extended period of idle time. This period could be hours or even, in some cases, as little as 30 minutes. Usually, when a SAS connection is terminated, all work datasets, LIBNAME references, macro variables and system settings are permanently lost. So, if a developer is in the midst of writing code, one step at a time, and has created numerous work datasets, macro variables, or non-permanent formats, if a session is involuntarily closed, all of that good work is lost. The only way (up until now) of restoring the environment was to re-execute the code up from the very beginning, up until the last written complete step. If, as is very common today in the world of BIG DATA, we are dealing with large work datasets, or very long code, this can possibly represent a significant investment of time, and can be a great source of frustration. INTRODUCTION Starting with SAS 9.4, a new procedure, called PRESENV, is now available which will store the current environment, including all work datasets, any compiled macro code, the values of any macro variables, and all temporary formats, as is, and enable it to be restored to a new SAS session at a later time. This paper will describe the use of the PRESENV system option and PRESENV procedure, in a practical, chronological sequence. The first section will describe what is required to preserve the environment. Afterwards, the next section will describe the steps needed to restore the environment once a new EG session is established.
I. USING PROC PRESNEV to PRESEVERE THE SAS ENVIRONMENT In order to save the SAS environment as is, only four simple pieces of code need to be added to any program being developed: For our purposes a simple SAS program, named presenv_example.sas will be used: options presenv; %let pgmname=presenv_example.sas; data work.cars; set sashelp.cars; proc summary data=cars nway; class make type; var msrp; output out=cars_mean(drop=_type freq_) mean(msrp)=avg_price; proc format; value cost low - 20000 = 'Inexpensive' 20000-40000 = 'Moderate' 40000 - high = 'Expensive'; data cars_mean_ftmd; set cars_mean; budget=put(avg_price,cost.); libname bkuploc '/directory1/directory2/directory3/tmpbkup/'; filename restore /directory1/directory2/directory3/restoration_pgm.sas'; proc presenv permdir=bkuploc sascode=restore show_comments;
Explanation of Program Statements Required The PRESENV system option is required to enable PROC PRESENV to preserve the current SAS Environment. When the PRESENV option is set, SAS begins to collect the data that is needed to preserve the environment. Data is collected until either the NOPRESENV option is set or the SAS session ends. You can enable and disable the collection of data at any time during a SAS session. When NOPRESENV is set, data collection is suspended, but not discarded. The following elements of a SAS session are saved: All WORK library data sets, formats and catalogs global statement values macro variable values system option values LIBNAME points to the storage location where SAS will copy the contents of the WORK library when preserving the current SAS EG environment. Caution: This libref should never point to a storage location containing any other data, since prior to storing the SAS WORK datasets and catalogs, SAS will delete all of the contents of this library. Filename references the name and location of the code automatically created by SAS to restore the preserved environment. PROC PRESENV Initiates the preservation process. PERMDIR=libref specifies the location where the entire contents of the WORK library are written. SASCODE=fileref specifies the location where SAS will automatically write the program used to restore the environment. Optional Argument: SHOW_COMMENTS displays all global statements. Redundant global statements are commented out. If this option is not used, then the global statements are suppressed. Usage: During development, a programmer could leave the procedure PRSENV as the last executable step within the code while it is being developed.
Execution of PROC PRESENV When the PROC PRESENV contained in the above code is executed, the following log file NOTES are produced: WARNING: No matching members in directory. NOTE: PROCEDURE DATASETS used (Total process time): 0.01 seconds 0.01 seconds NOTE: Deleting BKUPLOC.CARS (memtype=data). NOTE: Deleting BKUPLOC.CARS_MEAN (memtype=data). NOTE: Deleting BKUPLOC.CARS_MEAN_FTMD (memtype=data). NOTE: Deleting BKUPLOC.FORMATS (memtype=catalog). NOTE: Deleting BKUPLOC.SASGOPT (memtype=catalog). NOTE: Deleting BKUPLOC.SASMAC1 (memtype=catalog). NOTE: PROCEDURE DATASETS used (Total process time): 0.03 seconds 0.01 seconds NOTE: Copying WORK.CARS to BKUPLOC.CARS (memtype=data). NOTE: There were 428 observations read from the data set WORK.CARS. NOTE: The data set BKUPLOC.CARS has 428 observations and 15variables. NOTE: Copying WORK.CARS_MEAN to BKUPLOC.CARS_MEAN (memtype=data). NOTE: There were 114 observations read from the data set WORK.CARS_MEAN. NOTE: The data set BKUPLOC.CARS_MEAN has 114 observations and 3 variables. NOTE: Copying WORK.CARS_MEAN_FTMD to BKUPLOC.CARS_MEAN_FTMD (memtype=data). NOTE: There were 114 observations read from the data set WORK.CARS_MEAN_FTMD. NOTE: The data set BKUPLOC.CARS_MEAN_FTMD has 114 observations and 4 variables. NOTE: Copying WORK.FORMATS to BKUPLOC.FORMATS (memtype=catalog). NOTE: Copying WORK.REGSTRY to BKUPLOC.REGSTRY (memtype=itemstor). NOTE: Copying WORK.SASGOPT to BKUPLOC.SASGOPT (memtype=catalog). NOTE: Copying WORK.SASMAC1 to BKUPLOC.SASMAC1 (memtype=catalog). NOTE: PROCEDURE COPY used (Total process time): 0.08 seconds
Explanation of Log File The very first action taken by PROC PRESENV is to prepare the location referenced by the libref in the PERMDIR statement, by deleting all of its contents. Note: Either or will be produced, but not both. If this is the very first execution of PROC PRESENV which references this location, its contents are empty, and there is nothing to delete. In this case, the log file will only contain the following: WARNING: No matching members in directory. If however, this is a subsequent execution of PROC PRESENV, and SAS has already previously copied datasets from the WORK Library here, they will need to be refreshed. To do so, SAS will delete the entire contents of the directory prior to performing another copy from the WORK Library here. WARNING: In either case, SAS will attempt to delete the entire contents of this directory. Therefore, it extremely critical that the library referenced by PERMDIR statement in the PRESENV procedure should never, under any circumstances, point to an existing location where any critical permanent data is stored. Instead, it should always be a location which is used only for the distinct purpose of saving the work datasets required by SAS to restore the environment. User created WORK data sets; temporary formats, CATALOGS and the contents of WORK.REGISTRY, WORK.SASGOPT, and WORK.SASMAC are copied to the location specified by the libref used in the PERMDIV statement of PROC PRESENV. Upon completion of the PRESENV procedure, this location now contains the following:
II. RESTORING THE SAS ENVIRONMENT SAS has made the restoration process exceedingly simple. Once a new EG session is established, the only action required by the developer is to execute the code which was automatically created by SAS and stored in the location referenced by filename in the SASCODE= statement of the PROC PRESENV procedure When the restoration_pgm.sas program is executed, SAS datasets which were copied in the previous session from the WORK Library to the library specified in the PRESENV procedure (bkuploc) are now copied back to the WORK library, and are ready to be used. The value of all previously defined macro variables and system options are also restored, and all macros, both system and user written, are recompiled. After executing the restoration_pgm.sas, the original development program, my_orig_pgm.sas can be reloaded back into EG, and development can now resume from where it was left off. The Restoration Source Code Created by PROC PRESENV OPTIONS presenv; LIBNAME BKUPLOC '/em_data1/ls_uat/em_data1/tds/admin/sasdata/tmpbkup/'; FILENAME RESTORE '/em_data1/ls_uat/em_data1/tds/admin/sas_code/restoration_pgm.sas'; %let saved_options=options %sysfunc(getoption(source)) %sysfunc(getoption(notes)) %sysfunc(getoption(obs,keyword)) %sysfunc(getoption(firstobs,keyword)); options obs=max firstobs=1; proc sql; create table _data_ as select distinct memname from dictionary.catalogs where libname='bkuploc' and memtype='catalog'; quit; data _null_; set; call execute( cats('proc catalog force c=bkuploc.',memname, '; copy out=work.',memname,'; quit;') ); proc delete; proc copy in=bkuploc out=work mt=data; quit;
data _null_; call symput('pgmname','/em_data1/ls_uat/em_data1/tds/admin/sas_code/presenv_example.sas' ); call symput('sasworklocation','222f6f70742f736173776f726b2f5341535f776f726b3245353430303030 343443425F6370707261303161303031352F5341535F776F726B4632444430303030343443425F63707072 61303161303031352F22'X); call symput('_clientapp','2753415320456e746572707269736520477569646527'x); call symput('_clientappabrev','eg' ); call symput('_clientmachine','2744544344423831324632303242354327'x); call symput('_clientprocessflowname','2750726f6365737320466c6f7727'x); call symput('_clientprojectname','2770726573656e762e65677027'x); call symput('_clientprojectpath','272f656d5f64617461312f6c735f7561742f656d5f64617461312f746 4732F41646D696E2F5341535F436F64652F70726573656E762E65677027'X); call symput('_clienttasklabel','2770726573656e765f6578616d706c6527'x); call symput('_clientuserid','277535373336383727'x); call symput('_clientusername','2747726f73732c2053746576656e27'x); call symput('_clientversion','27372e3130302e322e3334393127'x); call symput('_eg_workspaceinit','1' ); call symput('_sashostname','276370707261303061303831372e77656c6c73666172676f2e636f6d27'x); call symput('_sasprogramfile','272f656d5f64617461312f6c735f7561742f656d5f64617461312f746473 2F41646D696E2F5341535F436F64652F70726573656E765F6578616D706C652E73617327'X); call symput('_sasservername','2753415341707027'x); call symput('_metauser','u573687' ); call symput('afdsid','0' ); call symput('afdsname',trimn(' ')); call symput('aflib',trimn(' ')); call symput('afstr1',trimn(' ')); call symput('afstr2',trimn(' ')); call symput('fspbdv',trimn(' ')); options _last_=work.cars_mean_ftmd; One of the important things to note about the contents of the restoration program- restoration_pgm.sas is that it contains very little code from the original program presenv_example.sas, other that the LIBNAME associated with the PERMDIR statement and the FILEREF associated with the SASCODE statement of PROC PRESENV. All other SAS code statements found in restoration_pgm.sas are directly related to restoring the environment, and the original contents of the WORK library. For this reason, it is critical the original program, presenv_example.sas, be repeatedly saved as additional code is added by the developer.
Explanation of the Restore Program Statements: Re-assigns any LIBNAME and FILEREF statements used in the program containing PROC PRESENV. Minimally, the LIBNAME referenced in the PERMDIR statement and the FILEREF in the SASCODE statement need to be reassigned, so that they can be properly referenced within the restoration program itself. Reestablishes any system options used in the previous session. Determines the names of any CATALOGS which have been backed up to the PERMDIR location and places them into WORK.DATA1 Copies all macros and formats listed in WORK.DATA1 from the PERMDIV LIBNAME location, back into the WORK library Deletes WORK.DATA1 Copy all members with a member type of DATA from PERMDIR LIBNAME location to the WORK Library Restores all system and user macro variable values Reassign WORK.CARS_MEAN_FTMD as the last used dataset (last) WORK.CARS_MEAN_FTMD Log File Created From The Execution of the Restoration Program Upon execution of the above restoration program, the following log messages are produced: (for the purposes of brevity, the source code statements are not presented, as they are available above) NOTE: Libref BKUPLOC was successfully assigned as follows: Engine: V9 Physical Name: /em_data1/ls_uat/em_data1/tds/admin/sasdata/tmpbkup NOTE: Table WORK.DATA1 created, with 2 rows and 1 columns. NOTE: PROCEDURE SQL used (Total process time): NOTE: There were 2 observations read from the data set WORK.DATA1. NOTE: DATA statement used (Total process time): 0.02 seconds
0.01 seconds NOTE: CALL EXECUTE generated line. 1 + proc catalog force c=bkuploc.formats; copy out=work.formats; quit; NOTE: Copying entry COST.FORMAT from catalog BKUPLOC.FORMATS to catalog WORK.FORMATS. NOTE: PROCEDURE CATALOG used (Total process time): NOTE: Copying entry CHECKFMT.MACRO from catalog BKUPLOC.SASMAC1 to catalog WORK.SASMAC1. NOTE: Copying entry ECLIBASSIGN.MACRO from catalog BKUPLOC.SASMAC1 to catalog WORK.SASMAC1. NOTE: Copying entry ECLIBUNASSIGN.MACRO from catalog BKUPLOC.SASMAC1 to catalog WORK.SASMAC1. NOTE: Copying entry ENTERPRISEGUIDE.MACRO from catalog BKUPLOC.SASMAC1 to catalog WORK.SASMAC1. NOTE: Copying entry GETTSLVLPARAM2.MACRO from catalog BKUPLOC.SASMAC1 to catalog WORK.SASMAC1. NOTE: Copying entry _EG_CONDITIONAL_DROPDS.MACRO from catalog NOTE: Copying entry _EG_HIDENOTESANDSOURCE.MACRO from catalog NOTE: Copying entry _EG_RESTORENOTESANDSOURCE.MACRO from catalog NOTE: Copying entry _EG_WHEREPARAM.MACRO from catalog BKUPLOC.SASMAC1 to catalog WORK.SASMAC1. NOTE: Copying entry _SAS_POPCHARTSIZE.MACRO from catalog NOTE: Copying entry _SAS_PUSHCHARTSIZE.MACRO from catalog NOTE: Copying entry _SAS_VERCOMP.MACRO from catalog BKUPLOC.SASMAC1 to catalog WORK.SASMAC1. NOTE: Copying entry _SAS_VERCONDCODE.MACRO from catalog NOTE: PROCEDURE CATALOG used (Total process time): NOTE: Deleting WORK.DATA1(memtype=DATA). NOTE: PROCEDURE DELETE used (Total process time): NOTE: Copying BKUPLOC.CARS to WORK.CARS (memtype=data). NOTE: There were 428 observations read from the data set BKUPLOC.CARS. NOTE: The data set WORK.CARS has 428 observations and 15 variables. NOTE: Copying BKUPLOC.CARS_MEAN to WORK.CARS_MEAN (memtype=data).
NOTE: There were 114 observations read from the data set BKUPLOC.CARS_MEAN. NOTE: The data set WORK.CARS_MEAN has 114 observations and 3 variables. NOTE: Copying BKUPLOC.CARS_MEAN_FTMD to WORK.CARS_MEAN_FTMD (memtype=data). NOTE: There were 114 observations read from the data set BKUPLOC.CARS_MEAN_FTMD. NOTE: The data set WORK.CARS_MEAN_FTMD has 114 observations and 4 variables. NOTE: Copying BKUPLOC._PRODSAVAIL to WORK._PRODSAVAIL (memtype=data). NOTE: There were 8 observations read from the data set BKUPLOC._PRODSAVAIL. NOTE: The data set WORK._PRODSAVAIL has 8 observations and 6 variables. NOTE: PROCEDURE COPY used (Total process time): A careful examination of the log shows that SAS is restoring the environment by performing a series of copies from the bkuploc to the WORK LIBRARY. Upon completion, all previously created WORK datasets are again accessible. Formats have been restored, and the macro code and variable values can be used again. In short, everything that is needed to resume development from the point where it was stopped is now once again available. Once the restoration program has been executed, the only remaining step is to open the original development program and continue where coding was previously left off. CONCLUSION SAS has provided a relatively simple way to preserve your EG session and operate in an environment where your EG session my not be persistent. It is also important to note that even if your environment is persistent, other events may cause you to lose connectivity to the SAS server from your local machine, such as an automatic reboot. In that event, PROC PRESENV might be used simply overnight, as a precautionary measure. Other situations might be entirely voluntary, such as a vacation, or a plan to take a break for an extended period of time. Regardless of the reason, there are probably many practical situations in which PROC PRESENV can be extremely useful, and provide an added layer of stability to a very dynamic work environment.
Contact Information Steven Gross Business Consultant, Wells Fargo Technology and Data Services Wells Fargo Commercial Capital Credit Risk Modeling COE 5000 Riverside Drive, Suite 300 East, Irving, TX 75039 4314 Office: (469) 299 7752 Cell: Steven.gross@wellsfargo.com Acknowledgements Flexibility by Design: A Look at New and Updated System Options in SAS 9.4, Jan Squillace, SAS Technical Support, Cary, NC Further Reading PRESENV Procedure Base SAS 9.4 Procedures Guide, Seventh Edition SAS Institute 2017 Notices SAS is a registered trademark of SAS Institute Inc, in the U.S. and other countries