Providing Users with Access to the SAS Data Warehouse: A Discussion of Three Methods Employed and Supported

Transcription

1 Providing Users with Access to the SAS Data Warehouse: A Discussion of Three Methods Employed and Supported Cynthia A. Stetz, Merrill Lynch, Plainsboro, NJ Abstract A Data Warehouse is stored in SAS datasets on a server running a UNIX operating system. The user community is heterogeneous, containing power SAS System users, casual SAS System users, and non-sas users. Supporting them all at their own comfort level requires the deployment of different access methods. Fortunately, with the SAS system, we have a variety of tools available for data access. This paper will discuss three access methods we have in use and are actively supporting, the pros and cons of each, and how best you can help your users choose the best method for their needs (which may mean choosing one for them!). We will explore the use of XVision connectivity software, (which allows the user to log on to the UNIX environment as if they were sitting at the server console), SAS/CONNECT from a PC/SAS session to the UNIX server, and ODBC (Open Database Connectivity) access to the SAS datasets, which allows any ODBC compliant enduser tool (such as Microsoft Query ) to tap into the SAS data store. Environment The Marketing Systems Department of the Merrill Lynch Private Client Market Planning Group manages a 600 gigabyte data warehouse, located on a Sun Sparc 2000 workstation running the Solaris 2.5 operating system. The workstation (nicknamed GEKKO) is connected to the Windows NT network via the TCP/IP protocol. Desktop PCs run the Windows NT operating system, version 4.0. The SAS System for UNIX, version 6.12, is installed on the Sparc 2000, and the PCs have access to the SAS System for Windows, version 6.12, which is installed on a network server. See figure 1 for a graphical representation of the physical environment. The primary data source for the warehouse is an MVS DB2 database, from which 15 gigabytes of data are extracted each month. Additional data flows into the warehouse from various sources, including Excel Spreadsheets and Microsoft Access databases located on the Windows NT Network, and external data loaded directly onto the Sparc 2000 via tape or compact disk. These additional data sources may be refreshed monthly, quarterly, or annually, as available and/or required. The current data joins historical data to form a rich data store much in demand by information analysts throughout Merrill Lynch Private Client, both within the Market Planning Group and from several other departments. Access to the data warehouse is provided via one of three methods we have chosen to deploy in our environment, based on the diversity of our users needs. Some users may have more than one of these access methods installed, and they will chose one depending on the nature of the task they wish to perform. The portability of the SAS language over multiple computer platforms allows them the flexibility to work in whichever environment best suits their immediate requirements. Environment File Server Figure 1 Windows Clients User Community Token Ring Gekko MPG Web Server Data Center Router MIDAS Token Ring The access methods will be presented in the context of user type for which each is best suited. WAN 1

2 Here, the author will take a moment to be a little preachy - know thy users. Since there is no need for any data warehouse unless there are people who find it useful and who are comfortable using it, providing excellent user support must be one of the primary goals of the data warehouse support staff. Determining into which category a user will fit may be a difficult task. We have had the most success by listening to the users, asking them quantitative questions about the kinds of analysis they plan to do, frequency of use, and current SAS experience level. We then provide continuing user support in the form of monthly data warehouse user group meetings, data and software bulletins, tips and techniques documentation (which we publish on our internal WEB site, to maximize their availability), and by offering our services for one-to-one assistance. The last is especially useful for users who are transitioning from the batch mainframe environment to an interactive environment, and for PC/SAS users encountering the idiosyncrasies of the UNIX environment for the first time. A little handson help goes a long way to increase users confidence with the new tools they now have at their disposal, which invariably leads to increased use of the data warehouse. And, that s the ultimate goal, now, isn t it? Supporting SAS Power Users : XVision connectivity software The first type of user we define is the power user. These are users that expect to access the data warehouse on a more or less daily basis, and will be performing intensive analysis on large amounts of data. For these users we install the XVision connectivity software on their PC, which allows them to logon to a virtual UNIX workstation. This software provides a window management interface to the SAS system, which appears quite similar to the SAS Display Manager System for the Personal Computer. Therefore the transition from the PC/SAS environment to the UNIX/SAS environment is a fairly easy one. These users are also ones that are not averse to learning the limited number of UNIX commands which will allow them to make the most of this environment. Common tasks they need to accomplish are removing program files (since there is no capability for this from within a SAS session), creating or removing directories under their home directory, and submitting and managing background processes. In order to give the user this access method, we install the XVision software on their PC, create a UNIX userid and password for them within the SUN environment, provide them with a directory on the SUN machine to contain their profile files, and populate that directory with the standard version of the autoexec.sas file. This autoexec.sas file contains a %INCLUDE statement referencing a file that contains the LIBNAME commands for all of the directories that are part of the production data warehouse. In this way, we can control which libraries and library names the user sees as being part of the warehouse. When new data becomes available, we have only to add it to the one file referenced by all of the autoexec.sas %INCLUDE statements to change the users view of the warehouse. This simplifies our support effort, while ensuring a consistent view of the warehouse for all users. We also include a sasautos specification in the users autoexec.sas file, to give them autocall access to in-house written macros. See Appendix A for samples of these files. To make this method more user friendly, we create a shortcut icon on their PC desktop that does all of the initialization tasks necessary for the XVision session, such as designating the host system to which to connect, logging them on, and starting the interactive SAS session in the Open Look Window Manager. This setup eliminates the need for them to know either their userid or password for the UNIX session. Advantages to the users of the UNIX workstation emulation method include the ability to submit multiple processes, or programs, for batch processing, while continuing to work in the interactive SAS session. This is accomplished by opening a command tool window in the Open Look Window Manager, and either executing the SAS program in the foreground (which will tie up that window for the duration of the job), or in the background, through use of a UNIX script file. In this way, users can maximize their productivity. We supply sample UNIX script files to get them started, an example of which you will find in Appendix B. One disadvantage to using this method is the requirement that the user become comfortable in the UNIX environment. As mentioned before, there are some tasks that cannot be controlled from within the 2

3 SAS session, and require at least a rudimentary knowledge of UNIX commands. One way we have tried to help in this regard is by creating cheat sheets, if you will, of often used UNIX commands, which we distribute to the users. Another disadvantage of this method is that there is no direct way to share data the user may have on his or her PC with the UNIX session. The data must be transferred to the UNIX server, using the file transfer capability that is built into the XVision product. Also, the UNIX session can only print to printers that have an IP address, and of which it is aware. In our environment, that is not much of an issue, since we have several printers distributed throughout the department, all accessible from the UNIX server. But, there are users who have printers connected directly to their PCs what they wish to use, and for them, and others who prefer to work in the PC/SAS world, we offer the second type of data warehouse access: SAS/CONNECT. An Alternative for PC/SAS users: SAS/CONNECT and Remote Library Services For the user who needs access to the data warehouse, but would rather work in the SAS System for the PC environment, we provide SAS/CONNECT capabilities. Using this access method, from an open PC/SAS session, the user signs onto their userid on the UNIX server and runs a SAS session there. In order to do that, they need to identify the UNIX server to the SAS/CONNECT system using the IP address, specify the connection protocol they are using (which in our case is TCP ), and have a script file on their PC which mimics the events that the user would go through when telnet ing to the UNIX host. We create the UNIX userid, password and autoexec.sas file on the UNIX server, the script file on their PC, and supply them with code containing the parameters for the SAS/CONNECT signon command. The SAS/CONNECT product includes sample script files for all of the supported protocols, which can be found in the saslink subdirectory. After we have it set up, all the user needs to know is what code to execute when he or she wants to connect to the data warehouse. If desired, they can add it into their PC/SAS autoexec.sas file. See Appendix C for a sample of this code. Once connected, users can submit code (using the RSUBMIT command) to be run on the UNIX server. This allows them to take advantage of the more powerful computing platform, and receive the results back on their PC, while still maintaining their code locally. Any procedure output comes back to their PC output window, and can be printed locally. They can easily transfer SAS data sets and flat files to and from the remote session using the UPLOAD and DOWNLOAD procedures of SAS/CONNECT, which allows them to combine warehouse data with local data. They can also browse the remote data warehouse by using Remote Library Services, which is done by assigning a SAS LIBNAME statement in the local SAS session to data residing on the remote UNIX system. They can also, inadvertently, create a great deal of network traffic by improper use of Remote Library Services (RLS). Once the LIBNAME is assigned to the remote data, the local session sees that data as it does local data, and can execute programs directly on the remote data. If the code submitted requests large amounts of data, that data will be moved across your network to the user s PC, clogging up your network and filling up the user s disk space. Therefore it is imperative that users understand how SAS/CONNECT remote processing and RLS work, and for which tasks each is appropriate. The best way to ensure that they do is for the support staff to proactively offer information on maximizing the efficient use of these tools. NON-SAS Users and the SAS Data Warehouse: Open Database Connectivity(ODBC) access The SAS ODBC driver for the SAS system allows ODBC compliant Windows applications, such as Microsoft Query, or Andyne s GQL, open access to SAS datasets. Since our data resides remotely from the local machines, the users use the SAS ODBC Driver to connect to a SAS/SHARE server that has the SAS/SHARE*NET software installed. The SAS/SHARE server runs on a UNIX userid dedicated for this purpose, and can be managed from any other UNIX SAS session using the OPERATE procedure. This procedure allows the support staff to check the status of the server, see who is using it, and allocate or release SAS libraries as needed. If the server needs to be brought down for 3

4 some reason, the QUIESCE procedure command can be used to gradually terminate the resource, by not allowing any additional users to access it while letting work in progress complete. This is the preferred way to stop a server, to minimize impact on the users. When starting the SAS/SHARE server, we assign it a name that is then associated with the SAS ODBC driver on the users PCs. The SAS/SHARE server must be re-started any time the UNIX host system is re-booted. Appendix D contains an example of how a server is started, and a PROC OPERATE sample also. The SAS ODBC driver, which is available in both 16 and 32 bit versions, comes with the base SAS software. We install it on the user s PC, and, through the ODBC Administrator, identify the desired SAS/SHARE server (since multiple SAS/SHARE servers can exist in the environment). The user then opens the desired application, and can access the SAS remote data warehouse as any other ODBC source. Warehouse metadata is stored as SAS datasets in the data warehouse, which makes it accessible to the ODBC user as to any other user. An advantage of this method is that access is seamless, which is important for this user type. However, none of the processing abilities of the more powerful UNIX environment are available to the ODBC user, which is a disadvantage of this access method. Summary The SAS data warehouse can be a powerful tool in the hands of the users. In order to help users make the most of its and their capabilities, it is the responsibility of the warehouse support staff to supply user access methods that suit the users. There are many variables to consider when deciding what modes of access should be supported in each systems environment. It may be tempting to try to apply a one-size-fits-all approach to the access method question, to minimize support requirements. But, unless the user population is homogeneous and static, more than one access method will most likely be required. Keeping in mind the relative strengths and weaknesses of the various methods available, the desires of the user community, and the abilities of the support staff, the data warehouse manager can deploy the right mix of products in her or his environment. Trademarks SAS, SAS/CONNECT, SAS/SHARE, and SAS/SHARE*NET are registered trademarks or trademarks of the SAS Institute Inc. in the USA and other countries. Other brand and product names are registered trademarks or trademarks of their respective companies. References SAS Institute Inc. (1995), Client/Server Computing with the SAS System Tips & Techniques, Cary NC: SAS Institute Inc. SAS Institute Inc. (1995), SAS Companion for the UNIX Environment and Derivatives, Cary NC: SAS Institute Inc. SAS Institute Inc. (1993), SAS/CONNECT software: Remote Library Services, Companion for the UNIX Environment and Derivatives, Cary NC: SAS Institute Inc. Author Contact Information Cynthia A. Stetz Merrill Lynch Private Client Market Planning 800 Scudders Mill Rd. Section 3H Plainsboro, NJ cynthia_stetz@ml.com 4

5 Appendix A /*** XVision users autoexec.sas file ***/ %include /datawarehouse/libraries.sas ; options sasautos = ( /datawarehouse/macros,!sasroot/sasautos ); run; /***************************************/ /*** /datawarehouse/libraries.sas ***/ /*** warehouse metadata ***/ libname metadata /datawarehouse/metadata/ ; /*** directory number one of the warehouse ***/ libname direct01 /datawarehouse/directory01/ ; /*** directory number two, containing large files, 2gig+ ***/ libname direct02 /datawarehouse/directory02/ type=partition partsize=3.0g; /***************************************/ Appendix B # sample UNIX script file for executing a SAS program in the background # please note the use of the pound sign for UNIX script comments # first, delete the log and list files from the last execution of this program rm /homelib/myprogram.log rm /homelib/myprogram.lst # execute sas, giving the program name /apps/sas /homelib/myprogram.sas -autoexec /homelib/alternativeautoexec.sas # print the log and list files on the default printer lp /homelib/myprogram.log lp /homelib/myprogram.lst Appendix C /*** sample of code used to SAS/CONNECT to a remote data warehouse ***/ /*** from a PC/SAS session, and enable remote library services ***/ /* specify location of the connection script, the IP address of the server, */ /* and the connection protocol to be used */ filename rlink c:\saslib\tcpunix.scr ; %let server1= ; options comamid=tcp remote=server1; /* sign on to the server and initialize SAS session there */ signon; 5

6 /* assign data warehouse libraries for the remote session */ rsubmit; /*** warehouse metadata ***/ libname metadata /datawarehouse/metadata/ ; /*** directory number one of the warehouse ***/ libname direct01 /datawarehouse/directory01/ ; /*** directory number two, containing large files, 2gig+ ***/ libname direct02 /datawarehouse/directory02/ type=partition partsize=3.0g; endrsubmit; /* assign remote libraries to libnames on local session, using the remote engine */ /*** warehouse metadata ***/ libname metadata remote /datawarehouse/metadata/ server=server1; /*** directory number one of the warehouse ***/ libname direct01 remote /datawarehouse/directory01/ server=server1; /*** directory number two, containing large files, 2gig+ ***/ libname direct02 remote /datawarehouse/directory02/ type=partition partsize=3.0g server=server1; Appendix D /** sample code used to start a SAS/SHARE server **/ /** this code is executed from a userid dedicated for this purpose **/ /** allocate the production libraries **/ %include /datawarehouse/libraries.sas ; /** start the server, giving it a name that will be used **/ /** in the PC ODBC connection **/ proc server serverid=sasodbc1 log=all msgn dtf=datetime22.2; run; /****************************************************/ /** proc operate can be used from any UNIX SAS session **/ /** (except the one actually running the server) **/ proc operate serverid=sasodbc1; display server; display l _all_; display u _all_; run; 6