Acquisition and Management of Market Data for SAS Risk Dimensions

Transcription

1 Acquisition and Management of Market Data for SAS Risk Dimensions Shankar Yaddanapudi, SAS Consultant, Washington DC ABSTRACT SAS Risk Dimensions is an Enterprise Risk Management solution to help financial institutions manage their risk. To implement a market risk solution, SAS Risk Dimensions requires two major inputs, portfolio data and market data. The acquisition and management of market data presents a significant challenge in a typical SAS Risk Dimensions implementation. Availability of accurate, up to date and relevant market data is essential for a successful implementation of any market risk system. The market data is used to price various instruments in the portfolio, and then risk measures like Value-at-Risk can be computed using either a Monte Carlo approach or a historical simulation approach. This paper describes a SAS based interface built to acquire required market data from Bloomberg. This interface allows a batch job to request and download market data automatically from the Bloomberg server every business day. This paper also discusses transformations performed on the raw market data, to create data structures that could be used by SAS Risk Dimensions engine in its computations. Finally some recommendations are made to help implement a cost-effective, robust and extensible application. INTRODUCTION SAS has a whole range of products under the umbrella of Enterprise Risk Management covering Credit, Market and Operational risks. SAS Risk Dimensions is a major component in this platform and can be used to manage credit and market risk. To implement a market risk system it is critical to have in place a process to obtain relevant market data which is current and accurate. Financial firms go to great lengths in securing market data, Bloomberg and Reuters being two major vendors of such data. The common practice in financial industry is to build custom market data engines for trading purposes, typically based on C++. These engines have the advantage of speed. However, for risk management purposes a SAS based interface offers several advantages, like total integration from acquisition of data to computation and reporting of risk metrics, and ease of use in customizing and maintaining the application. An example of using SAS to gather market data can be found in Phil Busby (2004) 1. In the following sections we describe a spreadsheet controlled, SAS based process which requests and fetches the market data from the Bloomberg server. A brief description of portfolio files is given first, followed by a description of the mechanism to request and download data from Bloomberg. Then the transformations performed on the raw market data are described. Issues like missing market data are discussed. An attempt is made to present a high level data management picture while also discussing some implementation issues that require prior planning. PORTFOLIO DATA The portfolio files contain a description of trading activities conducted across the firm. For the purpose of this paper, the most important information contained in the portfolio data is the listing of financial instruments traded for that day. The goal here is to request current and historical market data as required by the portfolio. For example, if portfolio consists of only two equities, then the process should just request prices of those two equities. As will be seen later, the very first step then, is to parse the portfolio data, and create a list of securities for which market data is needed. BLOOMBERG PROTOCOL TO REQUEST MARKET DATA While Bloomberg provides several mechanisms to request market data, for an automated batch Risk Management system, the most convenient method is to use the FTP method. A brief description of the steps required to FTP data from Bloomberg server follows. The first step is to create a request file, which contains a description of the required market data, in a format defined by Bloomberg. These files end with an extension.req, and are uploaded to Bloomberg server by FTP. Once a.req file is placed on the Bloomberg server, Bloomberg examines the file, and if the file is in correct format, generates a.copied file. And finally Bloomberg retrieves the appropriate market data and places the data in a 1

2 .reply file. Once a.reply file appears on the Bloomberg server, the file can be downloaded for further processing. It needs to be noted that, a reply file is not always guaranteed to be generated. There might be several reasons for this, for example, incorrect format of the request file. BLOOMBERG REQUEST FILE CREATION Bloomberg has defined a set of rules for creating request files. The request file should consist of three parts: a standard header, a list of fields needed, and a list of securities for which the market data should be retrieved. The standard header contains information about user account and password, provided by Bloomberg when the account is initiated. The list of fields is essentially the market data information that is being requested, for example it could be closing price of an equity option, PX_LAST. Bloomberg provides more than a thousand fields, covering a range of financial instruments, including derivatives, bonds and equities. The approach presented here makes use of all the required input information stored in an excel spreadsheet. This information includes user account name, what kind of securities will be requested, whether historical data is being requested and so on. The advantage of this approach is that the process can be controlled by business users through Excel spreadsheet, and the code need not be modified for adding new security types. Since the request file is essentially a text file in pre-specified format, it is straightforward to use DATA _NULL_ to create a request file. A simple strategy is to use DATA _NULL_ to create a file with standard header, and then append list of fields and list of securities to this flat file, in subsequent steps. Figures 1A and 1B show sample code to generate an example request file. This code only presents a basic sketch of the code, and needs to be customized to handle various types of requests. The data from the spreadsheet if first loaded into request header data set. This information is written out to a text file. Then information about the fields being requested is appended to this flat file, followed by a list of securities. TRANSFER OF DATA FROM BLOOMBERG Once the request file is created, it needs to be uploaded to the Bloomberg server. This is a critical step and unlike other steps in the process, this step depends to a certain extent on external resources, i.e., interactions with Bloomberg server. SAS provides filename FTP engine to transfer files to a remote server. As an alternative, FTP can also be achieved by using a group file handling functions, which include fopen, flcose, fread and fget. Figure 2 demonstrates a sample code to upload a file using file handling functions. A similar approach can be used to download files (Shankar Yaddanapudi, NESUG 2009) 2. PROCESSING RAW DATA FROM BLOOMBERG Once the market data reply files have been downloaded from Bloomberg server, further processing is needed. The files are in encrypted form, and a tool provided by Bloomberg needs to be used to used to decrypt the files, and if the files contain historical data, they need to uncompressed as well. Once the file is decrypted and unzipped, then further processing is needed to extract the data from the file. The file contains all the information needed, i.e., the security names and the fields and the data values. Figures 3A and 3B present sample SAS code to achieve these tasks. RECOMMENDATIONS The design decisions made for the market data acquisition process would impact both the batch processing time, and sometimes the cost incurred as well. Usually the market data required for a typical RMS are closing prices, which would be available after market closes, which is 4PM EST in US. In the writer s experience it would be best to schedule the market data requests later than 6PM to obtain faster response from Bloomberg. In addition, it might help to: List all the securities of a particular type, in a single request file. Make sure only those items which are needed are requested. Separate out request files by whether data is being requested for a single day, or for a historical period (historical requests cost more). Avoid multiple simultaneous logins to Bloomberg server by users, especially when the automated batch job is running. Make sure security data in portfolio is clean and accurate. 2

3 Promptly respond to change notifications from Bloomberg. Another issue to consider is the file format, in which the historical market data is stored. This is an issue that needs to be decided depending on the preferences of business users, but here are some options: SAS data sets: The advantage would be faster processing time, thus saving batch time. XML files: The advantage is that XML is increasingly the format of choice across the industry. The files, however tend to larger compared, thus increasing the processing time. CSV files: The advantage with flat files is that, any person or application can read these files without specialized software. CONCLUSIONS A SAS based interface to acquire market data from Bloomberg has been presented. The approach is very flexible and allows the user to request different kinds of market data using a single interface. The interface can be used in a Risk Management System as part of an automated batch system to generate various risk reports. REFERENCES 1. Busby, Phil (2004). Automated Data Collection Using SAS and FTP. SESUG 2004, Data Presentation, Paper DP Yaddanapudi, Shankar (2009). A Secure Automated FTP Process to Transfer Data from a Remote Server. NESUG 2009, Coder s Corner, Paper CC03. ACKNOWLEDGMENTS Thanks to Ron Holanek, CEO of The Financial Risk Group, Inc ( ), for providing many of the underlying ideas and the design framework. Author would also like to thank Jaroslav Havlicek at SAS Institute, for providing valuable suggestions. DISCLAIMER The contents of this paper do not represent the opinions, recommendations, or practices of SAS Institute. CONTACT INFORMATION Please contact the author if you have any questions or comments: Shankar Yaddanapudi Paradigm Infotech Columbia, MD shankar.stat@gmail.com 3

4 Fig 1A. Sample Code to create a Bloomberg Request File /* create header section for request file */ data _null_; file "C:\NESUG2010\infiles\testrequest.req"; set request_header; /* Bloomberg standard header statements */ put 'START-OF-FILE'; put 'FIRMNAME=' firmname; put 'FILETYPE=pc' ; put 'REPLYFILENAME=testrequest.reply'; put 'RUNDATE= '; /* secmaster stands for security master database--this contains */ /* end of day pricing data */ put 'SECMASTER=' secmaster; /* yellowkey option tells Bloomberg what database needs to be searched */ /* it can be Index, Corp, Mkt etc..it is case sensitive */ /* this matches with function keys present in standard Bloomberg terminal */ if missing(yellowkey) ne 1 then do; put 'YELLOWKEY=' yellowkey; put 'DERIVED=' derived; /* outputformat will be bulklist for certain kinds of data being requested like call schedules */ put 'outputformat=' outputformat; /* PROGRAMFLAG indicates frequency of request--for MRS this is always oneshot */ put 'PROGRAMFLAG=oneshot'; /* If historical data is requested then add DATERANGE to request file */ daterange= 'DATERANGE='!!"&HISTORY_DATE_START"!!' '!!"&HISTORY_DATE_END"; put daterange; put 'HIST_FORMAT=horizontal'; /* there are two majorbloomberg programs: getdata and gethistory */ /* gethistory is used for historical requests */ put 'PROGRAMNAME=gethistory'; 4

5 Fig 1B. Sample Code to create a Bloomberg Request File /* create fields section and list all unique fields to be requested */ data _null_; file " C:\NESUG2010\infiles\ testrequest.req" mod; set request_fields end=eof; if _n_=1 then put 'START-OF-FIELDS'; put field_nm; if eof then put 'END-OF-FIELDS'; /* append list of securities to request file */ data _null_; file " C:\NESUG2010\infiles\ testrequest.req " mod; length secid1 $120 SECID $120; set request_symbol_list end=eof; by request_id; if first.request_id then put 'START-OF-DATA'; temp=strip(security_id!!" "!!SECURITY_ID2!!" "!!SECURITY_ID3); SECID1=compbl(strip(strip(temp)!!' '!!compress(put(request_start_date1,yymmdd10.), '-/')!!' '!!compress(put(request_end_date1,yymmdd10.),'-/'))); put SECID1; /* add closing statements to request file */ if last.request_id then do; put 'END-OF-DATA'; put 'END-OF-FILE'; 5

6 Fig 2. Code to demonstrate downloading a file from a remote server %let outdir=c:\nesug\code; %let repfile=testfile.txt; %let ntries = 4; %macro getfile(); %local iter1 fsize; %let iter1=0; %let fsize=0; %do %while (&iter1 le &ntries and &fsize ne 1); filename outfile "&outdir.\&repfile"; %let iter1=%eval(&iter1+1); data _null_; length rec $ 32767; rec = ''; rc1 = filename( "fhandle", "&repfile", "ftp", "host='&server1'" " cd='/'" " user='&username' pass='&password'" ); file_id = fopen('fhandle','i'); if file_id ne 0 then check=fread(file_id); else check=1; file outfile; if file_id ne 0 then do; do while (check eq 0); rc3 = fget(file_id,rec,32767); if rc3 eq 0 then do; reclen = length(rec); put rec $varying. reclen; check=fread(file_id); rc4 = fclose(file_id); if rc3 eq 0 then call symput('fsize',put(1,best.)); % filename outfile clear; %mend getfile; 6

7 Fig 3A. Sample Code to Process Reply File from Bloomberg data work.replyfile; length line $32767; infile " C:\NESUG2010\infiles\testrequest.clear" missover pad lrecl=32767; input line $32767.; data reply1(keep=bb_symbol date charvalue field_nm); length date bb_symbol $ 32; set work.replyfile end=eof; retain dataflag 0; retain fieldflag 0; retain nempty 0; length field_nm $ 32; retain field_nm; retain maxfield 0; /* number of data fields */ length field1-field&maxfield $ 32; array field[&maxfield] field1-field&maxfield; retain field1-field&maxfield; length charvalue $ 32; if line =: 'START-OF-FIELDS' then do; fieldflag = 1; delete; if line =: 'END-OF-FIELDS' then do; fieldflag = 0; delete; if line =: 'START-OF-DATA' then do; dataflag = 1; delete; if line =: 'END-OF-DATA' then do; dataflag = 0; delete; if fieldflag then do; /* load the field name array */ maxfield = maxfield + 1; field[maxfield] = trim(line); 7

8 Fig 3B. Sample Code to Process Reply File from Bloomberg if dataflag then do; bb_symbol= scan((scan(line,1,' ')),1,' '); security_rc =input(scan(line,2,' '),5.); /* Third value is the count of the number of fields requested. */ fieldcount = input(scan(line,3,' '),5.); date = scan(line,4,' '); if fieldcount=. then fieldcount=0; do fieldnum=1 to fieldcount; charvalue =scan(line,4+fieldnum,' '); if charvalue in ('N.A.','N.S.','N.A') then charvalue=''; field_nm = field[fieldnum]; output; 8