Exporting Access Databases with Indexes and Keys Roger S. Cohen, New York State Dept of Tax and Finance, Albany, New York

Transcription

1 Exporting Access Databases with Indexes and Keys Roger S. Cohen, New York State Dept of Tax and Finance, Albany, New York ABSTRACT SAS Access for Microsoft Office Applications allows SAS programs to import and export tables to Microsoft Access MDB files. SAS does not currently provide a means for exporting tables that contain keys or indexed fields. This paper explains hot to create a standalone Visual Basic application that may be used within SAS to provide this capacity. PURPOSE Microsoft Access is a very popular database application. It has many wonderful features for creating forms, reports and queries and displaying data. SAS is a wonderful system for accessing data in many forms, manipulating it complex ways, producing analysis and creating specialized datasets. We have produced extremely capable applications by marrying the capabilities of both programs. In the New York State Tax Department, all (or virtually all) of our employees who use computers have Microsoft Office, including Microsoft Access installed on their computers. SAS is installed on only about twenty. We have found it very useful to create applications for groups of users that use a Microsoft Access front end, but are populated with data created or updated nightly by a SAS application. We use Microsoft s SQL Server to house the data for our larger and more mature projects, but because of institutional and other reasons it still makes sense for us to develop many of our tools using basic Access MDB files and tables. Our applications update automatically. We use Windows scheduler to initiate our SAS applications each night or early morning after the processing system files they read have updated. In the final stages of the nightly process our SAS programs export files to Access MDB files so that they are ready for users when they arrive in the morning. One problem that we had to overcome in the development of these processes was how to create Access tables that were properly indexed. SAS, Version 9 does not have any capability that we were able to find that would allow it to export an indexed MDB table. Instead, we created a standalone Visual Basic 6 application complied to an executable EXE file and called with all the necessary information it needed passed in the command line. Then, within any SAS program, we could include one or more X statements following the PROC EXPORT to call this executable and add whatever indexing or keys that were needed. METHOD Below is the Visual Basic, ver. 6 code used to create the executable file. Attribute VB_Name = "Module1" Public Sub Main() indexfunc End Sub Public Function indexfunc() As String On Error GoTo cmdindexfunc_err Dim dbs As Database Dim tbl As TableDef Dim idxvbl As Index Dim cmdstr As String 1

2 cmdstr = Command() If cmdstr = "" Or InStr(cmdstr, "?") > 0 Then MsgBox "INDEXGEN <Database Name>,<Table Name>,<Variable Name>" GoTo cmdindexfunc_exit End If cpos = InStr(cmdstr, ",") dbstr = Mid(cmdstr, 1, cpos - 1) cmdstr = Mid(cmdstr, cpos + 1) cpos = InStr(cmdstr, ",") tblstr = Mid(cmdstr, 1, cpos - 1) cmdstr = Mid(cmdstr, cpos + 1) vblstr = cmdstr dbstr = Replace(dbstr, """", "") tblstr = Replace(tblstr, """", "") vblstr = Replace(vblstr, """", "") 'cmdstr = Mid(cmdstr, cpos + 1) 'MsgBox "Database: " & dbstr & vbcrlf & _ "Table: " & tblstr & vbcrlf & _ "Variable: " & vblstr Set dbs = OpenDatabase(dbstr) Set tbl = dbs.tabledefs(tblstr) 2

3 Set idxvbl = tbl.createindex(vblstr & "Index") idxvbl.fields.append idxvbl.createfield(vblstr) tbl.indexes.append idxvbl 'SysCmd acsyscmdsetstatus, " " 'indexfunc = " " GoTo cmdindexfunc_exit cmdindexfunc_err: If Not (Err = 3284) Then If Not (Err = 3262) Then If Not (Err = 0) Then MsgBox MsgBox(Error() & Err) 'Ignore already indexed error 'MsgBox Err.Number & Err.Description cmdindexfunc_exit: End Function The program is compile to INDEXGEN.EXE and the syntax for invoking it is the following: INDEXGEN <Database Name>,<Table Name>,<Variable Name> This program may be run from Windows Command Prompt Window, from within a BAT file, from Run on the Windows Start Menu or within a SAS program by using the X statement. Below is an example of how a data table may be exported: Example: Exporting an Indexed MDB File from SAS: DATA MakeMeAnMDBtbl; /* Contains variable IndexMe1 and IndexMe2 which need to be indexed */ run; PROC EXPORT DATA = MakeMeAnMDBtbl OUTTABLE= NowAnMDBtlb DBMS= access replace; DATABASE= c:\path\myaccessapp.mdb ; OPTIONS NOXWAIT XSYNC; X C:\bin\INDEXGEN c:\path\myaccessapp.mdb, NowAnMDBtlb, IndexMe1; X C:\bin\INDEXGEN c:\path\myaccessapp.mdb, NowAnMDBtlb, IndexMe2; In example 1, we are primarily interested in the dataset, FileA. We merged with FileB and are surprised to find that Test has more records than FileA, although we used the in= option. In this simplest example (By group of 1 variable), we look for records with duplicate values for Ind_1. We find that TestB is not empty. After examination of TestB, we add a second indicator variable in order to uniquely define a record and run GetDup a second time. This time TestC is empty but we may not be done. Upon examination of TestB, we decide Ind_2 does not serve any useful purpose. The highest value of Ind_2 is an appropriate value to use. We replace the empty TestC with the appropriate filter of TestB. Example Use #2: By group of multiple variables %GetDup (fromfile=est2004, id=ind_1 Ind_2 Ind_3, tofile=dup2004); 3

4 The three variables, Ind_1 Ind_2 Ind_3, were expected to uniquely identify records in the dataset, est2004, which has over 1 million records. Executing %GetDup results in dataset, dup2004, of 1000 records. What the heck? Sometimes it s easier to examine the dataset in MS Excel. PROC EXPORT DATA = Dup2004 OUTFILE = "A:\Dev\Estimates " DBMS=EXCEL2000 REPLACE; SHEET="Dup2004"; run; Perhaps including Address to the previous By group will differentiate each record. %GetDup (fromfile=est2004, id= Ind_1 Ind_2 Ind_3 Address, tofile=dup2004b); Dup2004b has the same 1000 records. No help. Replace Address with Name. %GetDup (fromfile=est2004, id= Ind_1 Ind_2 Ind_3 Name, tofile=dup2004c); This time Dup2004c is empty. In fact, Ind_1 Ind_2 Ind_3 Name, is a By group that uniquely identifies records. Example Use #3: DATA Jun; SET Jun02 Jun15; run; The datasets Jun02 and Jun15 are the result of processes that should be independent and should produce unique records By Id Batch. Let s test the appending of these two datasets. %GetDup (fromfile=jun, id=id Batch, tofile=chk);* no duplicates; Good, Chk is an empty dataset. Example Use #4: A source file of tax returns were identified as having general exceptions. More detail was needed, which required JOINing the source file with a database of specific tax returns. Base was the result of that JOIN. Variables 1-2 provided an index of unique records from the source file, and we examine Base using GetDup. %GetDup(fromfile=base, id=var1 var2, tofile=testdup); We find that Testdup is not empty. Initially, we thought the duplicates resulted from two inconsequential variables, Date1 and Ind1 from the return dataset. No problem. Simply delete them from a Test dataset, then sort it using the nodup option. DATA test;set testdup;drop Date1 Ind1; PROC SORT DATA=test NODUP;BY var1 var2;run; In fact, Test was still not empty. Further analysis revealed that a subset of the source returns had gone through multiple stages of review, which were delineated by Date1 and Ind1. Those records had different values for system calculated variables, which was data that we wanted to keep. Consequently, we should retain Date1 and Ind1 and recognize that the records in Testdup are duplicates over the subset of (Var1, Var2), but their small differences are significant. CONCLUSION The problems and challenges of duplicate records often go well beyond just eliminating redundant records. The determinant of what constitutes a duplicate or potential duplicate may be based on a subset of variables or relevant differentiators. Once determined, the duplicates may require special processing or examination 4

5 rather than simple purging. This paper has presented a simple, general purpose method for identifying duplicate records based upon a selected set of variables. CONTACT INFORMATION Roger S. Cohen, Ph.D. Director of Operations Research and Analysis New York State Department of Taxation and Finance Harriman State Campus, Bldg 9, Room 928 Albany, NY Phone: (518) Fax: (518) roger_cohen@tax.state.ny.us Kenneth Gold Assistant Director of Operations Research and Analysis New York State Department of Taxation and Finance Harriman State Campus, Bldg 9, Room 928 Albany, NY Phone: (518) Fax: (518) Kenneth_gold@tax.state.ny.us SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. Other brand and product names are trademarks of their respective companies. 5