ENTERPRISE GUIDE 4.3 ( SAS 9.2 TS2M3)

Transcription

1 ENTERPRISE GUIDE 4.3 ( SAS 9.2 TS2M3) Implementing an Automated Analytic Process Edmonton SAS User Group May 11, 2011 John Kirtz Snowy Owl Software jk@snowyowlsoftware.com SAS is the registered trademark of SAS Institute Inc, North Carolina, USA 1

2 INTRODUCTION Presentation Objectives Encourage traditional SAS programmers to explore enterprise guide 4.3 Real Programmers use the SAS editor! Come over to the dark side, succumb to the power of 4.3 Why I actually started to use 4.3 Who is using EG 4.3? Who is aware of new features in EG 4.3? Demo EG can be used for more than developing SAS programs Why I should have started to use EG earlier No need to be a systems or SAS expert Daily / weekly data updates Analysis projects that need to be repeated 2

3 INTRODUCTION Presentation Outline Introduction 1. Review EG and SAS configuration used 2. Some features of Enterprise Guide 4.3 Come over to the dark side 3. Implement a sample automated analytic process using EG 4.3 Scenario Data processing ( hook in external SAS programs) Analysis / Report Output products ( EG Tasks no code) Product Delivery ( EG Report and objects no code) Automate (EG Schedule process no code) 4. Closing Remarks 3

4 SAS And Enterprise Guide Installation Software Used For Presentation Windows 7 Ultimate SAS 9.2 M3 Installation Foundation :Base / Graph / QC Enterprise Guide 4.3 default configuration installed with SAS Foundation Strictly using features easily available to SAS programmers No need for IT support No SAS BI Software No Stored Processes Many more capabilities with SAS BI installed Access to metadata Centralized data / projects Roles based processing Full use of Stored Processes 4

5 Some Features Of Enterprise Guide 4.3 Intellisense Come over to the dark side Very efficient for slow typist and bad speller now use much longer more descriptive datasets names Allows easy exploration of all possible choices New formats, informats, functions always being added New procs and procs I had forgotten about Hard to remember options and syntax for procs I was familiar with Examples Break old habits Found I always tended to use what I already knew Use favourite colours as opposed to checking available Allowed me to update my coding elements More powerful influence than it seems on the surface Just show what is available No attempt to detail how to access 5

6 Some Features Of Enterprise Guide 4.3 Integrated Syntax Help Come over to the dark side 6

7 Some Features Of Enterprise Guide 4.3 Syntax Suggestion and Auto-complete Come over to the dark side Available Functions and Context Based Parameter Help 7

8 Some Features Of Enterprise Guide 4.3 Come over to the dark side Available Libraries and Data Members Parentheses Matching 8

9 Some Features Of Enterprise Guide 4.3 Come over to the dark side Macro Variables and Macro Functions 9

10 Some Features Of Enterprise Guide 4.3 Come over to the dark side Colour Selection Format Selection 10

11 Some Features Of Enterprise Guide 4.3 Come over to the dark side Other Useful Features Finding Errors In Log Ctrl+E : next error Ctrl+Shift+E : previous error Analyse Program Execution Detailed Report Procedure and DATA steps File and data sources Process flow representation so-so results in this release Clean Up Program Formatting Problems see below 11

12 Some Features Of Enterprise Guide 4.3 Come over to the dark side Format Program Code (before) 12

13 Some Features Of Enterprise Guide 4.3 Come over to the dark side Format Program Code (after) 13

14 Scenario Snowy Owl Feed produces hog feed on five production lines at their Fort Saskatchewan Plant. Decided to test implementing NIR based QA/QC on one of the production lines Machine was installed on a line and set-up to test a sample every 3 minutes At the end of each hour the machine outputs a batch jdx file containing scan and constituent data The data in this file is to be used to generate distinct reports to NIR research and Product QA/QC dept If test is successful then NIR machines will be installed on all five lines. 14

15 Process Flow Product Sampled ( 3 min) QA/QC Data Marts Extract Report Data Const Listing Shew Hart EG Report QA/QC EG NIR Machine Batch JDX File ( 1 hour) Batch Dataset Upload Data Constituent Dataset Scan Dataset EG Analytic Tasks Scan Plots Scan CV EG Report NIR EG Import Data File Problem with Process Support Data Acquisition Analysis and Reports Report Distribution Exception Handling 15

16 SAS Tasks Create an Enterprise Guide project Develop SAS program to read JDX data into SAS datasets Batch, Constituent Concentration, Nir Scan data External program so it can be used elsewhere Develop SAS program to load batch datasets into historical SAS data marts Code separate as may switch to relational database later ( blob structure for NIR data) External program so it can be used elsewhere Develop SAS program to extract report data from historical data marts Code separate as may switch to relational database later External program so it can be used elsewhere Implement required analysis reports Use analytic EG tasks Implement required report distribution Use EG objects Separate task for each dept Test the process manually Automate the project Use project schedule object 16

17 Create Project : Add Autoexec Process 17

18 Data Acquisition : Data File Structure Jcamp-Dx Format ( see handout) Documented at ASCII Divided into sections 1. File Header contains batch-file info For each Sample there is a distinct section 2. Sample header Identify sample Scan parameters 3. Sample Constituent Data Predicted from scan Equations for constituents can be purchased or developed internally 4. NIR Scan Data value for each wavelength captured by the machine array of values defined by sample header 18

19 Data Acquisition : Information to be Captured 1. File Header ##TITLE= UltraHog1A ##JCAMP-DX= 4.24 ##DATA TYPE= LINK ##BLOCKS= 50 ##ORIGIN= Unity InfoStar Converter ##OWNER= FortSaskLineA3 ##DATE= 05/05/2011 ##TIME= 12:00:00 2. Sample Header ##TITLE= UltraHog1A ##JCAMP-DX= 4.24 ##DATA TYPE= NEAR INFRARED SPECTRUM ##ORIGIN= Unity InfoStar Converter ##OWNER= FortSaskLineA3 ##DATE= 05/05/2011 ##TIME= 12:03:00 ##SAMPLE DESCRIPTION=FortSaskLineA3 ##SPECTROMETER/DATA SYSTEM= UNITY ##XUNITS= NANOMETERS ##YUNITS= ABSORBANCE ##XFACTOR= 1 ##YFACTOR= ##FIRSTX= 680 ##LASTX= Sample Constituent ##CONCENTRATIONS= (NCU) (<Protein >, ,?) (<Starch >, ,?) (<Energy >, ,?) (<DEC >, ,?) (<Lysine >, ,?) (<TotalFiber >, ,?) 4. Scan NIR Data ##DELTAX= 1 ##XYDATA= (X++(Y..Y)) ##END= 19

20 Data Acquisition : Data Capture and Storage Develop programs Link in execution order 20

21 Analysis and Reports : Extract Report Data Develop program to extract required data from data marts Link to UpdateDatabase program to insure correct execution order 21

22 Analysis and Reports : Extract Report Data Execute HogFeedQAQC process flow to create analysis datasets ( needed to develop analytic tasks) 22

23 Analysis and Reports : Batch Constituent Listing Select ConstituentWideX dataset, click on Data List task 23

24 Analysis and Reports : Constituent Shewhart Charts Select ConstituentX dataset, click on Individual Measurements Chart task 24

25 Analysis and Reports : NIR Scan Plots Select NIRWAVELENGHTX dataset, click on Line Plot task 25

26 Analysis and Reports : NIR Wavelength Statistics Select NIRWAVELENGHTX dataset, click on Summary Table task 26

27 Analysis and Reports : Analysis Tasks Implemented 27

28 Analysis and Reports : QA/QC Dept Report Using Report Object 28

29 Analysis and Reports : NIR Dept Report Using Report Object 29

30 Report Distribution : Create PDF Version of QA/QC Report Use Export Option of QA/QC Report menu to create a PDF file 30

31 Report Distribution : Create PDF Version of NIR Report Use Export Option of NIR Report menu to create a PDF file 31

32 Report Distribution : Ready to Implement s 32

33 Report Distribution : Administration 33

34 Report Distribution : QA/QC Report Use Send To Option of QAQC Depart Report.pdf object Recipient as step in project 34

35 Project : Complete Use Export Option of NIR Report menu to create a PDF file 35

36 Project : Schedule 36

37 Closing Remarks Not Perfect (Yet)! SAS putting a lot of effort into EG still a few bugs but hot fixes are on the way. No task or method to shut down automated EG before it has completed the process no equivalent to ENDSAS statement need to set up a global status macro to conditionally execute steps no date for a fix for this An automated project that contains a conditional task may hang requires a Task Manager shutdown hot fix on the way in May (2011) Icon in process flow window indicates if there was an error but process cannot access this information SAS is aware of this and are planning to remedy it but no date available Problems with: DDE X statement and systask (possible permissions issue) SAS statements that do not work well in a SAS batch program Interactive environments such as the Report window SAS/AF Applications %WINDOW Statements DATA step debugger 37

38 Closing Remarks Not Perfect (Yet)! Add-ins Export to Excel works if you do not have Access to pc file formats Access to ODS Graphics Designer allow you run the designer from within an EG session Custom Add-ins can develop your own using Visual Basic or C# 38

39 MANAGING A PROCESS WITH E.G. 4.3 John Kirtz Snowy Owl Software jk@snowyowlsoftware.com The End! 39