Electricity Forecasting Full Circle

Similar documents
Integrating Large Datasets from Multiple Sources Calgary SAS Users Group (CSUG)

How to use UNIX commands in SAS code to read SAS logs

A Macro that can Search and Replace String in your SAS Programs

Useful Tips When Deploying SAS Code in a Production Environment

Top Coding Tips. Neil Merchant Technical Specialist - SAS

A SAS Macro Utility to Modify and Validate RTF Outputs for Regional Analyses Jagan Mohan Achi, PPD, Austin, TX Joshua N. Winters, PPD, Rochester, NY

Using an ICPSR set-up file to create a SAS dataset

EXAMPLE 3: MATCHING DATA FROM RESPONDENTS AT 2 OR MORE WAVES (LONG FORMAT)

A Macro to Create Program Inventory for Analysis Data Reviewer s Guide Xianhua (Allen) Zeng, PAREXEL International, Shanghai, China

Your Own SAS Macros Are as Powerful as You Are Ingenious

Shawn Dorward, MVP. Getting Started with Power Query

The Output Bundle: A Solution for a Fully Documented Program Run

Implementing external file processing with no record delimiter via a metadata-driven approach

Automatically Configure SDTM Specifications Using SAS and VBA

Using DDE with Microsoft Excel and SAS to Collect Data from Hundreds of Users

Cramped for Drive Space? Save Space with the Auto-Compress Macro

Matt Downs and Heidi Christ-Schmidt Statistics Collaborative, Inc., Washington, D.C.

Formula Training (Intermediate)

A SAS Macro for Producing Benchmarks for Interpreting School Effect Sizes

Shawn Dorward, MVP. Getting Started with Power Query

Data Transfer from Microsoft Access to SAS Made Easy

Reading in Data Directly from Microsoft Word Questionnaire Forms

Data Edit-checks Integration using ODS Tagset Niraj J. Pandya, Element Technologies Inc., NJ Vinodh Paida, Impressive Systems Inc.

Workbooks (File) and Worksheet Handling

Use That SAP to Write Your Code Sandra Minjoe, Genentech, Inc., South San Francisco, CA

SAS Drug Development Program Portability

Missing Pages Report. David Gray, PPD, Austin, TX Zhuo Chen, PPD, Austin, TX

Purchase this book at

SAS 101. Based on Learning SAS by Example: A Programmer s Guide Chapter 21, 22, & 23. By Tasha Chapman, Oregon Health Authority

Using a Control Dataset to Manage Production Compiled Macro Library Curtis E. Reid, Bureau of Labor Statistics, Washington, DC

The Demystification of a Great Deal of Files

A SAS Macro to Create Validation Summary of Dataset Report

OASUS Spring 2014 Questions and Answers

Utilizing SAS for Cross- Report Verification in a Clinical Trials Setting

Principles of Automation

Quick and Efficient Way to Check the Transferred Data Divyaja Padamati, Eliassen Group Inc., North Carolina.

BUSINESS ANALYTICS. 96 HOURS Practical Learning. DexLab Certified. Training Module. Gurgaon (Head Office)

Paper CC16. William E Benjamin Jr, Owl Computer Consultancy LLC, Phoenix, AZ

Internet, Intranets, and The Web

Automate Secure Transfers with SAS and PSFTP

Macros are a block of code that can be executed/called on demand

TIPS AND TRICKS: IMPROVE EFFICIENCY TO YOUR SAS PROGRAMMING

Using GSUBMIT command to customize the interface in SAS Xin Wang, Fountain Medical Technology Co., ltd, Nanjing, China

Using Maps with the JSON LIBNAME Engine in SAS Andrew Gannon, The Financial Risk Group, Cary NC

SeUGI 19 - Florence WEB Enabling SAS output. Author : Darryl Lawrence

Dictionary.coumns is your friend while appending or moving data

Using SAS to Control the Post Processing of Microsoft Documents Nat Wooding, J. Sargeant Reynolds Community College, Richmond, VA

Program Validation: Logging the Log

A Macro To Generate a Study Report Hany Aboutaleb, Biogen Idec, Cambridge, MA

Base and Advance SAS

A simplistic approach to Grid Computing Edmonton SAS Users Group. April 5, 2016 Bill Benson, Enterprise Data Scienc ATB Financial

SAS System Powers Web Measurement Solution at U S WEST

Spotfire Advanced Data Services. Lunch & Learn Tuesday, 21 November 2017

Data Quality Review for Missing Values and Outliers

ABSTRACT INTRODUCTION MACRO. Paper RF

Check Please: An Automated Approach to Log Checking

Abstract. Background. Summary of method. Using SAS to determine file and space usage in UNIX. Title: Mike Montgomery [MIS Manager, MTN (South Africa)]

Biostatistics & SAS programming. Kevin Zhang

Efficiently Join a SAS Data Set with External Database Tables

ET01. LIBNAME libref <engine-name> <physical-file-name> <libname-options>; <SAS Code> LIBNAME libref CLEAR;

CC13 An Automatic Process to Compare Files. Simon Lin, Merck & Co., Inc., Rahway, NJ Huei-Ling Chen, Merck & Co., Inc., Rahway, NJ

SAS (Statistical Analysis Software/System)

Techniques for Writing Robust SAS Macros. Martin Gregory. PhUSE Annual Conference, Oct 2009, Basel

work.test temp.test sasuser.test test

SAS (Statistical Analysis Software/System)

5348 Vegas Drive Las Vegas, NV 89108, U.S.A. Tel: ; Fax: Website:

Taming a Spreadsheet Importation Monster

Better Metadata Through SAS II: %SYSFUNC, PROC DATASETS, and Dictionary Tables

Contents of SAS Programming Techniques

Give me EVERYTHING! A macro to combine the CONTENTS procedure output and formats. Lynn Mullins, PPD, Cincinnati, Ohio

Microsoft Access Vba Copy Table Structure Only

A Practical Introduction to SAS Data Integration Studio

INTRODUCTION TO SAS HOW SAS WORKS READING RAW DATA INTO SAS

Quicker Than Merge? Kirby Cossey, Texas State Auditor s Office, Austin, Texas

Validation Summary using SYSINFO

Developing Data-Driven SAS Programs Using Proc Contents

While You Were Sleeping, SAS Was Hard At Work Andrea Wainwright-Zimmerman, Capital One Financial, Inc., Richmond, VA

So Much Data, So Little Time: Splitting Datasets For More Efficient Run Times and Meeting FDA Submission Guidelines

Importing CSV Data to All Character Variables Arthur L. Carpenter California Occidental Consultants, Anchorage, AK

ABSTRACT INTRODUCTION TRICK 1: CHOOSE THE BEST METHOD TO CREATE MACRO VARIABLES

Best Practice for Creation and Maintenance of a SAS Infrastructure

Creating databases using SQL Server Management Studio Express

AURA ACADEMY SAS TRAINING. Opposite Hanuman Temple, Srinivasa Nagar East, Ameerpet,Hyderabad Page 1

Autodesk Vault and Data Management Questions and Answers

Create Metadata Documentation using ExcelXP

Chapter 2: Getting Data Into SAS

Leave Your Bad Code Behind: 50 Ways to Make Your SAS Code Execute More Efficiently.

MedDRA Dictionary: Reporting Version Updates Using SAS and Excel

Someone Changed My SAS Visual Analytics Report! How an Automated Version Control Process Can Rescue Your Report and Save Your Sanity

Data Science Services Dirk Engfer Page 1 of 5

Text Generational Data Sets (Text GDS)

Reporting Template. By Denis Fafard Business Analyst WCB - Alberta

Getting the Most from Hash Objects. Bharath Gowda

SAS Macro Dynamics - From Simple Basics to Powerful Invocations Rick Andrews, Office of the Actuary, CMS, Baltimore, MD

Techno Expert Solutions An institute for specialized studies!

Certkiller.A QA

Procedure for Stamping Source File Information on SAS Output Elizabeth Molloy & Breda O'Connor, ICON Clinical Research

Fall 2012 OASUS Questions and Answers

How a Code-Checking Algorithm Can Prevent Errors

AN INTRODUCTION TO MACRO VARIABLES AND MACRO PROGRAMS Mike Zdeb, School of Public Health

Transcription:

Electricity Forecasting Full Circle o Database Creation o Libname Functionality with Excel o VBA Interfacing Allows analysts to develop procedural prototypes By: Kyle Carmichael

Disclaimer The entire presentation and associated views are solely that of the analyst and not of ATCO Power. Permission has been granted by ATCO Power for the analyst to share some examples from his work at ATCO Power. Forward-Looking Information: o Certain statements contained in this presentation may constitute forward-looking information. Forward-looking information is often, but not always, identified by the use of words such as "anticipate", "plan", "estimate", "expect", "may", "will", "intend", "should", and similar expressions. Forward-looking information involves known and unknown risks, uncertainties and other factors that may cause actual results or events to differ materially from those anticipated in such forward-looking information. The Corporation believes that the expectations reflected in the forward-looking information are reasonable, but no assurance can be given that these expectations will prove to be correct and such forward-looking information should not be unduly relied upon. Any forward-looking information contained in this presentation represents the Corporation's expectations as of the date hereof, and is subject to change after such date. The Corporation disclaims any intention or obligation to update or revise any forward-looking information whether as a result of new information, future events or otherwise, except as required by applicable securities legislation.

Agenda o o o Database Creation o Looping Macro (Based on Time) o Import engine (XML) o DOS commands (X Commands) Program Integration o Libname functionality (With Excel) o SAS Add-in (Controlling with VBA) Example Procedure

The Database Looping Macro %macro database_loop; %let starttime = %sysfunc(date()); /*Define Start (Today)*/ %let endtime = %sysevalf(&start + 7);/*Define End (Today+7 Days) %do %until( &time > %sysevalf(&endtime)); /* Start of Macro*/ %let time = %sysfunc(date()); /* Current time %XML_IMPORT_MACRO data _null_ ; Call sleep(5,1); Run; %end; /* Sleep for 30 seconds*/ /*End of Macro*/

1. Determine Filenames %macro get_filenames(location); filename pipedir pipe "dir ""%unquote(&location)"" /b " lrecl=32767; data filenames; infile pipedir truncover; input files $char1000.; run; %mend; %get_filenames(location); 2. Count the Number Files proc sql noprint; select count(files) into: count from filenames; quit; 3. Record the Filenames as macro variables proc sql; select files into :File1 - :File&count from filenames; quit;

4. Use a SAS engine to import the data (XML, Proc Import, ect.) %do i=1 %to &count; libname in xml (location\&&file&i); data scrape&i; set in.anything /*Only the in is important, the name does not matter*/ run; 5. Merge the different data files into a file in a permanent library %if the first file %then %do Create initial permanent data table (ex: sasuser.input) %else %do Append to the created table (proc append) 6. Use X Command to archive and clear xml files and proc datasets to clear work library X Copy location\&&file&i New_Location\NewName X Del /f /s /q location\&&file&i proc datasets lib=work; delete scrape&i; run;

Data Is Now In Database

The Libname Statement 1. Pull in filtering parameters from an excel workbook Filename parms excel location\parameters.xls Data parms; Set parms. Sheet1$ n; Run; 2. Convert the parameters to macro variables Proc sql; select year into: year from parms; run; Proc sql; select month into: month from parms; run; Proc sql; select model into: model from parms; run;

3. Filter the data and run analysis histogram, distribution, means, etcetera Data filtered; Set sasuser.input (where=(year = &year month=&month model=&model)); Run; proc means data=filtered; by year month model; output out = mean_output mean= max= p75= \autoname; run; 4. Send the data to an output excel file libname output excel location\model_output.xls Proc datasets lib=output; Delete exceloutput; Run; Data output.exceloutput Set mean_output; Run; Libname output clear;

Data Is Now In Excel

VBA Interfacing o VBA can control SAS Stored Processes Dim SAS As Object Set SAS =Application.COMAddIns.Item("SAS.OfficeAddin.Loader.ConnectProxy").Object SAS.Refresh Analysis_Stored_Process o o Only the name of the stored process has to be used In Visual Basic Editor you may have to go to Tools References to update library for SAS VBA SAS Function to see if connection is working Dim SAS As Object Set SAS =Application.COMAddIns.Item("SAS.OfficeAddin.Loader.ConnectProxy").Object SAS.HelloWorld

The Control Panel & Process

Open or email Excel with VBA Excel Control Panel Start Software Program with VBA Transfer Status Information Send Query Information Analysis Stored Process XML Stored Process Send Data to preformatted excel with Libname functionality Import XML files and clean directory

Questions & Contact Information Kyle Carmichael Business Analyst ATCO Power Phone:(403) 245-9972 E-mail: kyle.carmichael@atcopower.com