Please don't Merge without By!!

Size: px
Start display at page:

Download "Please don't Merge without By!!"

Transcription

1 ABSTRACT Please don't Merge without By!! Monal Kohli Have you ever merged datasets and forgotten a by Statement, looked at the results and thought wow % match but when you started validating the results they were all jumbled up. The culprit here is the BY" statement. It is a really powerful statement and its placement can be critical in achieving correct merge results. In this paper we will discuss the MERGENOBY SAS option and other common errors that can during a merge. INTRODUCTION Merging datasets is a very common routine in any kind of data analysis and is often overlooked as too-simple-to - mess-up. In reality, a single merge statement incorporates some of the very key concepts in SAS programming such as implied RETAIN, by variable processing and temporary variables in Program Data Vector. In this paper, we will visit all the key concepts around merging datasets and validating the results of the merge. We will look at the Option MERGENOBY which serves as a watchdog, monitoring any errors occurring due to missed BY statement in the Merge. The results obtained when we miss a BY statement in a MERGE are analogous to a ONE-ONE merge in many ways. In this paper the following concepts will be addressed: One-One merges Temporary Variables, Validating Match Merge results and how to use MERGENOBY system option to overcome merging errors in SAS datastep. Let s start our journey by taking a close look at One-One merges. MERGING DATASETS WITHOUT A BY STATEMENT INTENTIONALLY ONE-ONE MERGE A good example is the best sermon. Let s take an example where we merge two datasets without an explicit BY statement. In the example below, we are trying to append School and graduation dates to the students in dataset setone using the information from dataset settwo. These two datasets are merged without a BY statement because in this particular case we are just appending information and not really merging by a key variable. However there are few key things to note in the below example: 1. First, the number of observations in the final dataset is equal to the number of observations in the dataset with largest number of observations. 2. If there is a common variable in the two datasets, the value is overwritten by the value in the right dataset. SAS Code I: In the below code we merge two datasets setone and settwo. Both the datasets have different set of variables and the code tries to combine two datasets without a common key. When SAS combines two datasets without a common variable, the PDV is initialized with the descriptor portion of both the datasets, in this case ( studentid, cohort, Schoolname and GraduationDate) and the first record from setone is combined with the first record from settwo, second with the second and so on until there are no observations left in the second dataset. data setone input studentid 8. cohort $1. put studentid= cohort= cards A B A B C B A data settwo input SchoolName $3. GraduationDate $9. put SchoolName= GraduationDate= cards proc print data=setone title "Setone Contents" proc print data=settwo title "Settwo Contents" data combinedone_two merge setone settwo put studentid= cohort= schoolname= graduationdate= _error_= _n_= 1

2 Figure I - Log for above SAS Code I 178 data setone 179 input studentid 8. cohort $ put studentid= cohort= 181 cards studentid= cohort=a studentid= cohort=b studentid= cohort=a studentid= cohort=b studentid= cohort=c studentid= cohort=b studentid= cohort=a NOTE: The data set WORK.SETONE has 7 observations and 2 variables data settwo 192 input SchoolName $3. GraduationDate $ put SchoolName= GraduationDate= 194 cards NOTE: The data set WORK.SETTWO has 8 observations and 2 variables proc print data=setone 206 title "Setone Contents" 207 NOTE: There were 7 observations read from the data set WORK.SETONE. NOTE: PROCEDURE PRINT used (Total process time): 208 proc print data=settwo 209 title "Settwo Contents" 210 NOTE: There were 8 observations read from the data set WORK.SETTWO. NOTE: PROCEDURE PRINT used (Total process time): 211 data combinedone_two 212 merge setone settwo 2

3 Cont.. Figure I - Log for above SAS Code I 213 put studentid= cohort= schoolname= graduationdate= date= _error_= _n_= 214 studentid= cohort=a ort=a _ERROR_=0 _N_=1 studentid= cohort=b _ERROR_=0 _N_=2 studentid= cohort=a _ERROR_=0 _N_=3 studentid= cohort=b SchoolName=NYU olname=nyu GraduationDate=Fall2010 _ERROR_=0 _N_=4 studentid= cohort=c _ERROR_=0 _N_=5 studentid= cohort=b _ERROR_=0 _N_=6 studentid= cohort=a _ERROR_=0 _N_=7 studentid=. cohort= _ERROR_=0 _N_=8 NOTE: There were 7 observations read from the data set WORK.SETONE. NOTE: There were 8 observations read from the data set WORK.SETTWO. NOTE: The data set WORK.COMBINEDONE_TWO has 8 observations and 4 variables. When a merge statement is specified without a BY statement, the internal key used for matching is the _N_ temporary variable. With every iteration in the PDV, SAS matches _N_ from the first dataset to _N_ of the second dataset. In the absence of respective _N_ (missing record) from the second dataset, the values from the last iteration retained in the PDV are outputted as the final value. In this case, we have seven observations in the first dataset and eight observations in the second dataset with the last observation coming from the second dataset. This was an example with no common variable in the two datasets. Let s analyze an example with a common variable between the two datasets. In the below example, we have a common variable Cohort in the two datasets. However we only want to output the matches i.e. for every record in dataset setone there is a corresponding record in settwo. To accomplish this we use the concept of temporary variables available in the Program Data Vector. The IN= datastep option creates a variable that indicates whether the dataset contributed to the current observation. The value of the IN variable is only available during the life of dataset but not outputted in the final dataset. To output the IN variable to the final dataset, we need to assign the value to another variable. Let s look at an example of how to retain the IN variables in the current dataset. To output all the variables in PDV, use _ALL_. SAS Code II: In the below code we merge two datasets setone and settwo. Both the datasets have different set of variables and the code tries to combine two datasets without a common key. We use the SAS dataset option IN= in the merge statement. The IN variables a and b are temporary variables and are only available as long as there are observations in settwo dataset. The below code shows the value in PDV before and after the merge. When the last observation is read from settwo dataset, there is no respective observation in dataset setone, hence the value of a=0 and b=1. In the below code the temporary variables a and b are not outputted to dataset combinedone_two. data combinedone_two put '' put _all_ merge setone(in=a) settwo(in=b) put '' put _all_ 3

4 Figure II - Log for above SAS Code II 421 data combinedone_two 422 put '' 423 put _all_ 424 merge setone(in=a) settwo(in=b) 425 put '' 426 put _all_ 427 ****** * DATA IN PDV BEFORE MERGE *********** a=0 b=0 studentid=. cohort= SchoolName= GraduationDate= _ERROR_=0 _N_=1 a=1 b=1 studentid= cohort=a _ERROR_=0 _N_=1 a=1 b=1 studentid= cohort=a _ERROR_=0 _N_=2 a=1 b=1 studentid= cohort=b _ERROR_=0 _N_=2 a=1 b=1 studentid= cohort=b _ERROR_=0 _N_=3 a=1 b=1 studentid= cohort=a _ERROR_=0 ROR_=0 _N_=3 a=1 b=1 studentid= cohort=a _ERROR_=0 _N_=4 a=1 b=1 studentid= cohort=b all2010 _ERROR_=0 _N_=4 a=1 b=1 studentid= cohort=b _ERROR_=0 _N_=5 a=1 b=1 studentid= cohort=c ationdate=fall2010 _ERROR_=0 _N_=5 a=1 b=1 studentid= cohort=c _ERROR_=0 _N_=6 a=1 b=1 studentid= cohort=b _ERROR_=0 _N_=6 a=1 b=1 studentid= cohort=b _ERROR_=0 _N_=7 a=1 b=1 studentid= cohort=a _ERROR_=0 _N_=7 a=1 b=1 studentid= cohort=a _ERROR_=0 _N_=8 a=0 b=1 studentid=. cohort= _ERROR_=0 _N_=8 a=0 b=1 studentid=. cohort= _ERROR_=0 _N_=9 NOTE: There were 7 observations read from the data set WORK.SETONE. NOTE: There were 8 observations read from the data set WORK.SETTWO. NOTE: The data set WORK.COMBINEDONE_TWO has 8 observations and 4 variables. 4

5 MERGING DATASETS WITH A BY STATEMENT COMMON ERRORS When there is a common variable in two datasets and we want to combine the variables from both of the datasets using the common variable as the key, we use a MERGE with a BY statement. Let s look at some common errors and their explanation. 1. When the BY Statement is the Culprit Sorting before Merging Does this error message look familiar: ERROR: BY variables are not properly sorted? Match merging by a common variable involves a merge statement followed by a BY statement. The By statement tells SAS that data is sorted by the BY variable in both of the datasets. So let s consider a case when the data is left unsorted. In the below example, we are merging two datasets setone and settwo by studentid. In the first merge, we missed sorting the two datasets. The log shows the values in the PDV and errors caused due to the nonsorted BY group variable. The match merge produces an error message and stops executing. SAS Code III In the below code, we are merging the datasets setone and settwo by a common variable studentid. The main purpose of this example is to get an understanding of the data through the PDV changes from before vs. after the merge. In the below match merge datastep, SAS reads the values in the By group in ascending order. In the first iteration, studentid= is read into PDV and there is no matching value in the settwo dataset, hence the second value in the by group is read. The second value is , there is a match for this value in the settwo datatset but the first value in the dataset is , and not Therefore, it is essential to sort the datasets before using a BY group in a match merge. Figure III is a snapshot of the log before the data is sorted. Figure IV is the snapshot of the log after the data was sorted. SETONE studentid cohort A B Figure IIIA - Log for above SAS Code III SETTWO School Graduation studentid Name Date NYU Fall NYU Fall2012 data merge1 put '****** DATA IN PDV BEFORE MERGE ***********' put _all_ merge setone settwo by studentid put '****** DATA IN PDV AFTER MERGE ***********' put _all_ studentid= cohort= SchoolName= GraduationDate= FIRST.studentid=1 LAST.studentid=1 _ERROR_=0 _N_=1 studentid= cohort=a SchoolName= GraduationDate= FIRST.studentid=1 LAST.studentid=1 _ERROR_=0 _N_=1 studentid= cohort=a SchoolName= GraduationDate= FIRST.studentid=1 LAST.studentid=1 _ERROR_=0 _N_=2 ERROR: BY variables are not properly sorted on data set WORK.SETTWO. studentid= cohort=b SchoolName=NYU GraduationDate= Date=Fall2010 FIRST.studentid=1 LAST.studentid=1 _ERROR_=1 _N_=2 NOTE: The SAS System stopped processing this step because b of errors. NOTE: There were 2 observations read from the data set WORK.SETONE. NOTE: There were 2 observations read from the data set WORK.SETTWO. WARNING: The data set WORK.MERGE1 may be incomplete. When this step was stopped there were 1 observations and 4 variables. WARNING: Data set WORK.MERGE1 was not replaced because b this step was stopped. 5

6 Figure IIIB - Log for above SAS Code after sorting by studentid studentid= cohort=a SchoolName= GraduationDate= FIRST.studentid=1 LAST.studentid=1 _ERROR_=0 _N_=1 studentid= cohort=a SchoolName= GraduationDate= FIRST.studentid=1 LAST.studentid=1 _ERROR_=0 _N_=2 studentid= cohort= SchoolName=NYU GraduationDate=Fall2011 FIRST.studentid=1 LAST.studentid=1 _ERROR_=0 _N_=2 ****** DATA IN PDV BEFORE B MERGE *********** studentid= cohort= SchoolName=NYU GraduationDate=Fall2011 FIRST.studentid=1 LAST.studentid=1 _ERROR_=0 _N_=3 studentid= cohort=b FIRST.studentid=1 LAST.studentid=0 _ERROR_=0 _N_=3 studentid= cohort=b FIRST.studentid=1 LAST.studentid=0 _ERROR_=0 _N_=4 studentid= cohort=b SchoolName=NYU GraduationDate=Fall2012 FIRST.studentid=0 LAST.studentid=1 _ERROR_=0 _N_=4 studentid= cohort=b SchoolName=NYU GraduationDate=Fall2012 FIRST.studentid=0 LAST.studentid=1 _ERROR_=0 N_=5 NOTE: There were 2 observations read from the data set WORK.SETONE. NOTE: There were 3 observations read from the data set WORK.SETTWO. NOTE: The data set WORK.MERGE1 has 4 observations o and 4 variables. 2. When Length is the Culprit Variable lengths in the BY Variable Does this warning message look familiar WARNING: Multiple lengths were specified for the BY variable first by input data sets? This often happens when we try to merge datasets with BY variables having different lengths. The length of the BY variable used during matching is determined by the first dataset in the merge. Caution must be taken when specifying the datasets in the merge statement when the lengths of the variables are different. Let s look at an example. In the example below, datasets One and Two are merged by first and last names, but in this case the lengths of the two variables are different and the merge produces a dataset with zero records. SAS Code IV In the SAS Code below, the length of the first and last variables in the dataset one is 8, and the length of these two variables in dataset two is 10.The length of the first and last variables in dataset three is 8 (and not 10). The length of the variable in the output dataset is dictated by the length in the dataset first defined in the merge statement. In this case the length in the first dataset in the merge statement is 8, and the length of first and last variables in dataset three is 8. ONE--length first last $ First Last Nancy Grace James Smith TWO-- length first last $ School Graduation First Last Name Date Nancy Grace James Smith NYU Fall2011 data three merge one(in=one) two(in=two) by last first if one and two proc print data=three 6

7 Figure IV - Log for above SAS Code IV WARNING: Multiple lengths were specified for the BY variable last by input data sets. This may cause unexpected results. WARNING: Multiple lengths were specified for the BY variable first by input data sets. This may cause unexpected results. NOTE: There were 2 observations read from the data set WORK.ONE. NOTE: There were 2 observations read from the data set WORK.TWO. NOTE: The data set WORK.THREE has 2 observations and 4 variables. The dataset three looks like this: School Graduation first last Name Date Nancy Grace James Smith NYU Fall2011 What happens if we swap the positions of the datasets in the merge statement? The warning message in the log disappears because the length of the variable is longer in dataset two and SAS sets the length of the two variables as 8. So even if the length in dataset one is 6, there is no concern of data being truncated or invalid results being produced. Figure V - Log for above SAS Code with dataset positions switched data three merge two(in=two) one(in=one) by last first if one and two NOTE: There were 2 observations read from the data set WORK.TWO. NOTE: There were 2 observations read from the data set WORK.ONE. NOTE: The data set WORK.THREE has 2 observations and 4 variables. 3. When Common Variables are the culprit MSGLEVEL OPTION There are instances when we mistakenly merge two datasets with common variables and after reviewing the log we figure out that the values from the second dataset overwrote the values in the first dataset. This would come to our attention only if we printed the records in the log. Otherwise, it could slip through the cracks. Lets look at an example and visit a solution to resolve this problem if we missed displaying the data in the log. SAS Code V In the SAS code below, dataset one is being merged with dataset two and they share a common variable cohort1. During the merge, the values of cohort1 in dataset one are replaced by values in dataset two. The two datasets are being merged by the common variable studentid. 7

8 ONE studentid cohort1 cohort A D B E TWO studentid cohort1 cohort C F D H data three merge one(in=a) two(in=b) by studentid if a and b proc print data=three THREE studentid cohort1 cohort2 cohort C D F D E H How to detect this problem? SAS provides an option MSGLEVEL =I which writes a warning to the SAS log whenever a MERGE statement would cause variables to be overwritten. Let s try using this option before the merge: SASCODE VI options msglevel=i data three merge one(in=a) two(in=b) by studentid if a and b Figure VI - Log for above SAS Code with MSGLEVEL options msglevel=i data three merge one(in=a) two(in=b) by studentid if a and b INFO: The variable cohort1 on data set WORK.ONE will be overwritten by data set WORK.TWO. NOTE: There were 2 observations read from the data set WORK.ONE. NOTE: There were 2 observations read from the data set WORK.TWO. NOTE: The data set WORK.THREE has 2 observations and 4 variables. The Final Savior the Mergenoby OPTION Finally, we consider the cure for all missing BY accidents. SAS provides a system option called MERGENOBY which provides three ways to detect a problem in a match merge with a missing BY statement. Let s take a peek at all three settings. 1. MERGENOBY= NOWARN The value NOWARN for option MERGENOBY does not write any warnings in the log when a BY statement is missing in the merge. Below is the code for NOWARN. 8

9 SASCODE VII data one $ cards A B data two $ cards C D H options mergenoby=nowarn data three merge one(in=a) two(in=b) if a and b proc print data=three Below is the log for above match merge Three. There is no warning or error issued in the log to let the user know that a BY statement was missed in creating dataset three. In dataset three, there are only two observations and basically the merge worked as a One-to-One merge. Figure VII - Log for above SAS Code for NOWARN Setting options mergenoby=nowarn data three merge one(in=a) two(in=b) if a and b NOTE: There were 2 observations read from the data set WORK.ONE. NOTE: There were 3 observations read from the data set WORK.TWO. NOTE: The data set WORK.THREE has 2 observations and 2 variables. DATASET THREE studentid cohort C D 2. MERGENOBY= WARN In this iteration we will change the value of the option Mergenoby to WARN and watch the impact. The value WARN enables SAS to write a warning in the log when a BY statement is missing in the merge datastep. data one $ cards A B data two $ cards C D H options mergenoby=warn data three merge one(in=a) two(in=b) if a and b proc print data=three SASCODE VIII In the below code, we changed the value of mergenoby option to WARN. In the below code, we changed the value of the mergenoby option to WARN. Below is the log for above from match merge Three. There is a warning in the log to let the user know that a BY statement was missed in creating dataset three. In dataset three, there are only two observations and basically the merge worked as a One-on-One merge. 9

10 Figure VIII - Log for above SAS Code for WARN Setting options mergenoby=warn data three merge one(in=a) two(in=b) if a and b WARNING: No BY statement was specified for a MERGE statement. NOTE: There were 2 observations read from the data set WORK.ONE. NOTE: There were 3 observations read from the data set WORK.TWO. NOTE: The data set WORK.THREE has 2 observations and 2 variables. DATASET THREE studentid cohort C D 3. MERGENOBY= ERROR In the below code, we changed the value of option mergenoby to ERROR. data one $ cards A B data two $ cards C D H options mergenoby=error data three merge one(in=a) two(in=b) if a and b proc print data=three In this iteration we will change the value of the option Mergenoby to ERROR and watch the impact. SASCODE IX Figure IX - Log for above SAS Code for ERROR Setting ERROR: No BY statement was specified for a MERGE statement. NOTE: The SAS System stopped processing this t step because b of errors. WARNING: The data set WORK.THREE may be incomplete. When this step was stopped there were 0 observations and 2 variables. WARNING: Data set WORK.THREE was not replaced because this step was stopped. NOTE: DATA statement t used (Total process time): 10

11 In the above log for match merge three, there is an error in the log which shows that a BY statement was missed in creating dataset three. This warning prevents a merge with incorrect values from being executed. It is always a good idea to have a basic understanding of the data in the datasets used in a merge. If there are duplicates, the results may be unreliable. CONCLUSIONS We looked at various intricacies involved in a simple match merge and how an absence of BY statement can affect the results. SAS provides options like MSGLEVEL and MERGENOBY to detect and validate the results produced in a match merge. It is always a good idea to sum up the matches and nonmatches from the datasets in the match merge to validate the results in a match merge datastep. Additionally we will review some other variations of the match merge errors and solutions during my presentation in NESUG REFERENCES How merge really works, Bob Virgile, Nesug 1999 Preventing Data Loss when Combining SAS Datasets, John Cantrell, Pharmasug Set, Match, Merge Don t You Love SAS, Peter Eberhardt, Sesug 2007 Base SAS Procedures Guide, Second Edition, Volumes 1-4, SAS Institute, Inc., March, 2006,Cary, NC ACKNOWLEDGMENTS Many thanks to Hans Kroger and John Cohen. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. Other brand and product names are registered trademarks or trademarks of their respective companies. CONTACT INFORMATION Your comments and questions are valued and encouraged. Please contact the author at: Monal Kohli NewYork, NY monal.kohli@gmail.com * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * 11

Countdown of the Top 10 Ways to Merge Data David Franklin, Independent Consultant, Litchfield, NH

Countdown of the Top 10 Ways to Merge Data David Franklin, Independent Consultant, Litchfield, NH PharmaSUG2010 - Paper TU06 Countdown of the Top 10 Ways to Merge Data David Franklin, Independent Consultant, Litchfield, NH ABSTRACT Joining or merging data is one of the fundamental actions carried out

More information

Checking for Duplicates Wendi L. Wright

Checking for Duplicates Wendi L. Wright Checking for Duplicates Wendi L. Wright ABSTRACT This introductory level paper demonstrates a quick way to find duplicates in a dataset (with both simple and complex keys). It discusses what to do when

More information

Beginner Beware: Hidden Hazards in SAS Coding

Beginner Beware: Hidden Hazards in SAS Coding ABSTRACT SESUG Paper 111-2017 Beginner Beware: Hidden Hazards in SAS Coding Alissa Wise, South Carolina Department of Education New SAS programmers rely on errors, warnings, and notes to discover coding

More information

Merging Data Eight Different Ways

Merging Data Eight Different Ways Paper 197-2009 Merging Data Eight Different Ways David Franklin, Independent Consultant, New Hampshire, USA ABSTRACT Merging data is a fundamental function carried out when manipulating data to bring it

More information

9 Ways to Join Two Datasets David Franklin, Independent Consultant, New Hampshire, USA

9 Ways to Join Two Datasets David Franklin, Independent Consultant, New Hampshire, USA 9 Ways to Join Two Datasets David Franklin, Independent Consultant, New Hampshire, USA ABSTRACT Joining or merging data is one of the fundamental actions carried out when manipulating data to bring it

More information

Anatomy of a Merge Gone Wrong James Lew, Compu-Stat Consulting, Scarborough, ON, Canada Joshua Horstman, Nested Loop Consulting, Indianapolis, IN, USA

Anatomy of a Merge Gone Wrong James Lew, Compu-Stat Consulting, Scarborough, ON, Canada Joshua Horstman, Nested Loop Consulting, Indianapolis, IN, USA ABSTRACT PharmaSUG 2013 - Paper TF22 Anatomy of a Merge Gone Wrong James Lew, Compu-Stat Consulting, Scarborough, ON, Canada Joshua Horstman, Nested Loop Consulting, Indianapolis, IN, USA The merge is

More information

Not Just Merge - Complex Derivation Made Easy by Hash Object

Not Just Merge - Complex Derivation Made Easy by Hash Object ABSTRACT PharmaSUG 2015 - Paper BB18 Not Just Merge - Complex Derivation Made Easy by Hash Object Lu Zhang, PPD, Beijing, China Hash object is known as a data look-up technique widely used in data steps

More information

Cleaning up your SAS log: Note Messages

Cleaning up your SAS log: Note Messages Paper 9541-2016 Cleaning up your SAS log: Note Messages ABSTRACT Jennifer Srivastava, Quintiles Transnational Corporation, Durham, NC As a SAS programmer, you probably spend some of your time reading and

More information

Merge with Caution: How to Avoid Common Problems when Combining SAS Datasets

Merge with Caution: How to Avoid Common Problems when Combining SAS Datasets MWSUG 2017 Paper SA06 Merge with Caution: How to Avoid Common Problems when Combining SAS Datasets Joshua M. Horstman, Nested Loop Consulting, Indianapolis, IN ABSTRACT Although merging is one of the most

More information

Locking SAS Data Objects

Locking SAS Data Objects 59 CHAPTER 5 Locking SAS Data Objects Introduction 59 Audience 60 About the SAS Data Hierarchy and Locking 60 The SAS Data Hierarchy 60 How SAS Data Objects Are Accessed and Used 61 Types of Locks 62 Locking

More information

Essentials of PDV: Directing the Aim to Understanding the DATA Step! Arthur Xuejun Li, City of Hope National Medical Center, Duarte, CA

Essentials of PDV: Directing the Aim to Understanding the DATA Step! Arthur Xuejun Li, City of Hope National Medical Center, Duarte, CA PharmaSUG 2013 - Paper TF17 Essentials of PDV: Directing the Aim to Understanding the DATA Step! Arthur Xuejun Li, City of Hope National Medical Center, Duarte, CA ABSTRACT Beginning programmers often

More information

Merge with Caution: How to Avoid Common Problems when Combining SAS Datasets

Merge with Caution: How to Avoid Common Problems when Combining SAS Datasets Paper 1746-2018 Merge with Caution: How to Avoid Common Problems when Combining SAS Datasets Joshua M. Horstman, Nested Loop Consulting ABSTRACT Although merging is one of the most frequently performed

More information

HASH Beyond Lookups Your Code Will Never Be the Same!

HASH Beyond Lookups Your Code Will Never Be the Same! ABSTRACT SESUG Paper 68-2017 HASH Beyond Lookups Your Code Will Never Be the Same! Elizabeth Axelrod, Abt Associates Inc. If you ve ever used the hash object for table lookups, you re probably already

More information

If You Need These OBS and These VARS, Then Drop IF, and Keep WHERE Jay Iyengar, Data Systems Consultants LLC

If You Need These OBS and These VARS, Then Drop IF, and Keep WHERE Jay Iyengar, Data Systems Consultants LLC Paper 2417-2018 If You Need These OBS and These VARS, Then Drop IF, and Keep WHERE Jay Iyengar, Data Systems Consultants LLC ABSTRACT Reading data effectively in the DATA step requires knowing the implications

More information

Table Lookups: Getting Started With Proc Format

Table Lookups: Getting Started With Proc Format Table Lookups: Getting Started With Proc Format John Cohen, AstraZeneca LP, Wilmington, DE ABSTRACT Table lookups are among the coolest tricks you can add to your SAS toolkit. Unfortunately, these techniques

More information

USING SAS HASH OBJECTS TO CUT DOWN PROCESSING TIME Girish Narayandas, Optum, Eden Prairie, MN

USING SAS HASH OBJECTS TO CUT DOWN PROCESSING TIME Girish Narayandas, Optum, Eden Prairie, MN Paper RF-12-2014 USING SAS HASH OBJECTS TO CUT DOWN PROCESSING TIME Girish Narayandas, Optum, Eden Prairie, MN ABSTRACT Hash tables are in existence since SAS 9 version and are part of data step programming.

More information

Programming Beyond the Basics. Find() the power of Hash - How, Why and When to use the SAS Hash Object John Blackwell

Programming Beyond the Basics. Find() the power of Hash - How, Why and When to use the SAS Hash Object John Blackwell Find() the power of Hash - How, Why and When to use the SAS Hash Object John Blackwell ABSTRACT The SAS hash object has come of age in SAS 9.2, giving the SAS programmer the ability to quickly do things

More information

Debugging 101 Peter Knapp U.S. Department of Commerce

Debugging 101 Peter Knapp U.S. Department of Commerce Debugging 101 Peter Knapp U.S. Department of Commerce Overview The aim of this paper is to show a beginning user of SAS how to debug SAS programs. New users often review their logs only for syntax errors

More information

Andrew H. Karp Sierra Information Services, Inc. San Francisco, California USA

Andrew H. Karp Sierra Information Services, Inc. San Francisco, California USA Indexing and Compressing SAS Data Sets: How, Why, and Why Not Andrew H. Karp Sierra Information Services, Inc. San Francisco, California USA Many users of SAS System software, especially those working

More information

Basic SAS Hash Programming Techniques Applied in Our Daily Work in Clinical Trials Data Analysis

Basic SAS Hash Programming Techniques Applied in Our Daily Work in Clinical Trials Data Analysis PharmaSUG China 2018 Paper 18 Basic SAS Hash Programming Techniques Applied in Our Daily Work in Clinical Trials Data Analysis ABSTRACT Fanyu Li, MSD, Beijing, China With the development of SAS programming

More information

How MERGE Really Works

How MERGE Really Works How MERGE Really Works Bob Virgile Robert Virgile Associates, Inc. Overview Do your MERGEs produce unexpected results? Three basic DATA step concepts resolve the complexities of MERGE: compile and execute,

More information

The DATA Statement: Efficiency Techniques

The DATA Statement: Efficiency Techniques The DATA Statement: Efficiency Techniques S. David Riba, JADE Tech, Inc., Clearwater, FL ABSTRACT One of those SAS statements that everyone learns in the first day of class, the DATA statement rarely gets

More information

SAS Log Summarizer Finding What s Most Important in the SAS Log

SAS Log Summarizer Finding What s Most Important in the SAS Log Paper CC-037 SAS Log Summarizer Finding What s Most Important in the SAS Log Milorad Stojanovic RTI International Education Surveys Division RTP, North Carolina ABSTRACT Validity of SAS programs is an

More information

Tackling Unique Problems Using TWO SET Statements in ONE DATA Step. Ben Cochran, The Bedford Group, Raleigh, NC

Tackling Unique Problems Using TWO SET Statements in ONE DATA Step. Ben Cochran, The Bedford Group, Raleigh, NC MWSUG 2017 - Paper BB114 Tackling Unique Problems Using TWO SET Statements in ONE DATA Step Ben Cochran, The Bedford Group, Raleigh, NC ABSTRACT This paper illustrates solving many problems by creatively

More information

PROC FORMAT: USE OF THE CNTLIN OPTION FOR EFFICIENT PROGRAMMING

PROC FORMAT: USE OF THE CNTLIN OPTION FOR EFFICIENT PROGRAMMING PROC FORMAT: USE OF THE CNTLIN OPTION FOR EFFICIENT PROGRAMMING Karuna Nerurkar and Andrea Robertson, GMIS Inc. ABSTRACT Proc Format can be a useful tool for improving programming efficiency. This paper

More information

Step through Your DATA Step: Introducing the DATA Step Debugger in SAS Enterprise Guide

Step through Your DATA Step: Introducing the DATA Step Debugger in SAS Enterprise Guide SAS447-2017 Step through Your DATA Step: Introducing the DATA Step Debugger in SAS Enterprise Guide ABSTRACT Joe Flynn, SAS Institute Inc. Have you ever run SAS code with a DATA step and the results are

More information

Quality Control of Clinical Data Listings with Proc Compare

Quality Control of Clinical Data Listings with Proc Compare ABSTRACT Quality Control of Clinical Data Listings with Proc Compare Robert Bikwemu, Pharmapace, Inc., San Diego, CA Nicole Wallstedt, Pharmapace, Inc., San Diego, CA Checking clinical data listings with

More information

Pay Stub On-line. Systems Utilization. San Diego County Office of Education

Pay Stub On-line. Systems Utilization. San Diego County Office of Education Pay Stub On-line I. Before Logging on II. To Register III. Log In IV. Forgot Password V. Direct Deposit Pay Stubs VI. Change Email Address VII. Change Password VIII. Troubleshooting IX. Sign Out Systems

More information

One-Step Change from Baseline Calculations

One-Step Change from Baseline Calculations Paper CC08 One-Step Change from Baseline Calculations Nancy Brucken, i3 Statprobe, Ann Arbor, MI ABSTRACT Change from baseline is a common measure of safety and/or efficacy in clinical trials. The traditional

More information

How to Create Data-Driven Lists

How to Create Data-Driven Lists Paper 9540-2016 How to Create Data-Driven Lists Kate Burnett-Isaacs, Statistics Canada ABSTRACT As SAS programmers we often want our code or program logic to be driven by the data at hand, rather than

More information

Using SAS 9.4M5 and the Varchar Data Type to Manage Text Strings Exceeding 32kb

Using SAS 9.4M5 and the Varchar Data Type to Manage Text Strings Exceeding 32kb ABSTRACT Paper 2690-2018 Using SAS 9.4M5 and the Varchar Data Type to Manage Text Strings Exceeding 32kb John Schmitz, Luminare Data LLC Database systems support text fields much longer than the 32kb limit

More information

The inner workings of the datastep. By Mathieu Gaouette Videotron

The inner workings of the datastep. By Mathieu Gaouette Videotron The inner workings of the datastep By Mathieu Gaouette Videotron Plan Introduction The base The base behind the scene Control in the datastep A side by side compare with Proc SQL Introduction Most of you

More information

Journey to the center of the earth Deep understanding of SAS language processing mechanism Di Chen, SAS Beijing R&D, Beijing, China

Journey to the center of the earth Deep understanding of SAS language processing mechanism Di Chen, SAS Beijing R&D, Beijing, China Journey to the center of the earth Deep understanding of SAS language processing Di Chen, SAS Beijing R&D, Beijing, China ABSTRACT SAS is a highly flexible and extensible programming language, and a rich

More information

CMISS the SAS Function You May Have Been MISSING Mira Shapiro, Analytic Designers LLC, Bethesda, MD

CMISS the SAS Function You May Have Been MISSING Mira Shapiro, Analytic Designers LLC, Bethesda, MD ABSTRACT SESUG 2016 - RV-201 CMISS the SAS Function You May Have Been MISSING Mira Shapiro, Analytic Designers LLC, Bethesda, MD Those of us who have been using SAS for more than a few years often rely

More information

From An Introduction to SAS University Edition. Full book available for purchase here.

From An Introduction to SAS University Edition. Full book available for purchase here. From An Introduction to SAS University Edition. Full book available for purchase here. Contents List of Programs... xi About This Book... xvii About the Author... xxi Acknowledgments... xxiii Part 1: Getting

More information

Ditch the Data Memo: Using Macro Variables and Outer Union Corresponding in PROC SQL to Create Data Set Summary Tables Andrea Shane MDRC, Oakland, CA

Ditch the Data Memo: Using Macro Variables and Outer Union Corresponding in PROC SQL to Create Data Set Summary Tables Andrea Shane MDRC, Oakland, CA ABSTRACT Ditch the Data Memo: Using Macro Variables and Outer Union Corresponding in PROC SQL to Create Data Set Summary Tables Andrea Shane MDRC, Oakland, CA Data set documentation is essential to good

More information

Understanding and Applying the Logic of the DOW-Loop

Understanding and Applying the Logic of the DOW-Loop PharmaSUG 2014 Paper BB02 Understanding and Applying the Logic of the DOW-Loop Arthur Li, City of Hope National Medical Center, Duarte, CA ABSTRACT The DOW-loop is not official terminology that one can

More information

A Utility Program for Quickly Identifying LOG Error or Warning Messages

A Utility Program for Quickly Identifying LOG Error or Warning Messages A Utility Program for Quickly Identifying LOG Error or Warning Messages Zhengyi Fang and Paul Gorrell Social & Scientific Systems, Inc., Silver Spring, MD ABSTRACT This paper will show you how to create

More information

Be Your Own Task Master - Adding Custom Tasks to EG Peter Eberhardt, Fernwood Consulting Group Inc. Toronto, ON

Be Your Own Task Master - Adding Custom Tasks to EG Peter Eberhardt, Fernwood Consulting Group Inc. Toronto, ON Paper AP05 Be Your Own Task Master - Adding Custom Tasks to EG Peter Eberhardt, Fernwood Consulting Group Inc. Toronto, ON ABSTRACT In Enterprise Guide, SAS provides a ton of tasks to tickle travels into

More information

Hello World! Getting Started with the SAS DS2 Language

Hello World! Getting Started with the SAS DS2 Language ABSTRACT SESUG Paper HOW190-2017 Hello World! Getting Started with the SAS DS2 Language Tricia Aanderud and Jonathan Boase, Zencos Consulting DS2 is an object-oriented programming language that is used

More information

Handling Numeric Representation SAS Errors Caused by Simple Floating-Point Arithmetic Computation Fuad J. Foty, U.S. Census Bureau, Washington, DC

Handling Numeric Representation SAS Errors Caused by Simple Floating-Point Arithmetic Computation Fuad J. Foty, U.S. Census Bureau, Washington, DC Paper BB-206 Handling Numeric Representation SAS Errors Caused by Simple Floating-Point Arithmetic Computation Fuad J. Foty, U.S. Census Bureau, Washington, DC ABSTRACT Every SAS programmer knows that

More information

Hash Objects for Everyone

Hash Objects for Everyone SESUG 2015 Paper BB-83 Hash Objects for Everyone Jack Hall, OptumInsight ABSTRACT The introduction of Hash Objects into the SAS toolbag gives programmers a powerful way to improve performance, especially

More information

Going Under the Hood: How Does the Macro Processor Really Work?

Going Under the Hood: How Does the Macro Processor Really Work? Going Under the Hood: How Does the Really Work? ABSTRACT Lisa Lyons, PPD, Inc Hamilton, NJ Did you ever wonder what really goes on behind the scenes of the macro processor, or how it works with other parts

More information

Introduction / Overview

Introduction / Overview Paper # SC18 Exploring SAS Generation Data Sets Kirk Paul Lafler, Software Intelligence Corporation Abstract Users have at their disposal a unique and powerful feature for retaining historical copies of

More information

WHAT ARE SASHELP VIEWS?

WHAT ARE SASHELP VIEWS? Paper PN13 There and Back Again: Navigating between a SASHELP View and the Real World Anita Rocha, Center for Studies in Demography and Ecology University of Washington, Seattle, WA ABSTRACT A real strength

More information

SESUG 2014 IT-82 SAS-Enterprise Guide for Institutional Research and Other Data Scientists Claudia W. McCann, East Carolina University.

SESUG 2014 IT-82 SAS-Enterprise Guide for Institutional Research and Other Data Scientists Claudia W. McCann, East Carolina University. Abstract Data requests can range from on-the-fly, need it yesterday, to extended projects taking several weeks or months to complete. Often institutional researchers and other data scientists are juggling

More information

PharmaSUG China Paper 70

PharmaSUG China Paper 70 ABSTRACT PharmaSUG China 2015 - Paper 70 SAS Longitudinal Data Techniques - From Change from Baseline to Change from Previous Visits Chao Wang, Fountain Medical Development, Inc., Nanjing, China Longitudinal

More information

SAS/STAT 13.1 User s Guide. The NESTED Procedure

SAS/STAT 13.1 User s Guide. The NESTED Procedure SAS/STAT 13.1 User s Guide The NESTED Procedure This document is an individual chapter from SAS/STAT 13.1 User s Guide. The correct bibliographic citation for the complete manual is as follows: SAS Institute

More information

Files Arriving at an Inconvenient Time? Let SAS Process Your Files with FILEEXIST While You Sleep

Files Arriving at an Inconvenient Time? Let SAS Process Your Files with FILEEXIST While You Sleep Files Arriving at an Inconvenient Time? Let SAS Process Your Files with FILEEXIST While You Sleep Educational Testing Service SAS and all other SAS Institute Inc. product or service names are registered

More information

Sorting big datasets. Do we really need it? Daniil Shliakhov, Experis Clinical, Kharkiv, Ukraine

Sorting big datasets. Do we really need it? Daniil Shliakhov, Experis Clinical, Kharkiv, Ukraine PharmaSUG 2015 - Paper QT21 Sorting big datasets. Do we really need it? Daniil Shliakhov, Experis Clinical, Kharkiv, Ukraine ABSTRACT Very often working with big data causes difficulties for SAS programmers.

More information

Arthur L. Carpenter California Occidental Consultants, Oceanside, California

Arthur L. Carpenter California Occidental Consultants, Oceanside, California Paper 028-30 Storing and Using a List of Values in a Macro Variable Arthur L. Carpenter California Occidental Consultants, Oceanside, California ABSTRACT When using the macro language it is not at all

More information

Updating Data Using the MODIFY Statement and the KEY= Option

Updating Data Using the MODIFY Statement and the KEY= Option Updating Data Using the MODIFY Statement and the KEY= Option Denise J. Moorman and Deanna Warner Denise J. Moorman is a technical support analyst at SAS Institute. Her area of expertise is base SAS software.

More information

Using the SQL Editor. Overview CHAPTER 11

Using the SQL Editor. Overview CHAPTER 11 205 CHAPTER 11 Using the SQL Editor Overview 205 Opening the SQL Editor Window 206 Entering SQL Statements Directly 206 Entering an SQL Query 206 Entering Non-SELECT SQL Code 207 Creating Template SQL

More information

Using Proc Freq for Manageable Data Summarization

Using Proc Freq for Manageable Data Summarization 1 CC27 Using Proc Freq for Manageable Data Summarization Curtis Wolf, DataCeutics, Inc. A SIMPLE BUT POWERFUL PROC The Frequency procedure can be very useful for getting a general sense of the contents

More information

12. Combining SAS datasets. GIORGIO RUSSOLILLO - Cours de prépara)on à la cer)fica)on SAS «Base Programming» 269

12. Combining SAS datasets. GIORGIO RUSSOLILLO - Cours de prépara)on à la cer)fica)on SAS «Base Programming» 269 12. Combining SAS datasets 269 Appending datasets in different situa)ons PROC PRINT DATA=Lib9_3.emps; PROC PRINT DATA=Lib9_3.emps2008; PROC PRINT DATA=Lib9_3.emps2009; PROC PRINT DATA=Lib9_3.emps2010;

More information

Paper PS05_05 Using SAS to Process Repeated Measures Data Terry Fain, RAND Corporation Cyndie Gareleck, RAND Corporation

Paper PS05_05 Using SAS to Process Repeated Measures Data Terry Fain, RAND Corporation Cyndie Gareleck, RAND Corporation Paper PS05_05 Using SAS to Process Repeated Measures Data Terry Fain, RAND Corporation Cyndie Gareleck, RAND Corporation ABSTRACT Data that contain multiple observations per case are called repeated measures

More information

PhUse Practical Uses of the DOW Loop in Pharmaceutical Programming Richard Read Allen, Peak Statistical Services, Evergreen, CO, USA

PhUse Practical Uses of the DOW Loop in Pharmaceutical Programming Richard Read Allen, Peak Statistical Services, Evergreen, CO, USA PhUse 2009 Paper Tu01 Practical Uses of the DOW Loop in Pharmaceutical Programming Richard Read Allen, Peak Statistical Services, Evergreen, CO, USA ABSTRACT The DOW-Loop was originally developed by Don

More information

Now That You Have Your Data in Hadoop, How Are You Staging Your Analytical Base Tables?

Now That You Have Your Data in Hadoop, How Are You Staging Your Analytical Base Tables? Paper SAS 1866-2015 Now That You Have Your Data in Hadoop, How Are You Staging Your Analytical Base Tables? Steven Sober, SAS Institute Inc. ABSTRACT Well, Hadoop community, now that you have your data

More information

Using an Array as an If-Switch Nazik Elgaddal and Ed Heaton, Westat, Rockville, MD

Using an Array as an If-Switch Nazik Elgaddal and Ed Heaton, Westat, Rockville, MD Using an Array as an If-Switch Nazik Elgaddal and Ed Heaton, Westat, Rockville, MD Abstract Do you sometimes find yourself using nested IF statements or nested SELECT blocks? Does the code become more

More information

Uncommon Techniques for Common Variables

Uncommon Techniques for Common Variables Paper 11863-2016 Uncommon Techniques for Common Variables Christopher J. Bost, MDRC, New York, NY ABSTRACT If a variable occurs in more than one data set being merged, the last value (from the variable

More information

Penetrating the Matrix Justin Z. Smith, William Gui Zupko II, U.S. Census Bureau, Suitland, MD

Penetrating the Matrix Justin Z. Smith, William Gui Zupko II, U.S. Census Bureau, Suitland, MD Penetrating the Matrix Justin Z. Smith, William Gui Zupko II, U.S. Census Bureau, Suitland, MD ABSTRACT While working on a time series modeling problem, we needed to find the row and column that corresponded

More information

Paper AD12 Using the ODS EXCEL Destination with SAS University Edition to Send Graphs to Excel

Paper AD12 Using the ODS EXCEL Destination with SAS University Edition to Send Graphs to Excel Paper AD12 Using the ODS EXCEL Destination with SAS University Edition to Send Graphs to Excel ABSTRACT William E Benjamin Jr, Owl Computer Consultancy LLC, Phoenix Arizona Students now have access to

More information

SAS Drug Development. SAS Macro API 1.3 User s Guide

SAS Drug Development. SAS Macro API 1.3 User s Guide SAS Drug Development SAS Macro API 1.3 User s Guide ii SAS Drug Development 4.3.1 and 4.3.2: SAS Macro API 1.3 User s Guide Copyright 2013, SAS Institute Inc., Cary, NC, USA All rights reserved. Produced

More information

Creating and Executing Stored Compiled DATA Step Programs

Creating and Executing Stored Compiled DATA Step Programs 465 CHAPTER 30 Creating and Executing Stored Compiled DATA Step Programs Definition 465 Uses for Stored Compiled DATA Step Programs 465 Restrictions and Requirements 466 How SAS Processes Stored Compiled

More information

The Three I s of SAS Log Messages, IMPORTANT, INTERESTING, and IRRELEVANT William E Benjamin Jr, Owl Computer Consultancy, LLC, Phoenix AZ.

The Three I s of SAS Log Messages, IMPORTANT, INTERESTING, and IRRELEVANT William E Benjamin Jr, Owl Computer Consultancy, LLC, Phoenix AZ. ABSTRACT Paper TT_14 The Three I s of SAS Log Messages, IMPORTANT, INTERESTING, and IRRELEVANT William E Benjamin Jr, Owl Computer Consultancy, LLC, Phoenix AZ. I like to think that SAS error messages

More information

Understanding the Concepts and Features of Macro Programming 1

Understanding the Concepts and Features of Macro Programming 1 Contents Preface ix Acknowledgments xi Part 1 Understanding the Concepts and Features of Macro Programming 1 Chapter 1 Introduction 3 What Is the SAS Macro Facility? 4 What Are the Advantages of the SAS

More information

PharmaSUG Paper CC22

PharmaSUG Paper CC22 PharmaSUG 2011 - Paper CC22 Importing and Parsing Comments From a PDF Document With Help From Perl Regular Expressions Joel Campbell, PPD, Inc., Wilmington, NC Ryan Wilkins, PPD, Inc., Wilmington, NC ABSTRACT

More information

Troy Martin Hughes ABSTRACT INTRODUCTION

Troy Martin Hughes ABSTRACT INTRODUCTION MWSUG 2016 - Paper 38 A Failure To EXIST: Why Testing for Data Set Existence with the EXIST Function Alone Is Inadequate for Serious Software Development in Asynchronous, Multiuser, and Parallel Processing

More information

Practical Uses of the DOW Loop Richard Read Allen, Peak Statistical Services, Evergreen, CO

Practical Uses of the DOW Loop Richard Read Allen, Peak Statistical Services, Evergreen, CO Practical Uses of the DOW Loop Richard Read Allen, Peak Statistical Services, Evergreen, CO ABSTRACT The DOW-Loop was originally developed by Don Henderson and popularized the past few years on the SAS-L

More information

How to Incorporate Old SAS Data into a New DATA Step, or What is S-M-U?

How to Incorporate Old SAS Data into a New DATA Step, or What is S-M-U? Paper 54-25 How to Incorporate Old SAS Data into a New DATA Step, or What is S-M-U? Andrew T. Kuligowski Nielsen Media Research Abstract / Introduction S-M-U. Some people will see these three letters and

More information

Submission-Ready Define.xml Files Using SAS Clinical Data Integration Melissa R. Martinez, SAS Institute, Cary, NC USA

Submission-Ready Define.xml Files Using SAS Clinical Data Integration Melissa R. Martinez, SAS Institute, Cary, NC USA PharmaSUG 2016 - Paper SS12 Submission-Ready Define.xml Files Using SAS Clinical Data Integration Melissa R. Martinez, SAS Institute, Cary, NC USA ABSTRACT SAS Clinical Data Integration simplifies the

More information

Using a Fillable PDF together with SAS for Questionnaire Data Donald Evans, US Department of the Treasury

Using a Fillable PDF together with SAS for Questionnaire Data Donald Evans, US Department of the Treasury Using a Fillable PDF together with SAS for Questionnaire Data Donald Evans, US Department of the Treasury Introduction The objective of this paper is to demonstrate how to use a fillable PDF to collect

More information

Programming & Manipulation. Danger: MERGE Ahead! Warning: BY Variable with Multiple Lengths! Bob Virgile Robert Virgile Associates, Inc.

Programming & Manipulation. Danger: MERGE Ahead! Warning: BY Variable with Multiple Lengths! Bob Virgile Robert Virgile Associates, Inc. Danger: MERGE Ahead! Warning: BY Variable with Multiple Lengths! Bob Virgile Robert Virgile Associates, Inc. Overview Normally, when a data step merges two data sets, any common variables will have the

More information

Note: Basic understanding of the CDISC ODM structure of Events, Forms, ItemGroups, Items, Codelists and MeasurementUnits is required.

Note: Basic understanding of the CDISC ODM structure of Events, Forms, ItemGroups, Items, Codelists and MeasurementUnits is required. Paper CC-018 Exploring SAS PROC CDISC Model=ODM and Its Undocumented Parameters Elena Valkanova, Biostat International, Inc, Tampa, FL Irene Droll, XClinical GmbH, München, Germany ABSTRACT The CDISC Operational

More information

Are you Still Afraid of Using Arrays? Let s Explore their Advantages

Are you Still Afraid of Using Arrays? Let s Explore their Advantages Paper CT07 Are you Still Afraid of Using Arrays? Let s Explore their Advantages Vladyslav Khudov, Experis Clinical, Kharkiv, Ukraine ABSTRACT At first glance, arrays in SAS seem to be a complicated and

More information

Foundations and Fundamentals. SAS System Options: The True Heroes of Macro Debugging Kevin Russell and Russ Tyndall, SAS Institute Inc.

Foundations and Fundamentals. SAS System Options: The True Heroes of Macro Debugging Kevin Russell and Russ Tyndall, SAS Institute Inc. SAS System Options: The True Heroes of Macro Debugging Kevin Russell and Russ Tyndall, SAS Institute Inc., Cary, NC ABSTRACT It is not uncommon for the first draft of any macro application to contain errors.

More information

The NESTED Procedure (Chapter)

The NESTED Procedure (Chapter) SAS/STAT 9.3 User s Guide The NESTED Procedure (Chapter) SAS Documentation This document is an individual chapter from SAS/STAT 9.3 User s Guide. The correct bibliographic citation for the complete manual

More information

SAS 9 Programming Enhancements Marje Fecht, Prowerk Consulting Ltd Mississauga, Ontario, Canada

SAS 9 Programming Enhancements Marje Fecht, Prowerk Consulting Ltd Mississauga, Ontario, Canada SAS 9 Programming Enhancements Marje Fecht, Prowerk Consulting Ltd Mississauga, Ontario, Canada ABSTRACT Performance improvements are the well-publicized enhancement to SAS 9, but what else has changed

More information

Gary L. Katsanis, Blue Cross and Blue Shield of the Rochester Area, Rochester, NY

Gary L. Katsanis, Blue Cross and Blue Shield of the Rochester Area, Rochester, NY Table Lookups in the SAS Data Step Gary L. Katsanis, Blue Cross and Blue Shield of the Rochester Area, Rochester, NY Introduction - What is a Table Lookup? You have a sales file with one observation for

More information

Can you decipher the code? If you can, maybe you can break it. Jay Iyengar, Data Systems Consultants LLC, Oak Brook, IL

Can you decipher the code? If you can, maybe you can break it. Jay Iyengar, Data Systems Consultants LLC, Oak Brook, IL Paper 11667-2016 Can you decipher the code? If you can, maybe you can break it. Jay Iyengar, Data Systems Consultants LLC, Oak Brook, IL ABSTRACT You would think that training as a code breaker, similar

More information

Paper CT-16 Manage Hierarchical or Associated Data with the RETAIN Statement Alan R. Mann, Independent Consultant, Harpers Ferry, WV

Paper CT-16 Manage Hierarchical or Associated Data with the RETAIN Statement Alan R. Mann, Independent Consultant, Harpers Ferry, WV Paper CT-16 Manage Hierarchical or Associated Data with the RETAIN Statement Alan R. Mann, Independent Consultant, Harpers Ferry, WV ABSTRACT For most of the history of computing machinery, hierarchical

More information

Amie Bissonett, inventiv Health Clinical, Minneapolis, MN

Amie Bissonett, inventiv Health Clinical, Minneapolis, MN PharmaSUG 2013 - Paper TF12 Let s get SAS sy Amie Bissonett, inventiv Health Clinical, Minneapolis, MN ABSTRACT The SAS language has a plethora of procedures, data step statements, functions, and options

More information

Taming a Spreadsheet Importation Monster

Taming a Spreadsheet Importation Monster SESUG 2013 Paper BtB-10 Taming a Spreadsheet Importation Monster Nat Wooding, J. Sargeant Reynolds Community College ABSTRACT As many programmers have learned to their chagrin, it can be easy to read Excel

More information

Kharagpur Site Online Problems 2013

Kharagpur Site Online Problems 2013 Kharagpur Site Online Problems 013 Problem #1: List Editing At the ACME University, as part of the Graduate course work, each student is required to undergo project work. The authorities always make an

More information

Create a Format from a SAS Data Set Ruth Marisol Rivera, i3 Statprobe, Mexico City, Mexico

Create a Format from a SAS Data Set Ruth Marisol Rivera, i3 Statprobe, Mexico City, Mexico PharmaSUG 2011 - Paper TT02 Create a Format from a SAS Data Set Ruth Marisol Rivera, i3 Statprobe, Mexico City, Mexico ABSTRACT Many times we have to apply formats and it could be hard to create them specially

More information

The G4GRID Procedure. Introduction APPENDIX 1

The G4GRID Procedure. Introduction APPENDIX 1 93 APPENDIX 1 The G4GRID Procedure Introduction 93 Data Considerations 94 Terminology 94 Using the Graphical Interface 94 Procedure Syntax 95 The PROC G4GRID Statement 95 The GRID Statement 97 The BY Statement

More information

Using PROC SQL to Calculate FIRSTOBS David C. Tabano, Kaiser Permanente, Denver, CO

Using PROC SQL to Calculate FIRSTOBS David C. Tabano, Kaiser Permanente, Denver, CO Using PROC SQL to Calculate FIRSTOBS David C. Tabano, Kaiser Permanente, Denver, CO ABSTRACT The power of SAS programming can at times be greatly improved using PROC SQL statements for formatting and manipulating

More information

Are Your SAS Programs Running You? Marje Fecht, Prowerk Consulting, Cape Coral, FL Larry Stewart, SAS Institute Inc., Cary, NC

Are Your SAS Programs Running You? Marje Fecht, Prowerk Consulting, Cape Coral, FL Larry Stewart, SAS Institute Inc., Cary, NC Paper CS-044 Are Your SAS Programs Running You? Marje Fecht, Prowerk Consulting, Cape Coral, FL Larry Stewart, SAS Institute Inc., Cary, NC ABSTRACT Most programs are written on a tight schedule, using

More information

Unravelling the Knot of Ampersands

Unravelling the Knot of Ampersands Paper 3285-2015 Unravelling the Knot of Ampersands Joe Matise, NORC at the University of Chicago ABSTRACT We've all heard it before: "If two ampersands don't work, add a third." But how many of us really

More information

SAS System Options are Your Friends Edward Heaton, Westat, Rockville, MD

SAS System Options are Your Friends Edward Heaton, Westat, Rockville, MD SAS System Options are Your Friends Edward Heaton, Westat, Rockville, MD Abstract Does SAS always do things the way you want? Have you ever made a simple little mistake of omission with disastrous consequences?

More information

The Proc Transpose Cookbook

The Proc Transpose Cookbook ABSTRACT PharmaSUG 2017 - Paper TT13 The Proc Transpose Cookbook Douglas Zirbel, Wells Fargo and Co. Proc TRANSPOSE rearranges columns and rows of SAS datasets, but its documentation and behavior can be

More information

Using SAS Macros to Extract P-values from PROC FREQ

Using SAS Macros to Extract P-values from PROC FREQ SESUG 2016 ABSTRACT Paper CC-232 Using SAS Macros to Extract P-values from PROC FREQ Rachel Straney, University of Central Florida This paper shows how to leverage the SAS Macro Facility with PROC FREQ

More information

The Future of Transpose: How SAS Is Rebuilding Its Foundation by Making What Is Old New Again

The Future of Transpose: How SAS Is Rebuilding Its Foundation by Making What Is Old New Again Paper 701-2017 The Future of Transpose: How SAS Is Rebuilding Its Foundation by Making What Is Old New Again Scott Mebust, SAS Institute Inc., Cary, NC ABSTRACT As computer technology advances, SAS continually

More information

SAS File Management. Improving Performance CHAPTER 37

SAS File Management. Improving Performance CHAPTER 37 519 CHAPTER 37 SAS File Management Improving Performance 519 Moving SAS Files Between Operating Environments 520 Converting SAS Files 520 Repairing Damaged Files 520 Recovering SAS Data Files 521 Recovering

More information

What is sorting? Lecture 36: How can computation sort data in order for you? Why is sorting important? What is sorting? 11/30/10

What is sorting? Lecture 36: How can computation sort data in order for you? Why is sorting important? What is sorting? 11/30/10 // CS Introduction to Computation " UNIVERSITY of WISCONSIN-MADISON Computer Sciences Department Professor Andrea Arpaci-Dusseau Fall Lecture : How can computation sort data in order for you? What is sorting?

More information

What s New in SAS Studio?

What s New in SAS Studio? ABSTRACT Paper SAS1832-2015 What s New in SAS Studio? Mike Porter, Amy Peters, and Michael Monaco, SAS Institute Inc., Cary, NC If you have not had a chance to explore SAS Studio yet, or if you re anxious

More information

The Impossible An Organized Statistical Programmer Brian Spruell and Kevin Mcgowan, SRA Inc., Durham, NC

The Impossible An Organized Statistical Programmer Brian Spruell and Kevin Mcgowan, SRA Inc., Durham, NC Paper CS-061 The Impossible An Organized Statistical Programmer Brian Spruell and Kevin Mcgowan, SRA Inc., Durham, NC ABSTRACT Organization is the key to every project. It provides a detailed history of

More information

SAS Macro Dynamics - From Simple Basics to Powerful Invocations Rick Andrews, Office of the Actuary, CMS, Baltimore, MD

SAS Macro Dynamics - From Simple Basics to Powerful Invocations Rick Andrews, Office of the Actuary, CMS, Baltimore, MD Paper BB-7 SAS Macro Dynamics - From Simple Basics to Powerful Invocations Rick Andrews, Office of the Actuary, CMS, Baltimore, MD ABSTRACT The SAS Macro Facility offers a mechanism for expanding and customizing

More information

The Path To Treatment Pathways Tracee Vinson-Sorrentino, IMS Health, Plymouth Meeting, PA

The Path To Treatment Pathways Tracee Vinson-Sorrentino, IMS Health, Plymouth Meeting, PA ABSTRACT PharmaSUG 2015 - Paper HA06 The Path To Treatment Pathways Tracee Vinson-Sorrentino, IMS Health, Plymouth Meeting, PA Refills, switches, restarts, and continuation are valuable and necessary metrics

More information

SAS Application to Automate a Comprehensive Review of DEFINE and All of its Components

SAS Application to Automate a Comprehensive Review of DEFINE and All of its Components PharmaSUG 2017 - Paper AD19 SAS Application to Automate a Comprehensive Review of DEFINE and All of its Components Walter Hufford, Vincent Guo, and Mijun Hu, Novartis Pharmaceuticals Corporation ABSTRACT

More information