STOP MERGING AND START COMBINING by Robert S. Nicol U.S. Quality Algorithms
|
|
- Sybil Jefferson
- 6 years ago
- Views:
Transcription
1 STOP MERGING AND START COMBINING by Robert S. Nicol U.S. Quality Algorithms There are many ways to combine data within the SAS system. Probably the most widely used method is the. While the merge is very powerful, it is often misunderstood and overused by the novice SAS programmer. This paper ilhjstrates some of the common data combination methods and provides guidelines for choosing the most effective technique. The data combination routines covered by this paper are formats, the, the update statement, the set statement, proc SQl, proc append, and proc datasets. This paper is intended to be used as an ongoing rapid reference and as such is designed to get you quickly to the relative facts and syntax. How do the fields in the source files relate to the resultant file? Are all of the fields in all of the files required? Does the same field appear in more than one file? Other considerations: If the files are related by 'keys', are the files sorted or indexed by those keys? How much memory is available to you? How often does this combination have to be performed? How large are the files? How often does the data change? The code must be maintainable. The best way to insure the results of any project is to start with a plan. This axiom also holds true in the design of a program. An integral part of planning a program is having a thorough understanding of the available data. Armed with your knowledge of the data and your operating environment, the methods used to combine the data should be evident. This paper reviews the processes that can be applied under three distinct sets of conditions: When you need to combine entire observations. When one file is a 'master', that is you need to end up with only those original observations. When the data drive the resultant observations. PRIMARY DECISION FACTORS How are the observations related to one another? Should the observations remain intact? Are they related ordinally? Is one file a 'master' with fields added or modified based upon other files? If the files are related by 'keys', which observations are to survive? The methods to follow are presented in increasing order of preference. However, the conditions at your site may dictate a different order. Additionally the methods presented here will combine all observations and all variables. Should you need to modify the observations you may be able to use data set options (where, obs, firstobs) or sub-setting statements(if, where, delete}. The variables in the output data set may be controlled by data set options(drop, keep, rename} or statements (drop, keep, rename). 171
2 COMBINING ENTIRE OBSERVATIONS INTERLEAVING FILES Interleaving is the process of combining entire data sets into one resultant data set. The order of the observations is controlled via a "by" variable. The total number of observations in the final data set is equal to the total observations in the contributing files. set statement +advantage The command is safe and straight forward. drawback All files must be presorted or indexed. data inter; set one twoj NOTE: The merge command can also be used to interleave data sets. However if 'by variable' matches are found, the resultant file will be in error. CONCATENATION Concatenation is the process of combining entire data sets into one resultant data set. All of the observations in one data set are followed by all of the observations in the other data set. The order of the observations is controlled by the order of the data sets in the concatenation command. The total number of observations in the final data set is equal to the total observations in the contributing files. procappend + advantage As the 'base' data set is not rewritten, the computer resource requirements are significantly reduced. +drawback No other processing can be perlormed. The base data set must be available for modification. If the step abends then the base data set may be damaged. proc append force base::one data::two; set statement advantage As the command is part of a data step, further processing can be accomplished without starting a new data or proc drawback All observations are read in and then written out which requires CPU usage and workspace. data concatj setonetwoi proc datasets advantage Since the 'base' data set is not rewritten, computer resource requirements are significantly reduced. drawback Only other 'proc dataset' processes can be performed. The base data set must be available for modification. If the proc abends the base data set may be damaged. proc datasets library=work forcej append base=one data=twoj THE OBSERVATIONS IN ONE ALE ARE THE ONLY REQUIRED OBSERVATIONS The observations in the 'base' data set are to be the only observations in resultant file. The fields in each observation are a combination of the fields in the source files. 172
3 procsql advantage The files do not have to be presorted. drawback The proc requires large amounts of work space. The syntax becomes tiring jf the files have a large number of fields. The secondary files must be pre-selected to have unique keys. procsql; create table oneto1 as select one.key. two.field_a from one. two where one.key = two.key union select key. field_a from one where key not in (select key from two) order by key advantage The statement handles large numbers of fields and files with simple syntax. As the command is part of a data step, further processing can be accomplished without starting a new data or proc drawback All files must be presorted or indexed. The secondary files must be sorted with the 'nodupkeys' option. NOTE: if the same variable occurs in more than one data set then the resultant value will come f rom the right most data set in the merge statement. update statement advantage Allows multiple 'transaction' records per 'master'. As the command is part of a data step, further processing can be accomplished without starting a new data or proc drawback Fields cannot be added to the master file. All files must be presorted or indexed. data manyto1 ; update one(ln=in_one) two; if in_one; formats advantage As the format(s) is created from the transaction file, the primary file does not have to be sorted or indexed. It may be possible not to modify the base file at all, but merely apply the formats at time of output. Because the format is applied within a data step, further processing can be accomplished without starting a new data or proc If the transaction file does not change often consider saving permanent file. drawback If many fields are to be added the overall complexity of the code and CPU resources will become burdensome. NOTE: ConSider writing a macro to create formats from data sets, thus reducing the coding to create formats data manyt01 ; merge one(in=in_one) two; if In_one i data cntltwo(keep= fmtname type start label); set two end:eof; format start $5.; fmtname='two_fmt'; type:'c'; 173
4 start=key; label=field_a; output; if eof then do; start='other'; label=' '; output; end; proc fonnat cntlln=cntuwo; data withfonn; set one; from_2 = put(key,stwo_fmt.}; RESULTANT OBSERVATIONS ARE DEPENPENT UPON THE KEYS In some situations you may need to let'the data do the talking. That is the number of records in the final data set will be a direct result of the number of matching keys in the source files. One-ta-Many The is typified by combining one record from a reference file with many observations from a detail file. procsql +advantage Files do not have to be presorted. +drawback Requires large amounts of work space. The syntax becomes tiring if the files have a large number of fields. proc sql ; create table manyto1 as select one.key, one.source1, two.field_a,two.source2 from one, two where one.key=two.key union select key, source1, field_a from one where key not in (select key from two) order advantage Many fields and files can be handled with simple syntax. As the command is part of a data step, further processing can be accomplished without starting a new data or proc drawback All files must be presorted or indexed. data manyt01 ; merge one(ln=in_one) two(in=in_two); if in_one and in_two; Many-ta-Many Many-to-many merges occur when you combine files that each have multiple occurrences of the key values. Caution: In many years of experience in different industries, I have noted the most many-to-many merges are caused by invalid data(or a lack of understanding of the data) rather then a true need to perform a many-ta-many combination. proc sql advantage It works. Files do not have to be presorted. drawback- Requires large amounts of work space. The syntax becomes lengthy if the files have many fields. proc sql; create table manytom as select two. *, three.source3, three.flekca from two(rename=(field_a=f1d_a2}), three where two.key=three.key order by key, fld_a2: 174
5 DO NOT USE MERGE FOR MANY TO MANY The uses the data from the right most data set that is still contributing data. Although the system only flags many to many merges as a warning, this will usually create erroneous data. Roll you own You may write your own 'many-to-many combiner'. This can be accomplished by performing a series of one to many merges, or by using indexes and pointers or by... POINTS TO REMEMBER While programming SAS there is no substitute for reading the log and testing your logic. If you start off with knowing your data(repeats of key values etc.) and the result that you need{only the observations in the base data set, etc.), then choosing the data combination technique should be straight forward. If a method does not teel right, it may not be. Test it. Then compare the results and the resource usage. CONTACT INFORMATION Robert S. Nicol telephone: (day) (evening) fax
INTRODUCTION TO SAS HOW SAS WORKS READING RAW DATA INTO SAS
TO SAS NEED FOR SAS WHO USES SAS WHAT IS SAS? OVERVIEW OF BASE SAS SOFTWARE DATA MANAGEMENT FACILITY STRUCTURE OF SAS DATASET SAS PROGRAM PROGRAMMING LANGUAGE ELEMENTS OF THE SAS LANGUAGE RULES FOR SAS
More informationMerge Processing and Alternate Table Lookup Techniques Prepared by
Merge Processing and Alternate Table Lookup Techniques Prepared by The syntax for data step merging is as follows: International SAS Training and Consulting This assumes that the incoming data sets are
More informationPROC FORMAT: USE OF THE CNTLIN OPTION FOR EFFICIENT PROGRAMMING
PROC FORMAT: USE OF THE CNTLIN OPTION FOR EFFICIENT PROGRAMMING Karuna Nerurkar and Andrea Robertson, GMIS Inc. ABSTRACT Proc Format can be a useful tool for improving programming efficiency. This paper
More informationCreate a Format from a SAS Data Set Ruth Marisol Rivera, i3 Statprobe, Mexico City, Mexico
PharmaSUG 2011 - Paper TT02 Create a Format from a SAS Data Set Ruth Marisol Rivera, i3 Statprobe, Mexico City, Mexico ABSTRACT Many times we have to apply formats and it could be hard to create them specially
More informationBASICS BEFORE STARTING SAS DATAWAREHOSING Concepts What is ETL ETL Concepts What is OLAP SAS. What is SAS History of SAS Modules available SAS
SAS COURSE CONTENT Course Duration - 40hrs BASICS BEFORE STARTING SAS DATAWAREHOSING Concepts What is ETL ETL Concepts What is OLAP SAS What is SAS History of SAS Modules available SAS GETTING STARTED
More informationBase and Advance SAS
Base and Advance SAS BASE SAS INTRODUCTION An Overview of the SAS System SAS Tasks Output produced by the SAS System SAS Tools (SAS Program - Data step and Proc step) A sample SAS program Exploring SAS
More informationChoosing the Right Technique to Merge Large Data Sets Efficiently Qingfeng Liang, Community Care Behavioral Health Organization, Pittsburgh, PA
Choosing the Right Technique to Merge Large Data Sets Efficiently Qingfeng Liang, Community Care Behavioral Health Organization, Pittsburgh, PA ABSTRACT This paper outlines different SAS merging techniques
More informationcapabilities and their overheads are therefore different.
Applications Development 3 Access DB2 Tables Using Keylist Extraction Berwick Chan, Kaiser Permanente, Oakland, Calif Raymond Wan, Raymond Wan Associate Inc., Oakland, Calif Introduction The performance
More informationChapter 6: Modifying and Combining Data Sets
Chapter 6: Modifying and Combining Data Sets The SET statement is a powerful statement in the DATA step. Its main use is to read in a previously created SAS data set which can be modified and saved as
More informationHow to Incorporate Old SAS Data into a New DATA Step, or What is S-M-U?
How to Incorporate Old SAS Data into a New DATA Step, or What is S-M-U? Andrew T. Kuligowski Nielsen Media Research Abstract / Introduction S-M-U. Some people will see these three letters and immediately
More information50 WAYS TO MERGE YOUR DATA INSTALLMENT 1 Kristie Schuster, LabOne, Inc., Lenexa, Kansas Lori Sipe, LabOne, Inc., Lenexa, Kansas
Paper 103-26 50 WAYS TO MERGE YOUR DATA INSTALLMENT 1 Kristie Schuster, LabOne, Inc., Lenexa, Kansas Lori Sipe, LabOne, Inc., Lenexa, Kansas ABSTRACT When you need to join together two datasets, how do
More informationHow to Incorporate Old SAS Data into a New DATA Step, or What is S-M-U?
Paper 54-25 How to Incorporate Old SAS Data into a New DATA Step, or What is S-M-U? Andrew T. Kuligowski Nielsen Media Research Abstract / Introduction S-M-U. Some people will see these three letters and
More informationGary L. Katsanis, Blue Cross and Blue Shield of the Rochester Area, Rochester, NY
Table Lookups in the SAS Data Step Gary L. Katsanis, Blue Cross and Blue Shield of the Rochester Area, Rochester, NY Introduction - What is a Table Lookup? You have a sales file with one observation for
More informationThe Problem With NODUPLICATES, Continued
The Problem With NODUPLICATES, Continued Jack Hamilton First Health West Sacramento, California JackHamilton@FirstHealth.com moredupsov.doc Wednesday, Wed Apr 28 1999 12:37 PM Page 1 of 11 What Should
More informationContents. About This Book...1
Contents About This Book...1 Chapter 1: Basic Concepts...5 Overview...6 SAS Programs...7 SAS Libraries...13 Referencing SAS Files...15 SAS Data Sets...18 Variable Attributes...21 Summary...26 Practice...28
More informationTable Lookups: From IF-THEN to Key-Indexing
Table Lookups: From IF-THEN to Key-Indexing Arthur L. Carpenter, California Occidental Consultants ABSTRACT One of the more commonly needed operations within SAS programming is to determine the value of
More informationPharmaSUG Paper PO12
PharmaSUG 2015 - Paper PO12 ABSTRACT Utilizing SAS for Cross-Report Verification in a Clinical Trials Setting Daniel Szydlo, Fred Hutchinson Cancer Research Center, Seattle, WA Iraj Mohebalian, Fred Hutchinson
More informationSorting big datasets. Do we really need it? Daniil Shliakhov, Experis Clinical, Kharkiv, Ukraine
PharmaSUG 2015 - Paper QT21 Sorting big datasets. Do we really need it? Daniil Shliakhov, Experis Clinical, Kharkiv, Ukraine ABSTRACT Very often working with big data causes difficulties for SAS programmers.
More informationS-M-U (Set, Merge, and Update) Revisited
Paper 3444-2015 S-M-U (Set, Merge, and Update) Revisited Andrew T. Kuligowski, HSN ABSTRACT It is a safe assumption that almost every SAS user learns how to use the SET statement not long after they re
More informationPaper PO06. Building Dynamic Informats and Formats
Paper PO06 Building Dynamic Informats and Formats Michael Zhang, Merck & Co, Inc, West Point, PA ABSTRACT Using the FORMAT procedure to define informats and formats is a common task in SAS programming
More informationOUT= IS IN: VISUALIZING PROC COMPARE RESULTS IN A DATASET
OUT= IS IN: VISUALIZING PROC COMPARE RESULTS IN A DATASET Prasad Ilapogu, Ephicacy Consulting Group; Masaki Mihaila, Pfizer; ABSTRACT Proc compare is widely used in the pharmaceutical world to validate
More informationFormat-o-matic: Using Formats To Merge Data From Multiple Sources
SESUG Paper 134-2017 Format-o-matic: Using Formats To Merge Data From Multiple Sources Marcus Maher, Ipsos Public Affairs; Joe Matise, NORC at the University of Chicago ABSTRACT User-defined formats are
More informationQuicker Than Merge? Kirby Cossey, Texas State Auditor s Office, Austin, Texas
Paper 076-29 Quicker Than Merge? Kirby Cossey, Texas State Auditor s Office, Austin, Texas ABSTRACT How many times do you need to extract a few records from an extremely large dataset? INTRODUCTION In
More informationGet Started Writing SAS Macros Luisa Hartman, Jane Liao, Merck Sharp & Dohme Corp.
Get Started Writing SAS Macros Luisa Hartman, Jane Liao, Merck Sharp & Dohme Corp. ABSTRACT The SAS Macro Facility is a tool which lends flexibility to your SAS code and promotes easier maintenance. It
More informationSAS CURRICULUM. BASE SAS Introduction
SAS CURRICULUM BASE SAS Introduction Data Warehousing Concepts What is a Data Warehouse? What is a Data Mart? What is the difference between Relational Databases and the Data in Data Warehouse (OLTP versus
More informationSQL, HASH Tables, FORMAT and KEY= More Than One Way to Merge Two Datasets
TF19 SQL, HASH Tables, FORMAT and KEY= More Than One Way to Merge Two Datasets David Franklin TheProgrammers Cabin.com Introduction Merging data is one of the most common data manipulation task done with
More informationContents of SAS Programming Techniques
Contents of SAS Programming Techniques Chapter 1 About SAS 1.1 Introduction 1.1.1 SAS modules 1.1.2 SAS module classification 1.1.3 SAS features 1.1.4 Three levels of SAS techniques 1.1.5 Chapter goal
More informationThe Building Blocks of SAS Datasets. (Set, Merge, and Update) Andrew T. Kuligowski FCCI Insurance Group
The Building Blocks of SAS Datasets S-M-U (Set, Merge, and Update) Andrew T. Kuligowski FCCI Insurance Group S-M-U What is S M U? 2 S-M-U What is S M U? Shmoo? 3 S-M-U What is S M U? Southern Methodist
More informationNO MORE MERGE. Alternative Table Lookup Techniques
NO MORE MERGE. Alternative Table Lookup Techniques Dana Rafiee, Destiny Corporation/DDISC Group Ltd. U.S., Wethersfield, CT ABSTRACT This tutorial is designed to show you several techniques available for
More informationSUGI 29 Data Warehousing, Management and Quality
Building a Purchasing Data Warehouse for SRM from Disparate Procurement Systems Zeph Stemle, Qualex Consulting Services, Inc., Union, KY ABSTRACT SAS Supplier Relationship Management (SRM) solution offers
More informationDitch the Data Memo: Using Macro Variables and Outer Union Corresponding in PROC SQL to Create Data Set Summary Tables Andrea Shane MDRC, Oakland, CA
ABSTRACT Ditch the Data Memo: Using Macro Variables and Outer Union Corresponding in PROC SQL to Create Data Set Summary Tables Andrea Shane MDRC, Oakland, CA Data set documentation is essential to good
More informationSAS CLINICAL SYLLABUS. DURATION: - 60 Hours
SAS CLINICAL SYLLABUS DURATION: - 60 Hours BASE SAS PART - I Introduction To Sas System & Architecture History And Various Modules Features Variables & Sas Syntax Rules Sas Data Sets Data Set Options Operators
More informationUsing PROC SQL to Calculate FIRSTOBS David C. Tabano, Kaiser Permanente, Denver, CO
Using PROC SQL to Calculate FIRSTOBS David C. Tabano, Kaiser Permanente, Denver, CO ABSTRACT The power of SAS programming can at times be greatly improved using PROC SQL statements for formatting and manipulating
More informationSimplifying Effective Data Transformation Via PROC TRANSPOSE
MWSUG 2016 - Paper SA05 Simplifying Effective Data Transformation Via PROC TRANSPOSE Arthur X. Li, City of Hope Comprehensive Cancer Center, Duarte, CA ABSTRACT You can store data with repeated measures
More informationSAS (Statistical Analysis Software/System)
SAS (Statistical Analysis Software/System) Clinical SAS:- Class Room: Training Fee & Duration : 23K & 3 Months Online: Training Fee & Duration : 25K & 3 Months Learning SAS: Getting Started with SAS Basic
More informationOmitting Records with Invalid Default Values
Paper 7720-2016 Omitting Records with Invalid Default Values Lily Yu, Statistics Collaborative Inc. ABSTRACT Many databases include default values that are set inappropriately. These default values may
More informationData Edit-checks Integration using ODS Tagset Niraj J. Pandya, Element Technologies Inc., NJ Vinodh Paida, Impressive Systems Inc.
PharmaSUG2011 - Paper DM03 Data Edit-checks Integration using ODS Tagset Niraj J. Pandya, Element Technologies Inc., NJ Vinodh Paida, Impressive Systems Inc., TX ABSTRACT In the Clinical trials data analysis
More informationKeeping Track of Database Changes During Database Lock
Paper CC10 Keeping Track of Database Changes During Database Lock Sanjiv Ramalingam, Biogen Inc., Cambridge, USA ABSTRACT Higher frequency of data transfers combined with greater likelihood of changes
More informationLeave Your Bad Code Behind: 50 Ways to Make Your SAS Code Execute More Efficiently.
Leave Your Bad Code Behind: 50 Ways to Make Your SAS Code Execute More Efficiently. William E Benjamin Jr Owl Computer Consultancy, LLC 2012 Topic Groups Processing more than one file in each DATA step
More information3. Almost always use system options options compress =yes nocenter; /* mostly use */ options ps=9999 ls=200;
Randy s SAS hints, updated Feb 6, 2014 1. Always begin your programs with internal documentation. * ***************** * Program =test1, Randy Ellis, first version: March 8, 2013 ***************; 2. Don
More informationSAS Online Training: Course contents: Agenda:
SAS Online Training: Course contents: Agenda: (1) Base SAS (6) Clinical SAS Online Training with Real time Projects (2) Advance SAS (7) Financial SAS Training Real time Projects (3) SQL (8) CV preparation
More informationUpdating Data Using the MODIFY Statement and the KEY= Option
Updating Data Using the MODIFY Statement and the KEY= Option Denise J. Moorman and Deanna Warner Denise J. Moorman is a technical support analyst at SAS Institute. Her area of expertise is base SAS software.
More informationTop 10 Ways to Optimize Your SAS Code Jeff Simpson SAS Customer Loyalty
Top 10 Ways to Optimize Your SAS Code Jeff Simpson SAS Customer Loyalty Writing efficient SAS programs means balancing the constraints of TIME Writing efficient SAS programs means balancing Time and SPACE
More informationEXAMPLE 3: MATCHING DATA FROM RESPONDENTS AT 2 OR MORE WAVES (LONG FORMAT)
EXAMPLE 3: MATCHING DATA FROM RESPONDENTS AT 2 OR MORE WAVES (LONG FORMAT) DESCRIPTION: This example shows how to combine the data on respondents from the first two waves of Understanding Society into
More informationSo Much Data, So Little Time: Splitting Datasets For More Efficient Run Times and Meeting FDA Submission Guidelines
Paper TT13 So Much Data, So Little Time: Splitting Datasets For More Efficient Run Times and Meeting FDA Submission Guidelines Anthony Harris, PPD, Wilmington, NC Robby Diseker, PPD, Wilmington, NC ABSTRACT
More informationA Cross-national Comparison Using Stacked Data
A Cross-national Comparison Using Stacked Data Goal In this exercise, we combine household- and person-level files across countries to run a regression estimating the usual hours of the working-aged civilian
More informationTechdata Solution. SAS Analytics (Clinical/Finance/Banking)
+91-9702066624 Techdata Solution Training - Staffing - Consulting Mumbai & Pune SAS Analytics (Clinical/Finance/Banking) What is SAS SAS (pronounced "sass", originally Statistical Analysis System) is an
More informationBeginning Tutorials. Paper 53-27
Paper 53-27 DATA Step vs. What s a neophyte to do? Craig Dickstein, Tamarack Professional Services, Weare, NH Ray Pass, Ray Pass Consulting, Hartsdale, NY ABSTRACT "What's all the buzz about Proc SQL?
More information1. Join with PROC SQL a left join that will retain target records having no lookup match. 2. Data Step Merge of the target and lookup files.
Abstract PaperA03-2007 Table Lookups...You Want Performance? Rob Rohrbough, Rohrbough Systems Design, Inc. Presented to the Midwest SAS Users Group Monday, October 29, 2007 Paper Number A3 Over the years
More informationComparison of different ways using table lookups on huge tables
PhUSE 007 Paper CS0 Comparison of different ways using table lookups on huge tables Ralf Minkenberg, Boehringer Ingelheim Pharma GmbH & Co. KG, Ingelheim, Germany ABSTRACT In many application areas the
More informationA SAS Macro Utility to Modify and Validate RTF Outputs for Regional Analyses Jagan Mohan Achi, PPD, Austin, TX Joshua N. Winters, PPD, Rochester, NY
PharmaSUG 2014 - Paper BB14 A SAS Macro Utility to Modify and Validate RTF Outputs for Regional Analyses Jagan Mohan Achi, PPD, Austin, TX Joshua N. Winters, PPD, Rochester, NY ABSTRACT Clinical Study
More informationIntroduction / Overview
Paper # SC18 Exploring SAS Generation Data Sets Kirk Paul Lafler, Software Intelligence Corporation Abstract Users have at their disposal a unique and powerful feature for retaining historical copies of
More informationCharacteristics of a "Successful" Application.
Characteristics of a "Successful" Application. Caroline Bahler, Meridian Software, Inc. Abstract An application can be judged "successful" by two different sets of criteria. The first set of criteria belongs
More informationSAS (Statistical Analysis Software/System)
SAS (Statistical Analysis Software/System) SAS Analytics:- Class Room: Training Fee & Duration : 23K & 3 Months Online: Training Fee & Duration : 25K & 3 Months Learning SAS: Getting Started with SAS Basic
More informationBasic SQL Processing Prepared by Destiny Corporation
Basic SQL Processing Prepared by Destiny Corporation SQLStatements PROC SQl consists often statements: from saved.computeg- l.select 2.vAlIDATE 3.DESCRIBE 4.CREATE S.DROP a.update 7.INSERT B.DElETE 9.ALTER
More information2. Don t forget semicolons and RUN statements The two most common programming errors.
Randy s SAS hints March 7, 2013 1. Always begin your programs with internal documentation. * ***************** * Program =test1, Randy Ellis, March 8, 2013 ***************; 2. Don t forget semicolons and
More informationCertkiller.A QA
Certkiller.A00-260.70.QA Number: A00-260 Passing Score: 800 Time Limit: 120 min File Version: 3.3 It is evident that study guide material is a victorious and is on the top in the exam tools market and
More information. NO MORE MERGE - Alternative Table Lookup Techniques Dana Rafiee, Destiny Corporation/DDISC Group Ltd. U.S., Wethersfield, CT
betfomilw tltlljri4ls. NO MORE MERGE - Alternative Table Lookup Techniques Dana Rafiee, Destiny Corporation/DDISC Group Ltd. U.S., Wethersfield, CT ABSTRACT This tutorial is designed to show you several
More information3. Data Tables & Data Management
3. Data Tables & Data Management In this lab, we will learn how to create and manage data tables for analysis. We work with a very simple example, so it is easy to see what the code does. In your own projects
More informationProgramming Beyond the Basics. Find() the power of Hash - How, Why and When to use the SAS Hash Object John Blackwell
Find() the power of Hash - How, Why and When to use the SAS Hash Object John Blackwell ABSTRACT The SAS hash object has come of age in SAS 9.2, giving the SAS programmer the ability to quickly do things
More informationFrom Manual to Automatic with Overdrive - Using SAS to Automate Report Generation Faron Kincheloe, Baylor University, Waco, TX
Paper 152-27 From Manual to Automatic with Overdrive - Using SAS to Automate Report Generation Faron Kincheloe, Baylor University, Waco, TX ABSTRACT This paper is a case study of how SAS products were
More informationContents. Overview How SAS processes programs Compilation phase Execution phase Debugging a DATA step Testing your programs
SAS Data Step Contents Overview How SAS processes programs Compilation phase Execution phase Debugging a DATA step Testing your programs 2 Overview Introduction This section teaches you what happens "behind
More informationUSING PROC SQL EFFECTIVELY WITH SAS DATA SETS JIM DEFOOR LOCKHEED FORT WORTH COMPANY
USING PROC SQL EFFECTIVELY WITH SAS DATA SETS JIM DEFOOR LOCKHEED FORT WORTH COMPANY INTRODUCTION This paper is a beginning tutorial on reading and reporting Indexed SAS Data Sets with PROC SQL. Its examples
More information9 Ways to Join Two Datasets David Franklin, Independent Consultant, New Hampshire, USA
9 Ways to Join Two Datasets David Franklin, Independent Consultant, New Hampshire, USA ABSTRACT Joining or merging data is one of the fundamental actions carried out when manipulating data to bring it
More informationLongitudinal Employer - Household Dynamics. Internal document No. IP-LEHD-BRB LEHD Business Register Bridge Technical documentation
Longitudinal Employer - Household Dynamics Internal document No. IP-LEHD-BRB-1.1.10 LEHD Business Register Bridge Technical documentation Code version : 1.1.10 Author : Hyowook Chiang, Kristin Sandusky,
More information12. Combining SAS datasets. GIORGIO RUSSOLILLO - Cours de prépara)on à la cer)fica)on SAS «Base Programming» 269
12. Combining SAS datasets 269 Appending datasets in different situa)ons PROC PRINT DATA=Lib9_3.emps; PROC PRINT DATA=Lib9_3.emps2008; PROC PRINT DATA=Lib9_3.emps2009; PROC PRINT DATA=Lib9_3.emps2010;
More informationFacilitate Statistical Analysis with Automatic Collapsing of Small Size Strata
PO23 Facilitate Statistical Analysis with Automatic Collapsing of Small Size Strata Sunil Gupta, Linfeng Xu, Quintiles, Inc., Thousand Oaks, CA ABSTRACT Often in clinical studies, even after great efforts
More informationTired of CALL EXECUTE? Try DOSUBL
ABSTRACT SESUG Paper BB-132-2017 Tired of CALL EXECUTE? Try DOSUBL Jueru Fan, PPD, Morrisville, NC DOSUBL was first introduced as a function in SAS V9.3. It enables the immediate execution of SAS code
More informationPaper Haven't I Seen You Before? An Application of DATA Step HASH for Efficient Complex Event Associations. John Schmitz, Luminare Data LLC
Paper 1331-2017 Haven't I Seen You Before? An Application of DATA Step HASH for Efficient Complex Event Associations ABSTRACT John Schmitz, Luminare Data LLC Data processing can sometimes require complex
More informationLocking SAS Data Objects
59 CHAPTER 5 Locking SAS Data Objects Introduction 59 Audience 60 About the SAS Data Hierarchy and Locking 60 The SAS Data Hierarchy 60 How SAS Data Objects Are Accessed and Used 61 Types of Locks 62 Locking
More informationCC13 An Automatic Process to Compare Files. Simon Lin, Merck & Co., Inc., Rahway, NJ Huei-Ling Chen, Merck & Co., Inc., Rahway, NJ
CC13 An Automatic Process to Compare Files Simon Lin, Merck & Co., Inc., Rahway, NJ Huei-Ling Chen, Merck & Co., Inc., Rahway, NJ ABSTRACT Comparing different versions of output files is often performed
More informationIntroduction to PROC SQL
Introduction to PROC SQL Steven First, Systems Seminar Consultants, Madison, WI ABSTRACT PROC SQL is a powerful Base SAS Procedure that combines the functionality of DATA and PROC steps into a single step.
More informationFrom An Introduction to SAS University Edition. Full book available for purchase here.
From An Introduction to SAS University Edition. Full book available for purchase here. Contents List of Programs... xi About This Book... xvii About the Author... xxi Acknowledgments... xxiii Part 1: Getting
More informationInterleaving a Dataset with Itself: How and Why
cc002 Interleaving a Dataset with Itself: How and Why Howard Schreier, U.S. Dept. of Commerce, Washington DC ABSTRACT When two or more SAS datasets are combined by means of a SET statement and an accompanying
More information%Addval: A SAS Macro Which Completes the Cartesian Product of Dataset Observations for All Values of a Selected Set of Variables
%Addval: A SAS Macro Which Completes the Cartesian Product of Dataset Observations for All Values of a Selected Set of Variables Rich Schiefelbein, PRA International, Lenexa, KS ABSTRACT It is often useful
More informationAn Efficient Method to Create Titles for Multiple Clinical Reports Using Proc Format within A Do Loop Youying Yu, PharmaNet/i3, West Chester, Ohio
PharmaSUG 2012 - Paper CC12 An Efficient Method to Create Titles for Multiple Clinical Reports Using Proc Format within A Do Loop Youying Yu, PharmaNet/i3, West Chester, Ohio ABSTRACT Do you know how to
More informationFile Systems. ECE 650 Systems Programming & Engineering Duke University, Spring 2018
File Systems ECE 650 Systems Programming & Engineering Duke University, Spring 2018 File Systems Abstract the interaction with important I/O devices Secondary storage (e.g. hard disks, flash drives) i.e.
More informationTaming a Spreadsheet Importation Monster
SESUG 2013 Paper BtB-10 Taming a Spreadsheet Importation Monster Nat Wooding, J. Sargeant Reynolds Community College ABSTRACT As many programmers have learned to their chagrin, it can be easy to read Excel
More informationPaper DB2 table. For a simple read of a table, SQL and DATA step operate with similar efficiency.
Paper 76-28 Comparative Efficiency of SQL and Base Code When Reading from Database Tables and Existing Data Sets Steven Feder, Federal Reserve Board, Washington, D.C. ABSTRACT In this paper we compare
More informationCleaning Duplicate Observations on a Chessboard of Missing Values Mayrita Vitvitska, ClinOps, LLC, San Francisco, CA
Cleaning Duplicate Observations on a Chessboard of Missing Values Mayrita Vitvitska, ClinOps, LLC, San Francisco, CA ABSTRACT Removing duplicate observations from a data set is not as easy as it might
More informationPROCESSING LARGE SAS AND DB2 Fll..ES: CLOSE ENCOUNTERS OF THE COLOSSAL KIND
PROCESSING LARGE SAS AND DB2 Fll..ES: CLOSE ENCOUNTERS OF THE COLOSSAL KIND Judy Loren, ASG, Inc. Alan Dickson, ASG, Inc. Introduction Over the last few years, a number of papers have been presented at
More informationChecking for Duplicates Wendi L. Wright
Checking for Duplicates Wendi L. Wright ABSTRACT This introductory level paper demonstrates a quick way to find duplicates in a dataset (with both simple and complex keys). It discusses what to do when
More informationOracle Database 10g: Introduction to SQL
ORACLE UNIVERSITY CONTACT US: 00 9714 390 9000 Oracle Database 10g: Introduction to SQL Duration: 5 Days What you will learn This course offers students an introduction to Oracle Database 10g database
More informationAn Annotated Guide: The New 9.1, Free & Fast SPDE Data Engine Russ Lavery, Ardmore PA, Independent Contractor Ian Whitlock, Kennett Square PA
An Annotated Guide: The New 9.1, Free & Fast SPDE Data Engine Russ Lavery, Ardmore PA, Independent Contractor Ian Whitlock, Kennett Square PA ABSTRACT SAS has been working hard to decrease clock time to
More informationHash Objects for Everyone
SESUG 2015 Paper BB-83 Hash Objects for Everyone Jack Hall, OptumInsight ABSTRACT The introduction of Hash Objects into the SAS toolbag gives programmers a powerful way to improve performance, especially
More informationQMF: Query Management Facility
A A Report - Figure 7... 1:26 ADD Sessions - Ending a Table Editor... 5:5 Adding Rows to a Table... 5:1 Adding Comments to an SQL Query... 3:5 ALIGN... 4:16 Arithmetic in Queries... 3:17 Available Tables
More informationProgramming Gems that are worth learning SQL for! Pamela L. Reading, Rho, Inc., Chapel Hill, NC
Paper CC-05 Programming Gems that are worth learning SQL for! Pamela L. Reading, Rho, Inc., Chapel Hill, NC ABSTRACT For many SAS users, learning SQL syntax appears to be a significant effort with a low
More informationSAS Macro Technique for Embedding and Using Metadata in Web Pages. DataCeutics, Inc., Pottstown, PA
Paper AD11 SAS Macro Technique for Embedding and Using Metadata in Web Pages Paul Gilbert, Troy A. Ruth, Gregory T. Weber DataCeutics, Inc., Pottstown, PA ABSTRACT This paper will present a technique to
More informationSetting Up a New Project
112 Setting Up a New Project This section provides assistance in setting up your ATLAS.ti project in the most efficient manner and for maximum work productivity. Objectives Working with ATLAS.ti involves
More informationUsing an ICPSR set-up file to create a SAS dataset
Using an ICPSR set-up file to create a SAS dataset Name library and raw data files. From the Start menu, launch SAS, and in the Editor program, write the codes to create and name a folder in the SAS permanent
More informationABSTRACT DATA CLARIFCIATION FORM TRACKING ORACLE TABLE INTRODUCTION REVIEW QUALITY CHECKS
Efficient SAS Quality Checks: Unique Error Identification And Enhanced Data Management Analysis Jim Grudzinski, Biostatistics Manager Of SAS Programming Covance Periapproval Services Inc, Radnor, PA ABSTRACT
More informationPharmaSUG 2018 Paper AD-08 Parallel Processing Your Way to Faster Software and a Big Fat Bonus: Demonstrations in Base SAS. Troy Martin Hughes
PharmaSUG 2018 Paper AD-08 Parallel Processing Your Way to Faster Software and a Big Fat Bonus: Demonstrations in Base SAS ABSTRACT Troy Martin Hughes SAS software and especially extract-transform-load
More informationAmie Bissonett, inventiv Health Clinical, Minneapolis, MN
PharmaSUG 2013 - Paper TF12 Let s get SAS sy Amie Bissonett, inventiv Health Clinical, Minneapolis, MN ABSTRACT The SAS language has a plethora of procedures, data step statements, functions, and options
More informationPlanting Your Rows: Using SAS Formats to Make the Generation of Zero- Filled Rows in Tables Less Thorny
Planting Your Rows: Using SAS Formats to Make the Generation of Zero- Filled Rows in Tables Less Thorny Kathy Hardis Fraeman, United BioSource Corporation, Bethesda, MD ABSTRACT Often tables or summary
More informationCLINICAL DATA PROCESSING EFFICIENCY TECHNIQUES
CLINICAL DATA PROCESSING EFFICIENCY TECHNIQUES Denis Michel, Janssen Pharmaceutica Inc. ABSTRACT Efficiency may be defined as conservl;!iion of resources in performing I;! II;!Sk. Optimizing.the use of
More informationDavid Franklin Independent SAS Consultant TheProgramersCabin.com
Countdown of the Top 10 Ways to Merge Data Trivia The film The Poseidon Adventure is based on a real life event that involved the Queen Mary in 1942 the ship was hit by a 92 foot wave which listed the
More informationWant to Do a Better Job? - Select Appropriate Statistical Analysis in Healthcare Research
Want to Do a Better Job? - Select Appropriate Statistical Analysis in Healthcare Research Liping Huang, Center for Home Care Policy and Research, Visiting Nurse Service of New York, NY, NY ABSTRACT The
More informationA Tool to Compare Different Data Transfers Jun Wang, FMD K&L, Inc., Nanjing, China
PharmaSUG China 2018 Paper 64 A Tool to Compare Different Data Transfers Jun Wang, FMD K&L, Inc., Nanjing, China ABSTRACT For an ongoing study, especially for middle-large size studies, regular or irregular
More informationThe inner workings of the datastep. By Mathieu Gaouette Videotron
The inner workings of the datastep By Mathieu Gaouette Videotron Plan Introduction The base The base behind the scene Control in the datastep A side by side compare with Proc SQL Introduction Most of you
More informationPlease Don't Lag Behind LAG!
Please Don't Lag Behind LAG! Anjan Matlapudi and J. Daniel Knapp Pharmacy Informatics and Finance PerformRx, The Next Generation PBM 200 Stevens Drive, Philadelphia, PA 19113 ABSTRACT A programmer may
More information