PCG MONITOR. Costing Training Session 01 (Data Loader) Last Updated 24/04/2018. Page 1

Transcription

1 PCG MONITOR Costing Training Session 01 (Data Loader) Last Updated 24/04/2018 Page 1

2 Table of Contents Purpose... 3 Background... 4 System Overview... 4 The Data Loader Template and the PCG Tools Menu... 6 Data Source Requirements... 7 Input Sources... 8 Adding a new Source Amending an existing Source Copying an existing Source Remove an existing Source Basic mapping of a data source columns into a system column More Advanced mapping methods Changes to the underlying data structure Template Validation Creating your XML file Front-End Navigation Log In Accessing the Data Loader Module Load Existing Model Create New Model Delete Model Set Up Process APPENDIX 1 - Commonly used formulae LEFT/RIGHT SUBSTRING RTRIM LTRIM REPLACE CASE STATEMENT CONCAT CAST / CONVERSION Page 2

3 Purpose The purpose of this document is to set out all you need to know to be able to manage Stage 2 of a standard implementation. DATA LOADER. Stage COST LEDGER DATA LOADER Stage No Sub Task Description 1 Assemble Cost Ledger 2 Identify all data feeds (including overhead allocation support data) Configure Main Data loader template (DLT) Configure References Data loader LEDGER 3 Build rules to map on classifications and allocation codes from the Cost Ledger Use Journals & Splits where necessary OVERHEADS 4 For each Overhead Driver - set up an overhead definition in the Costing Ledger Template (CLT) using overhead tables where necessary ACTIVITY DEFINITION 5 For each Activity Driver - set up an activity definition in the Costing Activity Template (CAT) Use Data Organise where necessary MATCHING 6 Data Matching & Unmatched Reallocation Use Data Organise / Data Conversion where necessary ANALYSE COSTS 7 Setup Analyse Cost Configuration template This will cover The Data Loader Template (DLT) layout and the PCG Tools menu Data Source considerations Adding links to a new data source file and limiting what data you load Mapping the columns on the source file into the PCG Monitor columns Using formulae in the Mappings Template Validation Creating a Data Loader Model Uploading the template Loading Data Finding the cause of problems with your data We will not be covering the Local Validation Rules and Grouping which we will cover at a later session. Page 3

4 Background System Overview PCG Monitor is an all in one web-based system that handles all aspects of the costing process in one place without the need to combine several different systems into one package. There are 3 major elements to the process before we can analyse our costs. 1. Data Load 2. Activity 3. Ledger Each of these steps are supported by 3 dedicated Excel based rule templates that will be managed by the user. The 3 templates are:- 1. DLT Data Loader Template 2. CAT Costing Activity Template 3. CLT Costing Ledger Template Page 4

5 Page 5

6 The Data Loader Template and the PCG Tools Menu The Data Loader Template (DLT) is an excel file that will contain information about the files you plan to load. No actual data is stored in the template itself, it s a set of instructions that will get actioned when you process the template from the front end. You will be setting up and amending an XLS file. You will then export the XLS file and save it as an XML file. The XML file is what you be uploading into the software ready for processing. You will be supplied with a blank DLT_Master Template. Copy this to a working folder and change the name (Template files can be named anything and can contain spaces in the name, but it will make it easier if you maintain the 3-character template code prefix at the start of the file and avoid spaces in the name e.g. DLT_1718_M12_v01.xls) Open your renamed version of the DLT which at this stage will be an empty template. You will notice there are several sheets in this template, there is however only one sheet that you will be moving to directly and that sheet is SUMMARY_OF_NAME_MAPS. With the DLT open you should see the PCG Tools menu to the right of the regular Home tab in Excel. The PCG Tools menus contains all the buttons that you need to help you configure and amend your template. Page 6

7 Data Source Requirements Before we add any sources to you Data Loader Template, there are a few points to consider regarding the raw data files that we will be loading data from 1. You will be able to load data from the following file formats.txt (A text file saved with a txt file extension delimited with commas (,) or pipes ( ) ).CSV (A text file saved with a csv file extension delimited with a comma).xls (A legacy excel format usually designated as Microsoft Excel Worksheet).XLSX (An Excel file).mdb (A table in a Microsoft Access Database 2003 and earlier).accdb (A table in a Microsoft Access Database 2007+) 2. When loading from an Excel spreadsheet the Data needs to be in a table format this means Column Names must be in row 1 Columns that contain data must have a column name Avoid spaces at the end of column names. It is OK to have other data and cell content around the table you want to load Sub totals and Grand Totals can be managed if necessary but are best avoided where possible. 3. When loading from Excel, the file needs to be closed during process. 4. SQL Tables and views need to be accessible to the user that the system runs as, as well as the user assembling the template. Both users need to have read access. 5. Text Files, Access Databases and Excel files need to be located in a folder that is visible to the PCG Monitor system user. Ideally you would copy the files to be loaded onto a shared folder on the costing server itself. 6. Try to use source files that are complete i.e. do not exclude private patients or diagnostic tests performed as Direct Access or for external organisations. 7. Excel columns that contain mixed format need special attention. When Microsoft SQL server loads data from an Excel template it will make assumptions about the data type that exists in the column based on the entries in the 8 top rows. Generally, this does not cause any issues, but you will have a problem if the you have numeric and text in the same column and all numeric numbers are at sorted to appear at the top of the column. An example being maybe specialty. If you had a series of rows with specialties such as 100, 101, 300 etc and then had entries such as COM, RHB etc. If all the number type codes were at the top SQL Server would read it as a numeric column and the text type entries will be removed but won t fail the load. To avoid this, ensure in mixed format columns that text entries appear in the top rows. Page 7

8 Input Sources All Activity loaded into PCG Monitor will get classified (by default or by rule) into 1 of 3 types a. CORE (this will be the ultimate Cost Object such as an Inpatient Episode, Community Contact or Direct Access Imaging Referral) b. SUPPORT (this will be one of many records related to a core object and may represent things that happen to a cost object such as a ward stay, a theatre event, a pathology test etc) c. EXCLUDED (Data that will not attract any costs during the calculation process) When we load raw data into the Data Loader we load each file separately, but we group files together into what we call INPUT SOURCES. Input sources allow us to understand what type of data you are loading. The main purpose of Input Sources is to streamline the grouping process (see later), but it also allows us to initially identify whether a source will be a CORE, SUPPORT or an EXCLUDED record (i.e. data loaded for reference purposes only) All the PCG Monitor INPUTSOURCES are made up of a 4 Character code. The full list is CORE APCE - Admitted Patient Care OPAE - Non-Admitted Consultations AEAE - Emergency Medicine NRDE - Renal Dialysis Groupable Input Sources ACCE - Adult Critical Care PCCE - Paediatric Critical Care NCCE - Neonatal Critical Care OBAE All other Activity SUPPORT SUPP Support Data EXCLUDED REFE Reference Data Page 8

9 If you have Admitted Patient Care data, you would load it into the APCE INPUTSOURCE. Similarly, if the source represented Outpatient attendances then you would load this into OPAE etc. If anything doesn t specifically fit into the groupable input sources (and it s a core source) then use OBAE. All Support type sources should go into SUPP and reference type sources should feed into REFE. Page 9

10 Adding a new Source Step 1 is to identify the file you wish to load and set up a link to the file. From the PCG Tools menu, click Add Partial Source This will bring up a box that will guide you through all the elements to be added. a) Enter a name for your source without using spaces. b) Using the drop-down box, select the Input Source that you want to load the file into. c) Next, we need to specify the location of the data that we will be loading. There are 2 methods here. Page 10

11 i) For all file-based sources (e.g. Excel, Txt, csv and Access Databases), click the Browse To File button. This will to bring up a File Explorer box, from which you can navigate to the relevant location. When you locate the file, double click it or select it and click Open. Note: The File Explorer box will default to showing file type Text Files (*txt;*.csv). To see all files, you need to change this to All Files (*.*) you can also choose to view Excel files or Access files only. Page 11

12 ii) For SQL Server tables or views, you need to paste (or type) an address of the database preceded by a PCG key word of sqlsource: The address will need to be of the form server.database. The overall entry therefore will be sqlsource:server.database. You will need to press tab on your keyboard to register this address. d) With the file location in place, the next step will depend on what type of file you have connected to. i) TXT or CSV files require you to state an Enclosing Character and a separating character. An Enclosing Character is a character that exists at the start and end of a data element in a file. Some files will be supplied with a double quote enclosing character. If this is the case, select DOUBLE QUOTE in the Enclosing Char drop down, otherwise choose NONE. A Separating Character is how text files separate characters into different columns. PCG Monitor will currently accept either COMMA (,) or PIPE ( ) or TAB. ii) XLS or XLSX files require you to select the sheet from a drop-down list that contains the data you wish to load. iii) Microsoft Access Database tables require you to select the table name from a dropdown list. iv) SQL Server Tables or Views require you to select a table or view from a drop-down list e) Next, we have an option to Use Source YES/NO. Each source that you add to your template can be set so that it will not show as a source on the front end and therefore won t be able to be loaded. If you are adding a new source it is likely that you will want to set this to YES. Page 12

13 f) Split to Daily Records again this is a YES/NO choice. This option is designed to improve the process of matching Support Records to Core Records in the costing module where the support record covers a period that extends more than a day and therefore there is a potential that the support event relates to more than one core event. A typical use for this would be the Ward Stay source that we will want to match to Inpatient Episodes. In the example below, we have a hospital 4 day stay consisting of 2 inpatient episodes, but all 4 days were spent on one ward therefore generating a single ward stay record. When we come to setting up the matching process later, we would be forced to match the full ward stay record to either one of the inpatient episodes which would leave one of them overvalued and the other undervalued. Day 1 Day 2 Day 3 Day 4 Episode 1 Episode 2 Ward Stay on Ward X When you set up your Ward Stay source in your Data Loader template, if you select YES in the Split to Daily Records option, the Ward Stay record from the above example will get turned into 4 records on load as per the diagram below. This in turn will allow each daily segment to be matched as a separate entity and therefore assigned to the correct episode. Day 1 Day 2 Day 3 Day 4 Episode 1 Episode 2 Ward Stay Day 1 Ward Stay Day 2 Ward Stay Day 3 Ward Stay Day 4 g) The final part of the source set up is optional. You can add a SQL Criteria if there is a requirement for the data that you want to load to be filtered in some way. This may be where you are connecting to a data warehouse table that contains multiple years for example. The content of this must use SQL syntax and should be written as if you were writing a where statement. The steps required to action this are i. Click the Choose Subset of Data button. You will see the below. Page 13

14 ii. iii. Delete the blue highlighted instructions. Press tab on your keyboard to see a drop-down list of all the columns in your data source. iv. Start to type the column you are looking for to filter the drop-down list Page 14

15 v. Choose the column you want the criteria to apply to and press return. This will place the column in the white box. vi. You can now type the criteria that should apply as per the example below. Page 15

16 vii. You can then add further criteria by typing and / or then pressing tab again, selecting another column and continuing. Take care to consider the type of data that exists in the column that you are applying the criteria to determine whether you need to use single quotes or not. h) Once all elements are in place click Add Source to register the source. This will take you back to the SUMMARY_OF_NAME_MAPS sheet and you will see the entry you have created. As you build up the list of sources, they will all appear on this tab together. Page 16

17 Amending an existing Source If you need to amend a source that you have already added, click anywhere in the line for that source and then press Edit Partial Source from the PCG Tools menu to bring up the completed Entry box. With this in place, you can make changes to any setting or re-browse to a different file location. The only exception to this is Input Source. Once a source has been linked to an Input Source, this can t be changed. To do this you need to remove the old source entry and add it again. Once all entries are in place click Edit Source to register your new copied source. Copying an existing Source If you want to copy a source that you have already added. Click anywhere in the line of the source you want to copy and click Copy Partial Source from the PCG Tools Menu. The Entry box here, will give you a full replica of all the settings and you can again choose to amend any setting. You will have to change the Source Name as a minimum as all Source Names need to be unique. Once again you are unable to change the Input Source setting. Once all entries are in place click Copy Source to register your new copied source. Remove an existing Source If you want to remove a source that you have already added. Click anywhere in the line of the source you want to copy and click Remove Partial Source from the PCG Tools menu. Note: There is no undo functionality within the PCG templates. Please maintain strong version control and make new versions with new names of each template before making any substantial changes. Page 17

18 Basic mapping of a data source columns into a system column There are 2 parts to adding a new source to your template. The first is to provide the information to enable PCG Monitor to find the file itself which we dealt with above. The second step is to map the columns from the source table into relevant columns in the destination table (i.e. the PCG table) When you have added a new source on the SUMMARY_OF_NAME_MAPS sheet, you will notice that the Source Name is a blue hyper link. Click the blue hyper link to move to the column map sheet. To get back to the main SUMMARY_OF_NAME_MAPS sheet, click the Blue hyper link on the mapping sheet. If you have put more than one source into an INPUT SOURCE, you will see each of the sources on the same mapping sheet side by side. On the mapping sheet there are 3 fixed columns on the left (2 grey and one light blue) and a column that is headed by the Source Name that you clicked on. The PCG Monitor process takes Raw Data and loads it into an Input Source. The costing system will then pull all those Input Sources together into an Event table for costing purposes. Raw Input Source Event Page 18

19 The white column under the Source Name is where we will specify the columns in the raw data. The light blue column represents the columns in the Input Source that we need to map your raw columns to. The grey column A, represents the Event columns that the Input Source will flow into. In the main the light blue and the grey columns are the same but there are exceptions. As an example, you can see in the below image, TREATMENT_FUNCTION_CODE maps into 2 columns in EVENT CHARGE_TFC and TREATMENT_CODE. The other grey column ( C ) tells you the data type of the PCG Monitor columns. The task now is to add your raw data columns to the White column. The first thing to note is that all we need as a minimum is to put something in the Episode Start Date and Episode End Date columns. Note: In PCG Monitor the word Episode is generic it does not necessarily relate to Finished Consultant Episodes. Every file needs an Episode Start Date and an Episode End Date even if those 2 dates are the same If you are planning to use PCG Monitor for grouping, you will need to ensure you populate the relevant columns for each groupable input source as defined in the Reference Cost grouper manuals for the grouper you plan to run. An example being the 2017/18 Reference Cost Grouper Manual which can be found from the link below _reference_costs_grouper_ user_manualv1.0.pdf Certain fields are also required to meet CTP requirements as well but not having these should not prevent you from proceeding. The system will be able to cost and report anything. Other than that, you can put anything in any column subject to the data types. There is no one way of setting up a new source, but the best way is to work through each of the raw columns in order and find a suitable PCG Column. Note: The PCG Columns are not fixed. We are able to add and amend the structure of the Input Source tables and the Event table if necessary this includes changing the data types. From this tab you will have full access to the list of columns in your raw source. In the instructions below, we refer to Alt+F6 which is the way you access help across all the PCG Templates and Tab. When we say Alt+F6, we mean press the Alt key on your keyboard, and while keeping that key depressed, click the F6 button as highlighted below. Page 19

20 Where we say press Tab on your keyboard, it means the key highlight below The steps to add a column map are as follows 1) Enter the white cell in the column for your source on the row for the PCG column that you want to populate. In the example below, we are planning to put data into the PCG column called LOCAL_PATIENT_IDENTIFIER_1. A lot of these columns are named to follow the Data Dictionary, so you should be able to find what need. Remember that because it s an Excel spreadsheet you can use the standard find function within excel (i.e. Ctrl+F) Page 20

21 2) Press Alt+F6 to bring up a work box 3) As instructed at the top of this box, press Tab on your keyboard to access a drop-down list of all columns in your data source. 4) You can either expand this drop-down list in full by clicking the down arrow to the right of the drop-down box or start to type any part of the column you are looking for. As you start to type the list will reduce and you will see entries that match what you are typing Page 21

22 5) When you find the column you want, select it and press return once on your keyboard to add it to the working area. 6) Press Return on your keyboard again and this will place the selection into the cell on the template. 7) Now you can move on to the next column and repeat. Note that it is possible to populate a mapping cell without using the Alt+F6 function. If you know the column name you want to can simply type it in to the cell directly. Page 22

23 You can also use standard Excel copy and Paste. For example, if you are loading an Outpatient attendance data set, this will have an attendance date. As mentioned above we need to have a date in both Episode Start Date and Episode End date. For your Outpatient data set you will be mapping Attendance Date into both these PCG Columns. Once you have added the first date, you can simply copy and paste the column map into the second column. You can also manually change a cell that includes a mapping. More Advanced mapping methods The method outlined above will suffice for most cases. As you build up a template and attempt to load your sources you will come across reasons why you may need to manipulate the content in your source data slightly. For this we allow you to use a keyword rawsql: If this is placed at the start of a map cell then the system will run whatever is written after it. This means you can add any type of formula you need (as long as it is SQL!). There are only a few formulae you will generally need to use, and most are straightforward. To highlight how to do this, we can follow an example. We may have identified that in general one of your raw columns contains text that is less than 50 characters but there are a couple of records where the content is greater than 50-characters and you are mapping this into a PCG column that only takes up to 50 characters. This type of scenario will prevent the file from loading. We can address this by using a SQL function called LEFT. In SQL this helpfully has the same structure as Left in Excel, so most people should be familiar with this. Note: If you are unsure of the syntax of any SQL function, google is the best resource. Just type the function name followed by transact-sql into google and try to use the docs.microsoft.com pages. Page 23

24 Having read the Microsoft resource, we know now that to use the left function to take up to the first 50-characters from a column only the formula would be left(mycolumn,50) If we now wanted to apply this formula to our Local Patient ID example above, we would select the white cell again on the LOCAL_PATIENT_IDENTIFIER_1 line. Press Alt+F6 and we will be back in the working area. The easiest thing to do here is to delete the column entry that is in the work area so you are back to a blank sheet. Click the use rawsql button in the bottom left hand to add the keyword (note you can type the keyword yourself just be aware that it is case sensitive and must end with a colon ( : ) Now you can start to type your formula. In this case you would type left( Page 24

25 At this point the formula requires us to put the column name here. You can type this manually, but you can also use Tab to find the column from the list. If we press Tab now and start to type the column name we are looking for we see the below Select the column you need and press return once on your keyboard to add the column to your formula. You can then continue to finish typing the formula. Page 25

26 Now finally press return once more to place the completed formula in the cell. At this stage, if you need to amend the formula you can correct it directly in the cell or press Alt+F6 to get back to the work area. Changes to the underlying data structure All drop-down lists of table names are created at the point you set up a new source. If the underlying data structure of the table you are connected to changes after this (for example if you add a new column to your excel template or change the name of one of the columns) you need to refresh the template to update the field list. To do this simply click the Refresh columns button from the PCG Tools. Page 26

27 Template Validation The Data Loader Template can be validated for 2 things. 1) Does the source file exist? 2) Do the columns referred to in the map sheets exist in the source file? It should be noted that the syntax of any formulae placed after a rawsql: keyword will not be validated. Any issues with formulae will be highlighted during the upload process see later. Validation can be run using the PCG Tools menu. You can either validate just the mapping sheet you are on Or you can validate the whole of the template by expanding the Export & Validate menu using the small down arrow at the bottom left of the Export & Validate button and choosing Validate All If any validation errors are found, the System Info part of the PCG Tools menu will show a status of ERRORS! The word ERRORS in the menu is a hyper link that when clicked will take you to a sheet that lists all errors that exist in the template The Error summary will describe what the issue is and the Error Number column is itself a hyper link which if clicked will take you straight to the problem which will be highlighted in red Page 27

28 Creating your XML file Once you have got a template to a state to where you would like to run it through the system, you need to save the file as an XML. To do this simply press the Export button from PCG Tools If there are any unvalidated rules in your template, you will be warned of this, at which point you can opt to validate prior to export. You will be prompted in this way on every unvalidated sheet. To avoid this, click the Use this response for further unvalidated sheets box before you click the Validate Sheet and Continue button. After validation (or if the template is already validated) you will be prompted to save the file in the same location as the xls template, with the same name, but with a file type of XML. Just click Save. Page 28

29 Page 29

30 Front-End Navigation Access to the web front end (i.e. the part of the software that you will interact with) will be via Web Browser (either Chrome or Internet Explorer 11+) During installation you will have been given a web address that you can enter directly or save as a book mark. Log In The first screen that you will be presented with is the Log-In Screen. Your admin user will have set you up with a log-in name and initial password. Enter those details here and click login. Page 30

31 Accessing the Data Loader Module Once you have logged in you will see the Home Page, from here you will access the Data Loader module via the Data Management button. As you move your mouse to the Data Management Box, the box label changes to a single module access called Data Loader. Click this button and you will come to the screen below Page 31

32 Load Existing Model If you already have models that you have previously created, you can enter these by choosing Load Model. At which point you will be presented with a drop-down list of every Data Loader that exists on your server. Select the model you want to enter and click the Load button Create New Model To create a fresh new model, choose New Model option. The first screen you will see here can in most cases be ignored. The settings here allow you to save the backend database to a location other than the system default. To pass over this screen just click the Next button Page 32

33 At the next screen you are prompted to enter a Model name. The name needs to be unique, at least 5 characters long and cannot contain spaces. Standard practice is to maintain some sort of naming convention to keep things organised, such as YYYY_MXX_vXX (for example 1718_M03_v01). If you maintain a structured naming convention then as you start to enter your new name, you will see a dropdown list of similar names. You can then select one of these and amend the name as appropriate. Once you have a unique acceptable name, a green tick will appear. Click the Create Model button that you should now see. Page 33

34 Delete Model To remove a model that you no longer wish to keep, click on Delete Model. Again, you will see a drop-down list of all models that exists on your server, select the model you wish to remove and click Delete Model Set Up Once you have created a new model (or entered an existing model), you will see a series of tabs that you can access. The first one from the left being Set Up. Page 34

35 This tab has one main function and that is to browse and upload your DLT (Data Loader Template). You will browse to the xml version of your template. After browsing to the file and clicking Open, the Upload button will appear. Click this button to upload the template. Page 35

36 Process Having uploaded your DLT (Data Loader Template) you can now move to the Process tab to start the data load Process. Here you will see 3 sub tabs. Core / Support / Reference. Any files that you load via the Input Source SUPP will appear in the Support tab. All these files will by default be classified as Support. Files loaded via the Input Source REFE will appear in the Reference tab. All REFE files will by default be classified as Excluded. Any files loaded via any other Input Source (e.g. APCE / OPAE etc) will appear in the Core tab and will by default be classified as Core. Load a source To load a source, place a tick in the check box to the left of the source and click Process. This will start a process that will connect to the raw data and copy the contents to a PCG database in line with the details stored in the Data Loader template. Page 36

37 After the file has loaded successfully, you will see a green tick together with some record totals for the file and details of when the file was loaded. At this stage there is no further connection to the raw data. This means you will have created a copy of the data frozen at the point you loaded it. If the underlying raw data changes this will not change the data that is stored in the data loader. The costing system will source it s data from the Data Loader tables not the raw data. Page 37

38 The blue text next to each source are click-able hyper-links. Source will open a new tab giving details of the source file such as location and record numbers. View will open new tab that shows a grid view of the loaded data. Note that the functionality of this view is limited. To fully interrogate the data, it is advisable to load the source into a costing model and use the Activity Validation tab from there to inspect the data. Reload Load a source If at any stage the underlying data in the source data has changed and you wish to refresh the content stored in the data loader, simply tick the source that you want to refresh and press the process button again. This will remove the old data and completely replace it with the new content from the raw file. Remove data for a source If you wish to remove the contents of a source, then tick the source and press Reset. This will keep the source showing as a loadable option but there will be no details shown against the source and the data will be removed from the Data Loader database. Errors If there are any errors during a load for a particular source, instead of a green tick you will now see a red cross and depending on the type of error you may also get access to an Error link that will take you to a log file that will detail the errors that have been logged by the system. Page 38

39 You can do further checks on the cause of errors from the Log tab (see Source Validation section below). Source Validation From within the Log tab you can access a feature called Source Validation. The purpose of this is to pin point the cause of a source that is loading with an Error. Click on the Source Validation link as highlighted below. This will now show you a list of every source in your Data Loader model. Here you can tick one or more (or all) of the sources and click the Validate Button. This will run through a process of checking the data for the things that will generate an error on load. This could be loading a column of data into a PCG column that expects the content to be dates but a few of the dates are not legitimate dates. You may be loading a column into a 50-character field where a few of the records being loaded contain entries Page 39

40 of more than 50 characters. The validation will also highlight issues that have not necessarily create a load error. After running the validation, you will either see a green tick meaning all the checks have been passed or a red cross where at least 1 issue has been identified. Clicking the View Validation Report will give a comprehensive breakdown of all checks and issues for that source. Page 40

41 APPENDIX 1 - Commonly used formulae LEFT/RIGHT Left will truncate the contents of the field and take just a set number of characters from the LEFT. Right will truncate the contents of the field and take just a set number of characters from the RIGHT. LEFT ( character_expression, integer_expression ) RIGHT ( character_expression, integer_expression ) SUBSTRING This is the equivalent to the MID function in Excel. This will take a set number of characters starting from a given start point SUBSTRING ( expression,start, length ) RTRIM Returns a character string after truncating all trailing spaces RTRIM ( character_expression ) LTRIM Returns a character expression after it removes leading blanks LTRIM ( character_expression ) REPLACE Replaces all occurrences of a specified string value with another string value. REPLACE ( string_expression, string_pattern, string_replacement ) e.g. replace(mycol,, _ ) this will replace all spaces in a column with an under-score CASE STATEMENT This is the equivalent to an if statement in Excel. CASE WHEN Expression THEN result ELSE alternative_result END e.g. case when MYCOL like %ABC% then 1 else 0 end - this will return a 1 if the column contains ABC in any position otherwise it will return a zero CONCAT Returns a string that is the result of concatenating two or more string values CONCAT ( string_value1, string_value2 [, string_valuen ] ) e.g. CONCAT ( 'Happy ', 'Birthday ', 11, '/', '25' ) will return Happy Birthday 11/25. Page 41

42 CAST / CONVERSION Change the datatype of a column from one data type to another. Most useful for converting date formats. But has many other uses. Page 42