TUTORIAL - Importing and Editing. using the ESdat Environmental Database System. For use with ESDAT version 4

Transcription

1 TUTORIAL - Importing and Editing using the ESdat Environmental Database System For use with ESDAT version 4 (version 4 software updates available from

2 Function Tutorial - Importing and Editing Data Table of Contents Table of Contents... 1 Summary... 2 Pre-requisites... 2 Introduction... 2 The Import view... 3 Import Templates... 4 Import Options... 5 Exercise 1: Import Templates - Adding New data... 5 Exercise 2: Import Templates - Updating or Overwriting data... 7 Editing and Deleting... 9 Exercise 3: Editing data... 9 Exercise 4: Deleting Data... 9 Creating Drop-Downs Exercise 5: Creating a custom drop down menu Importing documents Exercise 5: Importing documents ing Import Templates Further Tutorials Support@ESDAT.com Web: Page 1

3 Summary Function Tutorial - Importing and Editing The tutorial will highlight the general principles of importing data into ESdat using templates. It will also illustrate methods for editing, overwriting and deleting data. Imports of specific data-types such as laboratory data, environmental standards, and historical chemistry are covered in the specific Topic Tutorials, for which this is a prerequisite. The tutorial applies to both Access and SQL versions of ESdat. The tutorial is conducted using the Access version of ESdat. It will use the Sample Hydrogeology and Contam database for any examples provided. Pre-requisites This tutorial assumes previous completion of Tutorial 1, and builds on Importing and Editing concepts in greater depth. Introduction ESdat users typically have a range of data that has been collected. Such information may include: Laboratory reports in ESdat format Chain of Custody forms indicating field duplicates, blanks and so on locations and co-ordinate information basemaps in a variety of GIS format field chemistry results borehole logs stratigraphic interpretations water levels (+ NAPL if applicable). This range of data must be imported into the ESdat in a consistent and coherent way to ensure data integrity is maintained. In addition, some data may not be in the correct format, data may be missing and/or need correcting. Importing, editing and deleting data are therefore key skills for efficient use of ESdat functionalities. Support@ESDAT.com Web: Page 2

4 The Import View ESdat provides a number of templates that can be used to bring data into ESdat databases. These are accessed through the Import view. The Import view provides a range of Import Options for importing data and/or information into ESdat. The blue area of the Import view provides buttons for generating blank Import Templates for the major types of data imported into ESdat: Locations Wells Chemistry Field or Historical Groundwater levels GW/NAPL data, and Timevariant TOC Borehole/Geological logging - Standard and Geotech. The All Data Tables section at the bottom of the blue areas provides a list of all data tables or reference tables (labelled as zref_ ) in the database. The brown area provides for importing from the PLog PDA software. The orange area of this view has templates to allow for the import of files as shown. Clicking on any of these options will present you with a dialogue box to allow you to choose the relevant files, with the exception of Documents which will provide you with a Blank Import Template for entering the details of one or more documents. Support@ESDAT.com Web: Page 3

5 Import Templates Clicking on any of the buttons in the blue area of the Import View will produce a blank Import Template. Import templates will vary in appearance, depending on the data type that has been selected. For example: The Locations Blank Import template looks like this; The Groundwater Levels GW/NAPL Blank Import template looks like this: Open blank Import Templates for Locations, Wells and Field Chemistry and review the Field Names (column headings). Note the changes in Field Names for different data types. Some Field Names are coloured with a double green colour. The combination of these double green cells is known as the Primary Key. Only one record can exist for each combination of values in the Primary Key. It is not possible to import two records with the same combination of values in these fields. In the Location example above, this is the case for the LocCode field, meaning any LocCode value can only ever be entered once (per ESdat Site/Project). In the GW/NAPL (ground water) example, there are several fields that combine to produce a unique combination for each record. In this case, this means that we can only have one record for a specific combination of Location, Well, Date/Time and measurement method. Measurement method is included as it is possible that a dip measurement and logger measurement may occur at the same time in the same well. A single green colouring means the data is required, although it may be populated by a default. Where it is populated by a default a comment is included with the field name (hover over the red triangle). The Measurement Method has a default. In order to further explore the use of templates to import data, we now need to use the example data provided for you during ESdat installation. Support@ESDAT.com Web: Page 4

6 Import Options In the Upload Data dialogue box, the following Record Actions will be available. The Add New choice will only import data for a new combination of primary key values (e.g. a location cannot be entered twice in the Location Table). Overwrite Existing instructs ESdat to find the existing record for that Primary Key and overwrite all other values for that record. It will overwrite any existing values with a blank. Update Existing operates similarly to Overwrite except that any blank cells in the Import Template are ignored, thus any existing values are not overwritten with a blank value. This is generally safer than Overwrite. Add New and Update Existing will add new records to ESdat by importing data for a new combination of primary key values AND search for an existing occurrence of the primary key values (if records are found with the same Primary Key), and then update the record with any new data found from the import template. (this option is only available for specific Import Templates, such as Environmental Standards) Require Parent Records means that higher Level (Parent) data must be present. In the above example, a valid Location must already be in the database. This option is useful if you are confident you have a complete list of Location Codes/Well Codes in the database, and want to ensure consistent naming. Create Missing Parent Records will create higher level data if it is missing. In the above example, a Location will be added if it is not already in the database. For more information on importing specific Data-Types see the tutorials on specific topics section. Exercise 1: Import Templates - Adding New data Open ESdat, and select the Sample Hydrogeology and Chem database. See here if you need guidance. To create an import template, click the Import View Type button in the View Type bar. The Import View Type screen will appear. Click on the GW/NAPL button (located to the right of the Groundwater Levels label. ESdat will open Excel and produce the following template: Support@ESDAT.com Web: Page 5

7 We are going to paste or enter data into this template and then import the data into ESdat. LocCode WellCode Date_Time Water_Elevation Units Comment BH08 A 8-May m Test Comment 1 BH08 A 8-Jun m Test Comment 2 Copy and Paste the above data to the relevant columns in the WaterLevel Import Template workbook you just created in ESdat. Arrange the data in the correct columns. Go to the Add Ins tab at the top of the page and click on this to open the Custom Toolbar Click on the Import button. Select Add New and then click Import from the Update Data dialogue box that opens. Data will now populate the database. Your template should now look the one below. The light green shading indicates rows that have been successfully imported. One data row will be imported to the database with no errors (shaded pale green), but the other (shaded purple) will cause an error. The comment in Cell A6 (indicated by the red triangle) will tell you that the record already exists (i.e. data already exists in the database for this Primary Key ). Support@ESDAT.com Web: Page 6

8 In the case of such errors, it is useful to compare the data you are trying to import with the data already in the database. To do this: Click on the icon in Excel. In the Upload Data dialogue box that appears, select the option checkbox Show comparison if the data being imported already exists, then click Import. A new (separate) worksheet will be created and shown (look for the sheet WaterLevel_Comparison at the bottom of the Excel worksheet). The new Import data will be shown to the left of the Existing data (some of the data will be off the screen; so scroll to the right to see all). Differences will be highlighted in purple (Note: some differences are fields which have been filled in with a default). Leave Excel open, but change back to the Water Level worksheet for the next exercise. Exercise 2: Import Templates - Updating or Overwriting data View the Water Level information in ESdat (Select WL1_WaterLevels_AHD). Double Click on BH08 in the LocCode column, and A in the WellCode Column, and click Apply Filter. Note that only the row that was imported from the Excel Import Template has the comment in the data. Support@ESDAT.com Web: Page 7

9 Change back to the Water Level Import Template in Excel: Click on the icon in Excel, change the Record Action option from Add New to Update Existing. Click Import. The line that was previously purple in the spreadsheet are now shaded light green, indicating successful import of data. Switch back to ESdat and click Apply Filter (to refresh the data). Confirm that both comments are now listed in the data in ESdat. Change back to the Water Level Import Template in Excel: Click on the icon in Excel and click Import with all defaults. Nothing will happen, as the data is shaded Green, meaning ESdat considers it is already successfully imported and can be ignored. Remove the Green shading using normal Excel tools and retry the import. You will receive the Record already Exists error, as this data is already imported. Delete the comments Test Comment 1 and Test Comment 2 from the data in the Import Template and import with Update Existing Switch back to ESdat and click Apply Filter (to refresh the data). Confirm that both comments are still listed in the data in ESdat. Update Existing will not replace existing data with blanks from Import Templates. Remove the Green shading using normal Excel tools and retry the import, this time using Overwrite Existing. Switch back to ESdat and click Apply Filter (to refresh the data). Confirm that both comments are now missing in the data in ESdat. Overwrite Existing will replace existing data with blanks from Import Templates. Close Excel without saving Support@ESDAT.com Web: Page 8

10 Editing and Deleting There will be times when you want or need, to edit or delete data from within ESdat. We will use the same data as above, with the same filter applied. Exercise 3: Editing data Still looking at the Water Level data. Click on the Allow Editing icon. The button should turn red to indicate you have entered Edit Mode. Select two of the Comments cells (in adjacent rows) and type a comment. Press Enter. Both rows will now have the new comment you have added. You can also select a whole column, and all data in that column (that is shown after filtering) will have the new value. Select the whole Comments column (just like in Excel, click the grey header with the column name) Type some text and press Enter. The whole column will update. Press Remove Filter You will notice only the rows that were previously filtered for will have the comment that was added. Exercise 4: Deleting Data You could also choose to delete whole rows here if you wished. Using the same data as above: If you are not in Edit Mode click on the Allow Editing icon. The button should turn red to indicate you have entered Edit Mode. Right click on the grey row-header on the far left of any row with a LocCode value of BH08. Select Delete Row. The row will be deleted. Note other data related to BH08 still remains (this is important for below) Change to the Locations Data-Type button Click on the Allow Editing icon. The button should turn red to indicate you have entered Edit Mode. Right click on the grey row-header on the far left of the row with a LocCode value of BH08. Select Delete Row. Support@ESDAT.com Web: Page 9

11 You will get an error message telling you that the record cannot be deleted because another table has related records. This means that other data relates to Location BH08, and hence it cannot be deleted from the Location table. This other data for BH08 needs to be explicitly deleted first before Location BH08 can be deleted. Creating Drop-Downs It may be useful to create drop-down data entry menus, in order to encourage (or enforce) data entry. The main way in which drop-downs can be populated is through use of the zref_drop_downs table. Other approaches are available (see the ESdat Help files). Exercise 5: Creating a custom drop down menu Imagine that you need to identify two purposes for monitoring LIC_Test (for licence testing) and COUNCIL_Test (for local council testing). In order to provide these options as drop down menu choices: Using the same Locations data as above: Right click on any cell in the Purpose column, and select Add to zref_drop_downs. A dialogue box Create a drop-down Value will open, prompting you to provide a drop-down value for Location.Purpose. Note the name must be limited to less than 30 characters. Type LIC_Test for the value. Click OK. Click on the same, or another cell in the Purpose column. This time, enter COUN_Test in the drop down value dialogue box. Now click on the Allow Editing icon (in the Data toolbar along the left edge of the screen). The button should turn red to indicate you have entered edit mode. Click on any cell in the Purpose Column and a drop down menu arrow will appear, click the dropdown, and the two choices you have entered should appear. The change you just made above will carry over to the Locations Import Template. To see this: Click on the Import view, and select the Location Blank Import Template. The Import template will be generated, this time showing the two options for Purpose that you have just created. Close the template without saving. Support@ESDAT.com Web: Page 10

12 Importing documents Documents can be stored and retrieved through ESdat databases. The location of these documents is at the users discretion. There are differences between the Access and SQL versions of ESdat for storing documents. In Access versions, only the path to the file is stored in the database. The SQL Server version has two columns/options for storing documents: Document_Path can be used to simply store the path to the document in ESdat Document can be used to make a copy of the document and store it in ESdat. ESdat can be configured (using Setup - Document Management) to store it either internally in the database, or in a centralised folder. Exercise 5: Importing documents Here the Access version will be used as an example. Open the Import view of the Sample Hydrogeology and Contam database in ESdat. Click on the Documents button in the orange Files section of the Import view. Complete the requited data in the template (as above) and click on Import under Add Ins. A template will appear. It will have the following Columns: LocCode: will offer a drop down menu for data input. Add or select location information as required. The LocCode doesn t need to be entered if the document doesn t relate to a specific location. Document Type: this could be photo, report etc. Description: e.g. Original Photo monitoring point etc. Document_Path: Double click and either navigate to the required file or enter the file name. Click the Import button (under Add Ins). Documents are viewed through ESdat by selecting the Documents output listed under the Locations Data-Type button. Double clicking on the Document Path (or Document in SQL Server) field will launch the document. ing Import Templates Blank Import templates, generated as Excel files, can be ed to other parties (field site offices, consultants offices etc.) to allow population with data. To do this, you can simply create a blank Import Template in Excel, and attach it to an . Alternatively, a Send Template to checkbox box is at the top right of the blue section of the Import page. If this is checked, when you click a Template it will be automatically Support@ESDAT.com Web: Page 11

13 attached to an ready to send. This only works if you use MS Outlook for ing, and if Outlook is configured to support external applications attaching files. Further Tutorials For a list of other tutorials available with ESdat see Support@ESDAT.com Web: Page 12