EDIT QUICK START USER GUIDE

Size: px

Start display at page:

Download "EDIT QUICK START USER GUIDE"

Rose Fleming
5 years ago
Views:

1 EDIT QUICK START USER GUIDE Version 1.0 May

2 TABLE OF CONTENTS 1 INTRODUCTION WHAT IS EDIT? HOW DOES EDIT WORK? OBJECTS IN EDIT DOMAINS FORMATS DATASETS PROGRAMS JOBS USING EDIT -FROM AN USER S PERSPECTIVE IMPORT DATASETS CREATE AND EXECUTE A JOB JOB RESULTS

1 INTRODUCTION 1.1 WHAT IS EDIT? EDIT is an IT tool which has been developed by Eurostat for data validation/edit across the European Statistical System (ESS), including Eurostat.

3 1 INTRODUCTION 1.1 WHAT IS EDIT? EDIT is an IT tool which has been developed by Eurostat for data validation/edit across the European Statistical System (ESS), including Eurostat. - What is data validation? An activity aimed at verifying whether the value of a data item comes from the given set of acceptable values. - What is data editing? The activity aimed at identifying erroneous entries and correcting them if necessary for example: the response is missing or incorrect. The use of the EDIT tool across the different statistical domains has significantly increased in Eurostat. The ESS validation strategy aims to place data validation at national level. Member States are currently using the EDIT IT application in a number of statistical domains on a voluntary basis. 1.2 HOW DOES EDIT WORK? EDIT can be installed locally on a PC by means of a downloadable package (standalone) 1. In order to gain access to EDIT and to obtain the package please drop an to ESTAT- VALIDATION@ec.europa.eu. Prospective EDIT users should be aware that ECAS 2 credentials are needed before access to the tool can be granted. Upon entering EDIT a number of actions are required in order to proceed to the validation process: 1) The user selects the file containing the dataset on which (s)he wants to run a set of validation rules. Upon its selection, the dataset will be imported into the system (EDIT); 2) The user then chooses a validation program (a set of validation rules) that will be applied to the imported dataset; 3) EDIT will make available the outcome of the validation process through the Job results; 4) Finally, if needed, the user will be able to export the validated dataset as well as the validation results themselves. Thus, from the user 3 perspective, the workflow of the EDIT tool can be summarised as follows: 1 Currently only the standalone version of EDIT is distributed to Member states Two other roles are foreseen in the application : Programmer and Administrator

4 2 OBJECTS IN EDIT 2.1 DOMAINS When a prospective user is granted access to EDIT (s)he is given permission to a specific statistical Domain. Let s take the example of Balance of Payments Statistics (BoP). This statistical domain contains all the relevant EDIT objects which are required to perform the validation, i.e.: Formats, Code Lists, Datasets, Programs and Jobs - but specific to the BoP Domain. Thus, any object used by EDIT for BoP statistics will not be available for other domains such as SILC (Social Inclusion and Living Conditions) or Tourism statistics. 2.2 FORMATS A Format is the object that contains information on the structure of the dataset. EDIT needs to know the number of variables and the type of information contained in each variable. Only when EDIT has this information it is able to import a dataset. Let s take the example of a dataset which contains three variables: Geo, Sex and Education Attainment. The format defined for this dataset could be as follows: 2.3 DATASETS A Dataset is a two dimensional table composed by rows and columns. Columns correspond to the variables present in the file and which are defined in the format. The rows correspond to the records present in the dataset. 2.4 PROGRAMS A Program contains a set of validation rules and functionalities. EDIT will make available a list of validation rules which can be selected by the user for starting the validation process. 2.5 JOBS Jobs can be launched by the EDIT user. The jobs connect the dataset imported to a set of validation rules (stored in the program). The validated file and error report related to the job are available to the user after the job has finished

5 - 5 -

3 USING EDIT -FROM AN USER S PERSPECTIVE After gaining access to the EDIT tool and downloading the specific standalone application, the user can start using EDIT.

Then he chooses a set of validation rules which are made available to him/her while specifying the dataset(s) to be validated, under the Jobs object. Then the validation job is executed.

6 3 USING EDIT -FROM AN USER S PERSPECTIVE After gaining access to the EDIT tool and downloading the specific standalone application, the user can start using EDIT. Based on the EDIT workflow, the user first selects and uploads the dataset which s(he) wants to validate by means of Import Datasets. Then he chooses a set of validation rules which are made available to him/her while specifying the dataset(s) to be validated, under the Jobs object. Then the validation job is executed. The results of the validation process will be made available within the same EDIT job. These steps are explained in more detail below: 3.1 IMPORT DATASETS In order to import a dataset, the user needs to click on Datasets and then Import Datasets. Then he fills in the information accordingly, knowing that: 1. In section File Import, the user selects the file that he wants to upload to EDIT. The user can manually set up the value for Max number of errors (by default is 100). This number will act as a threshold if errors are already found in the dataset prior importation; 2. In section File Properties, the user selects the input file format type. EDIT accepts GESMES, CSV, FLR or SDMX formats-type; 3. If the chosen format is CSV or FLR the user will be asked to enter the specific parameters applying to file; - 6 -

4. In section File Fields, (s)he selects the format -that contains information about the structure of the dataset- from the drop-down menu Dataset Format; 5.

7 4. In section File Fields, (s)he selects the format -that contains information about the structure of the dataset- from the drop-down menu Dataset Format; 5. In section Select Fields, (s)he selects which variables available in the chosen format apply to the dataset (s)he is about to import; 6. In section Dataset, (s)he can either Create a New dataset or Repopulate an Existing one. In the first case, (s)he needs to enter a name for the new dataset. Alternatively, (s)he would pick an Existing Dataset on the drop-down menu. The user is now ready to import the dataset by clicking on the Import button, located at the bottom right of the screen. There are three possible scenarios: FAILED it means that there are errors in the dataset the user is trying to import (i.e. total number of errors is larger than the limit stated on Max number of errors), preventing EDIT from importing the dataset. In order to check the error list (number of records dropped), the user should click on the yellow triangle in the Actions column. COMPLETED (with a yellow triangle in the Actions column) it means that only a part of the dataset was imported. In order to check the error list (number of records dropped), the user should click on the yellow triangle in the Actions column. COMPLETED it means that the entire dataset was imported. The imported dataset can be checked by clicking on the column. icon, located in the Actions - 7 -

3.2 CREATE AND EXECUTE A JOB After the user launches a job, EDIT will run a set of validation/imputation rules and/or apply data file operations to a specific dataset.

The Program Search section displays a search function which allows filtering of available programs.

8 3.2 CREATE AND EXECUTE A JOB After the user launches a job, EDIT will run a set of validation/imputation rules and/or apply data file operations to a specific dataset. In order to browse the list of available programs, the user should click on Jobs and then Search Jobs. The Program Search section displays a search function which allows filtering of available programs. The Program List section shows the list of programs according to the search criteria (if defined) or displays the full list if no filter parameters have been entered. Once the program to be used was chosen, (s)he will click on icon, located in the Actions column in order to create the job. Then EDIT displays the screen below on which (s)he is requested to: 1. Provide a Job Name and a Description (the latter is optional) in section General Information; 2. Select the dataset that was previously imported, under Dataset Instance; 3. Click on Execute, located at the bottom right of the screen to start the process. EDIT will display the following screen where the user can monitor the validation process

When COMPLETED is displayed in the Status column, the user is able to access the Job Results by clicking on the blue circle located in the Actions column. 3.

Several options are available on the Error Report screen, including the filter and export functions. The EDIT process ends here.

9 When COMPLETED is displayed in the Status column, the user is able to access the Job Results by clicking on the blue circle located in the Actions column. 3.3 JOB RESULTS Job Results contain information on the Error Reports, Input Datasets and Output Datasets. The user can click on the icon to display the Error Report. Several options are available on the Error Report screen, including the filter and export functions. The EDIT process ends here. For security reasons data cannot be modified manually in the system. Whenever errors are found the user will have to modify the corresponding values in the original dataset and load it again in order to reinitiate the validation process

EDIT 2014 Users Manual

EUROPEAN COMMISSION EUROSTAT Directorate B: Methodology; corporate statistical and IT Services Unit B-3: IT for statistical production EDIT 2014 Users Manual Date: 03.12.2014 Version: 01.20 Commission