SDMX training. How to set up an SDMX architecture for reporting and dissemination October Francesco Rizzo

Transcription

1 SDMX training How to set up an SDMX architecture for reporting and dissemination October 2014 Francesco Rizzo Hands-on session

2 Mapping Assistant workflow STEP 0 MSDB Connection DDB Connection STEP 1 Load SDMX Structures (CategoryScheme, DataFlow, DSD) STEP 2 Map local database schema (Dataset) STEP 3 Map local Concepts to SDMX ones (Mapping Set) STEP 4 Map local Codes to SDMX ones (Transcoding)

3 Mapping Assistant Dataset functionalities STEP 0 MSDB Connection DDB Connection STEP 1 Load SDMX Structures (CategoryScheme, DataFlow, DSD) STEP 2 Map local database schema (Dataset) STEP 3 Map local Concepts to SDMX ones (Mapping Set) STEP 4 Map local Codes to SDMX ones (Transcoding) Query Editor Custom Query.Stat Editor DB Tables.STAT Data provider

4 STEP 0 MSDB connection Database type: SqlServer Server: localhost\sqlexpress Database: MSDB_SDMXEXPGROUP_OCTOBER2014 User Id: mauser Password: mauser

5 STEP 0 DDB connection Connection Name: CONN_DDB_SDMXEXPGROUP Database type: SqlServer Server: localhost\sqlexpress Database: DDB_SDMXEXPGROUP_OCTOBER2014 User Id: mareader Password: mareader

6 Step 1 Load SDMX structural files

7 The Data Structure Definition (STS)

8 Load SDMX Structures To create SDMX artefacts interactively Category Schemes Concept Schemes Code Lists Hierarchical Code Lists KeyFamilies/DSDs

9 PAY ATTENTION - Loading SDMX Structure files The SDMX Information Model details some artefacts that represent Structural Metadata. They are described in XML Schemes and materialized in XML messages (SDMX-ML Structural messages) Each SDMX-ML structure message can contain one or more SDMX artefact. Mapping Assistant accepts Structural files containing one or more the following artefacts: CategoryScheme ConceptScheme Code Lists KeyFamiles/DataStructures HierarchicalCodeLists If a file (to be loaded through the MA) contains an artefact that refers to other artefacts, then those artefacts must be loaded before. Furthermore the referred artefacts must have the attribute isfinal= "true Only KeyFamiles/DataStructures with the attribute isfinal= "true can be mapped

10 Exercise load/create structural metadata 1. Import the Category Scheme file: ISTAT_TRAINING+IT1+2.0.xml 2. Under the Category IND add anew category [Add] [Add Category] Identifier: IND_TURN_ORD Language (Name): en (Turnover and orders), it (Fatturato ed ordinativi) 3. Import the file: STS_ISTAT\STS_MARKET+IT1+3.0.xml 4. Delete the DataFlow (STS_MARKET) created automatically under the category uncategorized 1. Create a new DataFlow under the Category IND_TURN_ORD, as follows: ID: IND_TURN_DNT AgencyID: IT1 Version: 1.0 Is Final: true Language (Name): en (Turnover D-N-T), it (Fatturato Nazionale-Estero-Totale) DSD: IT1+STS_MARKET+3.0

11 Step 2 Map local database schema (Dataset)

12 Dissemination Database (DDB) Turnover in industry

13 PAY ATTENTION - Create a Dataset through the «Query Editor» Datasets One or more DDB connections Columns that stores local statistical concepts TYPE A Change name in the Dataset columns Associate/delete descriptions

14 Exercise - Create a Dataset 1. Create a new Dataset (DS_IND_TUR_DNT) through the Query Editor on the connexion CONN_DDB_SDMXEXPGROUP 2. Select the following tables: VW_IND_TURN_D, VW_IND_TURN_N, VW_IND_TURN_T (TYPE A example) 3. Add to the table VW_IND_TURN_D a fixed column (MARKET) with the value NAZ 4. Order the column of the table VW_IND_TURN_D (bring the column MARKET to the 4 th position)

15 Exercise - Create a Dataset using the UNION 5. Click on the UNION button 6. Select all the columns of the all tables 7. Drag the head of the table VW_IND_TURN_D to the table VW_IND_TURN_N, and from the table VW_IND_TURN_N to the table VW_IND_TURN_T 8. Activate a «preview» to check that the UNION was created correctly

16 Exercise - Add a constraint 9. Add a Constraint: INDICATOR_TURN =IND_TURN 10. Activate a «preview» to check that the UNION was created correctly

17 Exercise Change names at the Dataset columns and Associate dimension descriptions to codes 9. Change the name and the description in the columns of the Dataset (e.g. name: ACTIVITY and description Economic Activity) 10. Associate dimension descriptions to codes

18 PAY ATTENTION: how to create a Dataset (Query Editor) - schema D

19 PAY ATTENTION: how to create a Dataset with «Custom Query»

20 Step 3 Map local concepts to SDMX one (Mapping set)

21 Create a MappingSet 1. Create a new Mapping Set (MS_IND_TURN_D-N-T) on Dataset DS_IND_TUR_DNT 2. Map the concepts as follows:

22 Step 4 Map local codes to SDMX one (Transcoding)

23 Exercise - Mapping of the Codes (Transcoding) 1. Transcoding as follows (except Market) Local concepts SDMX concepts - M FREQ IT IT GEO/REF_AREA N, W, Y N, W, Y ADJ/ADJUSTMENT IND_TURN TURN INDICATOR_TURN/STS_INDICATOR All All ACTIVITY/STS_ACTIVITY 1 STS_INSTITUTION 2005 STS_BASE_YEAR 1990M TIME_PERIOD.. - A OBS_STATUS P1M TIME_FORMAT MARKET

24 Exercise verify the transcoding 2. Check [View mapped Data] if the transcoding was performed correctly 3. Transcode the Market concept: NAZ, E, T D, E, T 4. Check [View mapped Data] again in order to verify that transcoding was performed correctly

25 Exercise - Set the DataFlow in produzione 5. Cliccare destro sopra il Data Flow da mettere in produzione e scegliere l opzione Set in Production

26 PAY ATTENTION: how to transcode through a script $Component_UNIT = $Column_BASE_PER + "100"

27 Exercise - stet up the TestClient

28 Exercise - use the TestClient Set the SDMX message Set the channel Only Dataflow In production are visualized Add filters Extract SDMX datasets Query WS To test

29 PAY ATTENTION how to set up the Web Service Set up the connection string within the WEG.CONFIG file (insert the suitable Mapping Store, UserID and Password)

30 Exercise - use the Web Service

31 PAY ATTENTION how to set up the Web Client Set up the web service end points within the WEG.CONFIG file

32 Exercise - use the Web Client

33 Exercise configure the TestClient to interact with the WS

34 Exercise use the TestClient to interact with the WS