Tips & Tricks Data Entry Tl Hw t imprt files frm Excel r Access int the DET 2 This dcument explains hw data prepared in ther prgrams (e.g. Excel, Oracle, Access,...) can be imprted t the Data Entry Tl. It is written based n the experience n the DET versin 3.3 and will be adapted accrding new versins. Yu might need t cntact yur IT department in rder t help yu. 0. General verview The DET writes and reads all price bservatins in XML files. XML files are nthing but text files that can be read in a web-brwser r text editr. The cntent f the files is structured accrding t definitins laid dwn in a "Key Family" (r "Data Set Definitin"). Thus, a database prgram needs the Key Family t understand the cntent f an XML file. The PPP Key Family is cmpatible with SDMX standards (see www.sdmx.rg). Because f this cmpatibility, it is pssible t cnvert XML files int CSV files (and back) that can be read by Excel r Access. It is thus pssible t prepare price bservatins in a CSV file and then cnvert this int a XML file that can be imprted int the DET, prvided the CSV file fllws the Key Family definitins. This dcument prvides yu with an verview f hw this can be managed. Here belw yu see the verview f hw files can be exchanged between the DET and ther databases. A tl named "SDMX cnverter" can be fund n CIRCA (see annex A) and is used fr the cnversin f the XML file t the CSV file (and vice versa). T use this cnvertr, yu als need t have the Key Family file (KF) and a header file (HF). Bth files are available in the zipfile n the DET hmepage. The cntent f the zip-file is displayed belw. The HF-file is called "header.prp" and the KFfile is called "CGS_Cuntry_Structure_v2.5.xml" The diagram n the next page describes the whle cycle f starting and ending at the DET with the CSV file as an in between stage, in which this CSV file acts as the gateway t yur database(s). Obviusly, nce yu have created the CSV file and yu are familiar with its frmat, yu can skip this step fr future uplads. ESTAT-C6-PPP-DB@ec.eurpa.eu 1/7
DET XML XML KF SDMX cnverter SDMX cnverter KF HF CSV Access Excel Other DB 1. Exprt ut f the Data Entry tl G t the Data Entry tl, imprt yur SUA-file and fill in (at least) 1 cmplete bservatin as an example. G t "Dataset" select "Exprt" and save yur file (e.g. Exprt.xml). Please ntice that the DET adds autmatically a prefix t the file name that cntains the abbreviatin and the year f the survey, e.g. PERAP2009Exprt.xml. If a file with the same name already exists the prefix is nt added. ESTAT-C6-PPP-DB@ec.eurpa.eu 2/7
2. Use the SDMX-cnverter t cnvert t CSV Open the SDMX-cnverter (Annex A) and fill in the fllwing fields : Input : select the input file (PREAP2009Exprt.xml, see step 1) and the frmat (CROSS SDMX) Output : select the utput file (e.g. Exprt.csv) and the frmat (CSV) Use Registry : Select "False" DSDFilename : This is the KF-file in the zip-file n the DET hmepage, named "CGS_Cuntry_Structure_v2.5.xml" Click n the "Cnvert"-buttn ESTAT-C6-PPP-DB@ec.eurpa.eu 3/7
Yu'll see the fllwing message when the cnversin was successful : 3. Fill the CSV with the data in yur wn DB's Since yu have the CSV-file, yu als have its frmat (structure definitin). Yu can easily imprt this file in e.g. MS Access and fill it with yur wn bservatins. If yu use Access r Excel fr data entry, then the best way wuld be t prgram a macr that exprts yur data in the frmat f the CSV file. The cmplete definitin can be fund in Annex B. Pay attentin t clumn 18 (VAT rate) which shuld be defined as a field with duble precisin. Pay als attentin t the fllwing rules : Dn't add clumns (e.g. dn't add a primary key) Dn't use qutes (' r ") t define text fields Each line in yur CSV file shuld end with a semi cln (;) Field delimiter = semi cln (;) The delimiter fr cncatenated fields (e.g; Specify) is a pipe ( ) Exprt yur CSV-file nce yu have filled it with the data f yur wn DB's, e.g. Exprt_DB.csv 4. Use the SDMX-cnverter t cnvert t SDMX Open the SDMX-cnverter again (Annex A) and fill in the fllwing fields : Input : select the input file (Exprt_DB.csv) and the frmat (CSV) Output : select the utput file (e.g. SDMXData_db.xml) and the frmat (CROSS SDMX) Use Registry : Select "False" DSDFilename : The same as in step 2 (CGS_Cuntry_Structure_v2.5.xml) Additinally, yu need t specify the fllwing : Input Parameters : this file (header.prp) can be fund in the zip-file n the DET Hmepage : Check if the CSV delimiter is filled in crrectly (;) ESTAT-C6-PPP-DB@ec.eurpa.eu 4/7
Prefix f the namespace : cgs Uncheck the default namespace Namespace : urn:sdmx:rg.sdmx.infmdel.keyfamily.keyfamily=estat:ppp_cgs click n "Cnvert" Again, yu'll see the fllwing message when the cnversin was successful : ESTAT-C6-PPP-DB@ec.eurpa.eu 5/7
5. Re-imprt the SDMX file in the DET Yu can easily imprt the created SDMX file int the DET. Yu just need t use the feature Imprt in the menu Dataset. 6. Gd t knw Dn't frget t update yur wn Survey Details in the DET n clicking "Settings" > "Cntact Infrmatin" Pay attentin that each bservatin in the CSV-file has 2 recrds : ne fr the measure "Price" and ne fr the measure "Quantity". Analyze the CSV file very carefully befre updating/appending data. Yu can als change the mapping f the clumns in the SDMX-cnverter and als the delimiter. Hwever, it might be easier t create in yur wn DB a query which matches the clumns perfectly. ESTAT-C6-PPP-DB@ec.eurpa.eu 6/7
ANNEX A : SDMX-cnverter The SDMX-cnverter (Versin 2.5.0) and dcumentatin can be fund n the fllwing CIRCA link here belw. http://circa.eurpa.eu/public/irc/dsis/stne/library?l=/x-dis/tls/sdmx_cnverter&vm=detailed&sb=title ANNEX B : Descriptin f the CSV-file Example 1 clumn A Name f the DATASET SRVIC 2 clumn B Name f the COUNTRY IS 3 clumn C Item cde 11.03.14.1.01.aa 4 clumn D N f bservatin 1 5 clumn E P(rice) r Q(uantity) f the bservatin P 6 clumn F Year 2008 7 clumn G Value f clumn E 20 8 clumn H Year and mnt f bservatin (with hyphen inbetween) 2008-6 9 clumn I Currency EUR 10 clumn J Brand 11 clumn K Mdel 12 clumn L Shp Type (if clumn E = P, else "null") 13 clumn M Shp Identifier (if clumn E = P, else blank) 14 clumn N Specify : name f Specify parameter & "="&value. Cncatenated 15 clumn O Separatr used in clumn N 16 clumn P Cmments Cmment 1 NUMBER OF HOURS WORKED=2.5 TRAVEL COSTS=0 MATERIALS=125 17 clumn Q Cncatenatin f OTHER & PRICE COLLECTOR OTHER=Other 1 COLLECTOR=Price Cllectr 1 18 clumn R VAT : Rate r null 0.20 19 clumn S Representativity : TRUE r FALSE (r null) TRUE 20 clumn T Flag 21 clumn U Item Cmments Item Cmments 22 clumn V Cmments fr lcal use LOCALCOMMENTS=Cmments fr lcal use 23 clumn W Finalized (TRUE r blank) TRUE ESTAT-C6-PPP-DB@ec.eurpa.eu 7/7