Collection of Raw Data

Transcription

1 Collection of Raw Data TASK FORCE MEETING N 2 27 March 2000 ITEM 3.3 Doc. CoRD 009 IQML: A presentation for CoRD meeting No. 2 For information

2 IQML Overview Chris Nelson Dimension EDI

3 IQML Goal To improve the accuracy and timeliness of statistical data collection from enterprises and individuals whilst at the same time reducing the burden of statistical reporting on enterprises.

4 IQML Objectives To research the realities of metadata interchange and object standards in order to facilitate an active contribution to the metadata interchange standards by implementing, in software, chosen aspects of the international standards for metadata interchange (eg OIM from the metadata coalition, CWM from the Object Management Group), and carrying out trials in the area of intelligent questionnaires. To harness the emerging technologies to facilitate the automatation, user friendliness, and application integration, of raw data collection demands of collection agencies. To assist raw data collection agencies to build collection instruments in a variety of forms (e.g. CATI, CAPI) using a common metadata model which will facilitate the development of and access to a common metadata repository.

5 IQML Partners - management and software University of Edinburgh project co-ordinators questionnaire design tool Dimension EDI standards (OMG, ebxml, EEG6) architecture and adherence to standards metadata repository tool

6 IQML Partners - Software Comfact XML definition for questionnaires questionnaire visualisation tool data base extraction tool Desan survey management tool social survey experience

7 IQML Partners - Users CSO, Ireland collection of balance of payment data from financial institutions Statistics Norway collection of data from municipalities (direct from databases and via electronic forms) National Technical University of Athens assist Greek statistics office to implement

8 Goals CWMI RFP (OMG) Establish an industry standard specification for common warehouse metadata interchange Provide a generic mechanism that can be used to transfer a variety of warehouse metadata RFP A complete specification of the syntax and semantics needed to export/import warehouse metadata and the common warehouse metamodel: Metamodel Interchange format (in XMI) API (in IDL)

9 IQML - R&D Implement OMG Common Warehouse Metamodel (CWM) for questionnaires Influence CWM specification for questionnaires Interrogate CWM compliant databases (e.g. Oracle Express, IBM DB2) to discover structure and content and map to questionnaire structure

10 Common Warehouse Metamodel What it is Who supports it Problem domain Role on the IQML project How we can influence it

11 CWMI RFP Schedule RFP: 17/9/98 LOI: 15/4/99 Initial Submission: 17/9/99 Final Submission: 11/2/2000 Adoption: 6/2000 (Oslo OMG meeting) Commercial availability: 6/2000 or earlier

12 CWM Joint Submission Team IBM (Dan Chang, J. J. Daudenarde, Debra LaVergne) Unisys (Don Baisley, Sridhar Iyengar, Doug Tolbert) NCR (Bruce McLean, Vihelm Rosenqvist) Hyperion (John Poole) Oracle (David Last, David Mellor) UBS (Hans-Peter Hoidn, Jeffrey Peckham) Genesis (David Frankel, Phil Longden) Dimension EDI (Chris Nelson, Anders Tornqvist)

13 CWM Supporting Companies Deere Sun Microsystems Hewlett-Packard Company Data Access Technologies InLine Software Aonix Hitachi

14 Data Warehouse Lifecycle

15 Warehouse Process Warehouse Operation UML 1.3 (Foundation, Behavioral_Elements, Model_Management) XML Record- Oriented MDDB Relational Object- Oriented (UML) CWM Metamodel Warehouse Management Foundation Data Resource Data Analysis Base Transformation Business Nomenclature Data Mining OLAP Information Visualization Software Deployment Bus Info Type Map ping Data Types Expre ssion Keys Index

16 Relevance of CWM to Raw Data Collection Data Analysis Data Resource Object- Oriented (UML) Business Nomenclature Relational Information Visualization Transformation Record- Oriented Data Mining MDDB OLAP XML

17 CWM and Statistics Data Analysis OLAP Transformation Data Resource Relational XML

18 CWM Extensions CWM provides interchange of the common portions of warehouse tool meta models CWM should be used as the new foundation of a warehouse tool model Tool X Metamodel CWM Metamodel

19 Information Set - Relationships 0..1 InformationSet version : String / rule : Rule / infosetadmin : InfoSetAdministration 1 * 0..1 /namespace InfoSetAdministration priority : String acknowledgement : Boolean / date : InfoSetDate / responsibleparty : ResponsibleParty / document : Document /modelelement * * /modelelement 1..* +date +infosetadmin * InfoSetDate type : String format : String datetime : String /ownedelement * * Rule ruleexpression : ExpressionNode type : String * 0..1 Segment regionsequence : ProcedureExpression / rule : Rule /responsibleparty * ResponsibleParty (from Business * /document Document (from BusinessInformation) * /cube /cuberegion * SegmentRegion / rule : Rule / renderedobject : RenderedObject /modelelement 0..1 /renderedobject * RenderedObject (from InformationVisualization)

20 Information Set - Inheritance Schema (from Olap) Cube (from Olap) CubeRegion (from Olap) InformationSet Segment SegmentRegion ModelElement (from Core) InfoSetAdministration InfoSetDate Rule

21 Influencing the CWM Current CWM Specification Information Set (IS) is a CWM Extension Package IS is a sub set of OLAP, with rules and visualization Revision Task Force (RTF) of OMG will oversee changes will be formed after adoption by OMG (June 2000)

22 WWW Address for CWM /schedule/cwmi_rfp.html.

23 5th Framework IQML Project Designer Metadata Repository Metadata Manager API Common Warehouse Metamodel Bus XML Survey Manager API Extensible Metdata Interchange (XMI)XML Visualizer DataBase Extractor Possible Architecture Other Source e.g. relational database XMI Generator Other XML Source 3rd Party Products Other CWM compliant Source

24 IQML Trial 1 - mid 2000 Designer Metadata Manager Survey Manager Visualizer DataBase Extractor XMI Generator API Common Warehouse Metamodel Bus API Metadata Repository XML Extensible Metdata Interchange (XMI)XML Other Source e.g. relational database

25 IQML Trials 2/3 - late 2001 Designer Metadata Manager Survey Manager Visualizer DataBase Extractor XMI Generator API Common Warehouse Metamodel Bus API Metadata Repository XML Extensible Metdata Interchange (XMI)XML Other Source e.g. relational database

26 IQML Project - Experimentation Designer Metadata Manager Survey Manager Visualizer DataBase Extractor XMI Generator API Common Warehouse Metamodel Bus API Metadata Repository XML Extensible Metdata Interchange (XMI)XML Other Source e.g. relational database CWM compliant Source

27 IQML Project - Open Standards Designer Metadata Manager Survey Manager Visualizer DataBase Extractor XMI Generator 3rd Party Products API Common Warehouse Metamodel Bus API Metadata Repository XML Extensible Metdata Interchange (XMI)XML Other Source e.g. relational database Other XML Source CWM compliant Source

28 Other /Survey XML Specifications Codebook Bureau of Census BLS Statistics Canada Various Universities Other EU Projects