Spectrum Miner. Version 8.0. Quadstone Metadata Markup Language

Transcription

1 Spectrum Miner Version 8.0

2 Copyright 2017 Pitney Bowes Software Inc. All rights reserved. MapInfo and Group 1 Software are trademarks of Pitney Bowes Software Inc. All other marks and trademarks are property of their respective holders. USPS Notices Pitney Bowes Inc. holds a non-exclusive license to publish and sell ZIP + 4 databases on optical and magnetic media. The following trademarks are owned by the United States Postal Service: CASS, CASS Certified, DPV, elot, FASTforward, First-Class Mail, Intelligent Mail, LACS Link, NCOA Link, PAVE, PLANET Code, Postal Service, POSTNET, Post Office, RDI, Suite Link, United States Postal Service, Standard Mail, United States Post Office, USPS, ZIP Code, and ZIP + 4. This list is not exhaustive of the trademarks belonging to the Postal Service. Pitney Bowes Inc. is a non-exclusive licensee of USPS for NCOA Link processing. Prices for Pitney Bowes Software's products, options, and services are not established, controlled, or approved by USPS or United States Government. When utilizing RDI data to determine parcel-shipping costs, the business decision on which parcel delivery company to use is not made by the USPS or United States Government. Data Provider and Related Notices Data Products contained on this media and used within Pitney Bowes Software applications are protected by various trademarks and by one or more of the following copyrights: Copyright United States Postal Service. All rights reserved TomTom. All rights reserved. TomTom and the TomTom logo are registered trademarks of TomTom N.V HERE Fuente: INEGI (Instituto Nacional de Estadística y Geografía) Based upon electronic data National Land Survey Sweden. Copyright United States Census Bureau Copyright Nova Marketing Group, Inc.

3 Copyright Portions of this program are Copyright by Nova Marketing Group Inc. All Rights Reserved Copyright Second Decimal, LLC Copyright Canada Post Corporation This CD-ROM contains data from a compilation in which Canada Post Corporation is the copyright owner Claritas, Inc. The Geocode Address World data set contains data licensed from the GeoNames Project ( provided under the Creative Commons Attribution License ("Attribution License") located at Your use of the GeoNames data is governed by the terms of the Attribution License, and any conflict between your agreement with Pitney Bowes Software, Inc. and the Attribution License will be resolved in favor of the Attribution License solely as it relates to your use of the GeoNames data. ICU Notices Copyright International Business Machines Corporation and others. All rights reserved. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, provided that the above copyright notice(s) and this permission notice appear in all copies of the Software and that both the above copyright notice(s) and this permission notice appear in supporting documentation. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF THIRD PARTY RIGHTS. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. Except as contained in this notice, the name of a copyright holder shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Software without prior written authorization of the copyright holder. 3

4 Table of Contents 1 - Introduction Purpose 6 Who should read this guide 6 Related documentation Quadstone Metadata Markup Language (QMML) formats XML formats 8 QMML schema definition 8 Main element types 8 Document metadata 9 QMML variants QMML examples QMML variant examples 14

5 1 - Introduction In this section Purpose 6 Who should read this guide 6 Related documentation 6

6 Introduction Purpose This document provides additional reference information on (QMML). Who should read this guide This book is intended for Spectrum Miner Administrators and provides additional Spectrum Miner XML schema definition information. Related documentation Document Location Spectrum Miner 8.0 Administration Guide 6

7 2 - Quadstone Metadata Markup Language (QMML) formats In this section XML formats 8 QMML schema definition 8 Main element types 8 Document metadata 9 QMML variants 11

8 (QMML) formats XML formats Quadstone provides a single XML schema for describing representation of metadata for the Quadstone System. This is the format, or QMML. In the present version of the Quadstone System, only a subset of Quadstone file and interchange formats are represented as XML documents, but QMML provides a general framework to allow extension to cover future Quadstone XML representations, as they become available in future releases. The QMML framework describes how to build documents to act as representations of Quadstone metadata state. These documents can contain a lot of detailed information (tracking audit history, maintaining descriptions of relationships between different components, and so on). QMML schema definition The QMML schema definition is shipped as part of the qmml extension product, and can be found in the <smhome>\server\qs8.0\integration\schemas\qmmlcore.xsd file. The current version of the QMML schema definition is DRAFT3, so the required schema URI is Note that QMML is defined by this XML schema definition alone. There is no equivalent description provided as a DTD. The schema defines a qmml XML namespace, in which a number of main element types are defined. A valid QMML document must contain one of these top-level QMML tags, in the QMML namespace. Main element types All QMML documents take the form of a overall top-level element tag, from the QMML namespace defined in the QMML schema. QMML thus defines a family of related document types. The following top-level element types are available in the current version of the QMML schema definition. Future versions will extend to include additional elements, to allow more extensive representations of general Quadstone System metadata. 8

9 (QMML) formats The <qmml:rules> element tag is used for QMML documents that describe rules. These documents are used by the RealTime framework. The <qmml:x-fdl> element tag is used for QMML documents that provide easy interchange with Quadstone Field Derivation Language (FDL) files. The <qmml:expressions> element tag is used as an intermediate form for representing the structure implicit in Quadstone Field Derivation Language expressions. It is a useful building block for converting from FDL to query languages used by external third-party products. Document metadata The QMML formats provide mechanisms for tracking origin and version history of documents, and of individual components within documents. The detailed specification of exactly which components can carry document audit-trail metadata is contained within the QMML schema definition. Components that support document metadata may contain <info> elements to hold this information. These elements take the following form, with the detailed metadata encoded according to the Dublin Core standard via RFC2731. <info> <description> optional textual description </description> <comment> optional textual comment </comment> <history> <metadata type="creation modification state-change" <meta name="dc.element" content="content"/>... <meta name="qc.element" content="content"/>... </metadata> </history> </info> Metadata tags The creation metadata element is used for describing the initial creation of a document (or elements within a document). The modification metadata element is used for tracking subsequent modifications to the document (or to elements within it). The state-change metadata element is used for tracking changes of state, similar to a general modification, but where the change takes the form of a change to a single named property. A 9

10 (QMML) formats state-change element requires an additional property attribute, to specify the name of the property whose state has been changed, (together with QC.State.Previous and QC.State.Value meta elements). This is used in cases such as tracking the live state of a ruleset within a qmml:rules document. Meta tags The following standard Dublin Core <meta> tag encodings can be used: DC.Creator is used in a creation metadata-element to give the name (real name, or login account name) of the creator of a QMML document or component. DC.Description is used for short summary descriptions of QMML documents, in any of the metadata elements. A longer textual description can be used in the additional <description> element. DC.Date.Created is used to record the creation-date of a document or component. The ISO8601 standard is used for formatting of dates. DC.Date is used to record additional dates (for merge or modification history), also formatted according to the ISO8601 standard. DC.Source is used in some cases to record the original source of information. In addition, some QMML-specific extension <meta> tags are used: QC.User is used in any of the metadata elements to record the name (real name, or logic account name) of the user responsible for a creation, merge or modification event. It can duplicate the information already provided via use of the standard Dublin Core DC.Creator tag. QC.Host is used in any of the metadata elements to record the hostname of the computer on which the document was created or modified. QC.Application is used in any of the metadata elements to record the name of the Quadstone System application used to create or modify a document. QMML documents created via other applications may omit this. QC.Application.Operation is (optionally) used to specify a particular operation that was applied. It is used to disambiguate in cases where the QC.Application element does not on its own provide sufficient detail (such as with the qsqmmledit utility, which can be used for many different QMML manipulation operations). QC.Application.Version is (optionally) used to specify the version of the application used to modify the document. QC.State.Value is used to describe the current (new) state of a property within a state-change metadata element. QC.State.Previous is used to describe the previous (old) state of a property within a state-change metadata element. Note that all metadata (apart from detailed description and state-change information), is encoded in the Dublin Core RFC2731 HTML form of <meta> elements, wrapped up inside QMML <history> 10

11 (QMML) formats and <metadata> blocks. The Dublin Core RDF/XML encodings (Resource Description Framework using XML) are not currently used. QMML variants Individual QMML variants build structure out of the set of elements available in the QMML schema definition. Each variant is designed to be targeted for a particular purpose, and will use appropriate elements to represent the information required for that purpose. Some global underlying structure (such as document metadata support) is shared by all variants. The full specification of what elements are provided, where they may occur, what they may contain, and what attributes they take, is given in the QMML schema definition file. A variant is defined by its top-level tag. The following variants are currently supported: The qmml:rules document type QMML documents for representing rules may contain the following main internal tags: The <info> element is used to carry general document metadata. This use is just a special case of use of this element type within a number of different components, in order to carry tracking information. It is described in some detail in the section above. The <focus> element is used to describe data origins (usually for a model) in terms of a description of a particular Quadstone System focus file. It includes specific information such as the pathname of the.ftr file, the TML that was used to generate this focus (if built using TML) and audit trail tracking information. It does not include any internals information (such as the lists of fields). The <fields> element is used to describe lists of fields in a table (and, typically, a table that corresponds to a focus). It does not include any detailed analytic metadata just the name and type for each field, and the FDL derivation expression for those fields that are derived. The <functions> element is used to describe global FDL function state (corresponding to FDL functions that have been loaded in to a Decisionhouse session from a function library file). This is the background function state against which any FDL expressions built during that session are defined. The <rulesets> element is used to describe rulesets. Each ruleset consists of a named collection of rules. A rule is defined in terms of a result field, which corresponds exactly to a derived field (as a reference to an entry in the <fields> element). 11

12 (QMML) formats The qmml:x-fdl document type QMML documents for FDL interchange may contain the following main internal tags: The <fdl-function> element is used to define a single FDL function expression. It is used in QMML documents that represent FDL. This may appear multiple times. The <fdl-derivation> element is used to define a single FDL field-derivation expression. This may appear multiple times. The qmml:expressions document type QMML documents for interchange of FDL structure representation may contain the following main internal tags: The <functions> element is used to describe global FDL function state, in exactly the same way as for a <qmml:rules> document. The<variables> element is used to declare the list of field names used in the derivation (including both input fields and result fields). The<expression> element contains either a single expression tree or a list of expression trees, depending on whether the document represents a number of named derivations or a single unnamed derivation. A list is given by an <expressionlist> subelement, which may in turn contain <assign> subelements. The full description of how FDL structure is represented is given in the QMML schema definition file. The structure within an <expression> element can contain elements corresponding to all of the syntactic items in the FDL language, such as literal values, conditionals, expression lists, variable assignments, and so on. This structure is identical to the parsed format that can be added to a qmml:rules document using the qsqmmledit -parse utility. 12

13 3 - QMML examples In this section QMML variant examples 14

14 QMML examples QMML variant examples Here are some simple examples for each of the QMML variant formats. For ease of readability, the initial standard XML processing instruction: <?xml version="1.0" encoding="utf-8"?> has been omitted. Similarly, the QMML namespace declaration attributes have been omitted. A fully valid QMML document would include the following in the main element tag: <qmml:elementname xmlns=" xmlns:xsi=" xsi:schemalocation=" qmml.xsd"> qmml:rules Here is an example of a QMML document for representing rules: <qmml:rules> <info> <history> <metadata type="creation"> <meta name="qc.application" content="none"/> <meta name="qc.application.version" content="none"/> <meta name="dc.date.created" content=" t10:36:17"/> <meta name="qc.host" content="demohost"/> <meta name="qc.user" content="demouser"/> <meta name="dc.creator" content="demouser"/> </metadata> </history> </info> <focus name="finance_10000" platform="win32"> <focuspath>f:/quadstone/foci/finance_10000</focuspath> </focus> <fields> <field dhtype="string(6)" id="_1" name="gender" type="string"> <info> <history> <metadata type="creation"> <meta name="qc.application" content="decisionhouse"/> 14

15 QMML examples <meta name="qc.application.version" content="4.0c"/> <meta name="dc.date.created" content=" t10:36:17"/> </metadata> </history> </info> <fdl> <text> case Sex_MF eq "M": "Male"; Sex_MF eq "F": "Female"; default: null </text> </fdl> </field> <field dhtype="string(1)" id="_2" name="sex_mf" type="string"/> <field dhtype="real" id="_3" name="income" type="real"/> </fields> <rulesets> <ruleset id="_0" live="false" name="default"> <info> <history> <metadata type="creation"> <meta name="qc.application" content="none"/> <meta name="qc.application.version" content="none"/> <meta name="dc.date.created" content=" t10:36:17"/> <meta name="qc.host" content="demohost"/> <meta name="qc.user" content="demouser"/> <meta name="dc.creator" content="demouser"/> </metadata> </history> </info> <fieldlist> <fieldref idref="_1" name="gender" type="string"/> </fieldlist> </ruleset> </rulesets> </qmml:rules> qmml:x-fdl Here is the same example, given as an unnamed FDL expression for the derived field: <qmml:x-fdl> <fdl-derivation> case Sex_MF eq "M": "Male"; Sex_MF eq "F": "Female"; 15

16 QMML examples default: null </fdl-derivation> </qmml:x-fdl> qmml:expressions Here is the same example, given as an unnamed FDL expression tree: <qmml:expressions> <variables> <variable id="_1" name="sexmf" type="string"/> </variables> <expression type="string"> <choose source="case" type="string"> <when type="string"> <condition type="integer"> <binaryop op="eq" type="integer"> <expression type="string"> <fieldref idref="_1" name="sexmf" type="string"/> </expression> <expression type="string"> <stringliteral type="string"><![cdata[m]]></stringliteral> </expression> </binaryop> </condition> <expression type="string"> <stringliteral type="string"><![cdata[male]]></stringliteral> </expression> </when> <when type="string"> <condition type="integer"> <binaryop op="eq" type="integer"> <expression type="string"> <fieldref idref="_1" name="sexmf" type="string"/> </expression> <expression type="string"> <stringliteral type="string"><![cdata[f]]></stringliteral> </expression> </binaryop> </condition> <expression type="string"> <stringliteral type="string"><![cdata[female]]></stringliteral> </expression> </when> <otherwise type="string"> 16

17 QMML examples <nullliteral type="string"/> </otherwise> </choose> </expression> </qmml:expressions> 17

18 3001 Summer Street Stamford CT USA Pitney Bowes Software Inc. All rights reserved