DDI Manual. Instructions for Developing and Extending the DDI Specification

Similar documents
SIF Data Model Specification Development, Review, Approval and Versioning Processes

IVOA Document Standards Version 0.1

Administrative Guideline. SMPTE Metadata Registers Maintenance and Publication SMPTE AG 18:2017. Table of Contents

LOUGHBOROUGH UNIVERSITY RESEARCH OFFICE STANDARD OPERATING PROCEDURE. Loughborough University (LU) Research Office SOP 1027 LU

Stylus Studio Case Study: FIXML Working with Complex Message Sets Defined Using XML Schema

University of California, San Francisco Helen Diller Family Comprehensive Cancer Center Policy and Procedure. PRMS Amendment Submission Policy

Teiid Designer User Guide 7.5.0

Symbiant Tracker Auditee Guide. Version 4.4. Auditee Guide

Idealist2018 Project. Ideal-ist Partner Search System - Manual for Proposers

Standard Setting and Revision Procedure

Community Design Guidelines Document (CDG)

CPSC 444 Project Milestone III: Prototyping & Experiment Design Feb 6, 2018

Release Management Process and Implementation Guidelines

Report to the IMC EML Data Package Checks and the PASTA Quality Engine July 2012

Code Administration Code of Practice

BPMN Working Draft. 1. Introduction

ELECTRONIC SUBMISSION FRAMEWORK (ESF)

Beginning To Define ebxml Initial Draft

UBL Library Content Methodology

Exchange Network Schema Conformance Report Preparation and Review Process

PROJ 302. Project Report, Poster and Digest Guidelines. for Industrial Engineering Students. Summer 2017

Video Services Forum Rules of Procedure

Quark XML Author September 2016 Update for Platform with Business Documents

ISO/IEC JTC 1 N Replaces: ISO/IEC JTC 1 Information Technology

CGI Deliverables Approval and Maintenance Process

ENTERPRISE SYSTEMS APOLLO (ANU POLLING ONLINE) USER GUIDE

Development System (UL CSDS)

Document Shepherds: Doing it well

Using UML To Define XML Document Types

Overview of Sentence Order Reference Document Development Process

Certification Process. Version 1.0

ERP/CRM System Implementation Methodology

Government of Ontario IT Standard (GO ITS)

Paper for Consideration by CHRIS. Cooperation Agreement Between IHO and DGIWG

DCMI Abstract Model - DRAFT Update

Quark XML Author October 2017 Update for Platform with Business Documents

Frequently Asked Questions (FAQs) Relating to the SNIA s New IP Policy v4

Catherine Hosage Norman, Ph.D., RAC. January 11, 2012

RECOMMENDATION OF THE BOARD GOVERNANCE COMMITTEE (BGC) RECONSIDERATION REQUEST DECEMBER 2013

Annex C. Developing New Engine Oil Performance Standards for API Certification Mark. C.1 General. C.2 Auto/Oil Advisory Panel

ISO INTERNATIONAL STANDARD

The Internet Society. on behalf of. The IETF Administrative Oversight Committee. Request for Proposal. RFC Editor RFC Format CSS Design

Reviewers Guide on Clinical Trials

Government of Ontario IT Standard (GO ITS) GO-ITS Number 56.3 Information Modeling Standard

JCRB CMC-REVIEW WEB PAGE MANUAL TABLE OF CONTENTS

IN THE UNITED STATES DISTRICT COURT FOR THE DISTRICT OF COLUMBIA

RIM Document Editorial Tasks

XML Task Group TG1 Transportation Subcommittee (X12I) Monday, 1 June, 2009 Cincinnati, OH. 3 Disapproved with comment.

DTD MIGRATION TO W3C SCHEMA

Briefing Paper: developing the DOI Namespace

Request for Proposal for Technical Consulting Services

Improving drafting findings and recommendations. Refresher seminar for experienced GHG expert reviewers, 8 March 2017 Suvi Monni UNFCCC secretariat

DON XML Achieving Enterprise Interoperability

USP Guideline on Use of Accelerated Processes for Revisions to the Food Chemicals Codex 1

Notes for authors preparing technical guidelines for the IPCC Task Group on Data and Scenario Support for Impact and Climate Analysis (TGICA)

Final Project Report

The IDN Variant TLD Program: Updated Program Plan 23 August 2012

This handbook contains directions on using tools and resources in WebAccess at CSM.

Sourcing - How to Create a Negotiation

Quark XML Author October 2017 Update with Business Documents

Architecture and Standards Development Lifecycle

Customer Guidance For Requesting Changes to SNOMED CT

Financial Planning Institute of Southern Africa SETTING THE STANDARD. Continuous Professional Development (Cpd) Policy

Quark XML Author for FileNet 2.5 with BusDocs Guide

API 1509 Annex C Ballot to Establish Auto Oil Process. Instructions

RDA Steering Committee and RSC Working Group Chairs

A SAMPLE PAPER SHOWING THE FORMAT REQUIRED FOR YOUR CONTRIBUTION TO THE SAGEEP 2015 PROCEEDINGS. Abstract. Submission Procedure

Guidance Document. UNC Modification Proposals Guidance for Proposers

Examination Guidelines for Design (Provisional translation)

The Dublin Core Metadata Element Set

Meredith Lichtenstein Cone, MPH Manager, Surveillance and Informatics Program May 8, 2018

DESIGN AND EVALUATION OF A GENERIC METHOD FOR CREATING XML SCHEMA. 1. Introduction

GUIDE TO CERTIFICATION

The Independent Stream an Introduction

EXAM PREPARATION GUIDE

UNH-IOL NVMe Testing Service

Data Models: The Center of the Business Information Systems Universe

The United Nations Crime Trends Survey (UN-CTS) Michael Jandl Statistics and Surveys Section UNODC

Schema Change Request ASWG Change Management Process Draft Version 0.2

Request For Proposal ONWAA Website & E-Learn Portal

OhioLINK DMSC EAD Task Force Meeting Minutes April 12, Location: Ohio State University, Thompson Library: Columbus, Ohio

UNESCO, Division for Planning and Development of Education Systems, Section for Sector Policy Advice and ICT in Education (ED/PDE/PAD)

Request for Qualifications for Audit Services March 25, 2015

ATNP Configuration Control Board (CCB) Procedures Document

ISO TC46/SC11 Archives/records management

EXAM PREPARATION GUIDE

2017 ICANN-IETF MoU Supplemental Agreement Introduction

FACETs. Technical Report 05/19/2010

The Internet Society. on behalf of. The IETF Administrative Oversight Committee REQUEST FOR PROPOSALS. for

PUBLIC CONSULTATION ON EBA XBRL TAXONOMY V2.1 EBA/CP/2014/ March Consultation Paper

GUIDELINES FOR THE USE OF THE e-commerce WORKFLOW IN IMI

JISC PALS2 PROJECT: ONIX FOR LICENSING TERMS PHASE 2 (OLT2)

ISO/IEC INTERNATIONAL STANDARD. Information technology Software asset management Part 2: Software identification tag

NISO STS (Standards Tag Suite) Differences Between ISO STS 1.1 and NISO STS 1.0. Version 1 October 2017

Cyber Security Standards Drafting Team Update

19. Bulleted and Numbered Lists

GNSO Council Report to the ICANN Board Locking of a Domain Name subject to UDRP Proceedings PDP

Document Title Ingest Guide for University Electronic Records

Mn/DOT Market Research Reporting General Guidelines for Qualitative and Quantitative Market Research Reports Revised: August 2, 2011

Change Control Process for European ectd Standards Version 1.1. December 2006

Transcription:

DDI Manual Instructions for Developing and Extending the DDI Specification Issued 4 May 2004 Modified 15 August 2005

2

Table of Contents 1. Overview 5 2. The DDI Model 5 2.1. Conceptual Specification 5 2.2. Technical Implementations 5 3. Definition of Roles 6 3.1. Director 6 3.2. Architect 6 3.3. Implementer 6 3.4. Technical Group 6 3.4.1. Code Creation 7 3.4.2. Review Role 7 3.5. Expert Committee Members 7 3.6. Public 7 4. Definition of Revision Types 7 4.1. Minor 7 4.1.1. Validating (non-invalidating) 7 4.1.2. Invalidating 8 4.1.3. Bug Fix 8 4.2. Major 8 4.2.1. Structural Enhancement 8 4.2.2. Expansion of Coverage 8 4.2.3. Revision of Conceptual Specification 8 5. Process for Revision 9 5.1. Determining Which Process to Follow 9 5.2. Minor Revision Process 9 5.2.1. Timeline for Approval Process 9 5.2.2. Contents of Proposal 10 5.2.3. Review Process 10 5.3. Major Revision Process 11 5.3.1. Timeline for Approval Process 11 5.3.2. Preparing a Proposal 11 5.3.3. Contents of Proposal 12 5.3.4. Review Process 13 6. Appendices 14 6.1. Minor Revision Form 14 6.2. Major Revision Form 17 3

4

1 Overview Every proposal for a modification to the existing DDI standard must go through the process for either major or minor version revisions. The broad outline for processing major version revision is provided in the Bylaws of the DDI Alliance and a procedure for minor version revisions was prepared in January 2004. Both procedures include a period for public comment. The major version revision requires a minimum of seven months from submission to the Director to final approval. The minor version revision process can take as little a month to approve depending on the type of revision required. To be considered by the Alliance, a proposal (major or minor) must be sponsored by an Alliance Member Organization. The proposal must include a complete draft statement of intent, conceptual specification, and intended functionality. The sponsoring organization designates an individual who will act as an Architect for the proposal. The Architect serves as the contact person for any questions or comments from the Director, Expert Committee members, or the public. After receiving the proposal, the Director initiates the review process. The actual code creation will be handled by an Implementer, who is responsible for translating the specifications of the request into the proper code (DTD, Schema, etc.) and maintaining the integrity of the DDI structure. The purpose of the following guidelines is to provide a structure for the preparation of both major and minor change proposals for review. Having a uniform structure for presenting proposals will help the Alliance to review, understand, and provide constructive comments for proposals as they work their way through the process. This should make the lengthy preparation and review process flow more smoothly and reduce the need for additional clarifications. 2 The DDI Model When discussing the DDI model, there needs to be a clear distinction between the Conceptual Specification and the various Technical Implementations. 2.1 Conceptual Specification This is the structured format and substantive content of the DDI model. Previously, this information has appeared in the Tag Library Documentation and in the comments in the DTD. The Conceptual Specification provides the logical framework of how the DDI is structured, what it is attempting to accomplish, and the interrelationships among parts of the model. This is basically the intent of the DDI explained in a structured format. 2.2 Technical Implementation This is a schema derived from the Conceptual Specification to be utilized to maintain structured metadata on a specific platform or data format. Examples of this are (for XML) DTDs, W3C Schemas, and RELAXNG. Other plausible technologies include various relational and hierarchical database schemas. There may be a significant number of comparable ways to accomplish the same validation and enforcement of the DDI 5

Specification. There is a one-to-many mapping of the DDI Conceptual Specification to its many Technical Implementations. 3 Definition of Roles 3.1 Director The Director receives proposals for change, distributes them to the Expert Committee for comment, and directs the voting process at various stages of the review. The Director has the power to accept or reject the result of a vote, but must provide justification for any decision contrary to that of the Expert Committee. Upon the advice of the Technical Group, the Director may choose to forgo minor revision reviews and institute the change requested. This power of discretion is generally to be used only in instances in which the approval of the Expert Committee is clear in advance and delay in incorporating the change is unnecessary. 3.2 Architect A proposal for major changes to the DDI is prepared by one or more Alliance Member Organization usually working within the Substantive Content Working Group structure. In developing a proposal, the group names an Architect for the proposal. This is the individual who will be responsible for organizing and responding to comments and questions during the review process. This person should be someone with a thorough understanding of the proposal and the problem it is trying to address. The Architect works closely with the Implementer (described below) in the iterative process of writing code and preparing the proposal for submission to the Expert Committee. The Architect is the contact person for the proposed revision and works closely with the Director in moving the proposal through the review process. The Architect is responsible for maintaining the Comment Log that records the comments, questions, and decisions or dispositions during the course of review. 3.3 Implementer The Implementer is a member of the Structural Reform Group (SRG) who works with the Architect and the proposing content-oriented Working Group to render the proposed features in a way that does not violate the conceptual design or model development procedures. The Implementer is responsible for providing justification for how the model was rendered and for specifying how goals and objectives of the proposal are or are not met by the model. The Implementer will work iteratively with the Architect and the proposing Working Group until they have a model both agree on. 3.4 Technical Group The current Structural Reform Group (SRG) is considered to be the Technical Group identified in the Bylaws. It is desirable, therefore, that most members of the SRG have the knowledge and expertise to perform the roles outlined for the Technical Group. 6

3.4.1 Code Creation In order to retain consistency and control of the model development, the SRG is responsible for all code creation. The SRG will create code for minor revisions shortly after submission of the request to the Director. For a major revision, the Working Group developing a proposal will specify the changes in substantive terms following the Major Revision Process, Contents of Proposal: Part I (see below), and then work with the SRG to create the XML (or other) code. The Implementer will be the person responsible for developing the code, but that person will be able to draw on the resources of the entire Technical Group in fulfilling that role. In addition to code, the SRG will provide the explanatory items listed in Contents of Proposal: Part II. 3.4.2 Review Role In order to retain consistency and control of the model development, the SRG will review all proposals to modify the DDI. The SRG will prepare a recommendation to the Director on whether or not the proposal should be adopted. If they conclude that the proposal would violate the conceptual design of the DDI, they can recommend that the Director veto the proposal. However, early collaboration on major change proposals between the proposing substantive Working Group and the SRG should obviate the need to recommend that a proposal be vetoed. 3.5 Expert Committee Members The members of the Expert Committee are responsible for reviewing all proposals, seeking clarification, providing constructive criticism and comments, and voting on the acceptance, rejection, or delay of the proposal through the review process. 3.6 Public All major revisions have a period of public review. The comments of DDI users provide valuable perspective both in the application of the contents of any proposal and in determining the type of explanatory material that needs to accompany the change. 4 Definition of Revision Types 4.1 Minor Minor revisions are changes relating to enhancement of the current model or alterations of a small number of elements or attributes. These may include, but are not limited to: Bug Fixes errors in the Technical Implementation that cause processing errors for users Additions/subtractions to controlled vocabularies Additions/subtractions of a few elements or attributes to facilitate processing of metadata 4.1.1 Validating (non-invalidating) A validating change is one that will not break the validation of a DDI instance against a Technical Implementation. These are often referred to as non-invalidating changes. 7

Older instances of the DDI specification will parse against the revised implementation; new instances using the changed item will require some form of version note. 4.1.2 Invalidating An invalidating change is one that could break the validation of a DDI instance against a Technical Implementation. Older instances of the DDI specification will not parse against the new, revised implementation if the altered element or attribute was used in a way not consistent with the new structure. Invalidating changes will require instructions and/or tools that will: Identify XML instances that are not valid Provide a means for updating earlier XML instances to meet the new standard 4.1.3 Bug Fix A Bug Fix is a correction to a Technical Implementation that addresses an error that conflicts with the Conceptual Specification. 4.2 Major Major changes enhance or expand significantly on a current section of the DDI or provide new functionality or coverage. All changes to the Conceptual Specification are considered major revisions. Major revisions should address a class of data files and not be limited to correcting for specific implementation or processing issues experienced by a single user. Unless a change meets the specific criteria for the expedited Minor Revision Process, it is considered major and therefore needs to be processed through the full review process. Major revisions are not invalidating in all instances. Substantial changes that were not invalidating have been made to the DDI DTD in the past. 4.2.1 Structural Enhancement Structural enhancements offer no additional coverage to the DDI in terms of data file types or classes of studies. Structural enhancements recommend changes that will add depth or detail to an existing area, thereby improving users ability to mark up, describe, and/or process information contained in the XML instance. 4.2.2 Expansion of Coverage Proposals to expand the coverage of the DDI to a new data type, storage format, or class of material fall into this category. For example, Version 2.0 of the DDI included new sections to describe the relationship of data items found in most aggregate data files. This could also include new levels of description such as families of data files, two- and threedimensional storage formats, or related materials. 4.2.3 Revision of Conceptual Specification The Conceptual Specification covers both the description of what the DDI model should ideally address and the design rules for how that is implemented. These design rules relate to issues such as how elements and attributes are used, the use of namespaces, or global types. The Conceptual Specification outlines the universe of what DDI defines. Changes to that coverage constitute a revision of the Conceptual Specification. Any proposed revision that lies outside of the current conceptual model or violates its design 8

principles will require an evaluation of and revision to the conceptual model prior to implementation. 5 Process for Revision 5.1 Determining Which Process to Follow Use the following decision tree to determine whether the Minor or Major Revision Process applies. If in doubt, use the Major Revision Process. Is it a Bug Fix? no Is it limited to expanding a controlled vocabulary? yes yes Use MINOR Revision no Is it limited to adding/removing 3 or fewer attributes for existing elements? yes no Is it limited to adding/removing 3 or fewer elements to existing 2 nd or 3 rd level hierarchies? (those contained within the current major divisions of the hierarchy)? no yes Use MAJOR Revision 5.2 Minor Revision Process 5.2.1 Timeline for Approval Process The following is a minimum timeline for minor revisions in which the public review is used. Additional time required for the technical review could delay this process. Week 1: Review by Expert Committee Chair and Technical Group or XML Consultant 9

Week 2-3: Public review Week 4: Vote 5.2.2 Contents of Proposal The Minor Revision process uses a standard submission form found in Appendix 6.1 of this manual. The information covered by the form is listed below. While the submitting group may provide suggested coding changes, the official coding change will be provided by the Technical Group during its review with the Expert Committee Chair. Working Name of Proposal (for reference purposes and EZBoard or other discussion thread) Working Group/Initiator Alliance Member Sponsors (2 minimum) Architect (name and email) Date of Submission Type of Change Proposed (bug fix, validating, or invalidating, etc.) Description of the Revision Rationale for Change Additional Information (optional) Revised Section of Outline Hierarchy Revised Section of Tag Library 5.2.3 Review Process Submission: The Working Group sends the proposal to the Chair of the Expert Committee for review. Technical Review: The Chair conducts a Technical Review involving the Alliance Structural Reform Working Group and/or an external consultant. The purpose of this review is to evaluate the implications of the proposed change on various Technical Implementations of the DDI and its fit with the current Conceptual Specification. The Expert Committee Chair sends the proposal to the Alliance Director. Director Review: For some minor changes, the Director has the discretion to implement the change immediately after the Technical Review, if the SRG so recommends. Public Review: The Director sends the proposal via email to the Expert Committee and posts the proposal on the public site (ddialliance.org) and on the Expert Committee s private bulletin board for a period of two weeks. Public review and comment is solicited through the ddi-users listserv (Expert Committee members should join that group to monitor public feedback). Committee members may comment publicly or privately. Voting: After two weeks of review and comment, each Expert Committee member has two weeks to vote Yes or No. A Yes vote indicates that the validity and usefulness of the proposed modification have been demonstrated and that the specification should now be accepted as a part of the DDI standard. A No vote indicates the case has not yet been made for the proposed modification. A No vote must be accompanied with comments to explain the vote. For the specification to be approved, at least one-half of 10

the Expert Committee must vote and two-thirds of those voting must vote Yes. The Director will consider the votes and make an informed decision as to whether to accept the specification or to restart the process at some earlier stage. Change Implementation: If the proposal is approved by the vote of the members of the Expert Committee, by the end of the two-week voting period the Director will ordinarily begin the process of incorporating the proposal into the DDI specification. However, the Director has the right to veto the proposal with a full explanation for that decision to the members of the Expert Committee. That veto, in turn, can be overridden by a two-thirds vote of the Steering Committee. Publication: If the proposal is accepted, a revised DDI specification is posted on the public site as the new development version or, if a development version already exists, the change is incorporated. The approved change is then incorporated into the next scheduled release of a new production version of the DDI specification. 5.3 Major Revision Process 5.3.1 Timeline for Approval Process The time required for preparation of the proposal prior to submission to the Director varies depending on the complexity of the proposal and the resulting working model. Consultation with the Expert Committee Chair and the SRG can begin at any point in the preparation of the proposal, but will need to take place in sufficient time for the preparation of the working model by the SRG. The approval process takes a minimum of seven months once a proposal is submitted to the Director. Remember that proposals intended for inclusion in major scheduled versions of the DDI must have final approval at least three months prior to the version date in order to be incorporated in that version. For example, a proposal for inclusion in a version release scheduled for January 1 should be submitted to the Director no later than March 1 of the previous year, so that final approval is received by October 1 and the three-month period of technical implementation and testing can take place. Month 1: Trial Review and Call for Objections Month 2-3: Technical Review Month 4: Vote on passing to Public Review and Proof of Concept stage Month 5-6: Public Review and Proof of Concept Month 7: Vote by Expert Committee and Final Approval by Director 5.3.2 Preparing a Proposal In developing a description of the issue to be addressed and the goal to be achieved with a proposed enhancement, collect as many examples as possible/practical and consult with producers and users of the data as well as those who will need to use the resulting XML descriptions as a basis for running programs and analysis. Be sure to address the issues of classes of data files as opposed to solving specific problems for specific applications or situations. 11

Review the current DDI to clarify those elements and/or relationships it can define, those that would require enhancement to meet specific needs, and those that are missing and need to be added to the model. This is a good point to set up a contact person within the Structural Reform Group who can provide technical support including information on changes or enhancements that are currently in development, implications or limitations of various technical implementations of the DDI (DTD, Schema, etc.), and the big picture perspective. This person can also be helpful in keeping the proposal development group informed of conflicting or closely related developments in other groups. Upon completion of Part I (described below) or earlier, contact the SRG for the assignment of a person (Implementer) who will create the working model to be submitted as Part II of the proposal. This working model will be based on the information provided in Part I and will be developed in close consultation with the Architect in order to accurately reflect the fields and relationships presented in Part I. This is an iterative process and may be initiated prior to the completion of the final draft of Part I. The Implementer is responsible for rendering the proposed features in a way that does not violate the conceptual design or model development procedures. The Implementer is responsible for providing justification for how the model was rendered and identifying how goals/objectives of the proposal are or are not met by the model. 5.3.3 Contents of Proposal Part I: Prepared by the Sponsoring Working Group or Initiator The contents listed below should appear with the headings of the major bullets. This makes it easier for review and comparison by other members of the committee and public. Working Name of Proposal (for reference purposes and EZBoard or other discussion thread) Working Group/Initiator Alliance Member Sponsors (3 minimum) Architect (name and email) Date of Submission Summary o Class of data files being addressed o Brief problem statement o Brief verbal summary of solution (for example, increased use of structured language, additional elements/attributes within current structure, new modules, any elements/attributes that would become invalid if the proposal is implemented as proposed, etc.) Issue being addressed (this is an expansion of the problem statement -- in many cases bulleted lists with explanations may be easier for reviewers to understand quickly than extensive text) End result sought (this is critical for the technical review and proof of concept -- we need to be able to evaluate whether the solution does what it sets out to do) Rational for change (justify the approach to the problem) Use cases 12

Background information (may include listing and location of previous discussion threads -- it should be clear enough so that someone can follow the reasoning behind decisions presented in the proposal) Revision request (SRG spreadsheet Contact an Implementer if you need assistance with this) Part II: Prepared by Technical Group/SRG Implementer in cooperation with the Architect and Working Group/Initiator -- Based on the contents of Part I Implementer (name) Working Model including: o Schema code o DTD (if applicable) o UML model derived from code Justification for how the model was rendered Statement of how the goals/objectives of the proposal are or are not met by the working model 5.3.4 Review Process Role of the Architect The Architect is responsible for shepherding the proposal through the review process. Tasks include: Serve as Primary Contact for all questions Create and maintain the Comment Log, in which comments are received, logged, brought to the group (including the Implementer), discussed, and decided upon Add disposition information to the Comment Log, at which point an issue is closed (the same comment or issue cannot be raised twice) Role of the Implementer Participate in the discussion of comments received during review Prepare revised working models when needed to reflect a change in the original model based on the comments Role of Expert Committee Review materials provided Provide comments through the Comment Log in order to track both comments and disposition of comments When providing comments, specify whether you are requesting clarification or alteration of the model Over the full review process, consider the following questions o Is the description of the issue being addressed complete? Is something missing or too narrowly defined? o Does the goal address the issue as described in the proposal? o Is the proposal adequate to achieve the result described? Should the goal be revised or proposal expanded? o Is anything critical missing? o Does it conflict with the broad conceptual model of the DDI? 13

6 Appendices 6.1 Minor Revision Form -- (Writable template version accessible at: http://www.ddialliance.org/users/dtd/revision.form.zip) Working Name of the Proposal: Working Group/Initiator: Alliance Member Sponsors (2 minimum): Architect (name and email): Date of Submission: Type of Change Proposed (see definitions below): (Choose one type.) Change to the Conceptual Specification Change to a Technical Implementation Bug Fix Validation Status (Indicate if the change is validating or invalidating.) Validating Invalidating Description of Revision: 14

Rationale for Change: Additional Information (optional): 15

Revised Section of DDI Summary Outline: Textual Description of Elements and Attributes for Tag Library: 16

6.2 Major Revision Form Spreadsheet Modeling Technique Introduction This is a simple technique for data modeling using an Excel spreadsheet. The intent is to capture all of the data needed to create both an XML schema, and also to create more other models based on the information provided, using UML or other methodologies. The spreadsheet has four columns: (1) Element Structure (Name) (2) Nesting Level (3) Data Type (4) Occurrence (5) Machine Actionable (6) Notes The contents of each column are detailed below. (1) Element Structure (Name) This is the name of the data construct (element or attribute in XML terms). This column is also used to indicate containership, using indentation. Thus, a child element will be indented (3 spaces), and the first un-indented element in the rows above is the parent element. As an example: For an Address element containing Addressee, Street, City, StateOrProvince, and Country, the first column of the spreadsheet will look like this: Address Addressee Street City StateOrProvince Country 17

If there is a construct that provides a choice, then that should be indicated using START CHOICE followed by each option (indented) and preceded by END CHOICE : for example, if we were to replace StateOrProvince with an actual choice between State or Province, then that would be indicated in the first column of the spreadsheet as: Address Addressee Street City START CHOICE State Province END CHOICE Country GPSCoordinates Latitude Longitude If there are choices that are part of a sequence, then START GROUP and END GROUP should be used, with the members of the group indented below it. Thus, we could have a more sophisticated content model: Address Addressee START CHOICE START GROUP Street City StateOrProvince Country END GROUP START GROUP Latitude Longitude END GROUP END CHOICE This example shows an Addressee element, followed by Street, City, StateOrProvince, and Country OR a group that held only Latitude and Longitude. Generally speaking, complex content models should be avoided if possible -- straight sequences are best suited for many purposes in XML Schemas, especially extension. This is not an aspect of modeling, however, but a constraint imposed by the schema language. As the example shows, the nesting can get quite deep, making for an unwieldy first column. After three or four layers of indentation, one may wish to simply produce another spreadsheet to give particular details. This can be easily done by indicating in the Notes column that details are in a separate spreadsheet, and starting the second spreadsheet with a repetition of the element that serves as the top-level entry in the second spreadsheet. 18

Each entry in the first column defines a single row in the spreadsheet. The reserved words START CHOICE, END CHOICE, START GROUP, and END GROUP are blank rows, because they do not mark objects in the data model, but serve to indicate the modeling of content groups. Note that no distinction is made between an XML attribute and an XML element here, although preferences may be stated in the Notes column. An attribute is simply a child data element in terms of the model. (2) Nesting Level Nesting level is used in addition the indication in the Element Structure (Name) column to indicate the nesting level of a given element. Given the examples used above, the nesting columns would be as follows: Address 1 Addressee 2 START CHOICE 2 START GROUP 3 Street 4 City 4 StateOrProvince 4 Country 4 END GROUP 3 START GROUP 3 Latitude 4 Longitude 4 END GROUP 3 END CHOICE 2 Note that the START CHOICE, START GROUP, etc. indicators also contain nesting rows, with the options contained within the choice or group having a nesting level one greater. 19

(3) Data Type A data type is a concept taken from object-oriented technology, but it is not a difficult concept to grasp. Data are often modeled in patterns, and these patterns can be given names. This is referred to in many cases as a class. The second column of the spreadsheet provides the name of the type. Types are of two sorts (at least for the purposes of modeling XML): simple and aggregate. A simple type, or primitive, refers to the format of contained data. Thus, if a modeling construct holds a string, then it would be of the String type. If an integer, then it holds the Integer type, and so on. Types can describe very specific numeric formats, or patterns of strings. Here is a partial list of types for use in the spreadsheet model: String Token (a string with no spaces) URL (or URN or URI, depending on which type of identifier is needed) Integer Decimal Code Date Time DateTime In the cases of String, Code, and Decimal, more information is needed: String: You may indicate constraints on the length of the string if desired. This is often not done, but some strings (say, for example, a US ZIP code) are of a very definite pattern. Other pattern constraints should de described in the Notes column.) Decimal: If possible, the number of positions after the decimal point should be indicated, and other parameters described (in the Notes column), such as whether or not exponential notation would need to be supported by that data field. Code: This is a list of specific values (typically tokens), which can be enumerated. Each code is a reference to a particular value (for example, US is a coded way of expressing The United States of America ). For long lists, you may wish to reference a separate spreadsheet. In addition to identifying the type, this column should also be used to indicate and restrictions on size of the fields using the following notation method: [TYPE]# - used to indicate a fixed length field. For instance, String32 indicates a string of 32 characters, exactly. [TYPE]..# - used to indicate a maximum length of a field. For instance String..32 indicates a string of 0 to 32 characters. 20

Decimal#.# - used to indicate number precision. The first number is the total whole digits, and the second is the number of digits after the decimal point. For instance Decimal10.5 indicates a decimal number with up to 10 digits to the left of the decimal place and 5 digits of decimal precision. Ultimately, every different simple type needed in the model will become named, and specific names for particular data patterns can be provided in the spreadsheet if this is useful -- simply document their definitions in the Notes column the first time they are used, and provide the name of the data pattern in the Type column. Every reference to that type later in the spreadsheet will be understood to refer to the simple type defined on first use. Aggregate types (also called complex types) are slightly different, referring to a pattern of child elements. Thus, if we have defined a construct called Address, we might refer to its content from other, similar constructs. If we have an element ShippingAddress that is different from OfficialAddress, we might simply refer to the Address aggregate type in my Type column, rather than having to redefine what the substructure of an Address is each time. Note that any defined aggregate element in the spreadsheet can be used also as a type construct if needed -- this distinction can be more fully modeled later, and it will be made explicit in the XML schema. Typically, aggregate elements (that is, elements not directly containing data) will not share types with other elements. In this case, the name in the Type column should be identical to the name in the Name column -- the element is its own type. (4) Occurrence The value in this field indicates whether the element is required or optional in terms of its parent s content model. Values should be expressed as follows: 1..1 Indicates that it is used once and only once (required) 0..1 Indicates that it may be used once, or not at all (optional) 1..n Indicates that it must be used once, but may be repeated (required and repeatable) 0..n Indicates that it need not be used, but may be repeated if it is used (optional and repeatable) m..n Indicates that it must be used within certain bounds, with m being the lower bound, and n being the upper bound (this is not easily expressible in DTD syntax, but is easy to do in XML schema) Again, this is always in reference to the parent. If we have a required element A that contains B and C, which are both optional, we would have a spreadsheet that looked like this: Name Type Occurrence 21

A A 1..1 B B 0..1 C C 0..1 Use this column also to indicate that the value is defaulted or fixed. In this case, write DEFAULT and provide the default value, or FIXED, and provide the fixed value. (5) Machine Actionable A Yes/No column used to indicate whether an element is intended to be acted upon by an automated process, or will only be used for human readable processing. (6) Notes Because of the simplicity of this modeling methodology, it is common to use the Notes column for a wide variety of purposes. Some of these purposes are described above, and are needed to explain the use of other columns. The primary purpose of the Notes field is to provide documentation for the data element one is modeling. Every row in the spreadsheet (other than blank rows for START GROUP, END GROUP, START GROUP, and END GROUP ) should contain the definition of the data element it represents. This is in addition to whatever other information may be contained. Thus, it is a good idea to have Definition: precede the paragraph that provides that definition, to set it apart from other Notes information. General Comments The emphasis of a data modeling is on the semantics of the constructs, not necessarily on the way in which the data will be displayed in an XML instance. It is to be expected that the various models produced from different groups will be normalized according to a standard design at a later stage in the process, so that the resulting XML schema will have a high degree of consistency. At this stage, however, the idea is to get a basic model to start with, so that issues of XML Schema design can be applied, and other, more formal models, rendered. As a format for capturing this model, an Excel spreadsheet or any other tabular format is appropriate. Word and HTML tables are also good ways to work. Each group should select a single format to work in for ease of use. 22

Example Element Structure (Name) Nesting Level Data Type Occurrence Machine Actionable Notes Address 1 0..n Addressee 2 string 1..1 Y This should be a reference to a key START CHOICE 2 1..1 START GROUP 3 Street 4 string 1..1 Street address City 4 string 1..1 Y City StateOrProvince 4 code 1..1 Y Coded state or province list Country 4 code 1..1 Y See URI for list of country codes END GROUP 3 START GROUP 3 Latitude 4 string 1..1 Y Degrees latitude Longitude 4 string 1..1 Y Degrees longitude END GROUP 3 END CHOICE 2 LastUpdated 2 datetime 0..1 Y Date address was last updated LastVerified 2 datetime 0..1 Y Date address was last verified 23