edmr2.0 XML Documentation Last Updated: 5/12/2010

Transcription

1 edmr2.0 XML Documentation Last Updated: 5/12/2010 Versions edmr2.0 vs edmr1.0 To differentiate between the old and new edmr versions we have designated the original version as edmr1.0 and the latest version as edmr2.0. Keep in mind that the new 2.0 version is not simply an updated version of the older 1.0 product. edmr2.0 has been completed revamped to accommodate an entirely new a radically different underlying database structure. The old DMR table structure was basically a flat-filed approach involving only one table. The new WYPDES version uses 6 individual normalized tables to represent the DMR information. See the section on "<Element> Hierarchy" below for a more detailed explanation. Contact Information The primary contacts related to the edmr application are as listed below: Lead Contact: Developer: Kevin Campbell edmr and Database Specialist, Wyoming DEQ/WQD kcampb@wyo.gov Steve Rea IT/Admin, Wyoming WDEQ srea@wyo.gov We recommend that you make every attempt to first contact Kevin Campbell if at all possible since he is the lead on this project. He will attempt to answer your question directly or if it involves specific technical issues he will contact the appropriate person or persons. edmr2.0 _XML - Page 1 of 32

2 Current Document Status This is an update of the original draft document that will continue to be updated as new information is made available. Currently, this document should be fairly complete; however, we will continue to add information to this document and re-copy it our web site as needed. This document can be obtained at the following URLs: Application Status There are several features of the edmr2.0 application that are not yet implemented. The following is a partial list of these items: Features Not Yet Implemented o TreeView CheckBox Set NoDischargeType Functionality. One new feature is to provide a check box on the tree-view control and each of the levels at and below the SubmissionPeriod(SubPd) level. There will be a Set No Discharge Type button provided and a drop-down list of the no discharge types. Selecting an appropriate no discharge type and pressing the Set NoDischarge Type button will result in the DMR.NoDischargeTypeID values being updated directly without having to move to the parameters grid. o TreeView CheckBox Select Individual Outfall Functionality. The checkboxes will also be used to select multiple outfall functionality on the parameters grid. There is a limit to how many records can be displayed on the parameters grid before the performance is negatively impacted, however, having to select to one outfall is often too restrictive. Therefore, one or more boxes at the outfall level can be selected in order to access multiple outfalls on the parameters grid. o Overall Status Form. A form will be provided that will indicate the overall status associated with each submission period. The proposed form navigation controls are as follows: Multi-select list box allowing the selection of "All" or multiple permits. Multi-select list or text box allowing the selection or indication of the desired year(s) Multi-select list box allowing the selection of the "All" or multiple submission frequencies (All, Monthly, Quarterly, Semi-Annually, Annually) Multi-select list box allowing the selection of "All" or multiple submission periods. edmr2.0 _XML - Page 2 of 32

3 The test and production site will continue to be updated as these features are brought online. We will notify users whenever the new features are made available and notes indicating the latest changes are posted inside the scrollable boxes on the login form. Login There are two versions of edmr2.0 the production site and the test site. Production Site. The production site is against the live WYPDES database. Any actions performed at this level are permanent. The production site can be accessed using the following URL: Test Site. Currently we are not offering a test site. If in the future it is determined that a test site is beneficial we may create a new test site at a later date. In order to log in either of the edmr2.0 sites you must have an active loginid, Password and PIN number. You may obtain this information from Kevin Campbell (see "Contact Information" above). New Terminology (Submission Periods vs Reporting Periods) One obstacle is the new terminology relating to what are referred to "Submission Periods" and "Reporting Periods". Under the new system a "Submission Period" is now what was referred to as a "Reporting Period" under the old system. And yet there is still a "Reporting Period" in the new system. Submission Period. A "Submission Period" is the time in months required for DMR data to be submitted back to DEQ. Typical values are 1 - Monthly; 2 - Bi-Monthly; 3 - Quarterly; 6 - Semi-Annually and 12 - Annually. The number of submission period months is downloaded within the <SubmissionFrequencyMonths> element of the <LimitSet> element and appears as the [SF] column on the various grid controls. Reporting Period. A "Reporting Period" is the time in months indicating the interval that data must be reported. Typical values are 1 - Monthly; 2 - Bi-Monthly; 3 - Quarterly; 6 - Semi-Annually and 12 - Annually. The number of reporting period months is downloaded within the <ReportingPeriodMonths> element of the <LimitSet> element and appears as the [RF] column on the various grid controls. edmr2.0 _XML - Page 3 of 32

4 Typical combinations of the two are as follows: Submission Period Reporting Period Description 6 1 Submit to DEQ every 6 months; report monthly 3 3 Submit to DEQ every 3 months; report quarterly 12 6 Submit to DEQ every 12 months; report semi-annually A reporting period can never be less than one and can never exceed the number of months in the submission period. XML vs Grid Approach As with the edmr1.0 system, you can move DMR data to the WDEQ servers in one of two ways -- either using XML files to upload data or by entering directly into grid controls on a webform. XML Upload. Using this approach you first download DMR data in XML format. Once downloaded, it is entirely up the user to decide what they will do with the data. Most current XML users use the downloaded data to populate their organizational database systems and in turn re-generate XML files which now include the measurement value data from their system. These re-generated XML files are in turn uploaded back the WDEQ database system. This is the same process that was used within the edmr1.0 system with the exception that the XML structure has changed. Grid Control. Alternatively, you can enter data directly in the grid controls provided. For smaller facilities this will be the preferred method. This again is substantively the same process that was used in the edmr1.0 system with the exception that changes have been implemented to accommodate the new WYPDES database structure. The document covers only the XML approach. There is a companion document related to the Grid control approach. Upload vs Submit For those not familiar with edmr1.0 an explanation is required relating to an "upload" vs a "submit". Upload. An upload is the process of moving XML data from your client computer to the DEQ database server. You simply navigate to the file containing the XML data and then press the Upload button. The upload process is very loosely coupled. You can include any number of DMR upload units and in any order. You can even re-upload the same file multiple times. Submit. The "Submit" process finalizes the edmr submittal process. All records must have been satisfactorily validated prior to initiating the "submit" process. The "submit" is characterized by copying all previously uploaded information to an edmr transaction log table that is the safe-storage mechanism for the edmr process. This transaction log table is granted only insert rights to protect against any unwarranted updates or deletes. edmr2.0 _XML - Page 4 of 32

5 DMRID Record IDs are the Fundamental Unit for XML Uploads With all of the basics concepts now covered it is time to note that the basic unit for XML uploads are at the "reporting period" level, i.e. at the DMRID level rather than at the "submission period" level, i.e. at the DMRSubmissionID level. This is in part because all processing on the web server is at the DMRID level but also because it provides the ultimate flexibility in terms of how an upload file is constructed. You can include any combination of DMRID values from any number of differing submission periods without worrying whether they belong to a particular submission period. All information relating to a particular DMRID, including DMRSubmissionID, will be gathered on the server-side. Note that submission period level information is included within the <LimitSet> element for reference purposes but that none of this information need be submitted back as part of the XML upload. Merge Process All XML uploads (and grid control "Save" operations) are facilitated using a "merge" process. This "merge" process is characterized by the creation of source and target datasets where the source dataset is comprised of the uploaded XML data and the target dataset is comprised of an accompanying set of records from the database tables. The source dataset is merged into the target dataset which results in a modified dataset where only those records having changed are included in the modified dataset. Only the modified dataset records actually get updated. The merge process then allows you to upload the same dataset of information multiple times without actually re-updating the same information over and over again. Schema Validation The new edmr2.0 system offers schema validation. You can either validate your XML files against the edmr.xsd (schema) file using the database webform utility or you can download the.xsd file and perform the validation yourself. The new schema validation includes "data type validation" as well as validation for "well-formed xml". The data type for each element has been defined within the schema file and will result in an error if the uploaded value is not the proper data type. The original edmr1.0 system did not include schema validation (particularly data type validation) and this one factor was responsible for more upload delays than any other single issue. Schema validation can be performed by navigating to the Upload webform (Upload/Validate). Under the "Browse for File to Validate or Upload" heading, browse to the file you wish to validate and then select either the "Validate XML (View)" or "Validate XML (Download) buttons. The "View" results will be displayed in the results window at the bottom of the form. The "Download" approach will prompt you for a local file location and download a text version of the schema validation information to your computer. Validation errors will be displayed in the results window at the bottom of the form. The message include the severity, message and line number of where the error occurred within the XML file. edmr2.0 _XML - Page 5 of 32

6 It is important to note that schema validation will be performed as part of the upload process, and the upload is allowed to proceed only if the schema validation succeeds. Upload Qualifications In order to perform an XML upload there are a list of qualifications that must be met prior to the upload actually occurring. These are listed in the order processed: Schema Validation As mentioned above under "Schema Validation" this validation must succeed prior to any other upload processing beginning. Any schema validation errors will be displayed in the upload forms processing window along with detailed messages relating to the individual schema elements. Line numbers are included. Submission Period Locked The edmr administrators have the ability to lock individual submission periods. Typically, these submission periods are locked at some date after the submission period is over. All DMRID values that are rejected for submission due to being locked will be displayed in the upload forms processing window along with a message indicating the reason(s). Ownership The ownership of each upload DMRID value will be checked to verify that they belong to the current login account. All DMRID values that are rejected based on the ownership check will be displayed in the upload forms processing window along with a message indicating the reaon(s) Overall Status Once a submission period has been submitted the overall status of the submission period will be either SUB or AMD. You cannot upload DMRID values once the submission process has occurred. You can only amend previously submitted records via the grid control. All DMRID values that are rejected based on the overall status will be displayed in the upload forms processing window along with a message indicating the reason(s). edmr2.0 _XML - Page 6 of 32

7 Example XML Download The following is an example of the new XML structure created for download. Note that the items highlighted in red and bolded are the items required for upload. All other items not bolded or highlighted in red are provided as informational items for download purposes only. They are not required for upload and in fact if included in the upload file will be ignored during the parsing of the XML data. <?xml version="1.0" encoding="utf-8"?> <DMRSet xmlns=" xmlns:xsi=" <DMR>.. <DMRID>68404</DMRID>.. <LimitSetID>6531</LimitSetID>.. <LimitSet>... <LimitSetID>6531</LimitSetID>... <WYPermitNumber>WY </WYPermitNumber>... <OutfallNumber>001</OutfallNumber>... <SubmissionFrequencyMonths>6</SubmissionFrequencyMonths>... <ReportingPeriodMonths>6</ReportingPeriodMonths>... <SubmissionPeriodBeginDate> </SubmissionPeriodBeginDate>... <SubmissionPeriodEndDate> </SubmissionPeriodEndDate>.. </LimitSet>.. <ReportBeginDate> </ReportBeginDate>.. <ReportEndDate> </ReportEndDate>.. <DMR_NoDischargeTypeID>10</DMR_NoDischargeTypeID>.. <DMRCommentText xsi:nil="true" />.. <DMRParameter>... <DMRParameterID>591589</DMRParameterID>... <DMRID>68404</DMRID>... <LimitID>30678</LimitID>... <Limit>.... <LimitID>30678</LimitID>.... <SampleTypeID>50</SampleTypeID>.... <SampleFrequencyTypeID>68</SampleFrequencyTypeID>.... <ParameterID>4949</ParameterID>.... <MonitoringLocationTypeID>2</MonitoringLocationTypeID>.... <LimitBeginDate> </LimitBeginDate>.... <LimitEndDate> </LimitEndDate>... </Limit>... <ActualSampleFrequencyTypeID>68</ActualSampleFrequencyTypeID>... <ReportedNumberOfExcursions>0</ReportedNumberOfExcursions>... <DMRValue>.... <DMRValueID>458453</DMRValueID>.... <DMRParameterID>591589</DMRParameterID>.... <LimitValueID>37241</LimitValueID>.... <LimitValue>..... <LimitValueID>37241</LimitValueID>..... <ValueTypeID>3</ValueTypeID> edmr2.0 _XML - Page 7 of 32

8 ..... <StatisticalBaseTypeID>77</StatisticalBaseTypeID>..... <Limit_ValueQualifierTypeID>6</Limit_ValueQualifierTypeID>..... <UnitID>12</UnitID>..... <Limit_Value>7500</Limit_Value>.... </LimitValue>.... <DMR_ValueQualifierTypeID xsi:nil="true" />.... <DMRValue_NoDischargeTypeID>10</DMRValue_NoDischargeTypeID>.... <MeasurementValue xsi:nil="true" />.... <DMRValueCommentText xsi:nil="true" />... </DMRValue>.. </DMRParameter>.. <DMRParameter>... <DMRParameterID>591590</DMRParameterID>... <DMRID>68404</DMRID>... <LimitID>30279</LimitID>... <Limit>.... <LimitID>30279</LimitID>.... <SampleTypeID>50</SampleTypeID>.... <SampleFrequencyTypeID>68</SampleFrequencyTypeID>.... <ParameterID>5920</ParameterID>.... <MonitoringLocationTypeID>2</MonitoringLocationTypeID>.... <LimitBeginDate> </LimitBeginDate>.... <LimitEndDate> </LimitEndDate>... </Limit>... <ActualSampleFrequencyTypeID>68</ActualSampleFrequencyTypeID>... <ReportedNumberOfExcursions>0</ReportedNumberOfExcursions>... <DMRValue>.... <DMRValueID>458455</DMRValueID>.... <DMRParameterID>591590</DMRParameterID>.... <LimitValueID>164017</LimitValueID>..... <LimitValue> <LimitValueID>164017</LimitValueID> <ValueTypeID>3</ValueTypeID> <StatisticalBaseTypeID>77</StatisticalBaseTypeID> <Limit_ValueQualifierTypeID>6</Limit_ValueQualifierTypeID> <UnitID>20</UnitID> <Limit_Value>10</Limit_Value>..... </LimitValue>.... <DMR_ValueQualifierTypeID xsi:nil="true" />.... <DMRValue_NoDischargeTypeID>10</DMRValue_NoDischargeTypeID>.... <MeasurementValue xsi:nil="true" />.... <DMRValueCommentText xsi:nil="true" />... </DMRValue>.. </DMRParameter>.. <DMRParameter>... <DMRParameterID> </DMRParameterID>... <DMRID>68404</DMRID>... <LimitID>30677</LimitID>... <Limit>.... <LimitID>30677</LimitID>.... <SampleTypeID>50</SampleTypeID> edmr2.0 _XML - Page 8 of 32

9 .... <SampleFrequencyTypeID>68</SampleFrequencyTypeID>.... <ParameterID>3533</ParameterID>.... <MonitoringLocationTypeID>2</MonitoringLocationTypeID>.... <LimitBeginDate> </LimitBeginDate>.... <LimitEndDate> </LimitEndDate>... </Limit>... <ActualSampleFrequencyTypeID>68</ActualSampleFrequencyTypeID>... <ReportedNumberOfExcursions>0</ReportedNumberOfExcursions>... <DMRValue>.... <DMRValueID>458451</DMRValueID>.... <DMRParameterID> </DMRParameterID>.... <LimitValueID>29061</LimitValueID>.... <LimitValue>..... <LimitValueID>29061</LimitValueID>..... <ValueTypeID>5</ValueTypeID>..... <StatisticalBaseTypeID>102</StatisticalBaseTypeID>..... <Limit_ValueQualifierTypeID>6</Limit_ValueQualifierTypeID>..... <UnitID>13</UnitID>..... <Limit_Value>8.5</Limit_Value>.... </LimitValue>.... <DMR_ValueQualifierTypeID xsi:nil="true" />.... <DMRValue_NoDischargeTypeID>10</DMRValue_NoDischargeTypeID>.... <MeasurementValue xsi:nil="true" />.... <DMRValueCommentText xsi:nil="true" />... </DMRValue>... <DMRValue>.... <DMRValueID> </DMRValueID>.... <DMRParameterID> </DMRParameterID>.... <LimitValueID>75757</LimitValueID>.... <LimitValue>..... <LimitValueID>75757</LimitValueID>..... <ValueTypeID>4</ValueTypeID>..... <StatisticalBaseTypeID>135</StatisticalBaseTypeID>..... <Limit_ValueQualifierTypeID>4</Limit_ValueQualifierTypeID>..... <UnitID>13</UnitID>..... <Limit_Value>6.5</Limit_Value>.... </LimitValue>.... <DMR_ValueQualifierTypeID xsi:nil="true" />.... <DMRValue_NoDischargeTypeID>10</DMRValue_NoDischargeTypeID>.... <MeasurementValue xsi:nil="true" />.... <DMRValueCommentText xsi:nil="true" />... </DMRValue>.. </DMRParameter>. </DMR> </DMRSet> The following is the same example as the above XML file but includes only those items required for uploading purposes. As you can see limiting the elements to just those required for uploading purposes will significantly decrease the quantity of information being transported. It is recommended that you limit the upload to just the required records. edmr2.0 _XML - Page 9 of 32

10 <?xml version="1.0" encoding="utf-8"?> <DMRSet xmlns=" xmlns:xsi=" <DMR>.. <DMRID>68404</DMRID>.. <DMR_NoDischargeTypeID>10</DMR_NoDischargeTypeID>.. <DMRCommentText xsi:nil="true" />.. <DMRParameter>... <DMRParameterID>591589</DMRParameterID>... <DMRValue>.... <DMRValueID>458453</DMRValueID>.... <DMR_ValueQualifierTypeID xsi:nil="true" />.... <DMRValue_NoDischargeTypeID>10</DMRValue_NoDischargeTypeID>.... <MeasurementValue xsi:nil="true" />.... <DMRValueCommentText xsi:nil="true" />... </DMRValue>.. </DMRParameter>.. <DMRParameter>... <DMRParameterID>591590</DMRParameterID>... <DMRValue>.... <DMRValueID>458455</DMRValueID>.... <DMR_ValueQualifierTypeID xsi:nil="true" />.... <DMRValue_NoDischargeTypeID>10</DMRValue_NoDischargeTypeID>.... <MeasurementValue xsi:nil="true" />.... <DMRValueCommentText xsi:nil="true" />... </DMRValue>.. </DMRParameter>.. <DMRParameter>... <DMRParameterID> </DMRParameterID>... <ActualSampleFrequencyTypeID>68</ActualSampleFrequencyTypeID>... <ReportedNumberOfExcursions>0</ReportedNumberOfExcursions>... <DMRValue>.... <DMRValueID>458451</DMRValueID>.... <DMR_ValueQualifierTypeID xsi:nil="true" />.... <DMRValue_NoDischargeTypeID>10</DMRValue_NoDischargeTypeID>.... <MeasurementValue xsi:nil="true" />.... <DMRValueCommentText xsi:nil="true" />... </DMRValue>... <DMRValue>.... <DMRValueID> </DMRValueID>.... <DMR_ValueQualifierTypeID xsi:nil="true" />.... <DMRValue_NoDischargeTypeID>10</DMRValue_NoDischargeTypeID>.... <MeasurementValue xsi:nil="true" />.... <DMRValueCommentText xsi:nil="true" />... </DMRValue>.. </DMRParameter>. </DMR> </DMRSet> edmr2.0 _XML - Page 10 of 32

11 Schema The following is the.xsd schema file as it exists at the time of this writing. <?xml version="1.0" encoding="utf-8"?> <xs:schema id="edmr_access_format_3" targetnamespace=" elementformdefault="qualified" xmlns=" xmlns:mstns=" xmlns:xs=" <xs:element name="dmrset"> <xs:complextype> <xs:sequence> <xs:element name="dmr" maxoccurs="unbounded"> <xs:complextype> <xs:sequence> <xs:element name="dmrid" type="" minoccurs="1" /> <xs:element name="limitsetid" type="" minoccurs="0" /> <!-- The following are the <LimitSet> level elements and are not required for upload. --> <xs:element name="limitset"> <xs:complextype> <xs:sequence> <xs:element name="limitsetid" type="" minoccurs="1" /> <xs:element name="wypermitnumber" type="xs:string" minoccurs="0" /> <xs:element name="outfallnumber" type="xs:string" minoccurs="0" /> <xs:element name="submissionfrequencymonths" type="" minoccurs="0" /> <xs:element name="reportingperiodmonths" type="" minoccurs="0" /> <xs:element name="submissionperiodbegindate" type="xs:date" minoccurs="0" /> <xs:element name="submissionperiodenddate" type="xs:date" minoccurs="0" /> </xs:sequence> </xs:complextype> </xs:element> <!-- The following two elements must appear at the <DMR> element level and cannot be included within the <LimitSet> element. A given <LimitSet> sequence will usually appear multiple times within a given download. This would result in multiple <ReportBeginDate> and <ReportEndDate> values appearing within what should be single <LimitSet> instance as identified by a LimitSetID. --> <xs:element name="reportbegindate" type="xs:date" minoccurs="0" /> <xs:element name="reportenddate" type="xs:date" minoccurs="0" /> <!-- The following are the <DMR> level elements and are required for upload. --> <!--<xs:element name="dmr_nodischargetypeid" type="" nillable="true" minoccurs="1" />--> <xs:element name="dmr_nodischargetypeid" type="nodischargetypeidtype" nillable="true" minoccurs="1" /> <xs:element name="dmrcommenttext" type="xs:string" nillable="true" minoccurs="1" /> <xs:element name="dmrparameter" maxoccurs="unbounded"> <xs:complextype> <xs:sequence> <!-- The following is the DMRParamterID(pk) and DMRID(fk) element and are required for upload. --> <xs:element name="dmrparameterid" type="" minoccurs="1" /> <xs:element name="dmrid" type="" minoccurs="1" /> <xs:element name="limitid" type="" minoccurs="0" /> <!-- The following are the <Limit> level elements and are not required. --> edmr2.0 _XML - Page 11 of 32

12 <xs:element name="limit"> <xs:complextype> <xs:sequence> <xs:element name="limitid" type="" minoccurs="0" /> <xs:element name="sampletypeid" type="" minoccurs="0" /> <xs:element name="samplefrequencytypeid" type="" minoccurs="0" /> <xs:element name="parameterid" type="" minoccurs="0" /> <xs:element name="monitoringlocationtypeid" type="" minoccurs="0" /> <!--Although the [UnitID] is stored within the [Limit] table there are two columns stored, [UnitID_C] and [UnitID_Q]. However, you do not know which one applies without examining the [LimitValue].[ValueQualifierID] value. Therefore, the [UnitID] must be displayed at the [LimitValue] table level. <xs:element name="unitid" type="" minoccurs="0" /> --> <xs:element name="limitbegindate" type="xs:date" minoccurs="0" /> <xs:element name="limitenddate" type="xs:date" minoccurs="0" /> </xs:sequence> </xs:complextype> </xs:element> <!-- The following are the <DMRParameter> level elements and are required for upload. --> <xs:element name="actualsamplefrequencytypeid" type="" nillable="true" minoccurs="1" /> <xs:element name="reportednumberofexcursions" type="" nillable="true" minoccurs="1" /> <xs:element name="dmrvalue" maxoccurs="unbounded"> <xs:complextype> <xs:sequence> <!-- The following are the DMRValueID(pk) and DMRParameterID(fk) elements and are required for upload. --> <xs:element name="dmrvalueid" type="" minoccurs="1" /> <xs:element name="dmrparameterid" type="" minoccurs="1" /> <xs:element name="limitvalueid" type="" minoccurs="0" /> <!-- The following are the <LimitValue> level element and are not required. --> <xs:element name="limitvalue"> <xs:complextype> <xs:sequence> <xs:element name="limitvalueid" type="" minoccurs="0" /> <xs:element name="valuetypeid" type="valuetypeidtype" minoccurs="0" /> <xs:element name="statisticalbasetypeid" type="" minoccurs="0" /> <!-- The following definition for <Limit_ValueQualifierTypeID> should not have nillable="true" but it has been changed temporarily to allow the latest data to work properly. --> <xs:element name="limit_valuequalifiertypeid" type="valuequalifiertypeidtype" nillable="true" minoccurs="0" /> <!--As explained above there are actually two [UnitID] fields in the [Limit] table; [UnitID_Q] and [UnitID_C]. However, the one to be used cannot be determined without examining [LimitValue].[ValueQualifierID] field. Therefore, the [UnitID] field is displayed at the [LimitValue] level.--> <xs:element name="unitid" type="" minoccurs="0" /> <xs:element name="limit_value" type="xs:double" nillable="true" minoccurs="0" /> </xs:sequence> </xs:complextype> </xs:element> <!-- The following are the <DMRValue> level elements and are required. --> edmr2.0 _XML - Page 12 of 32

13 /> <xs:element name="dmr_valuequalifiertypeid" type="" nillable="true" minoccurs="1" /> <xs:element name="dmrvalue_nodischargetypeid" type="nodischargetypeidtype" nillable="true" minoccurs="1" /> <xs:element name="measurementvalue" type="xs:double" nillable="true" minoccurs="1" /> <xs:element name="dmrvaluecommenttext" type="xs:string" nillable="true" minoccurs="1" </xs:sequence> </xs:complextype> </xs:element> </xs:sequence> </xs:complextype> </xs:element> </xs:sequence> </xs:complextype> <xs:key name="dmrid"> <xs:selector xpath="." /> <xs:field xpath="mstns:dmrid" /> </xs:key> <xs:keyref name="dmr_dmrparameter" refer="dmrid"> <xs:selector xpath=".//mstns:dmrparameter" /> <xs:field xpath="mstns:dmrid" /> </xs:keyref> <xs:keyref name="dmrparameter_dmrvalue" refer="dmrparameterid"> <xs:selector xpath=".//mstns:dmrvalue" /> <xs:field xpath="mstns:dmrparameterid" /> </xs:keyref> </xs:element> </xs:sequence> </xs:complextype> <xs:key name="dmrparameterid"> <xs:selector xpath=".//mstns:dmrparameter" /> <xs:field xpath="mstns:dmrparameterid" /> </xs:key> <xs:key name="dmrvalueid"> <xs:selector xpath=".//mstns:dmrvalue" /> <xs:field xpath="mstns:dmrvalueid" /> </xs:key> <xs:key name="limitsetid"> <xs:selector xpath=".//mstns:limitset" /> <xs:field xpath="mstns:limitsetid" /> </xs:key> <xs:keyref name="limitset_dmr" refer="limitsetid"> <xs:selector xpath=".//mstns:dmr" /> <xs:field xpath="mstns:limitsetid" /> </xs:keyref> <xs:key name="limitid"> <xs:selector xpath=".//mstns:limit" /> <xs:field xpath="mstns:limitid" /> </xs:key> <xs:keyref name="limit_dmrparameter" refer="limitid"> <xs:selector xpath=".//mstns:dmrparameter" /> <xs:field xpath="mstns:limitid" /> edmr2.0 _XML - Page 13 of 32

14 </xs:keyref> <xs:key name="limitvalueid"> <xs:selector xpath=".//mstns:limitvalue" /> <xs:field xpath="mstns:limitvalueid" /> </xs:key> <xs:keyref name="limitvalue_dmrvalue" refer="limitvalueid"> <xs:selector xpath=".//mstns:dmrvalue" /> <xs:field xpath="mstns:limitvalueid" /> </xs:keyref> </xs:element> <xs:simpletype name="valuetypeidtype"> <xs:restriction base="xs:integer"> <xs:enumeration value="1" /> <!-- Concentration 1 --> <xs:enumeration value="2" /> <!-- Concentration 2 --> <xs:enumeration value="3" /> <!-- Concentration 3 --> <xs:enumeration value="4" /> <!-- Quantity 1 --> <xs:enumeration value="5" /> <!-- Quantity 2 --> </xs:restriction> </xs:simpletype> <xs:simpletype name="valuequalifiertypeidtype"> <xs:restriction base="xs:integer"> <xs:enumeration value="1" /> <!--EQ (Equal To) --> <xs:enumeration value="2" /> <!-- EST (Estimated) --> <xs:enumeration value="3" /> <!-- GT (Greater Than) --> <xs:enumeration value="4" /> <!-- GTE (Greater Than or Equal To --> <xs:enumeration value="5" /> <!-- LT (Less Than) --> <xs:enumeration value="6" /> <!-- LTE (Less Than or Equal To) --> <xs:enumeration value="7" /> <!-- TOO (Too Numerous to Count) --> </xs:restriction> </xs:simpletype> <xs:simpletype name="nodischargetypeidtype_alternate"> <xs:union membertypes="nodischargetypeidtype"> <xs:simpletype> <xs:restriction base="xs:string"> <xs:enumeration value="null"/> </xs:restriction> </xs:simpletype> </xs:union> </xs:simpletype> edmr2.0 _XML - Page 14 of 32

15 <xs:simpletype name="nodischargetypeidtype"> <xs:restriction base="xs:integer"> <!-- <xs:length value="1" /> --> <xs:enumeration value="1" /> <!-- Wrong Flow --> <xs:enumeration value="2" /> <!-- Operation Shutdown --> <xs:enumeration value="3" /> <!-- Discharge to Lagoon/Groundwater --> <xs:enumeration value="4" /> <!-- Frozen Conditions --> <xs:enumeration value="5" /> <!-- No Influent --> <xs:enumeration value="6" /> <!-- Other (See Comments) --> <xs:enumeration value="7" /> <!-- Conditional Limit Not Required This Monitoring Period --> <xs:enumeration value="8" /> <!-- General Permit Exemption --> <xs:enumeration value="9" /> <!-- Below Detection Level --> <xs:enumeration value="10" /> <!-- No Discharge --> <xs:enumeration value="11" /> <!-- Lost Sample/Data Not Available --> <xs:enumeration value="12" /> <!-- Analysis Not Conducted/No Sample --> <xs:enumeration value="13" /> <!-- Insufficient Flow for Sampling --> <xs:enumeration value="14" /> <!-- Sampling Equipment Failure --> <xs:enumeration value="15" /> <!-- Invalid Test --> <xs:enumeration value="16" /> <!-- Land Applied --> <xs:enumeration value="17" /> <!-- Recycled But Not Entered --> <xs:enumeration value="18" /> <!-- Flood Disaster --> <xs:enumeration value="19" /> <!-- DMR Received but not Entered --> <xs:enumeration value="20" /> <!-- Not Quantifiable --> <xs:enumeration value="21" /> <!-- Administratively Resolved --> <xs:enumeration value="22" /> <!-- Fire Conditions --> <xs:enumeration value="23" /> <!-- Dry Lysimeter Well --> <xs:enumeration value="24" /> <!-- Parameter Value Not Reported --> edmr2.0 _XML - Page 15 of 32

16 </xs:restriction> </xs:simpletype> </xs:schema> edmr2.0 _XML - Page 16 of 32

17 <Element> / Table Hierarchy The element/table hierarchy found in the XML and.xsd files matches that of the new WYPDES database. <DMR> element / table ( :1) <LimitSet> element / table DMRID, pk LimitSetID, pk LimitSetID, fk (1: ) (1: ) <DMRParameter> element / table ( :1) <Limit> element / table DMRParameterID, pk LimitID, pk DMRID, fk LimitiD, fk (1: ) (1: ) <DMRValue> element / table ( :1) <LimitValue> element / table DMRValueID, pk DMRParameterID, fk LimitValueID, fk edmr2.0 _XML - Page 17 of 32 LimitValueID, pk There is a one-to-many (1: ) relationship between the <DMR> element/table and the <DMRParameter> element/table based on <DMRID> acting as a primary key (pk) within the <DMR> element/table and as a foreign key (fk) within the <DMRParameter> element/table. Expressed in words, one <DMR> element contains many <DMRParameter> elements. There is also a one-to-many (1: ) relationship between the <DMRParameter> element/table and the <DMRValue> element/table based on <DMRParameterID> acting as a primary key within the <DMRParameter> element/table and as a foreign key within the <DMRValue> table. Expressed in words, one <DMRParameter> element contains many <DMRValue> elements. There is a one-to-many (1: ) relationship between the <LimitSet> element/table and the <DMR> element/table based on <LimitSetID> acting as a primary key within the <LimitSet> element/table and a foreign key within the <DMR> element/table. There is also a one-to-many (1: ) relationship between the <Limit> element/table and <DMRParameter> element/table based on <LimitID> acting as a primary key within the <Limit> element/table and a foreign key within the <DMRParameter> element/table. And there is also a (1: ) relationship between the <LimitValue> element/table and the <DMRValue> element/table based on <LimitValueID> acting as a primary key within the <LimitValue> element/table and as a foreign key within the <DMRValue> element/table. The <DMR>, <DMRParameter> and <DMRValue> are elements where user data is entered and subsequently uploaded back to the edmr application.

18 The <LimitSet>, <Limit> and <LimitValue> elements are comprised of item requirements relating to each individual DMR-prefixed table record. These elements are not required to be uploaded back to the edmr application. In fact, if they are uploaded, they will simply be ignored by the edmr application. One consideration, however, is that including this data in the upload process will just increase the size of the data uploads and thereby possibly degrade the overall performance. After reviewing the above explanation relating to the actual relationships between elements it should now be noted that this is not the way the edmr XML files are structured. For readability purposes and file management purposes, it was decided that the actual element hierarchy for XML download purposes would be inverted. Instead of a <LimitSet> element acting as a container for multiple <DMR> elements, each <DMR> element contains one <LimitSet> element. And instead of a <Limit> element acting as a container for multiple <DMRParameter> elements, each <DMRParameter> element contains one <Limit> element. And instead of a <LimitValue> element acting as a container for multiple <DMRValue> elements each <DMRValue> element contains one <LimitValue> element. So instead of a XML structure that is constructed as follows: <LimitSet>. <DMR>.. <Limit>... <DMRParameter>.... <LimitValue>..... <DMRValue />.... </LimitValue>.... <LimitValue>..... <DMRValue />.... </LimitValue>... </DMRParameter>... <DMRParameter> </DMRParameter>.. </Limit>. </DMR>. <DMR>.. </DMR> </LimitSet <LimitSet> </LimitSet> We end up with a more readable DMR-centric XML structure that appears as follows: <DMR>. <LimitSet> edmr2.0 _XML - Page 18 of 32

19 . </LimitSet>. <DMRParameter>.. <Limit>.. </Limit>.. <DMRValue>... <LimitValue>... </LimitValue>.. </DMRValue>. </DMRParameter> </DMR> <DMR> </DMR> Although the latter structure is more manageable from a readability and file manipulation standpoint there is a downside. This structure results in duplications that must be dealt with when processing the XML data. Duplicate limit-series records must be accounted for and removed as part of the XML processing code. Or, if directly importing into Access tables the tables must be created prior to import with the primary keys identified. This will guarantee the duplicate limit-series records are removed during the import process. edmr2.0 _XML - Page 19 of 32

20 <DMR> element Reporting Period Records grid table <DMRID> element grid s [DMRID] field The <DMRID> element contains the primary key for the <DMR> element / [DMR] table. Required for Upload. YES Nullable. NO WYPDES FATAL Validation Errors: o Anything other than a non-negative integer. <LimitSetID> element grid s [LimitSetID] field The <LimitSetID> element is a foreign key reference to the <LimitSet> element or [LimitSet] table. Nullable. NO WYPDES FATAL Validation Errors: o Anything other than a non-negative integer. <DMR_NoDischargeTypeID> element grid s [NoDischType] field The <DMR_NoDischargeTypeID> element contains the foreign key reference to the [RefNoDischargeType] lookup table which has a primary key of NoDischargeTypeID. This field is used both as a flag to indicate that either a No Discharge condition has occurred or that a Discharge, No Analysis condition has occurred. The type of either No Discharge or Discharge, No Analysis is identified using the [????] field within the [RefNoDischargeType] lookup table. The [DMR].[NoDischargeTypeID] is limited to a subset of the total [RefNoDischargeType] table items. These legal entries are listed as follows: NoDischargeTypeID Description 10 No Discharge 2 Operation Shutdown 37 Outfall Not Constructed The dropdown-list on the grid control only contains these three items; however, it is possible to upload other integer values. If other than the legal values are uploaded the upload operation will complete but a WYPDES validation rule will result in the DMR level record edmr2.0 _XML - Page 20 of 32

21 receiving a status value of 'ERR'. Required for Upload. YES Nullable: yes WYPDES FATAL Validation Errors: o Value not one of those three integer values listed above as allowable. <DMRCommentText> element grid s [AddComment] field The edmr system is designed to append records to the existing underlying database comment field. For example, if the existing comment field is: <edmr dt= >this is a comment</edmr> and you add a new comment string: Add another comment. the resulting updated comment field will read: <edmr dt= >this is a comment</edmr><edmr dt= >add another comment.</edmr> Note that each additional comment added via the edmr application will include the dated <edmr> tag. For the <DMRCommentText> element, the value included within this element will be appended to the already existing comment. Also, since it is possible to upload the same XML file multiple times, the XML approach will check to see if the included value already exists as part of the existing comment. If it does, the <DMRCommentText> will NOT be re-appended. For the grid control, the value typed in the [CommentAdd] field will be appended to the existing comment. xs:string SQL Server Data Type. varchar(1000) Required for Upload. YES Nullable: yes WYPDES FATAL Validation Errors: o none. <ReportBeginDate> element [BeginDate] field within the Reporting Period Record, Parameter Records and Value Records grid tables edmr2.0 _XML - Page 21 of 32

22 The <ReportBeginDate> element is the reporting period begin date value. Each <DMR> element corresponds to one reporting period and the length of the reporting period is defined by the <ReportingPeriodMonths> element value. The <ReportBeginDate> value is calculated by adding one day and then subtracting the number of months identified in the <ReportingPeriodMonths> element. Note that even though the <ReportBeginDate> element is not contained within the <LimitSet> element it is not a required element for upload. This date value appears at this level because there is a one-to-many relationship between the <LimitSet> and <DMR> elements where one <LimitSet> element can appear in multiple <DMR> elements. Including this value in the <LimitSet> element would result in problems for users who directly import the XML data into Access tables. xs:date SQL Server Data Type. calculated value <ReportEndDate> element [BeginDate] field within the Reporting Period Record, Parameter Records and Value Records grid tables The <ReportEndDate> element is the reporting period end date value. Each <DMR> element corresponds to one reporting period and the length of the reporting period is defined by the <ReportingPeriodMonths> element value. Note that even though the <ReportEndDate> element is not contained within the <LimitSet> element it is not a required element for upload. This date value appears at this level because there is a one-to-many relationship between the <LimitSet> and <DMR> elements where one <LimitSet> element can appear in multiple <DMR> elements. Including this value in the <LimitSet> element would result in problems for users who directly import the XML data into Access tables. xs:date SQL Server Data Type. datetime <LimitSet> element appears at various locations within the edmr Parameters web page Note that the elements within the <LimitSet> table are provided for reference only. They are not required to be a part of the XML upload file and in fact will be ignored by the edmr processing if they are uploaded. <LimitSetID> element grid s [LimitSetID] field The <LimitSetID> element contains the primary key for the <LimitSet> element / [LimitSet] table. Nullable. NO WYPDES FATAL Validation Errors: o Anything other than a non-negative integer. edmr2.0 _XML - Page 22 of 32

23 <WYPermitNumber> element appears as header information on the edmrparameters web form The <WYPermitNumber> element is the WYPDES permit number identifying the permit in question. xs:string SQL Server Data Type. varchar(12) <OutfallNumber> element appears a header information on the edmrparameters web form The <OutfallNumber> element is the text value representing the outfall in question. xs:string SQL Server Data Type. varchar(6) <SubmissionFrequencyMonths> element grid s [SF] field The <SubmissionFrequencyMonths> element contains the number of months in the submission period. See the definition of a "Submission Period" include earlier in this documentation. <ReportingPeriodMonths> element grid s [RF] field The <ReportingPeriodMonths> element contains the number of months in the reporting period. See the definition of a "Reporting Period" earlier in this documentation. <SubmissionPeriodBeginDate> element grid s [RF] field The <SubmissionPeriodBeginDate> element contains the beginning date for the submission period. See the definition of a "Submission Period" earlier in this documentation. xs:date SQL Server Data Type. datetime <SubmissionPeriodEndDate> element grid s [RF] field The <SubmissionPeriodEndDate> element contains the ending date for the submission period. See the definition of a "Submission Period" earlier in this documentation. xs:date SQL Server Data Type. datetime edmr2.0 _XML - Page 23 of 32

24 <DMRParameter> element Parameter Records grid table <DMRParameterID> element grid s [DMRParamID] field The <DMRParameterID> element contains the primary key for the <DMRParameter> / [DMRParameter] table. Required for Upload. YES Nullable: NO WYPDES FATAL Validation Errors: o Anything other than a non-null integer. <DMRID> element grid s [DMRID] field The <DMRID> element is a foreign key reference to the <DMR> element / [DMR] table. Nullable: NO WYPDES FATAL Validation Errors: o Anything other than a non-null integer. <LimitID> element grid s [LimitID] field The <LimitID> element is a foreign key reference to the <Limit> element / [Limit] table. Nullable: NO WYPDES FATAL Validation Errors: o Anything other than a non-null integer. <ActualSampleFrequencyTypeID> element grid s [ActSampFq] field The <ActualSampleFrequencyTypeID> element contains a foreign key reference to the [RefSampleFrequencyType] lookup table which has a primary key of SampleFrequencyTypeID. This field is used to indicate the actual sample frequency that applies to the parameter being reported. edmr2.0 _XML - Page 24 of 32

25 Required for Upload. YES Nullable: NO WYPDES FATAL Validation Errors: o Anything other than a non-null integer. <ReportedNumberOfExcursions> element grid s [#Excursions] field The <ReportedNumberOfExcursions> is an attribute field containing the reported of exceedances (excursions) for the parameter being reported. Required for Upload. YES <Limit> element appears at various locations within the edmr Parameters web form Note that the elements within the <Limit> table are provided for reference only. They are not required to be a part of the XML upload file and in fact will be ignored by the edmr processing if they are uploaded. <LimitID> element grid s [LimitID] field The <LimitID> element contains the primary key for the <Limit> element / [Limit] table. Nullable. NO WYPDES FATAL Validation Errors: o Anything other than a non-negative integer. <SampleTypeID> element [SmpTy] field within the Parameter Records grid control The <SampleTypeID> element contains a foreign key reference to the [RefSampleType] lookup table having a primary key of SampleTypeID. This field defines the required sample type that applies to the parameter being reported. edmr2.0 _XML - Page 25 of 32

26 <SampleFrequencyTypeID> element [SmpFqTy] field within the Parameter Records grid control The <SampleFrequencyTypeID> element contains a foreign key reference to the [RefSampleFrequencyType] lookup table having a primary key of SampleFrequencyTypeID. This field is used to indicate the requested sample frequency type that applies to the parameter being reported. <ParameterID> element [Parameter] field within the Parameter Records grid control. The <ParameterID> element contains a foreign key reference to the [RefParameter] lookup table having a primary key of ParameterID. The [Parameter] field within the grid control displays a drop-down list text description relating to the ParameterID value. The field defines the parameter that applies to the record indicated. <UnitID> element [Units] field within the Parameter Records grid control. The <UnitID> element contains a foreign key reference to the [RefUnit] lookup table having a primary key of UnitID. The [Units] field within the grid control displays a drop-down list text description relating to the UnitID value. The field defines the units that applies to the parameter record indicated. <LimitBeginDate> element appears as header information on the edmr Parameters web form The <LimitBeginDate> is the beginning date of the LimitBeginDate-LimitEndDate date range that is used as a subset of the permit effective and expiration date values. Any Effective-Expiration date range can be broken up into multiple LimitBeginDate-LimitEndDate ranges. These ranges provide a means of providing separate parameter lists and requirements for the individual LimitBeginDate- LimitEndDate ranges. xs:date SQL Server Data Type. datetime <LimitEndDate> element appears as header information on the edmr Parameters web form edmr2.0 _XML - Page 26 of 32

27 The <LimitEndDate> is the ending date of the LimitBeginDate-LimitEndDate date range that is used as a subset of the permit effective and expiration date values. Any Effective-Expiration date range can be broken up into multiple LimitBeginDate-LimitEndDate ranges. These ranges provide a means of providing separate parameter lists and requirements for the individual LimitBeginDate-LimitEndDate ranges. xs:date SQL Server Data Type. datetime <DMRValue> element Value Records grid table <DMRValueID> element grid s [DMRValueID] field The <DMRValueID> element contains the primary key for the [DMRValue] table. Required for Upload. YES Nullable: NO WYPDES FATAL Validation Errors: o Anything other than a non-null integer. <DMRParameterID> element grid s [DMRParameterID] field The <DMRParameterID> element is a foreign key reference to the <DMRParameter> element / [DMRParameter] table. Nullable: NO <LimitValueID> element grid s [LimitValueID] field The <LimitValueID> element is a foreign key referencing the <LimitValue> element / [LimitValue] table. Nullable: NO <DMR_NoDischargeTypeID> element grid s [NoDischType] field The <DMR_NoDischargeTypeID> element contains the foreign key reference to the [RefNoDischargeType] lookup table which has a primary key of NoDischargeTypeID. edmr2.0 _XML - Page 27 of 32

28 This field is used both as a flag to indicate that either a No Discharge condition has occurred or that a Discharge, No Analysis condition has occurred. The type of either No Discharge or Discharge, No Analysis is identified using the [????] field within the [RefNoDischargeType] lookup table The edmr application contains a WYPDES validation item that will check if the [DMRValue].[NoDischargeTypeID] matches that of the [DMR].[NoDischargeTypeID]. If the corresponding [DMR].[NoDischargeTypeID] is non-null and the [DMRValue].[NoDischargeTypeID] does not match the [DMR].[NoDischargeTypeID] the condition will result in a record status of 'ERR'. Required for Upload. YES <DMR_ValueQualifierTypeID> element grid s [ActValQual] field The <DMR_ValueQualifierTypeID> element contains a foreign key reference to a subset of the [RefValueQualifierType] lookup table having a primary key of ValueQualifierTypeID. The <DMR_ValueQualifierTypeID> element is a qualifier value for the <MeasurementValue> element as opposed to the <Limit> element's <Limit_ValueQualifierTypeID> element which acts as a qualifier to the <LimitValue> element. For consistency with the EPA ICIS system, this field was originally implemented as dropdown list including the following four choices: Equal To Estimated TooNumerousToCount Yes/No However, it was decided for the WYPDES system only the "Equal To" item is relevant and therefore the dropdown list has been limited to only this item. It was also decided that because other items may need be added to this list in the future that we will maintain the dropdown list functionality. To reiterate, the only legal entry for this field is as follows: Equal To Any other entry will result in a WYPDES validation status of "ERR". SQL Server Data Type. Required for Upload. int YES <MeasurementValue> element [Value] field on the Value Parameters grid control The <MeasurementValue> element is used to enter the numerical value that is to be updated. xs:double SQL Server Data Type. double edmr2.0 _XML - Page 28 of 32

29 Required for Upload. YES <DMRValueCommentText> element [Comment] field on the Value Parameters grid control The edmr system is designed to append records to the existing underlying database comment field. For example, if the existing comment field is: <edmr dt= >this is a comment</edmr> A`nd you add a new comment string: Add another comment. the resulting updated comment field will read: <edmr dt= >this is a comment</edmr><edmr dt= >add another comment.</edmr> Note that each additional comment added via the edmr application will include the dated <edmr> tag. For the <DMRCommentText> element, the value included within this element will be appended to the already existing comment. Also, since it is possible to upload the same XML file multiple times, the XML approach will check to see if the included value already exists as part of the existing comment. If it does, the <DMRCommentText> will NOT be re-appended. For the grid control, the value typed in the [CommentAdd] field will be appended to the existing comment. xs:string. SQL Server Data Type. varchar(1000) Required for Upload. YES <LimitValue> element appears at various locations within the edmr Parameters web form Note that the elements within the <LimitValue> table are provided for reference only. They are not required to be a part of the XML upload file and in fact will be ignored by the edmr processing if they are uploaded. <LimitValueID> element grid s [LImitValueID] field The <LimitValueID> element contains the primary key for the <LimitValue> element / [LimitValue] table. edmr2.0 _XML - Page 29 of 32

30 Nullable: NO WYPDES FATAL Validation Errors: o Anything other than a non-null integer. <ValueTypeID> element doesn t appear on grid The <ValueTypeID> element contains a foreign key reference to the [RefValueType] lookup table which has a primary key of ValueTypeID. <StatisticalBaseTypeID> element [SBCode] field on the Value Records grid control The <StatisticalBaseTypeID> element contains a foreign key reference to the [RefStatisticalBaseType] lookup table having a primary key of StatisticalBaseTypeID. <Limit_ValueQualifierTypeID> element [Qual] field on the Value Records grid control The <Limit_ValueQualifierTypeID> element contains a foreign key reference to the [RefValueQualifier] lookup table which has a primary key of ValueQualifierTypeID. This is the companion to the <DMRValue> element s <DMR_ValueQualifierTypeID> element. <LimitValue> element [ActValQual] field on the Value Records grid control The <LimitValue> element contains the specified limit for the record in question. Note that a <LimitValue> of -1 indicates that this a Report Only item. xs:double SQL Server Data Type. double Sample Grid Control Page The following is provided merely as a quick reference look at the grid structure. Notice that the three tiers -- Reporting Periods, Parameters, and Values -- are represented using three separate grid controls. edmr2.0 _XML - Page 30 of 32

31 Refer to the documentation related to the Treeview/Grid approach. edmr2.0 _XML - Page 31 of 32

32 edmr2.0 _XML - Page 32 of 32