CIM Certification Program. Deborah May The Open Group

Transcription

1 CIM Certification Program Deborah May The Open Group

2 Agenda Certification Program Overview of program Test Suite Overview of Test Suite Beta Release DMTF 2002 Developers' Conference June 10-13, 2002 Page 2

3 The Open Group Global Vendor/Technology Neutral IT Consortium Bringing IT suppliers & buyers together to solve enterprise interoperability problems Develop consensus and facilitate interoperability Provide certification services and facilitate procurement of certified products CONSORTIA Evolve & integrate specs and open source technologies Provide consortia infrastructure services DMTF 2002 Developers' Conference June 10-13, 2002 Page 3

4 Free Standards Group LSB (Linux) Certification Experience - Examples DMTF 2002 Developers' Conference June 10-13, 2002 Page 4

5 Agenda Certification Program Overview of program Summary of program deliverables Certification Policy document Test Suite Overview of Test Suite Beta Release DMTF 2002 Developers' Conference June 10-13, 2002 Page 5

6 Program Summary Core Documents - form foundation of program Certification Policy Compliance Specification Implementation Components used by suppliers Certification Program Guide Problem Reporting & Interpretations System Registration Package (Registration Form, Agreements) Implementation Components used by operational staff Operations Manual Test Suite Validation Procedures DMTF 2002 Developers' Conference June 10-13, 2002 Page 6

7 Certification Policy Creation and operation of the program Achieving and maintaining certification Processes to address problems during the certification process Broad enough to cover future certification needs Does not include anything: That is likely to change frequently Is not subject to DMTF review & approval processes DMTF 2002 Developers' Conference June 10-13, 2002 Page 7

8 CIM Certification Certifying a MOF MOF file and a unique identifier string Version of the standard CIM Schema to which the MOF is compliant MOF may represent the model implemented in a software service/feature of the supplier s product Requirements for conformant MOF are specified in CIM Compliance Specification For DMTF members only DMTF 2002 Developers' Conference June 10-13, 2002 Page 8

9 Terminology Specification Authority (SA) Develops, maintains, interprets specifications Test Suite Maintenance Authority (TSMA) Certification Authority (CA) Manages day-to-day operations of program Ensures repeatability, reproducibility, and objectivity (fairness) Maintains test suites DMTF 2002 Developers' Conference June 10-13, 2002 Page 9

10 Certification Process Pre-certification activity 1a. Supplier obtains test suite Compliance Specification Certification Program Guide 1b. Supplier performs development & QA testing Certification Policy Certification Agreement 1c. Supplier becomes familiar with program requirements A DMTF 2002 Developers' Conference June 10-13, 2002 Page 10

11 Certification Process Registration A Registration Form MOF Certification Agreement (signed) Fees (if any) 2. Supplier registers for certification Problem with Registration 3. CA checks initial certification information B OK DMTF 2002 Developers' Conference June 10-13, 2002 Page 11

12 Certification Process Testing and Audit Testing problem B 4. Supplier performs formal testing Source of problem? Test results Specification, Test Suite or Certification System problem Conformance Requirements not met 5. CA audits certification information Audit successful 6. Notification & publication of certification Supplier may hold back publication for period of time Supplier submits PR via Problem Reporting & Interpretations Process Certification Register Certificate DMTF 2002 Developers' Conference June 10-13, 2002 Page 12

13 Certification Awarded based on validation of both test results and supplier s claim of conformance Conformance demonstrated through use of test suites Testing provides an indicator of compliance DMTF 2002 Developers' Conference June 10-13, 2002 Page 13

14 Supplier Obligations Supplier must warrant and represent that MOF meets, and will continue to meet, the applicable conformance requirements Which means that: MOF conforms to an industry standard specification MOF will remain conformant throughout the life of the MOF If there is a non-conformance, the MOF will be fixed in a timely manner DMTF 2002 Developers' Conference June 10-13, 2002 Page 14

15 Duration & Renewal Certification is valid only during period in which supplier is DMTF member Certification must be renewed annually in conjunction with renewal of DMTF membership Supplier must demonstrate that Certified MOF continues to meet conformance requirements DMTF 2002 Developers' Conference June 10-13, 2002 Page 15

16 Certification Register Public, web-accessible record of all Certified MOFs Includes: Name of supplier Unique identifier for MOF Version of the standard CIM Schema against which the MOF was certified Contents of MOF file DMTF 2002 Developers' Conference June 10-13, 2002 Page 16

17 Test suite acceptance process Documented process for formally accepting new test suites for use in the program New test suite releases must: Undergo a beta testing process Faults found are rectified prior to approval process Be approved by DMTF prior to use for formal testing CA maintains list of currently approved test suites on its web site DMTF 2002 Developers' Conference June 10-13, 2002 Page 17

18 Test Suite updates Maintenance release Fix test suite problems or interpretations of the spec 2-week beta test period Once approved, replace existing releases after 3- month overlap period Enhancement release Functional upgrade as well as extensions to coverage 3-month beta test period Once approved, replaces existing releases after 6- month overlap period DMTF 2002 Developers' Conference June 10-13, 2002 Page 18

19 Problem Resolution Problem Reporting & Interpretations Process Allows any supplier to raise questions about the specification, test suite or certification system and get definitive answers Answers must be timely Answers must have industry consensus Answers must be available to all members Anonymity must be preserved DMTF 2002 Developers' Conference June 10-13, 2002 Page 19

20 Problem Resolution Supplier files a Problem Report for a problem found during certification process CA categorizes PR Test Suite or Certification System issues Specification issue Preliminary Review by PR review group Opinion Submitted by TSMA or CA A 5 business days max 3 business days max DMTF 2002 Developers' Conference June 10-13, 2002 Page 20

21 Problem Resolution A Further Consideration Detailed Review by PR review group Initial PR response Resolution (Interpretation, TSD, CSD, Rejected) 10 business days max CA Notifies Supplier of Resolution Final Resolution? No Yes Supplier Satisfied? Yes Return to Certification Process Provide preliminary resolution to Supplier (if possible) No Supplier invokes appeals process DMTF 2002 Developers' Conference June 10-13, 2002 Page 21

22 Appeals Process Suppliers can appeal decisions made by CA or DMTF Two-level appeals process: Technical Review & Board Review Deterministic timeframes for response, escalation Supplier raises appeal via CA CA arranges review & monitors process Anonymous Review? Yes - CA represents supplier No - Supplier presents case for review DMTF 2002 Developers' Conference June 10-13, 2002 Page 22

23 Confidentiality All information is confidential between supplier and CA during certification process Once certification is achieved, it s public information - supplier info and MOF entered into certification register Test results always confidential Option to remain confidential for up to 6 months DMTF 2002 Developers' Conference June 10-13, 2002 Page 23

24 Re-certification Scope of certification: submitted MOF Any modifications to MOF constitute new MOF Includes modification, addition, or removal of any class, property or method Requires re-testing and re-certification Option to replace existing entry in certification register or add new one Renamed MOF (modify identifier) with no changes to MOF requires amendment to certification register only DMTF 2002 Developers' Conference June 10-13, 2002 Page 24

25 Agenda Certification Program Overview of program Test Suite Overview of Test Suite Beta Release DMTF 2002 Developers' Conference June 10-13, 2002 Page 25

26 Test Suite Validates submitted MOF file against CIM Specification and a specified version of the standard CIM Schema Formal Testing Test suite hosted on CA s web site Testing performed by supplier Results submitted to CA for audit Pre-certification Testing Downloadable commandline version of test suite available DMTF 2002 Developers' Conference June 10-13, 2002 Page 26

27 Example Walkthrough of test suite execution using the web-based version User enters URL of test suite server into their browser Initial test suite interface will prompt for user and MOF information DMTF 2002 Developers' Conference June 10-13, 2002 Page 27

28 User Information DMTF 2002 Developers' Conference June 10-13, 2002 Page 28

29 Upload MOF file User specifies version of standard CIM schema to validate against Browse file system to select MOF file to upload Selecting Submit will upload the MOF file to test server & execute test suite MOF to be tested must be contained in a single file DMTF 2002 Developers' Conference June 10-13, 2002 Page 29

30 Upload MOF file DMTF 2002 Developers' Conference June 10-13, 2002 Page 30

31 View Results Upon completion of test execution, results will be displayed in browser by class Each property & method in the class will be listed with detail on the name, result and information on failures (if any). Properties will also indicate attribute as required, key or extended DMTF 2002 Developers' Conference June 10-13, 2002 Page 31

32 View Results DMTF 2002 Developers' Conference June 10-13, 2002 Page 32

33 Submit results to CA At bottom of results window, user has option to submit results to CA for certification or delete the results If user is satisfied with results Enter brief description of results for CA Select Submit Results stored on test suite server & notification sent to CA If user wishes to delete results, they will be removed from server & user may repeat testing DMTF 2002 Developers' Conference June 10-13, 2002 Page 33

34 Submit results to CA DMTF 2002 Developers' Conference June 10-13, 2002 Page 34

35 CIM Certification Test Suite Beta Release Accessible via the WBEM tools area of the DMTF web site: DMTF 2002 Developers' Conference June 10-13, 2002 Page 35

36 Questions?