Conformity Assessment Schemes and Interoperability Testing (1) Keith Mainwaring ITU Standardization Bureau (TSB) Consultant Moscow, 9-11 november 2011 Contents The benefits of conformity assessment Conformity assessment schemes The guidelines - ISO CASCO The practice - telecommunications sector conformity assessment schemes Interoperability testing Summary and conclusions 1
The Benefits of Conformity Assessment Why? Conformity Assessment Standardise 2
Why Standardise? Compatibility between equipment from different sources Manufacturers Users (e.g. network operators) Interface characteristics & system behaviour Between systems Between components within a system Ensure interoperability Reduce barriers to trade Provides governments and regulators with best practices Benefits of Standardisation Encourage adoption of technology Reduce risks Help level the industry playing field & encourages competition Create economies of scale Improve efficiency Increase size of market Overall benefit of the industry 3
Why test conformance? Avoid equipment incompatibilities due to: Different interpretations of a specification Errors in implementation Choice of different options Conformance Testing Methodology X.290 Series OSI conformance testing methodology and framework for protocol Recommendations for ITU-T applications X.290 General concepts X.291 Abstract test suite specification X.292 The Tree and Tabular Combined Notation (TTCN-2) X.293 Test realization X.294 Requirements on test laboratories and clients for the conformance assessment process X.295 Protocol profile test specification X.296 Implementation conformance statements 4
Formal Description Techniques Z.100 - Z.109 Specification and Description Language (SDL) Z.110 Z.119 Application of formal description techniques Z.120 Z.129 Message Sequence Chart (MSC) Z.150 Z.159 User Requirements Notation (URN) Z.160 Z.179 Testing and Test Control Notation (TTCN-3) Why assess conformity and certify? Give confidence that products requirements are met Benefits the user as they can make better purchase decisions Benefits the supplier as products may more easily gain market acceptance Helps level the industry playing field & encourages competition WTO Technical Barriers to Trade Agreement recognises the important contribution that conformity assessment schemes can make by improving efficiency of production and facilitating the conduct of international trade Provides governments and regulators with best practices 5
Design of conformity assessment scheme Certification scheme should be related to the degree of risk associated with non-compliance considering such aspects as safety, health or environmental impacts, durability, compatibility and suitability for intended use Cost should not be excessive Should ensure transparency and neutrality Conformity Assessment Schemes The guidelines - ISO CASCO The practice - telecommunications sector conformity assessment schemes 6
Conformity Assessment Guidelines ISO Conformity Assessment Committee (CASCO) ISO/IEC 17000 Series Conformity assessment ISO/IEC 17000:2004 Vocabulary and general principles. ISO/IEC 17001:2005 Impartiality - Principles and requirements ISO/IEC 17002:2004 Confidentiality - Principles and requirements ISO/IEC 17003:2004 Complaints and appeals - Principles and requirements ISO/IEC 17004:2005 Disclosure of information - Principles and requirements ISO/IEC 17007: 2009 Guidance for drafting normative documents suitable for use for conformity assessment ISO/IEC 17011:2004 General requirements for accreditation bodies accrediting conformity assessment bodies ISO/IEC 17021:2011 Requirements for bodies providing audit and certification of management systems ISO/IEC 17025:2005 General requirements for the competence of testing and calibration laboratories ISO/IEC 17030:2003 General requirements for third-party marks of conformity ISO/IEC 17040:2005 General requirements for peer assessment of conformity assessment bodies and accreditation bodies ISO/IEC 17050-1:2007 Supplier's declaration of conformity - Part 1: General requirements ISO/IEC 17050-2:2004 Supplier's declaration of conformity - Part 2: Supporting documentation 7
ISO/IEC conformity assessment guides ISO/IEC Guide 23 Methods of indicating conformity with Standards for third Party certification Systems ISO/IEC Guide 28 Conformity assessment - Guidance on a third-party certification system for products ISO/IEC Guide 60 Conformity assessment - Code of good practice ISO/IEC Guide 65 General requirements for bodies operating product certification schemes ISO/IEC Guide 67 Conformity assessment fundamentals of product certification ISO/IEC Guide 68 Arrangements for the recognition and acceptance of conformity assessment results ISO / IEC Guide 67 Conformity assessment fundamentals of product certification As products are designed, produced, distributed, used and ultimately disposed of, they may give rise to societal concerns. A very frequent concern is simply whether a product is what it appears to be. Concerns can involve such product attributes as safety, health or environmental impacts, durability, compatibility, suitability for intended purposes or for stated conditions, and similar considerations. Product certification benefits consumers as they are able to make better purchase decisions about products and also benefit suppliers as by demonstrating conformity they may more easily gain market acceptance. 8
Consideration of Risk The type of activity undertaken to demonstrate conformity of product with requirements is often determined by the consequences of nonconformity. When consequences are insignificant or not severe, society may (require) expect little or no demonstration of conformity of product since the problems generated can be easily addressed and solved after they occur. In these cases the supplier's claims may be sufficient but they may be complemented by third-party product certification on a voluntary basis. However, where the consequences of nonconformity are significant, society may demand completion of activities that demonstrate conformity to requirements prior to allowing the product on the market, concurrent with the product appearing on the market, or both. One method of providing such assurance is through product certification. Product Certification Should address the concerns of users by instilling confidence regarding fulfillment of requirements May be used by suppliers to market their products Should not require excessive resources and so be overly expensive 9
Who performs conformity assessment? 1 st party the supplier 2 nd party the user 3 rd party an independent body ISO define product certification as a 3 rd party conformance assessment activity ISO / IEC Guide 65 General requirements for bodies operating product certification schemes 3 rd party certification scheme Non-discriminatory accessible to all applicants whose activities fall within its declared field of operation. There shall not be undue financial or other conditions. Access shall not be conditional upon the size of the supplier or membership of any association or group Standard specifications required If explanation is required as to the application of these documents for a specific certification system, it shall be formulated by relevant and impartial committees or persons possessing the necessary technical competence, and published by the certification body. 10
ISO / IEC Guide 65 Requirements (1) Organization of certification body Impartial Transparent Legal entity Competent personnel Independence ensure that activities of related bodies do not affect the confidentiality, objectivity and impartiality of its certifications, and it shall not 1) supply or design products of the type it certifies, 2) give advice or provide consultancy services to the applicant as to methods of dealing with matters which are barriers to the certification requested, 3) provide any other products or services which could compromise the confidentiality, objectivity or impartiality of its certification process and decisions Complaints procedure ISO / IEC Guide 65 Requirements (2) Operations Specification of the technical requirements for conformity (including testing, sampling & inspection requirements) Monitor the suitability and competence of the organisations performing testing, inspection and certification/registration Subcontracting Agreement covering confidentiality etc Certification body keeps full responsibility Quality System Conditions and procedures for granting, maintaining, extending, suspending and withdrawing certification Internal audits and management reviews 11
ISO / IEC Guide 65 Requirements (3) Documentation Records Confidentiality Changes in certification requirements Appeals, complaints and disputes Application procedure Evaluation procedure Pre-evaluation, evaluation and report Decision on certification Surveillance Use of licenses, certificates and marks of conformity Complaints made to suppliers ISO/IEC Guide 65 Testing Options Initial testing of a product and assessment of its suppliers quality systems, followed by surveillance that takes into account the factory quality system and the testing of samples from the factory and the open market OR Initial testing and surveillance testing OR Type testing only 12
Outline of conformity assessment process End users gain independent assurance that products meet their requirements Certification body issues conformity assessment certificate; accredited & recognised internationally Testing Laboratory performs tests on products & issue test reports; accredited by a national authority and recognised internationally Standards Development Organisation produces base specification & associated test specifications IEC CA structure Conformity Assessment Board Policy Strategy Projects Working Groups IECEE System for conformity testing and certification of electrotechnical equipment and components IECEx System for certification to standards for equipment for explosive environments IECQ Approval and certification system for electronic components IEC Conformity Assessment Schemes 13
IEC Conformity Assessment Principles Peer assessment Mutual recognition On-line databases Thank You! (to be continued.) 14