OpenFlow Conformance. ONF White Paper. January 12, 2015

Transcription

1 OpenFlw Cnfrmance Test Prgram ONF White Paper January 12, 2015

2 OpenFlw Cnfrmance Test Prgram Versin 1.0 ONF Dcument Type: TR (Technical Recmmendatin) ONF Dcument Name: TR_OpenFlw_Cnfrmance_Test_Prgram_v.1.0 Disclaimer THIS SPECIFICATION IS PROVIDED AS IS WITH NO WARRANTIES WHATSOEVER, INCLUDING ANY WARRANTY OF MERCHANTABILITY, NONINFRINGEMENT, FITNESS FOR ANY PARTICULAR PURPOSE, OR ANY WARRANTY OTHERWISE ARISING OUT OF ANY PROPOSAL, SPECIFICATION OR SAMPLE. Any marks and brands cntained herein are the prperty f their respective wners. Open Netwrking Fundatin 2275 E. Bayshre Rad, Suite 103, Pal Alt, CA Open Netwrking Fundatin. All rights reserved. Open Netwrking Fundatin, the ONF symbl, and OpenFlw are registered trademarks f the Open Netwrking Fundatin, in the United States and/r in ther cuntries. All ther brands, prducts, r service names are r may be trademarks r service marks f, and are used t identify, prducts r services f their respective wners. Page 2 f Open Netwrking Fundatin. All rights reserved.

3 OpenFlw Cnfrmance Test Prgram Versin 1.0 Table f Cntents Table f Cntents... 3 Executive Overview... 4 Cnfrmant Prfiles and Test Specificatin v OpenFlw Cnfrmant Apprved Devices List... 6 ONF Apprved SDN Testing Labratries... 6 Prduct Testing Wrkflw... 7 Interperability Test Events... 8 Develpment Prcess Radmap and Milestnes... 8 Cnclusin Cntributrs References Page 3 f Open Netwrking Fundatin. All rights reserved.

4 OpenFlw Cnfrmance Test Prgram Versin 1.0 Executive Overview This paper describes the Open Netwrking Fundatin (ONF) OpenFlw Cnfrmance Testing Prgram, including its benefits, prcesses, and apprved devices. The ONF Testing and Interperability Wrking Grup seeks t accelerate the develpment and adptin f the OpenFlw Standard. The grup wrks tward this gal by ensuring standardizatin develpment thrugh testing and certificatin, fstering interperability amng vendr implementatins, and prviding an industry-recgnized certificatin fr OpenFlw cmpliance. In general, a test prgram is used t validate devices against a defined specificatin and prvides a base level f assurance that can save bth time and mney frm develpment (fr manufacturers) t deplyment (fr end-user custmers). The OpenFlw Cnfrmance Testing Prgram prvides a way fr netwrking vendrs t demnstrate cmpliance with the OpenFlw specificatin. An OpenFlw Cnfrmance certificate is the highest level f assurance t validate prduct cnfrmance with a specific versin f the OpenFlw specificatin. Netwrking vendrs can earn this certificate fr hardware r sftware devices that supprt the OpenFlw prtcl. The key bjective f the ONF Cnfrmance Testing Prgram is t be flexible and pen while maximizing the practical utility f testing and reducing the need fr redundant r excessive burden n vendrs and labratries. As such, the prgram accmmdates prduct families, requiring nly ne device within an established prduct family t be tested. This paper prvides infrmatin fr service prviders and enterprises t learn the latest status f apprved OpenFlw-enabled equipment and level f prfile supprt. Netwrk equipment manufactures will get a walk thrugh f the testing prcess, including ONF apprved labs, test specificatins, prfiles, and future develpment and prgram radmaps. CERTIFICATION BENEFITS TO SERVICE PROVIDERS AND ENTERPRISES The ONF Certificate f Cnfrmance prvides a trusted and independent level f cnfidence. It ensures that purchases are mre deplyment-ready, because they cnfrm t a particular versin f OpenFlw, which is the nly glbally acknwledged specificatin fr Sftware- Defined Netwrking (SDN). Other benefits f certificatin include: Increase in quality f deplyed prducts Cst reductin as prduct issues are fund prir t field deplyment Cllabratin with vendrs that are ONF members and at the frefrnt f SDN develpment and slutins CERTIFICATION BENEFITS TO NETWORK EQUIPMENT MANUFACTURERS Participating in the ONF OpenFlw Cnfrmance Testing Prgram prvides ONF members with bth technical and marketing benefits. Once a prduct passes the OpenFlw Cnfrmance test, a cmpany can use the trademarked OpenFlw Cnfrmant lg n sftware and hardware packaging, printed and web cllateral, and ther materials vital t getting custmers t ntice (and purchase) the prduct. Page 4 f Open Netwrking Fundatin. All rights reserved.

5 OpenFlw Cnfrmance Test Prgram Versin 1.0 The benefits f cnfrmance include: Reduced time t market due t a third-party inspectin that can find cnfrmance issues that were nt discvered during the internal develpment and QA prcesses Increased engineering expertise and talent as persnnel gain familiarity with additinal pen-surce tls and tests that can imprve all f their SDN prducts Increased sales pprtunities thrugh marketing that leverages the prven, glbally trusted OpenFlw Cnfrmant lg Glbal recgnitin as a thught leader in SDN slutins Reduced csts fr in-huse test plan develpment, implementatin, and executin IT thught leaders and custmers wh knw SDN knw and trust ONF and OpenFlw. Cnfrmant Prfiles and Test Specificatin v1.0.1 The OpenFlw Switch Test Specificatin v defines the requirements and crrespnding test prcedures that determine the cnfrmance f an OpenFlw v1.0.1 enabled switch. As the fundatin f the OpenFlw Cnfrmance Testing Prgram, this specificatin was develped and reviewed by test experts frm ONF member institutins. The test cases are derived frm the OpenFlw Switch Specificatin and the subsequent Errata v In rder t guarantee that the test specificatin is fully cnfrmant, test cases were carefully designed fr every single pint in the OpenFlw Switch Specificatin Every test case in the test specificatin has a Specificatin Reference field that references the relevant dcument, sectin, and page in the switch specificatin. The test case then verifies that the requirements f the dcument are crrectly implemented. There are three designated prfiles available fr the prgram t accmmdate specialized devices that may nt require the use f all 12-match fields utlined in the OpenFlw Switch Test Specificatin v1.0.1: Full Cnfrmance Layer 3 Cnfrmance Layer 2 Cnfrmance Please refer t Sectin 3.0 f the test specificatin fr a cmplete descriptin f prfile requirements. MANDATORY VS. OPTIONAL TEST CASES In each prfile there are mandatry and ptinal test cases. Per prfile requirements are utlined in each test sectin f the OpenFlw Switch Test Specificatin v Vendrs may request an apprved lab t perfrm additinal testing and reprting fr any available ptinal test cases. This will allw vendrs t indicate cnfrmance fr additinal ptinal feature supprt. Mandatry vs. ptinal test case requirements were based n the OpenFlw Switch Specificatin 1.0.0, but real-wrld hardware limitatins and industry demand were als taken Page 5 f Open Netwrking Fundatin. All rights reserved.

6 OpenFlw Cnfrmance Test Prgram Versin 1.0 int accunt. These requirements were carefully crafted based n vendr and user feedback during the pilt testing phase. OpenFlw Cnfrmant Apprved Devices List The OpenFlw Cnfrmant Prduct Registry 3 cntains devices that have met all the requirements fr the ONF Certificate f Cnfrmance as per the test prfile. The list includes each prduct s name, firmware versin, a simple descriptin, prduct family and ptinal supprt, as well as links t mre detailed ntes based n testing. Apprved devices are eligible t use the OpenFlw Cnfrmant lg. All usage f the lg must abide by the ONF trademark 4 and lg 5 plicies. ONF Apprved SDN Testing Labratries The fllwing labratries are authrized OpenFlw cnfrmance testing labs that are ISO cmpliant r have frmally applied fr ISO accreditatin as well as have fllwed the Lab Validatin and Prficiency requirements as defined by the ONF. BII Beijing Internet Institute, China Cntact: Li Zhen, zli@biigrup.cm CNLabs Criterin Netwrks, Bangalre, India Cntact: Kumar Jayaprakash, jp@criterinnetwrks.cm InCNTRE Indiana Center fr Netwrk Translatinal Research and Educatin, Indiana, USA Cntact: Rn Milfrd, rmilfrd@incntre.iu.edu Page 6 f Open Netwrking Fundatin. All rights reserved.

7 OpenFlw Cnfrmance Test Prgram Versin 1.0 NBL Netwrk Benchmarking Lab at Natinal Chia Tung University, Taiwan Cntact: Ying-Dar Lin, CATR Telecmmunicatin Technlgy Labs (CTTL) hsted by China Academy f Telecmmunicatin Research f MIIT, Beijing, China english.catr.cn Cntact: Chen Lei, chenlei@catr.cn UNH-IOL University f New Hampshire InterOperability Labratry, New Hampshire, USA Cntact: Erica Jhnsn, Erica.jhnsn@il.unh.edu Apprved labratries must have plicies and prcedures in place t perfrm OpenFlw testing per the defined test specificatins. All f the labratries are third party and independent. Each lab may ffer testing services fr a fee and shuld be cntacted fr mre details. VALIDATED TEST TOOLS The OFTest tl 6 has been validated as the first test tl t perfrm the ONF Cnfrmance Test Specificatin fr OpenFlw Switch Specificatin OFTest can be used t pre-test and gain cnfidence prir t cntracting with an apprved test lab. Cmmercial test tls will be available as the prgram grws and will be validated t the test specificatins by an ONF-apprved prcess. Prduct Testing Wrkflw Netwrk equipment manufacturers must fllw the steps depicted in Figure 1 t gain OpenFlw cnfrmant status. It shuld be nted that as f Nvember 2014, ONF extended cnfrmance testing t nn-members. Thus, cmpanies interested in receiving an OpenFlw Certificate f Cnfrmance, regardless f ONF membership, are entitled t use the OpenFlwcnfrmant lg with the respective versin f the OpenFlw specificatin upn successful cmpletin f the testing as described belw. Once the device supprts OpenFlw, the apprved lab will test t an agreed-upn prfile and wrk with the vendr t get 100% pass results fr mandatry test cases in the designated prfile. The lab will then issue a validatin f cnfrmance with a recmmendatin f certificatin, and will wrk with the vendr t cmplete an applicatin frm with detailed infrmatin t serve as a recrd f cmpletin. Testing may apply t prduct families as lng as the internals (firmware, chipsets, and sftware) are identical. Page 7 f Open Netwrking Fundatin. All rights reserved.

8 OpenFlw Cnfrmance Test Prgram Versin 1.0 The applicatin will be apprved by ONF and then added t the OpenFlw Cnfrmant Prduct Registry. Figure 1: Certificate f cnfrmance wrkflw Interperability Test Events ONF cnducts interperability test events twice a year t prvide vendrs the pprtunity t test their prducts with ther ONF members equipment. These events are cnsistently described as ne f the mst valuable benefits f an ONF membership. The equivalent f mnths f QA testing can be accmplished in a week s time. While cnfrmance has nt been a requirement fr participatin in these events, it is highly recmmended that vendrs self-test r cntract with ne f the apprved labs t test their prduct with an apprved testing tl prir t participatin in these interperability events. Pretesting helps cmpanies identify and fix cmmn issues that therwise cause significant delays during interperability testing, preventing participants frm getting the mst value ut f such events. These events als serve as a testing grund t validate the tls used fr test cnfrmance and prvide a frum fr feedback t the ONF specificatin writers. See the Testing and Interperability Wrking Grup page at fr mre infrmatin n these events, and white papers utlining cmmnly reprted interperability issues. Develpment Prcess Radmap and Milestnes In 2011, the ONF Testing and Interperability Wrking Grup (TIWG) was created, with a charter t develp an equally pen and cmprehensive testing and validatin prgram. A small number f cmpanies and a few vlunteer members designed a prcess whereby vendrs culd independently validate OpenFlw cnfrmance, frmulate benchmarks fr system and cmpnent perfrmance, and create resurces t develp an effective interperability matrix. ONF began spnsring PlugFests and ther technical gatherings s that cmpanies culd discver hw their OpenFlw devices (bth hardware and sftware) behaved in mre cmplex Page 8 f Open Netwrking Fundatin. All rights reserved.

9 OpenFlw Cnfrmance Test Prgram Versin 1.0 envirnments. Six such weeklng events have already taken place. The TIWG created the fllwing radmap fr its first 5 years (see Figure 2). Figure 2: ONF testing develpment radmap SWITCH CONFORMANCE DEVELOPMENT STATUS The test specificatin fr OpenFlw 1.3 is currently in develpment. The Testing and Interperability Wrking Grup identified ver 500 mandatry test cases and ver 600 additinal test cases fr ptinal parts f the OpenFlw Switch Specificatin v The wrking grup is nw in the prcess f finalizing these test cases and editing the test specificatin dcument. This well vetted prcess was als fllwed fr the v1.0 test specificatin develpment. One challenge fr the 1.3 testing prgram is the multi-table pipeline definitin. The wrking grup is currently analyzing ways t define meaningful prfiles fr the 1.3 specificatin, and hw t grup ptinal features in ways that are useful fr the market. These prfiles might target different device types, device classes, r applicatins/use cases. The wrking grup is als lking int TTP descriptins fr these prfiles t prvide an unambiguus way f describing the required functinality. The definitin f prfiles and functin grups will be accmpanied by a parallel pilt prgram in 2014Q3 & Q4 t ensure that the develped test cases are meaningful and applicable fr current hardware implementatins. This pilt prgram will be pen t all ONF member cmpanies develping test suites fr OpenFlw 1.3, and several selected, representative switch Page 9 f Open Netwrking Fundatin. All rights reserved.

10 OpenFlw Cnfrmance Test Prgram Versin 1.0 implementatins. The pilt prgram requires prviding the necessary engineering resurces t ensure that the prgram can be run effectively. T speed up prviding a first perable certificatin prgram, several prpsals were made at the ONF F2F member meeting in Santa Clara, Califrnia, in March These prpsals cvered fur majr changes: Separating mst f the ptinal features int subgrups that can be certified independently frm the main cnfrmance test suite. Several functinal grups were prpsed fr this. Examples include: Multiple cntrllers Auxiliary cnnectins Grup tables Meters and queues Advanced actins Advanced matching Multiple Tables Advanced Cunters Additinal Optinal Features Defining a base cnfrmance suite cvering the bare minimum f the functinality prvided by the 1.3 specificatin, and applied t a mst basic pipeline descriptin. This means: Only ne table (n pipeline, instructins, Gt, etc.) Only mandatry match fields Only mandatry actins and instructins In additin t the 13 mandatry match fields, the 27 ptinal match fields prvided by the spec might be differently supprted by different implementatins, and may als be srted int related functinal grups, fr example: Metadata VLAN IP QS Additinal L4 IPv4 helper IPv6 full Prvider backbne Cvering different pipelines and multiple table layuts in later iteratins f the 1.3 cnfrmance prgram. Page 10 f Open Netwrking Fundatin. All rights reserved.

11 OpenFlw Cnfrmance Test Prgram Versin 1.0 These changes will surely speed up develpment f a first 1.3 cnfrmance prgram, but there is a nn-negligible risk that this may nly be relevant t sftware implementatins and might nt prvide enugh cverage f custmer requirements t increase market acceptance. The prblem f defining relevant prfiles and TTPs fr different hardware pipelines used in the 1.3 cnfrmance prgram still exists, and needs t be slved fr brader cverage f hardware implementatins. TTPs are prpsed as ne pssible way t describe and define these different hardware pipelines. The Testing and Interperability Wrking Grup has defined a sample test suite frm the existing 1.3 test cases t meet the requirements f a basic cnfrmance prfile as utlined in the first tw prpsals. Prpsals 3 and 4 are intertwined, and depend n the result f further prfile discussins. Cnclusin New technlgies, regardless f hw widely they are being adpted, still require custmer trust in rder t thrive. End-user custmer acceptance and deplyment f new standards can be greatly assisted by the creatin f independent, third-party validatin f technical assertins made by prduct vendrs. 8 ONF s OpenFlw Cnfrmance Testing Prgram supplies that necessary verificatin f cnfrmance t the specificatin, and will sn prvide useful benchmarking f perfrmance characteristics, and device interperability that will prvide great benefits t custmers. As mre custmers insist n that validatin, a beneficial reinfrcing prcess thus begins: custmers ask fr certificatin, vendrs include certificatin, and mre trusted sales cycles allw this new market t flurish. Cntributrs Rick Bauer Michael Haugh Erica Jhnsn Rn Milfrd Pan Zhang Open Netwrking Fundatin / The Open Netwrking Fundatin is a nnprfit rganizatin funded in 2011, whse gal is t accelerate the adptin f pen SDN. ONF emphasizes the interests f end users thrughut the Data Center, Enterprise, and Carrier netwrk envirnments. Open Netwrking Fundatin, the ONF symbl, and OpenFlw are registered trademarks f the Open Netwrking Fundatin, in the United States and/r in ther cuntries. All ther brands, prducts, r service names are r may be trademarks r service marks f, and are used t identify, prducts r services f their respective wners. Page 11 f Open Netwrking Fundatin. All rights reserved.

12 OpenFlw Cnfrmance Test Prgram Versin 1.0 References The Adptin and Diffusin f Interrganizatinal System Standards and prcess Innvatins, by Matthew Nelsn and Michael J. Shaw. Available at Page 12 f Open Netwrking Fundatin. All rights reserved.