FPGA Verification How to improve verification without throwing everything away

Transcription

1 Test and Verification Solutions Experts in Verification FPGA Verification How to improve verification without throwing everything away Mike Bartley, TVS 1

2 Agenda Have you concerns with your current verification strategy? What is the impact? Will it survive increasing complexity? What is blocking your adoption of more advanced verification techniques? A look at techniques which add to your current approach 2

3 How do you currently verify your FPGA designs? 1. In the lab 2. A few directed simulations and then the lab 3. The software team do it 4. Using advanced verification techniques Constrained random, formal verification, mutations, 3

4 Do you recognise any of these concerns? My debug times in the lab are getting longer And I can t see how to reduce them Every time I fix a bug I introduce a new one My products are so complex that it is too hard to get all the bugs out before we go to market And to make a judgement call on going to market I have a lot of new design IP on my FPGA with complex interfaces I know I need a new verification strategy but my product deadlines are too tight Debug is at least 50% of development effort 4

5 Some advanced verification techniques Constrained Random Functional Code Formal Verification Regression results metrics Bug rate analysis Analysis of open issues Code review completion Mutation analysis Software running Are all requirements verified? Verification - it's all about confidence Mike Bartley, SNUG 2001 Which ones to adopt? 5

6 A few statistics Harry Foster at Verification Challenges % of all FPGA designs today have at least one embedded processor! External IP adoption increased by 69% New logic decreased by 34% 87% of all FPGAs today have two or more clock domains! Assertions Constrained-Random Simulation Code coverage 42% 36% 54% 69% 64% 72% FPGA Non-FPGA Listen to the 2012 survey Harry Foster at DVClub April 8th Functional coverage 46% 72% 0% 10% 20% 30% 40% 50% 60% 70% 80% Wilson Research Group and Mentor Graphics 2010 Functional Verification Study 6

7 The mechanics of an advanced test bench Test Functional Checker Monitor Stimulus generator constraint addr data Assertions Driver Design Under Test assert Active Passive Code 7

8 Adding value to your current test bench Functional Checker Monitor Assertions Active Existing Test Bench Design Under Test assert Existing Test Bench Passive Existing Test Bench Code 8

9 Add advanced techniques to your current test bench Technique Effort Value Code Functional Assertions Checker Constrained random Low effort to start measuring High effort to sign-off holes High effort to define a full coverage model High effort to implement the coverage model High effort to sign-off holes Effort varies with number of assertions Effort varies with sophistication of the checker High effort complex Needs a checker and fnal coverage Very useful when < 100% When 100% - need other data Check that major features are fully verified High value with well defined assertions High value for debug High value can write tests more quickly. Can consider pseudo random Very high 9

10 The mechanics of finding a bug in simulation Stimulate Propagate Design Under Test Mutation testing adds value in terms of test suite qualification. Actual Results Compare Expected Results Observe 10

11 Add advanced techniques to your current test bench Technique Effort Value Code Functional Assertions Checker Constrained random Mutation Analysis Low effort to start measuring High effort to sign-off holes High effort to define a full coverage model High effort to implement the coverage model High effort to sign-off holes Effort varies with number of assertions Effort varies with sophistication of the checker High effort complex Needs a checker and fnal coverage Low effort to adopt a tool High effort to run and analyse output Low effort for Do It Yourself Very useful when < 100% When 100% - need other data Check that major features are fully verified High value with well defined assertions High value for debug High value - High value can write tests more quickly. Can consider pseudo random Very high Very high if using tool discover quality of you verif. DIY will give useful feedback 11

12 The rise of design IP FPGA External IP increase by 138% from 2007 to 2010 Wilson Research Group and Mentor Graphics 2010 Functional Verification Study PCIe FPGA Into the lab Simulation PCIe FPGA PCIe PCIe hard ware VIP 12

13 The rise of design IP Design IP FPGA FPGA inter face Inter face VIP Simulation Into the lab FPGA Inter face hard ware 13

14 Summary Design Complexity Rises Debug takes over 50% Is your lab the best verification and debug environment? Passive test bench elements Allow you to keep your current test bench Add extra value Design IP Need to consider your verification IP strategy 2002 mean # gates 400K 2007 mean # gates 2.7M 2010 mean # gates 6.1M Wilson Research Group and Mentor Graphics 2010 Functional Verification Study 14

15 Q&A TVS was established in 2008 The TVS headquarters are in Bristol, UK with Offices in India, France and Germany TVS has over 80 verification engineers engaged around the world TVS has demonstrated it s ability to expand and contract teams according to client needs 15