Certitude Functional Qualification with Formal Verification. Jean-Marc Forey November PDF Free Download

Certitude Functional Qualification with Formal Verification Jean-Marc Forey November 2012 Springsoft Proprietary

Topics Case study presentation Why Verification Verification efficiency Formal verification Introduction to Certitude Functional Qualification Results obtained on the studied case Conclusion Q&A 2 Springsoft Proprietary

Case study presentation Synchronous fifo Clock, Reset_ ReadEn, WriteEn, DataIn FifoEmpty, FifoFull, DataOut All outputs are clocked Formal verification environment Focused on flags Leveraged from an other design 7 assertions Dynamic verification environment Focused on data path Directed test 3 Springsoft Proprietary

Why verification? Designs are made by humans Humans make mistakes Hence designs have bugs However customers / consumers want Bug free designs Safer, more reliable products More features, more powerful products How can providers satisfy the above constraints? => Have computers make designs => Wait for bugs to be reported by the users => Audit the design to squeeze out the bugs 4 Springsoft Proprietary

Verification Set of techniques to find bugs in designs Human contribution is primordial Specs, Architecture, Methodology... Implementation aspects assisted by computers For test generation To check simulation results To check which properties hold And provide counter examples, for those that don t But there are some little hurdles Size of the problem and available time How to measure verification? What is the meaning of the measure? How to combine measurements? 5 Springsoft Proprietary

Verification Verification is partial Art: what to verify, at which level, methods Risk management: what to skip, when to stop Combine techniques Verification too is subject to bugs Missing properties Missing / broken checkers Missing tests Missing scenarios Bugs in the VE hide bugs in the design How to gauge the efficiency of the verification? 6 Springsoft Proprietary

Verification efficiency Metrics to estimate what was done Bug rate Functional coverage Dynamic verification only Structural coverage(s) of the design Line, block, condition, expression, toggle, fsm, branch, path, MC/DC,??? Which one(s) make(s) sense with Formal verification? 7 Springsoft Proprietary

Metrics meaning Metrics to estimate what was done Really??? not covered => not verified But Covered doesn t mean Verified not (not covered => not verified) (verified => covered) How to combine metrics? Ex: line, toggle, fsm, functional How to combine data from dynamic / formal VE? What are the achievable scores for a given design? How much is a metric (score) telling about the VE effectiveness? 8 Springsoft Proprietary

Formal verification Some key questions about formal verification in the following slides 9 Springsoft Proprietary

Formal verification Properties How many are needed? Which ones? Only asked questions are answered... Have you got all the right questions? 10 Springsoft Proprietary

Formal verification Properties No (explicit) stimuli, but there are implicit ones All stimuli combinations are considered as being valid Unless restricted by constraints Doesn t mean they are all valid What if some combinations are illegal and no constraint was set All the properties are passing Isn t it strange if invalid inputs don t lead to output misbehaviours How many properties are missing? Which ones? Are the reached cover points valid??? Some may be reachable only through forbidden input combinations 11 Springsoft Proprietary

Formal verification Properties No (explicit) stimuli Execution traces / waveforms For counter examples only One has to believe the formal tool when it reports a pass 12 Springsoft Proprietary

Formal verification Properties No (explicit) stimuli Traces How much of the design is used to get the proofs? Which aspects of the design are verified? White box verification remains a misleading friend IU ID. B1 P1 B2 P2? 13 Springsoft Proprietary

Formal verification not (FifoFull && FifoEmpty) ReadEn WriteEn DataIn Cone Of Influence Reduced cone FifoEmpty FifoFull DataOut In the above example Inputs ReadEn/WriteEn can influence the FifoEmpty output Only 1 gate is needed to prove the property The property is necessary but not sufficient Cone of influence!= What is verified Reduced cone 14 Springsoft Proprietary

How do you find the reduced cone? Inject (artificial) bugs! At least one property fail Good All property passes Bad The same bug would be missed Can be very shocking Bugs with similar effect would be missed too Ex: not detected SA0 on FifoFull Missing properties? Over-constraints? More verification is needed Dynamic and/or formal 15 Springsoft Proprietary

Certitude principle Inject faults (artificial bugs) ReadEn WriteEn DataIn FifoEmpty FifoFull DataOut Run the verification At least 1 Fail => fault is detected (good) All Pass => fault is not detected (bad) 16 Springsoft Proprietary

How it Works Modifies RTL code to insert faults out1 = f(i1) out1 = 1 b0 // disconnect output and tie it to constant if (a) if (TRUE) // remove else branch f1(); f1(); else f2(); else f2(); a = b c a = b & c // change operator Same principle and definition is applicable to both dynamic and formal verification Combining data from dynamic and formal is immediate 18 Springsoft Proprietary

Case study results Fifo presented previously Formal verification Focused on flags Properties: P_correct_flag_interval: not(f1 ##1!F2[*0:14] ##1 F2) With F1=FifoEmpty and F2=FifoFull With F1=FifoFull and F2=FifoEmpty P_never_FE_FF: not (FifoEmpty && FifoFull) P_deassert_FE: WriteEn &&!ReadEn =>!FifoEmpty P_deassert_FF:!WriteEn && ReadEn =>!FifoFull P_FF_Stable:!ReadEn && FifoFull => FifoFull P_FE_Stable:!WriteEn && FifoEmpty => FifoEmpty 19 Springsoft Proprietary

Case study Qualification of the formal verification 28 NA and 14 ND out of 83 faults NA: not in the cone of influence of any property ND: all properties are proven on the modified design SA0 on FifoEmpty and FifoFull are ND A pair of properties don t seem effective p_correct_flag_interval: not(f1 ##1!F2[*0:14] ##1 F2) 20 Springsoft Proprietary

Case study Added 2 properties p_getfull:!readen throughout (WriteEn[->4]) => FifoFull p_getempty:!writeen throught (ReadEn[->4]) => FifoEmpty Got 2 failures (due to 2 bugs) for specific Read/Write pointer values Due to expressions like if ( && (ReadPtr+1==WritePtr) && ) Was replaced by `define ONE {{ADDR_WIDTH{1 b0}},1 b1} if ( && (ReadPtr+`ONE==WritePtr) && ) Qualification of the formal verification after fixes 34 NA and 2 ND out of 93 faults 21 Springsoft Proprietary

Case study Dynamic verification focused on data side Qualification of the dynamic verification 5 NA, 9 NP, 11 ND out of 83 faults NA: non activated NP: non propagated; no influence at the boundary of the design and test passed ND: output behaviour is different, and tests pass Improved VE and found 2 bugs on the data side Related to Read / Write in the memory when the fifo is empty/full Qualification of the dynamic verification after the fixes 2 NA, 2 NP, 2 ND faults out of 95 faults ND are detected by the static VE NP are in redundant code 22 Springsoft Proprietary

Conclusion Certitude helps understand what parts of the design are verified Even more important with formal verification Addressing verification issues pointed out by Certitude allows verifier to discover design bugs The bugs discovered with formal would probably not have been found with dynamic verification too corner case Easy to merge metrics from dynamic and formal verification environments Get a global picture of the verification strengths and weaknesses Address the issues where best suited Optimize the verification effort Certitude provides a strong unified metric for both dynamic and formal verification 23 Springsoft Proprietary

Q&A 24 Springsoft Proprietary

Certitude Functional Qualification with Formal Verification. Jean-Marc Forey November 2012