Code Coverage. Design Verification Techniques

Transcription

1 Code Coverage Design Verification Techniques

2 TestBench / Code Coverage / Cycle Based Simulation RTL Verification 2 RTL Verification Acceleration Emulation Circuit Validation Testbench Code Coverage Cycle-based simulation Property Checking Equivalence Checking Functional Verification Temporal Verification Static Timing Analysis Functional Verification

3 TestBench / Code Coverage / Cycle Based Simulation Code Coverage 3 RTL Verification Acceleration Emulation Circuit Validation Testbench Code Coverage Cycle-based simulation Property Checking Equivalence Checking Functional Verification Temporal Verification Static Timing Analysis Functional Verification

4 TestBench / Code Coverage / Cycle Based Simulation Motivation 4 Key points about simulation: How much of my design has been simulated? How can I reduce my test development time? How can increase my test efficiency? Functional Verification

5 TestBench / Code Coverage / Cycle Based Simulation Motivation 5 Definition Code coverage reports the way the design has been exercised during a given simulation Testbench DUT Source file If (a = 1 or b = 0 ) then z <= data(7 downto 0) + base else z <= base << 2; endif Exercised Never exercised! Functional Verification

6 TestBench / Code Coverage / Cycle Based Simulation Motivation 6 Goal Measure the test cases completeness Indicate whether the test exercises any construct or combination of construct Report holes in verification to refine existing test cases Coverage indicates the maturity of test cases Code coverage does not prove that a design is functionally correct! Functional Verification

7 TestBench / Code Coverage / Cycle Based Simulation Where in the Flow 7 Code Coverage Synthesis RTL Entry RTL = main usage rich variety of metrics Floorplan Place & Route Gate Level Netlist (Pre layout) Gate level: very restrictive Simple metrics only (e.g. toggle coverage) Extraction Gate Level Netlist (Post layout) + Parasitics Functional Verification

8 TestBench / Code Coverage / Cycle Based Simulation Inputs 8 The complete simulation environment Design Under Test (DUT) Testbench + components Testcases Metrics Functional Verification

9 TestBench / Code Coverage / Cycle Based Simulation Technique 9 Code coverage is a dynamic technique that is applied during a functional simulation Usually coming with a testbench in order to validate the tb coverage on the design under test The code coverage itself does not perform any functional verification on the design Functional Verification

10 TestBench / Code Coverage / Cycle Based Simulation Technique Instrument the design To add checkpoints at strategic locations to record whether a particular construct has been exercised Note: testbench itself is not instrumented Functional Verification

11 TestBench / Code Coverage / Cycle Based Simulation Technique Run the simulation Using existing testcases Cumulative traces are collected Test 1 Test n Testbench DUT Source file If (a = 1 or b = 0 ) then z <= data(7 downto 0) + base else z <= base << 2; endif Functional Verification

12 TestBench / Code Coverage / Cycle Based Simulation Technique Report coverage (metrics) Coverage metrics are then displayed Functional Verification

13 TestBench / Code Coverage / Cycle Based Simulation Technique 13 Statement coverage A counter is put on each statement Counter values are annotated in the HDL code These lines are not covered by any of the testcases Functional Verification

14 TestBench / Code Coverage / Cycle Based Simulation Technique 14 Signal coverage Counts and reports the number of time a signal changes values e.g.: to check a signal goes through all possible values e.g.: to check a signal makes all possible transitions Count c_state Functional Verification

15 TestBench / Code Coverage / Cycle Based Simulation Technique 15 Branch coverage Very close to statement coverage Determines if all possibilities from a control flow are executed if (b = a) then c := 1; end if; d := c; FALSE b = a? TRUE Assume test case forces b = a 100% statement coverage 50% branch coverage c := 1; d := c; Functional Verification

16 TestBench / Code Coverage / Cycle Based Simulation Technique 16 Path coverage Counts the paths in a process. A=1 B = 1 T T F F A=1 B = 1 T T T F F F F T 100% branch coverage F T a = 1? Op = 0 Op = data F T b = 1? 50% path coverage result = op result = 1.0 / op Functional Verification

17 TestBench / Code Coverage / Cycle Based Simulation Technique 17 Finite State Machine coverage State Transition Path State0 Cond0-1 Cond0-2 Cond2-1 State1 Cond1-2 State2 Cond1-3 Cond2-3 State3 Functional Verification

18 TestBench / Code Coverage / Cycle Based Simulation Technique 18 Condition coverage Shows which combinations in an expression are responsible for entering (or not) a branch Hard to reach 100% for this criteria! if ((a = 1) or (b = 2)) then z <= d; Assume test case sets a and b either to 0 or 1 100% statement coverage 100% branch coverage 50% condition coverage False b=2 True F Covered Not Covered a=1 T Covered Not Covered Functional Verification

19 TestBench / Code Coverage / Cycle Based Simulation Technique 19 Trigger coverage Counter on the signals of any waiting list. Line Text 122 p_fsm: process(a, b, c) Count Signal Transition 200 only a changed 0 only b changed 14 only c changed Construct entered 214 times Covered 2/3 cases: 66% coverage Functional Verification

20 TestBench / Code Coverage / Cycle Based Simulation Technique 20 The Coverage completes the TB : Refine the test case to reach higher coverage => Functional Verification coverage! Define the minimum number of test cases to get the higher coverage (regression issue) => Simulation run time! Spot the most frequently called routines => Help optimizing the source code to reduce simulation time! Help analyzing the code quality => Remove unused code! e.g. if (A and not(a)) : no branch condition flagged e.g. if ( ) A <= D(0 to 3) case A when Functional Verification

21 TestBench / Code Coverage / Cycle Based Simulation Limitations 21 Restriction on the design style Better coverage if no functions nor procedures Loops should be used with caution Simulation runtime impact (in %) e.g. small block (1300 gates) Verification Navigator NCSim Ultra 10 Statement Branch Condition Path Trigger Signal 25% 25% 12000%! 25% 25% ~2000% Functional Verification

22 TestBench / Code Coverage / Cycle Based Simulation Strategy 22 RTL Level: Metrics are dependent No need to look for 100% path coverage if statement and branch covers less than 100% 1. Start with Statement and Branch coverage 2. Increase test cases to reach 100%) 3. Go on with Condition only Gate level coverage No more construct exist at gate level! Statement, branch, conditions, : all useless! Only remaining metric Toggle (simplified metric from signal coverage where signal toggling (1 to 0 and 0 to 1 transitions) is counted) Usually done on flip-flops only Functional Verification

23 TestBench / Code Coverage / Cycle Based Simulation Tools 23 Plug-in tools Benefit: rich variety of metrics and report capabilities Drawback: simulation speed! (run through an API) Coverage tool Simulator API link Simulator native capability Reverse benefit and drawback! Functional Verification

24 TestBench / Code Coverage / Cycle Based Simulation Deliverables 24 Coverage reports Sorted by metric, indicates completeness of test Warning: Reminder coverage is not function! Minimum set of test cases to get the higher coverage Suppress redundant test cases Result is to reduce simulation run time with the best coverage! Functional Verification

25 TestBench / Code Coverage / Cycle Based Simulation 25 And what s more on tools? Functional Verification

26 TestBench / Code Coverage / Cycle Based Simulation Verification Closure Flow 26 Verification Plan Tests vmanager Deploy Analyze Control Project Progress Failure Triage Coverage Analysis Verification Environment Verification Environment Verification Environment DUT DUT DUT Specman Elite Specman + Elite Specman Simulation + Elite Simulation + Simulation No Verification Closure? Yes Done! Functional Verification

27 TestBench / Code Coverage / Cycle Based Simulation Analyze Simulation Failure Triage 27 Rapid identification of failures Multiple simulations analyzed together Sort and group by error message, time to failure, Eliminate extraneous common failures Select shortest failing run Easier, faster debugging Explore coverage contribution common to failing runs May point to the functionality causing the failure Functional Verification

28 TestBench / Code Coverage / Cycle Based Simulation Analyze Executable coverage plan Coverage levels are annotated in the plan Track and Report Coverage 28 Total coverage measurement Tracks progress against milestones Allows aspects/phases, with different coverage goals HTML reports For , project website Functional Verification

29 TestBench / Code Coverage / Cycle Based Simulation Analyze Advanced Coverage Analysis 29 Correlate coverage and scenarios Run more of the most effective Group coverage holes Clear picture of uncovered areas Easy to characterize required new scenarios Correlate various coverage metrics types Discrepancies between coverage levels in various metrics Functional Verification

30 TestBench / Code Coverage / Cycle Based Simulation Control Controlling Closure 30 Refine/enhance scenarios To target coverage holes revealed during analysis Refine coverage model Based on failure analysis and the correlation between different coverage metrics Create next session description Daily/weekly/ To target all or part of the verification plan goals esif Session Runner Functional Verification

31 TestBench / Code Coverage / Cycle Based Simulation Verification Management 31 Verification Plan DUT Verification Environment Simulation Simulation Session Simulation Session Verification Closure Yes No Spec Deploy Analyze Project Progress Failure Analysis Coverage Analysis Functional Verification

32 Cycle Based Simulation Design Techniques

33 Code Coverage / Cycle Based Simulation / Property Checking - ABV RTL Verification 2 RTL Verification Acceleration Emulation Circuit Validation Testbench Code Coverage Cycle-based simulation Property Checking Equivalence Checking Functional Verification Temporal Verification Static Timing Analysis

34 Code Coverage / Cycle Based Simulation / Property Checking - ABV Cycle-based simulation 3 RTL Verification Acceleration Emulation Circuit Validation Testbench Code Coverage Cycle-based simulation Property Checking Equivalence Checking Functional Verification Temporal Verification Static Timing Analysis

35 Code Coverage / Cycle Based Simulation / Property Checking - ABV Motivation 4 How can I run faster simulations?

36 Code Coverage / Cycle Based Simulation / Property Checking - ABV Motivation 5 Concept: traditional software simulation = event driven INPUT CLOCK OUTPUT Cycle-based concept reduce the number of calculation Cycle-based simulation is no more a specific technique, it is automatically integrated in every simulation tool when usable Used at RTL level only

37 Code Coverage / Cycle Based Simulation / Property Checking - ABV Deliverables 6 Cycle-based simulation in the verification flow First trials Event driven Cycle-based Faster simulation Sanity check (triggers, constructs, Combinational loops) Logic Synthesis Event driven gate level simulation Acceleration Emulation

38 Universal Verification Methodology Design Verification Techniques

39 Verification Techniques / UVM / Property Checking - ABV Overview 2 Introduction RTL Verification Circuit Validation Verification Platform Static Timing Analysis

40 Verification Techniques / UVM / Property Checking - ABV A Global Methodology 3 RTL Verification Acceleration Emulation Circuit Validation Testbench Code Coverage Cycle-based simulation Property Checking Equivalence Checking Functional Verification Temporal Verification Static Timing Analysis

41 Verification Techniques / UVM / Property Checking - ABV Motivation 4 How can I put in place a reusable solution of verification?

42 Verification Techniques / UVM / Property Checking - ABV Motivation 5 What is simulation versus functional verification? Simulation is the process of adding test vectors into a model of the design-under-test then observing how that model behaves A traditional test bench reads test vectors or commands from a file to change the values of the signals in the DUT over time, then dumps output from the DUT to compare versus expected. This process is limited and cannot support the reliable verification of very complex systems. An effective Verification Methodology is necessary!

43 Verification Techniques / UVM / Property Checking - ABV Motivation 6 Definition: UVM is a methodology for the functional verification of digital hardware UVM is explicitly simulation-oriented, but UVM can also be used alongside assertion-based verification, hardware acceleration or emulation The hardware to be verified would be described using Verilog, SystemVerilog, VHDL or SystemC This could be behavioral, register transfer level, or gate level

44 Verification Techniques / UVM / Property Checking - ABV Motivation 7 Objective: A good verification methodology starts with a statement of the function the DUT is intended to perform. From this is derived a Verification Plan UVM test benches are more than traditional test benches UVM test benches are complete verification environments composed of reusable verification components, and used as part of an overarching methodology of constrained random, coverage-driven, verification Ref. Doulos web introduction on UVM (

45 Verification Techniques / UVM / Property Checking - ABV Environment 8 Objective: UVM is using a library of components with defined role to describe the DUT environment Uvm_envt Uvm_subscriber Uvm_subscriber Uvm_agent Uvm_sequencer Uvm_agent Uvm_sequencer Uvm_driver Uvm_monitor Uvm_monitor Uvm_driver DUT

46 Verification Techniques / UVM / Property Checking - ABV Where in the Flow 9 Testbench With UVM Synthesis Floorplan Place & Route Extraction RTL Entry Gate Level Netlist (Pre layout) Gate Level Netlist (Post layout) + Parasitics RTL = main usage RTL simulation = mandatory Gate level: reuse same verification environment

47 Verification Techniques / UVM / Property Checking - ABV Technique 10 UVM drives a dynamic technique that is based on a functional simulation This methodology includes code coverage in order to check the design under test is well covered The UVM must reflect the functional design environment of the DUT and can be re-used at any step of the design

48 Verification Techniques / UVM / Property Checking - ABV Turning Simulation into Verification 11 A good verification methodology starts with a statement of the function the DUT is intended to perform. From this is derived a verification plan Verification of complex systems must be automated. Along with the verification plan, automated checking, functional coverage and analysis are explicitly addressed by SystemVerilog and UVM. Checkers and functional coverage model take time to create but result in better quality of verification.

49 Verification Techniques / UVM / Property Checking - ABV Verification Process Start with simple directed (non-random) tests to bring up the design, 2. Move to fully random tests to explore the state space and flush out as many bugs as possible with minimum human effort devoted to test writing. This will typically achieve much less than 100% functional coverage, \ 3. Define a series of tests, where constrains can shape the random stimulus to push the design into interesting corner cases. random stimulus alone is not enough to explore all the key use cases directed or highly constrained tests can be too narrow to give good overall coverage Constrained random stimulus is a compromise between the two extremes

50 Verification Techniques / UVM / Property Checking - ABV Checkers, 13 Constrained random verification relies on Checkers, Coverage and Constraints Automatic checkers ensure functional correctness Checkers can be implemented using SystemVerilog assertions or using regular procedural code. Assertions can be embedded within the design-under-test, placed on the external interfaces, or can be part of the verification environment (supported in UVM)

51 Verification Techniques / UVM / Property Checking - ABV, Coverage Coverage provides a measure of the functional completeness of the testing, and tells you when the goals defined in the verification plan are met. SystemVerilog offers two mechanisms for coverage: property-based coverage (cover directives) sample-based coverage (covergroups). The specification and execution of the coverage model is tied to the verification plan, annotated by simulation tools

52 Verification Techniques / UVM / Property Checking - ABV and Constraints 15 Constraints provide the means to reach coverage goals by shaping the random stimulus to push the DUT into interesting corner cases. Constrained random stimulus is still random, but the statistical distribution of the vectors is shaped to ensure the DUT is pushed into interesting corner cases. SystemVerilog offers dedicated features for expressing constraints, when UVM provides mechanisms that allow constraints to be part of a test rather then embedded within verification components.

53 Verification Techniques / UVM / Property Checking - ABV Verification Reuse 16 UVM facilitates the construction of verification environments (VE), by providing reusable components within a library of SystemVerilog classes, and by providing a set of guidelines for best practice Verification productivity is enhanced by reuse. UVM enable it by: Having a modular VE where each component is clearly defined, Allowing flexibility since each components are configured and used, Allowing imported components to be customized to the application, Having well-defined coding guidelines to ensure consistency. Verification components at all layers can be reused in different environments. Low-level driver and monitor components can be reused across multiple DUT. The whole verification environment can be configured top-down by multiple tests

54 Verification Techniques / UVM / Property Checking - ABV Test sequence 17 UVM is based on OVM (Open-source Verif Method), using systemverilog language (structure close to verilog) A Verification Environment sequence: Define the different steps in systemv - Interface to the design-under-test - Design-under-test (or DUT) - Verification environment (or test bench) - Transaction - Sequencer (stimulus generator) - Driver - Top-level of verification environment - Instantiation of sequencer and driver - Top-level module - Instantiation of interface, DUT, Test (which instantiates the verification environment) - Process to run the test

55 Verification Techniques / UVM / Property Checking - ABV Inputs 18 Design Under Test (DUT) Key signal drivers (clocks, resets) Components (models) Test (=stimuli vectors)

56 Verification Techniques / UVM / Property Checking - ABV Inputs: DUT 19 Design Under Test (DUT) RTL/gate description (VHDL, Verilog) Same code as for synthesis (except for hard blocks where a model for simulation should be provided) The hardware to be verified can be described using Verilog, SystemVerilog, VHDL or SystemC It can be behavioral, register transfer level, or gate level. UVM is based on system-verilog languages

57 Verification Techniques / UVM / Property Checking - ABV Limitations 20 Same as TB with less restriction Exhaustive tests do not exist! UVM includes random test to increase coverage

58 Verification Techniques / UVM / Property Checking - ABV Verification Environment Automated Verification 21 Stimulus Stimulus stimulus Scenarios Scenarios Scenarios Automatic Stimulus Generation Physical Layer Data and Assertion Checkers Device Generation Self Checking Coverage Monitor Coverage

59 Verification Techniques / UVM / Property Checking - ABV Tools 22 Same as TestBench! Classical functional simulators (Ncsim, Questa, Vcs ) Language: systemverilog Object language Mixed of classes (software, dynamic object) and module (hardware component, hierarchy)

60 Verification Techniques / UVM / Property Checking - ABV Deliverables 23 Same as TB! Result of the test Should clearly state the passing test cases and the failing ones (if any and why!) Coverage report to deliver as an evidence for test completeness Sign-off criterion Execution of test cases on RTL and gate with no difference Good verification = high coverage with no test failing

61 Property Checking - ABV Design Verification Techniques

62 Cycle Based Simulation / Property Checking ABV / Equivalence Checking RTL Verification 2 RTL Verification Acceleration Emulation Circuit Validation Testbench Code Coverage Cycle-based simulation Property Checking Equivalence Checking Functional Verification Temporal Verification Static Timing Analysis

63 Cycle Based Simulation / Property Checking ABV / Equivalence Checking Assertion Based Verification 3 RTL Verification Acceleration Emulation Circuit Validation Testbench Code Coverage Cycle-based simulation Property Checking Equivalence Checking Functional Verification Temporal Verification Static Timing Analysis

64 Cycle Based Simulation / Property Checking ABV / Equivalence Checking Motivation 4 of How can I ensure properties of my design are valid under all circumstances?

65 Cycle Based Simulation / Property Checking ABV / Equivalence Checking Motivation 5 Property Expected behavior of a circuit based on existing specifications Safety: a condition that is always true or false Liveness: a condition that must happen regardless of timing A property sometimes is hard to prove with simulation e.g. bus arbiter managing many masters -> heavy TB!

66 Cycle Based Simulation / Property Checking ABV / Equivalence Checking Motivation 6 Under all circumstances The verification should ensure the property would never violate, regardless of any stimulus Assertion technique is use to describe the property then check this property is correctly applied in the circuit Property Checking uses the Assertion Based Verification technique (ABV)

67 Cycle Based Simulation / Property Checking ABV / Equivalence Checking Why assertions? 7 Assertions can find hard to find bugs (deep in the design) Reduce debug time: assertions are very close to the root of bugs and stop on error Improve re-use by communicating design intent (embedded in IP) Travel with the design (executable specification) Improve verification efficiency Increase debugging productivity assertions are alive all the time Could be tool independent : both static and dynamic

68 Cycle Based Simulation / Property Checking ABV / Equivalence Checking Motivation 8 Protocol compliancy A grant must be generated at max 3 cycles after a request is asserted Operation CPU_READ and CPU_WRITE must never be asserted at the same time Design quality all flip-flop can be reset no flip-flop get stuck an enumerated signal goes through all values

69 Cycle Based Simulation / Property Checking ABV / Equivalence Checking Where in the Flow 9 Synthesis Micro-Architecture RTL Entry Micro-architecture: to check the behavior (mainly protocols) RTL = main target (operation or design quality) Floorplan Place & Route Extraction

70 Cycle Based Simulation / Property Checking ABV / Equivalence Checking Inputs 10 Design model for micro-architecture (usually C language) RTL (written in hdl) Assumptions about the design environment To limit the set of admissible sequences, the strategy is to specify valid operation mode: X and Y are never asserted at the same time once a request has been asserter, no other request is asserted before a grand is asserted

71 Cycle Based Simulation / Property Checking ABV / Equivalence Checking Properties 11 Properties to check Signals in property are those seen in the architecture document or at the block boundary The verification plan should include: The properties & environment in natural language Property: Req persists until ack is asserted Environment: Ack is asserted one cycle after req The properties & environment in the tool language PSL (Property Specification Language) SVA (System Verilog Assertion)

72 Cycle Based Simulation / Property Checking ABV / Equivalence Checking Assertion Based Verification Technique 12 ABV can be a Dynamic verification Property is verified during a simulation Overhead during simulation Or a Static verification = model checking Exhaustive and semi-exhaustive formal property checking No need to provide stimuli

73 Cycle Based Simulation / Property Checking ABV / Equivalence Checking Technique 13 Property checking process: Assertion Describe the environment and the properties to be checked (can/should be done before the DUT!) The tool determines whether a property holds under the adopted assumptions If a property is violated, a counterexample is automatically generated by the tool Different languages are involved for DUT, environment, property and tool

74 Cycle Based Simulation / Property Checking ABV / Equivalence Checking Technique 14 How to define an Assertion If constraints are too strong, a misbehaviour can be missed (false positive) If constraints are too weak, non-significant misbehaviours can be spotted out (false negative)

75 Cycle Based Simulation / Property Checking ABV / Equivalence Checking Who should write assertions? 15 Architect Verification Engineer Black-box assertions: interface Use of separate assertions to verify protocols or interface assertions Designer White-Box assertions, prefer inline assertions Comment assumptions on the interface and ensure they will be fulfilled when integrated Verify some implementation invariants

76 Cycle Based Simulation / Property Checking ABV / Equivalence Checking Assertion Languages 16 Many languages exist today: VHDL assert statement Only combinational OVL (Open Verification Library) Very simple, but not powerful PSL (IEEE Property Specification Language) OVA SVA HVL like e No static tools, running during TB simulation Others proprietary languages Tools specific (PEC, HPL, ITL )

77 Cycle Based Simulation / Property Checking ABV / Equivalence Checking Using existing HDL functionality 17 VHDL assert statement assert a<b report a must be smaller than b severity 0; Verilog no assert, but: always (a or b) begin if (a >= b) begin $display( error: a must be smaller than b ; $finish; end end Pros Use of existing language and tools Cons No temporal expressions ; limited to boolean logic only

78 Cycle Based Simulation / Property Checking ABV / Equivalence Checking Limitations 18 RTL coding style for property checking No unstable combinational loops Handling tri-state signals can be a problem Deep FIFO makes issues: Strategy is to use shallow FIFO for debug purpose An easy way to do that: declare FIFO size as a top level constant The FIFO can be verified separately and then replaced by a black-box Design size limit: Block-level Property checking is CPU (fixed point) and Memory intensive task

79 Cycle Based Simulation / Property Checking ABV / Equivalence Checking Tool 19 RuleBase (IBM) Historical tool on the market Still leading edge technology Many EDA tools today Magellan (Snps), IFV (Cadence Incisive), 0-in (Mentor) A lot of start-up

80 Cycle Based Simulation / Property Checking ABV / Equivalence Checking Deliverables 20 The implementation report should state Property has been fully checked and passes Property has been checked under certain restrictions and passes Property is known to fail under specified conditions Note: design may be correct for a given implementation. Violation conditions may be used in other systems

81 Equivalence Checking Design Verification Techniques

82 Property Checking ABV / Equivalence Checking / Acceleration Overview 2 Introduction RTL Verification Circuit Validation Verification Platform Static Timing Analysis

83 Property Checking ABV / Equivalence Checking / Acceleration Circuit Validation 3 RTL Verification Testbench Coverage Property Checking Circuit Validation Equivalence Checking Verification Platform Temporal Verification Static Timing Analysis Acceleration Emulation Functional Verification

84 Property Checking ABV / Equivalence Checking / Acceleration Equivalence Checking 4 RTL Verification Testbench Coverage Property Checking Circuit Validation Equivalence Checking Verification Platform Temporal Verification Static Timing Analysis Acceleration Emulation Functional Verification

85 Property Checking ABV / Equivalence Checking / Acceleration Motivation 5 How can I prove that my two netlists have the same functionality?

86 Property Checking ABV / Equivalence Checking / Acceleration Motivation 6 Goal To check a new implementation step preserves the functionality, fastest than simulation Advantage: Equivalence is ensured whatever the stimuli are Equivalence is an exhaustive technique Example Compare two different implementations of the same architecture Compare two successive releases of the same design

87 Property Checking ABV / Equivalence Checking / Acceleration Where in the Flow 7 Synthesis Floorplan Place & Route Extraction RTL Entry Gate Level Netlist (Pre layout) Gate Level Netlist (Post layout) + Parasitics Equivalence Checking (EC) can be performed between: RTL vs RTL RTL vs gate Gate vs gate (the fastest and most used verification)

88 Property Checking ABV / Equivalence Checking / Acceleration Where in the Flow 8 RTL sign-off flow Effort is put on the RTL simulation (the reference) EC is then used to prove the netlist after synthesis, place and route behaves the same RTL Gate1 (hierarchical) Gate2 (flat) Equivalence Checking Equivalence Checking

89 Property Checking ABV / Equivalence Checking / Acceleration Where in the Flow 9 Validation after every design modification RTL optimization, scan chain or clock tree insertion, timing optimization: EC can prove the function remain unchanged Design Migration EC is a fast way to compare two designs mapped on two different libraries

90 Property Checking ABV / Equivalence Checking / Acceleration Inputs 10 Design (HDL) Recommendation: keep the reference structure (hierarchy, naming scheme) for implementation to speed up the process Assumptions To limit the exploration space and speed up the process (e.g test mode disabled to avoid discrepancies on the scan chain) Name mapping If name changed for the same functionality, provide a mapping file that allows EC to retrieve the points to compare

91 Property Checking ABV / Equivalence Checking / Acceleration Technique 11 Equivalence Checking is a static mode verification No stimuli are required That make an exhaustive verification of the whole design EC does not check the functionality versus the specification The goal is to insure the functionality does not change between to design descriptions

92 Property Checking ABV / Equivalence Checking / Acceleration Technique 12 Comparison Made on memory points (flip-flop, latches, memories) and the output of both designs The logic cones ending to these points are compared Netlist A Netlist B Logic equivalence Not Gate equivalence!

93 Property Checking ABV / Equivalence Checking / Acceleration Technique 13 Checking result Pass implementation matches the reference Fail implementation is different (function) Tool provides vectors to prove the difference Abort runtime or memory capacity overflow or too difficult to compare Check is: successful if all compare points passes Partially successful if pass or abort only

94 Property Checking ABV / Equivalence Checking / Acceleration Limitations 14 RTL coding style for equivalence checking No combinational loops No bus keeper for maximum efficiency Design complexity Strategy: to perform hierarchical verification (check low level blocks, black-box, go to upper level ) Run Time Memory

95 Property Checking ABV / Equivalence Checking / Acceleration Limitations 15 Retiming EC is a combinational process. Thus the designs must match from a sequential point of view Some basic retiming is admissible VS + x + x Clock cycle boundary

96 Property Checking ABV / Equivalence Checking / Acceleration Limitations 16 Netlist sign-off EC must be completed with Static Timing Analysis Netlist A Netlist B + Equivalence Checking Arc1 Arc2 Clock period = 5 ns Static Timing Analysis ~ = Simulation In some cases (reset, combinational loops, non synchronism), the gate level simulation remains mandatory

97 Property Checking ABV / Equivalence Checking / Acceleration Tools 17 Cadence Conformal LEC Synopsys Formality Some other EDA tools

98 Property Checking ABV / Equivalence Checking / Acceleration Deliverables 18 Documentation Views checked (RTL, gate) and the result Views partially checked e.g memory, transistor view, e.g assumptions used to achieve EC and methods to prove those assumptions (e.g. test enable) Memory/CPU usage: to reproduce the run Scripting: List of compare points + compare points removed because known to be different

99 Acceleration Design Verification Techniques

100 Equivalence Checking / Acceleration / Emulation Overview 2 Introduction RTL Verification Circuit Validation Verification Platform Static Timing Analysis

101 Equivalence Checking / Acceleration / Emulation Verification Platform 3 RTL Verification Circuit Validation Verification Platform Testbench Coverage Property Checking Acceleration Emulation Equivalence Checking Temporal Verification Static Timing Analysis Functional Verification

102 Equivalence Checking / Acceleration / Emulation Acceleration 4 RTL Verification Circuit Validation Verification Platform Testbench Coverage Property Checking Acceleration Emulation Equivalence Checking Temporal Verification Static Timing Analysis Functional Verification

103 Equivalence Checking / Acceleration / Emulation Motivation 5 Man! This simulation lasts forever How can I simulate faster?

104 Equivalence Checking / Acceleration / Emulation Motivation 6 Goal Improve Software simulation run time which is too slow for today chips reach higher simulation cycles/s using Acceleration Performances Software simulation: ~100 cycle/s Accelerated simulation: ~3000 cycle/s Note: cycle/s = simulation cycle/clock wall second

105 Equivalence Checking / Acceleration / Emulation Where in the Flow 7 Synthesis Floorplan Place & Route Extraction RTL Entry Gate Level Netlist (Pre layout) Gate Level Netlist (Post layout) Anytime the design has reached a minimum maturity (netlist is roughly stable) Targets: RTL Gate level

106 Equivalence Checking / Acceleration / Emulation Inputs 8 Design: RTL/gate level A sub-system or the whole DUT is mapped onto the acceleration box Memories are usually mapped onto physical on-accelerator memories TestBench and test The same TB and stimuli as for classical simulation If TB is synthesizable can be mapped too (emulation)

107 Equivalence Checking / Acceleration / Emulation Technique: Principle 9 Acceleration implements the software simulation kernel on dedicated hardware (CPU + memories, FPGA, ) Simulation is then run on a workstation connected to hardware boxes Triggers: -process P_1 (a,b)=> z -Process clock (every 10ns) - -Process alu (opcond, z)=> accu Software simulation Acceleration

108 Equivalence Checking / Acceleration / Emulation Technique 10 Accelerating the design DUT If (a= 1 ) then Z <= c + d; Testbench DUT: port map Clock <= not(clock) after 5 ns MAPPING Accelerator Test

109 Equivalence Checking / Acceleration / Emulation Technique: Usage 11 Simulation speed-up (2X 20X) Non regression tests Acceleration is used to prove that a new netlist behaves as the previous release throughout long test sequences Toggle count Acceleration is used to count the flip-flop toggle (hence the coverage at gate level) Manufacturing Tests Accelerating ATPG vectors (very long sequences)

110 Equivalence Checking / Acceleration / Emulation Technique: Usage 12 Interesting tests with acceleration Reset and clock generator To ensure the validity of these critical blocks that cannot be verified outside of simulation Debug port verification To ensure the TAP controller can control and observe everything required for test TAP

111 Equivalence Checking / Acceleration / Emulation Technique: Usage 13 Timing verification with acceleration Not exactly functional verification Acceleration provides a way to run back-annotated gate level simulation (wires + cells) A way to check timing violations and bus contention Place & Route Extraction Gate Level Netlist (Post layout) + Parasitics

112 Equivalence Checking / Acceleration / Emulation Limitations 14 RTL coding for acceleration X, U and Z states are supported Metastability should be handled properly at design level (synchronization stage) Asynchronous designs simulate poorly: gated clock may introduce skew problems hard to solve data clock enable

113 Equivalence Checking / Acceleration / Emulation Limitations 15 RTL coding for acceleration Behavioral RTL cannot be accelerated (not synthesizable) Testbench should be synthesizable for maximum efficiency (emulation) Testbench Triggers: -Process clock (every 10ns) -Process reset(after 100 ns) -Process tb (read file) => a API link Accelerator

114 Equivalence Checking / Acceleration / Emulation Limitations 16 Debug cycle Any change in the design requires a re-mapping on the accelerator Debugging is not so easy as in software simulation Acceleration should be used whenever the design has reached a minimum maturity Acceleration box capabilities Number of I/O On-board Memory Size Gate count

115 Equivalence Checking / Acceleration / Emulation Tools 17 Example: AXIS (without back-annotation) Capacity Up to 8 boards of 448k cells + 32MByte memory + 32 Mbyte memory 448k 448k 448k 300k 300k 300k 300k 300k 300k 300k 300k 448k OR 448k + 32 Mb memory 448k 448k 448k 448k 1 single huge design (2.5 Mcells) 8 smaller designs in parallel

116 Equivalence Checking / Acceleration / Emulation Deliverables 18 The same as for simulation Report on test completeness: Table of failing/passing test cases Toggle count for flip-flops (optional)

117 Emulation Design Verification Techniques

118 Acceleration / Emulation / Stating Timing Analysis Verification Platform 2 RTL Verification Circuit Validation Verification Platform Testbench Coverage Property Checking Acceleration Emulation Equivalence Checking Temporal Verification Static Timing Analysis Functional Verification

119 Acceleration / Emulation / Stating Timing Analysis Emulation 3 RTL Verification Circuit Validation Verification Platform Testbench Coverage Property Checking Acceleration Emulation Equivalence Checking Temporal Verification Static Timing Analysis Functional Verification

120 Acceleration / Emulation / Stating Timing Analysis Motivation 4 How can I ensure the design will be working in the context of the real system?

121 Acceleration / Emulation / Stating Timing Analysis Motivation 5 Goals Hardware checks: functional verification to ensure the hardware is interacting correctly with the actual application Accuracy: the testbench is the actual application! Software checks To offer a software development platform before the silicon is ready Run Time: Fast! Up to 1 M cycle/s

122 Acceleration / Emulation / Stating Timing Analysis Motivation 6 Platform example: a video coding/decoding system Display Camera User interface: -Host program loading -Stimuli provider (data in) -Checker (data out) Host board: -Host CPU to control the chip Emulator: -The design is mapped here Interface: -Actual devices

123 Acceleration / Emulation / Stating Timing Analysis Where in the Flow 7 Synthesis Floorplan Place & Route RTL Entry Gate Level Netlist (Pre layout) RTL Level: design synthesis is handled by emulation tool Gate level: a translation library must be provided GTECH (Synopsys) CORELIB (ST) Others Extraction Gate Level Netlist (Post layout) + Parasitics

124 Acceleration / Emulation / Stating Timing Analysis Inputs 8 Design A minimum maturity is required Planning fine debug on the emulator is hard to conduct Mapping onto the emulator is not an easy task: it does takes time (several hours for a typical design) Long and complex task to map a design onto the emulator. Verification engineer is required! Some physical environment must be built

125 Acceleration / Emulation / Stating Timing Analysis Technique 9 Principle Dynamique technique as Simulation Emulator = sea of programmable devices (FPGA) RTL + TB If (a= 1 ) then Z <= c + d; Synthesis Mapping Gates Evolution: mapping of the gate on processors (Palladium)

126 Acceleration / Emulation / Stating Timing Analysis Technique 10 Modes of operation In-circuit: good coverage emulator connected to the actual application platform Non-regression: easy debug Accelerated platform Everything (including TB) is mapped onto the emulator. Speed is higher but the TB must be often simplified (less coverage) Co-emulation: fast simulation When the TB is not synthesizable, emulator (as accelerator with a simulator) can co-emulate with an external TB. It offers a very fast simulation

127 Acceleration / Emulation / Stating Timing Analysis Limitations 11 Cost Price: up to 1 M US$ Less shareable than a SW simulator (but under continuous improvement) Time: expertise to map and drive the emulator Signals 2-state (0,1) behavior only

128 Acceleration / Emulation / Stating Timing Analysis Limitations 12 RTL coding for emulation Memories are mapped to their emulation model Need to write specific memory models Non-synthesizable constructs are not supported e.g. absolute time construct (pulse, delay) Signals crossing clock domains must always be re-synchronized Clock gating limited to clean gating (and-like) No X,U propagation in emulation Z is supported (must be resolved) All flip-flops must be controllable by reset

129 Acceleration / Emulation / Stating Timing Analysis Tool 13 Old: Mentor Celaro (until 2002) Up to 192 slots for 6M gates 128Mbyte memory (SRAM) Shareable: up to 4 designs in parallel Speed: up to 1 M cycle/s Size: up to 8 m 3, 2.5 Tons Cadence Palladium 16 M gates on 8 Gbytes memory Shareable: up to 16 designs in parallel Speed up to 1.8 M cycles/s Size: 1.4 m 3, 500 kg Mentor Veloce (2007) 16 M gates on 1 Gbytes memory per user Shareable: up to 4 users in parallel (128MG/8GB) Speed up to 1.5 M cycles/s Size: 1.3 m3, ~450 kg

130 Acceleration / Emulation / Stating Timing Analysis Deliverables 14 The same as for simulation!

131 Acceleration / Emulation / Stating Timing Analysis Prototyping 15 Goal: MAIN: early SW development - Validation To offer a SW development platform before the silicon is ready in order to debug the final application The testbench can be the actual application! HW/SW HW debugging is still possible (minor refinement) Speed: Up to 30 M cycle/s

132 Acceleration / Emulation / Stating Timing Analysis Prototyping 16 Platform example: an ARM7/Entry Level Navigation Project Prototyping Platform I2S Audio User interface: -Host program loading -RTOS (Nucleus) -SW Drivers ARM Integrator (ARM7TDMI) CDROM Drive LCD Display The ASIC is mapped onto FPGAs SRAM and SDRAM onto custom boards

133 Acceleration / Emulation / Stating Timing Analysis Limitations 17 Cost Price: ~$US 400K Need for external equipments: LSA, Oscilloscope, Clock and Pattern Generator, Not sharable as a SW simulator Time: expertise to map and drive the tool Debug Environment Poor visibility of internal nodes Observation nodes are decided at mapping time Limit = number of I/O, resources to make a node observable

134 Acceleration / Emulation / Stating Timing Analysis Tool 18 Aptix MP3/MP4 Up to 8 Mgates (10 Xilinx Virtex2V8000) Up to 10k visible nets at a time Custom Modules Aptix: SRAM, SDRAM, DPRAM, Integrator I/F, Replicas available, Speed: up to 30 M cycle/s ProDesign (Platinium) Eve (Zebu)

135 Acceleration / Emulation / Stating Timing Analysis Emulation vs Prototyping 19 Emulation Easy to map a design (1 to 3 weeks) but request a good expertise of the tool Easy for debug Available for huge design Prototyping High Speed simulation Long to map (1 to 3 months) More for early SW development than design validation

136 Static Timing Analysis Design Verification Techniques

137 Emulation / Stating Timing Analysis / Conclusion Overview 2 Introduction RTL Verification Circuit Validation Verification Platform Static Timing Analysis

138 Emulation / Stating Timing Analysis / Conclusion Temporal Verification 3 RTL Verification Testbench Coverage Property Checking Circuit Validation Temporal Verification Equivalence Checking Verification Platform Acceleration Emulation Static Timing Analysis Functional Verification

139 Emulation / Stating Timing Analysis / Conclusion Motivation 4 How can I ensure my design will work at the target frequency under all circumstances?

140 Emulation / Stating Timing Analysis / Conclusion Motivation 5 Goal: To check the design will reach the expected timing performances How: By ensuring any timing path meets the timing requirements Warning: STA does not check functionality!

141 Emulation / Stating Timing Analysis / Conclusion Motivation 6 Concept Check the data are available at the right time around the clock edge signal through static timing calculation INPUT CLOCK INPUT CLOCK SETUP HOLD OUTPUT

142 Emulation / Stating Timing Analysis / Conclusion Where in the Flow 7 Synthesis RTL Entry Floorplan Place & Route Gate Level Netlist (Pre layout) Pre layout gate level: to set up the timing environment Extraction Gate Level Netlist (Post layout) + Parasitics Post layout gate level: main target

143 Emulation / Stating Timing Analysis / Conclusion Where in the Flow 8 Pre layout accuracy Statistics based (Wire Load Model) R,C = f(area) Post layout accuracy R,C = extracted from layout

144 Emulation / Stating Timing Analysis / Conclusion Inputs 9 Design Gate level (Verilog, EDIF, VHDL) Analog blocks, memories have their own timing model Timing Annotation Gate and/or wire annotation As extracted from Place & Route Or as estimated in pre-layout analysis R1 C1 R2 C2

145 Emulation / Stating Timing Analysis / Conclusion Inputs 10 Design constraints Environment Driving cell Operating conditions Load SDF DSPF Input transition

146 Emulation / Stating Timing Analysis / Conclusion Inputs 11 Design constraints About environment Quantifies the manufacturing dispersion (oxide & metal thickness, ) Process Voltage Timing model for devices (transistors, ) is available for a range of voltage supply Temperature Timing model for devices (transistors, ) is available for a range of temperature

147 Emulation / Stating Timing Analysis / Conclusion Inputs 12 Design Constraints Timing Constraints Combo Logic Combo Logic Combo Logic Combo Logic Arrival Time Departure Time Clock period, waveform and phase shift

148 Emulation / Stating Timing Analysis / Conclusion Technique 13 Static Timing Analysis is a static technique, no stimuli are required STA insures the timing performances are reached on the functional design Note: The goal is NOT to check the functionality of the design: it must be associated with a verification technique to insure the full verification STA + EQ Check ~ TB simulation (static) (static) (dynamique)

149 Emulation / Stating Timing Analysis / Conclusion Technique: Delay Calculation 14 For gate level designs 0.4 ns Input transition A Z 0.09 pf Output capacitance Output capacitance Delay = ns Input transition Library (characterization) library.lib (STF) " , , , ", " , , , ", " , , , ", " , , , ", " , , , ", " , , , ", " , , , ", " , , , ", " , , , " , , , Transition = ns

150 Emulation / Stating Timing Analysis / Conclusion Technique: Delay Calculation 15 For custom blocks (analog, memories) Using timing models Min, Max timing Setup, hold time Access time CLOCK

151 Emulation / Stating Timing Analysis / Conclusion Technique: Block Level 16 Goal: check any timing path is within the constraints (min < T Path < max) Timing path Input port Clock pin of a sequential device Start Point Combo Logic Output port Data pin of a sequential device End Point

152 Emulation / Stating Timing Analysis / Conclusion Technique: Block Level 17 Clock management Clock pin Transition at the end points FFx skew T delay FFy

153 Emulation / Stating Timing Analysis / Conclusion Technique: Issuing reports 18 DRC issues Timing values reported by the tool are reliable if transition and capacitance are within the maximum used for library characterization Input transition Output capacitance Library (characterization) library.lib (STF) " , , , ", " , , , ", " , , , ", " , , , ", " , , , Max capacitance violation Manufacturing yield Max transition violation

154 Emulation / Stating Timing Analysis / Conclusion Technique: Issuing Reports 19 Coverage Type of Check Total Met Violated Untested setup ( 38%) 9 ( 1%) 1934 ( 62%) hold ( 37%) 17 ( 3%) 1935 ( 62%) All Checks ( 59%) 40 ( 7%) 4586 ( 34%) May be due to: - Environment assumption (e.g. report done in functional mode) - Design constructs (e.g. combinational loops cannot be verified)

155 Emulation / Stating Timing Analysis / Conclusion Technique:Top level 20 Hierarchical analysis is based on timing models for blocks Timing model may be extracted from the netlist Or carved netlist may be used Combo Logic Combo Logic Combo Logic Internal paths are removed

156 Emulation / Stating Timing Analysis / Conclusion Technique: Top Level 21 Clock tree impact at top level Clock tree balancing between blocks, IPs can be checked only at top level Block 1 Clock tree depth = 1.3 ns PAD IP Clock tree depth = 0.7 ns

157 Emulation / Stating Timing Analysis / Conclusion Limitations 22 RTL coding for Static Timing Analysis Asynchronous constructs cannot be verified Asynchronous inputs Paths between asynchronous clock domains Combinational loops

158 Emulation / Stating Timing Analysis / Conclusion Limitations 23 Accuracy of timing models Need a continuous improvement in the accuracy of timing models at standard cell level! (from linear -> NLDM -> CCS) Need a strong verification of timing model at block or IP level (ILM models) Runtime More than 1 day for flat chip level analysis Can be done in a hierarchical mode (if accurate models!) Note: always fastest than a simulation!

159 Emulation / Stating Timing Analysis / Conclusion Tools 24 Primetime (Synopsys) Accuracy proven versus transistor level timing analyzers Chip level capability (timing models management) Runtime: more than 1 day for large chips

160 Emulation / Stating Timing Analysis / Conclusion Deliverables 25 Reports Max capacitance, transition Coverage: including details about the untested nodes Clock skews all over the chip

161 Conclusion Design Verification Techniques

162 Stating Timing Analysis / Conclusion Training Objective 2 At the end of this class you are able to : Understand what is a Functional Verification What is the basic taxonomy of functional verification Discover the different Verification Techniques with a list of tools and languages Understand how to apply Vtechniques on a design Understand what is a temporal verification and the link with the functional verification

163 Stating Timing Analysis / Conclusion Verification in the Design Flow 3 Architecture -Architecture Property Checking RTL Entry Testbench Coverage Synthesis Floorplan Place & Route Extraction Equivalence Checking Gate Level Netlist (Pre layout) Equivalence Checking Gate Level Netlist (Post layout) + Parasitics Acceleration Emulation Prototyping Static Timing Analysis Layout Transistor

164 Stating Timing Analysis / Conclusion People involved in Verification 4 Architect - chip analysis - block partitioning -detailed specifications Designer (HW, SW) - block description (C, HDL) - block synthesis (Ass, Gate) - block validation (Simulation,Timing) - block implementation (P&R) - block verification (LVS, DRC) - simulation, acceleration -ABV - equivalence checking - Static Timing Analysis Verification Engineer - coverage driven verification -emulation - acceleration - equivalence checking - CAD tools, flow - ABV (property checking,..) - Training CAD Support Block & IP Integrator -chip assembly - chip verification vs spec

165 Stating Timing Analysis / Conclusion 5