1 ADVANCED DIGITAL IC DESIGN (SESSION 6) Digital Verification Basic Concepts
Need for Verification 2 Exponential increase in the complexity of ASIC implies need for sophisticated verification methods to be incorporated in the ASIC design process. 70% of time spent in verification Catch the bug as early as possible So catch it in simulation saves time and money. Imagine respin of a chip if the same bug is caught in Silicon the famous FDIV bug in Intel
Tradeoffs 3 Building test benches involves Tradeoffs between: Performance How fast we can test? Efficiency Test coverage and fault grading? Extensibility Evolution with the design Design and verification go together.
What is verification? 4 A process that ensures conformance of a design to some predefined set of expectations. The expectations defined by the specifications
Types of Verification 5 Functional verification of RTL Gate-level simulation, to verify that the synthesized netlist matches the expected functionality Formal Verification (equivalence checking) to make sure that the gate level netlist is equivalent to the RTL. Timing Verification, to verify that the design can run at speed this usually involves a static timing tool.
Functional Verification 6 Run at RTL level and Gate Level too. It involves at RTL level Specifying test cases Creating a test environment Creating the actual tests Ensure that all interesting cases are covered At gate level in addition to above it involves Gates to RTL equivalence checking (Formal Verification) Gates-to-RTL equivalence checking (Formal Verification), reset conditions, clocks, race conditions etc. are checked.
Test Benches 7 With invention of HDLs like Verilog, simulators can compute the responses which could be manually tested. t Use of bus functional models which read stimulus from files or procedurally created the stimulus and applied it to the DUT. The bus models and the environment around the DUT are collectively known as the test bench.
Functional Verification Flow 8 Design Specification Strategy Phase Verification Plan Test Bench Phase Regression and Coverage Regression and Coverage Phase
Strategy Phase 9 Identify Test Cases Level of abstraction of Test bench Stimulus Generation Policy Results monitoring Policy E.g. Cycle accurate, packet accurate. What level of stimulus does it receive? E.g. random, directed, directed random Real-time or post-processing
Strategy Phase (continued) 10 Finding Interesting test cases The exhaustive set of test cases for a million gate design may be in billions!!! The challenge is in narrowing that test space to manageable and practical set without compromising the functionality
Regression and Coverage 11 Regression Ability to run tests periodically in batches, so the stimulus must be easily reproducible and pass/fail be automatically detected. Coverage How much of the design has been tested. Back annotation The Verification engineer may look at coverage to modify The Verification engineer may look at coverage to modify or add more test cases. Get very close to 100% coverage is the objective.
Verification Flows 12 Depends on complexity of design Unit-level checks functionality Chip/system level checks the interconnection between different units. Stimulus generation and result-checking strategies depend on the level of abstraction on which verification is done.
Functions of the Test Bench 13 Generate the stimulus Apply stimulus to the DUT Check the results to verify that the test was successful that is, the output of the DUT conformed to expectations.
Generating Stimulus 14 Binary Stimulus sequence of 0s and 1s driven into the inputs of the DUT User Stimulus a directive from the user to a test bench Finally translates to binary stimulus If the binary stimulus is got from user-specified data then it is called directed testing If the binary stimulus is generated randomly then it is called If the binary stimulus is generated randomly then it is called random testing
15 Checking Results (visual checking) Always useful Works for simple designs Prone to human errors Not suitable for large design debug environment.
16 Checking Results (Post-processing of logs ) This is good because it does not affect the simulation performance, as it is post simulation. Logging outputs at different levels a very effective technique to locate faults without excessive log sizes. Difficulty is that the bug is located only after the whole test is completed. Necessary state information at time of the bug need not be necessarily available.
17 Checking Results (real-time monitoring ) Implementing real-time monitors that check results in a fly. Useful for debugging, as it may flag an error as soon as it is detected and hence state information can be easily inferred. Slows down the simulation
Test cases 18 Test cases for each test bench has to be identified. Three main ways for this: Test by features Test by interfaces Test by corner cases
Test by Features 19 Developed eloped by listing all the design features specified in the document for the DUT a complete functionality list for the DUT. Eliminating Redundancy The major time-reducing factor in feature testing Limitation: Missing interesting cases that are very hardware specific Challenges: Ensure that the specification used to generate the test-by-features test plan is complete and accurate Invent the test cases that best verify the features being tested.
Testing by Interfaces 20 Best suited for protocol testing Limitations: i i An interface test does not account for all possible states of the interfaced components. For a given test t sequence the interfaced components may behave differently in different states. To overcome the limitations a use of test-by-features test plan in coordination with test-by-interfaces test plan is highly beneficial.
Testing by corner cases 21 The boundary conditions of a DUT feature The boundary conditions of the DUT implementation of a feature
Summary of Test Plan Methodologies Testing by Features Testing by Interfaces Testing by corner cases To Generate the test plan List the documented feature of the DUT List all combinations of transactions on all interfaces Study the implementation of DUT features Advantages of the Test Plan Guarantees that advertised features are tested. Testing is not complete without full test-by-features coverage Exposes transactions implied by protocols supported at interfaces but not explicitly named in the DUT feature list. Uncovering test cases not easily inferred from specifications or protocols. Disadvantages of the Test Plan Missing corner cases Missing DUT features that require a sequence of transactions Requires detailed knowledge of DUT implementation 22
Which Test Plan to use? 23 Use all three as one complements the other in advantages and disadvantages All three methods helps achieve the ultimate goal for the verification engineer: A Fully tested bug-free design
Test Generation Schemes 24 Once test plan is listed, the next step is to generate stimulus to perform the test. Test generation schemes are the general methodologies used to generate this stimulus. The main schemes are: Exhaustive Directed Random and Directed Random Sliding window (sweep) The Zeroth Law of Test Generation: Any generated test should yg successfully run to completion and pass on an non-erroneous DUT
Reproducibility 25 A must for both passing and failing tests Debugging failed tests Regression for passing tests Reproducing random tests the randoms are only pseudo-random (Verilog random facilities are also pseudo random in nature).
26 Overview of Test Generation Schemes Complete Test Space Exhaustive Testing Directed Testing Random Testing Directed random Testing Sliding Window
Exhaustive Testing 27 All possible input combinations covered Generated Algorithmically Guarantees complete test space coverage I ibl ith d d i d t hibiti l Impossible with modern designs due to prohibitively large test spaces
Directed Testing 28 Testing of simple and completely unrelated features of a DUT is where this testing excels. Important test cases (Corner cases) have to kept in mind for Directed testing. Expected results must be known in advance.
29 Random and Directed Random Testing Unearthing the unexpected The most closest to a real-time in-field test Random tests are long Long Tests to be re-run on a bug being unearthed. They are necessary say to fill up a long FIFO. A generic framework generating different tests based on random seeds and hence reproducible. Careful initialization of random tests finds bugs more quickly starting at some known context Directed Random testing needs: Parameters and constraints Intelligent test generators with feedback
Sliding Window 30 Used in directed environment or random environment Sweep across parameters and transactions whose result depend upon ordering of them sweep across all possible permutations Sweeping all parameters in a large design is outof-reach and that is the biggest disadvantage.
Logging and Error Checking 31 Human testing of log dumps and waveforms Automated Checking Running tests all throughout without human intervention is a big advantage. Regression tests requires automated checking facility Log information hierarchical dump sufficient enough to retrieve the context of a system
Formal Verification 32 How to ensure full coverage Model Checking: Define a property like fairness and ensure that a model confirms to this property Equivalence checking: Checks whether all models are equivalent functionally