CS 250! VLSI System Design

Transcription

1 CS 250! VLSI System Design Lecture 6 Design Verification ! Professor Jonathan Bachrach! slides by John Lazzaro TA: Colin Schmidt www-inst.eecs.berkeley.edu/~cs250/ CS 250 L6: Design Verification UC Regents Fall 2013/14 UCB

2 IBM Power Million Transistors A complex design... First silicon booted AIX & Linux, on a 16-die system. 96% of all bugs were caught before first tape-out. How??? CS 250 L6: Design Verification UC Regents Fall 2013/14 UCB

3 Three main components... (1) Specify chip behavior at the RTL level, and comprehensively simulate it. (3) Technology layer: do the the electrons implement the RTL, at speed and power? (2) Use formal verification to show equivalence between VHDL RTL and circuit schematic RTL. Today, we focus on (1).

4 Today: Processor Design Verification Making a processor test plan Unit techniques State machine How to write test programs

5 Lecture Focus: Functional Design Test Not manufacturing tests... goal The processor design! correctly executes programs! written in the! Instruction Set Architecture Correct == meets the Architect s Contract Intel XScale ARM Pipeline, IEEE Journal of Solid State Circuits, 36:11, November 2001

6 Architect s Contract with the Programmer To the program, it appears that instructions execute in the correct order defined by the ISA. As each instruction completes, the! architected machine state appears to the program to obey the ISA. What the machine actually does is up to the hardware designers, as long as the contract is kept. Accelerator instructions must define a contract...

7 When programmer s contract is broken... Testing our financial trading system, we found a case where our software would! get a bad calculation. Once a week or so.!! Eventually, the problem turned out to be a! failure in a CPU cache line refresh. This! was a hardware design fault in the PC.!! The test suite included running for! two weeks at maximum update rate without error, so this bug was found. Eric Ulevik

8 A 475M$ Bug

9 Three models (at least) to cross-check. The contract specification The answer (correct, we hope). Simulates the ISA model in C. Fast. Better: two models coded independently. The Chisel RTL model Logical semantics of the Chisel model we will use to create gates. Runs on a software simulator or FPGA hardware. Chip-level schematic RTL Extract the netlist from layout, formally verify against Chisel RTL. Catches synthesis bugs. This netlist also used for timing and power. Where do bugs come from?

10 Where bugs come from (a partial list)... The contract is wrong. You understand the contract, create a design that correctly implements it, write correct Chisel for the design... The contract is misread. Your design is a correct implementation of what you think the contract means... but you misunderstand the contract. Conceptual error in design. You understand the contract, but devise an incorrect implementation of it... Chisel coding errors. You express your correct design idea in Chisel.. with incorrect Chisel semantics. Also: CAD-related errors. Example: Chisel-to-Verilog translation errors.

11 For your project: X dx/dt Do bit-by-bit compare of output movie from contract and Chisel. dy/dt y

12 Optic Flow Contract CS 250 L11: Design Verification UC Regents Fall 2012 UCB

13 Computing Specify all details of fixed-point arithmetic, so that bit-accurate hardware is efficient. y x t CS 250 L6: Design Verification UC Regents Fall 2012 UCB

14 Computing Be sure to specify what happens in the first frame (i.e. what happens on reset) y x t CS 250 L6: Design Verification UC Regents Fall 2012 UCB

15 (2) How to do optimization... Think carefully about commutativity, associativity, overflow and underflow, pros and cons of saturating arithmetic, etc. Reminder: All of your implementations need to be bit-accurate with your contract model... so define it carefully!

16 Four Types of Testing CS 250 L6: Design Verification UC Regents Fall 2013/14 UCB

17 Big Bang: Complete Processor Testing Top-down! complete processor Bottom-up! how it works Assemble the complete processor.!! Execute test program suite on the! processor.!! Check results. Checks contract model against Chisel RTL. Test suite runs the gamut from 1-line programs to boot the OS.

18 Methodical Approach: Unit Testing Top-down! complete processor unit Bottom-up! how it works Remove a block from the design.!! Test it in isolation against! specification. Requires writing a bug-free contract model for the unit.

19 Climbing the Hierarchy: Multi-unit Testing Top-down! complete processor multi-unit unit Bottom-up! how it works Remove connected blocks! from design.!! Test in isolation against! specification. Choice of partition determines if test is eye-opening or a waste of time

20 Processor Testing with Self-Checking Units Top-down! how it works complete processor processor!! with! self-checks multi-unit unit Bottom-up! Add! self-checking! to units!! Perform complete processor Self-checks are unit tests built into CPU, that generate the right answer on the fly. Slower to simulate.

21 Testing: Verification vs. Diagnostics Top-down! complete processor processor!! with! self-checks multi-unit unit Bottom-up! Verification: A yes/no answer to the question Does the processor have one more bug? Diagnostics: Clues to help find and fix the bug. Diagnosis of bugs found during complete processor is hard...

22 CPU program diagnosis is tricky... Observation: On a buggy CPU model, the correctness of every executed instruction is suspect. Consequence: One needs to verify the correctness of instructions that surround the suspected buggy instruction. Depends on: (1) number of instructions in flight in the machine, and (2) lifetime of non-architected state (may be indefinite ).

23 State observability and controllability Top-down! complete processor processor!! with! self-checks multi-unit unit Bottom-up! Observability: Does my model expose the state I need to diagnose the bug? Controllability: Does my model support changing the state value I need to change to diagnose the bug? Support!= yes, just rewrite the model code!

24 Writing a Test Plan CS 250 L6: Design Verification UC Regents Fall 2013/14 UCB

25 The timeline... Top-down! complete processor Plan in advance what tests to do when... Epoch 1 Epoch 2 Epoch 3 Epoch 4 processor!! with! self-checks multi-unit Time unit Bottom-up! processor! assembly! complete correctly! executes! single! instructions correctly! executes! short! programs

26 An example test plan... Top-down! complete processor processor!! with! self-checks multi-unit Epoch 1 Epoch 2 Epoch 3 Epoch 4 unit! early multi! unit! later processor!! with! self-checks multi-unit unit diagnostics processor!! with! self-checks multi-unit unit diagnostics complete processor verification processor!! with! self-checks diagnostics unit Bottom-up! processor! assembly! complete correctly! executes! single! instructions correctly! executes! short! programs Time

27 Unit Testing CS 250 L6: Design Verification UC Regents Fall 2013/14 UCB

28 Combinational Unit Testing: 3-bit Adder Cin Number of input bits? 7 A Sum Total number of possible input values? 2 7 = 128 B 3 Just test them all... Cout Apply test vectors! 0,1, to inputs. 100% input space coverage Exhaustive

29 Combinational Unit Testing: 32-bit Adder Cin Number of input bits? 65 A B Sum Total number of possible input values? 2 65 = 3.689e+19 Just test them all? Cout Exhaustive does not scale. Combinatorial explosion!

30 Test Approach 1: Random Vectors Cin how it works A B Sum Apply random! A, B, Cin to adder.!! Check Sum, Cout. Cout When to stop? Bug curve. Bug! Rate Bugs found per minute of Time

31

32 Test Approach 2: Directed Vectors A B Cin + Power Tool: Directed Random 32 Sum how it works Hand-craft! test vectors! to cover! corner cases!! A == B == Cin == 0 Cout Black-box : Corner cases based on functional properties. Clear-box : Corner cases based on unit internal structure.

33

34

35 State Machine Testing CPU design examples DRAM controller state machines Cache control state machines Branch prediction state machines CS 250 L6: Design Verification UC Regents Fall 2012 UCB

36 Testing State Machines: Break Feedback Rst Change Next State Combinational Logic D Q R D Q G D Q Y Isolate Next State logic. Test as a combinational unit. Easier with certain HDL coding styles... CS 250 L11: Design Verification UC Regents Fall 2012 UCB

37 Testing State Machines: Arc Coverage Rst == 1 Change == 1 R Y G R Y G Change == Change == 1 R Y G Force machine into each state.! Test behavior of each arc. Intractable for state machines with high edge density... CS 250 L11: Design Verification UC Regents Fall 2012 UCB

38 Regression Testing Or, how to find the last bug... CS 250 L6: Design Verification UC Regents Fall 2013/14 UCB

39 Writing complete CPU test programs Top-down! complete processor processor!! with! self-checks Single instructions with directed-random field values. Epoch 1 Epoch 2 Epoch 3 Epoch 4 processor!! with! self-checks White-box Instructions-inflight sized programs that stress design. processor!! with! self-checks complete processor multi-unit Time unit Bottom-up! processor! assembly! complete correctly! executes! single! instructions correctly! executes! short! programs Tests that stress long-lived non-architected state. Regression : re-run subsets of the test library, and then the entire library, after a fix.

40 torture tester 1 randomly generate legal instructions and check interleavings generate chunks of working code with abstract register names along with checks add r1 r2 r3 movspr fpflags r1... bit twiddle... mov r4 fd... fadd g1 g2 g3 fmul g4 g1 g2... allocate registers and interleave chunks make sure that results are the same as before avoid illegal instructions

41 another torture tester 2 gen modular expression trees of compute with testers could correspond to instructions of clumps code gen for these to make legal instructions can randomly generate many legal tests and checks check responses

42 fuzz 3 earliest versions were the monkey 1983 feed random events into mac programs to test for bugs in macpaint fuzz or fuzzing 1998 barton miller bombarding them with random data until they crashed crashme 1991 robustness of unix os by executing random machine instructions no substiture for formal proofs of correctness techniques mutation-based mutate legal data generation-based randomly generate test data based on models of input test reduction find minimal test case

43 notes 4 has to allow for timing differences could define in terms of epsilon make sure accelerator is robust catch bugs as early as possible as much easier! store seed and replay failing test limited by weakest link checksum support for constraints and coverage

44 Formal Verification: How does it work? Cadence Labs + EECS Adjunct Prof. Verity - a Formal Verification Program for Custom CMOS Circuits Andreas Kuehlmann Arvind Srinivasan David P. LaPotin Boolean Equivalence Checking. How can you show two RTL descriptions are equivalent using static techniques? We begin by formally defining the problem... CS 250 L11: Design Verification UC Regents Fall 2012 UCB

45 The formal problem statement... Verilog RTL State Machine C A : Combinational logic. S A : Flip-flops. Question: If we initialize flip-flops appropriately...!! For any sequence of! input vectors Xo, X1,... C B : Combinational logic. S B : Flip-flops. Schematic State Machine CS 250 L11: Design Verification x x A B x FSM A z A z A S A FSM B z B C A Clk Reset C B S B Clk Reset z B y A B y A brief sketch of how the tool goes about answering the question... c Will C ever be 1?!! If no, the Verilog and schematic RTLs are! Booleanequivalent. UC Regents Fall 2012 UCB

46 FSM Equivalence: A two-step algorithm (1) Flip-flop equivalence. We assume we can find all flip-flops in RTL for each FSM. If the FSMs do not have the same number of flip-flops, FSMs not EQ. x x A FSM A C A y A z A z A S A Clk Reset (2) Prove C A = C B. Like proving DeMorgan s theorem for a homework assignment... an exercise in symbolic manipulation. c Otherwise, run an algorithm to reorder S A (and thus C A ) to match S B. Trivial solution: the user names the flip-flops. But modern tools can do it automatically. CS 250 L11: Design Verification B x FSM B z B C B S B Clk Reset z B B y But the symbols are manipulated by a computer program... and the functions are very large! Rocket science would start here... UC Regents Fall 2012 UCB

47 Conclusion -- Testing Processors! Bottom-up test for diagnosis, top-down test for verification. Unit : avoiding combinatorial explosions. Complete CPU tests: write programs that stress the hard parts of the design. Make your plan early!