A Low Cost Checker for Matrix Multiplication

Transcription

1 A Low Cost Checker for Matrix Multiplication Lisbôa, C. A., Erigson, M. I., and Carro, L. Instituto de Informática, Universidade Federal do Rio Grande do Sul Abstract The implementation of a low cost checker, using an extension of a randomized verification technique is proposed as an approach to detect errors in matrix multiplication operations with a small overhead in both area and performance. The proposed extended technique is shown to be able to detect all errors affecting a single element of the product matrix, thereby providing a significant improvement in the fault detection capabilities of the original one. 1. Introduction During the past five decades, as technology evolved, the problem of designing reliable systems based on unreliable electronic components, instead of being solved, has aggravated. Despite (and because of) the continuous evolution in research, design, engineering, materials, manufacturing processes and test mechanisms, in the 21 st century the industry continues to face growing challenges in order to manufacture economically feasible components with acceptable levels of yield and reliability. Previously a concern only for mission critical applications, errors due to the effects of transient pulses produced by radiation and other interferences are now being considered by the design community as a concern for all kinds of applications, since these errors are much more likely to occur in designs based on future technologies. The technology evolution towards nanoscale leads to the possibility of manufacturing chips with up to devices. Not only the number of transistors, but also the speed of the circuits has increased with the advent of deep sub-micron technology. All together, those facts imply in higher sensitivity of combinational logic to soft errors, and, as shown in Figure 1, from [1], while the SER of SRAM memories remains almost stable with technological scaling, the SER of logic has been always increasing. Figure 1. SER evolution: SRAM x logic [1] For future technologies, solutions that impose high temporal or space redundancy will impair the ability to explore the main advantages of the technology evolution. Moreover, the extreme reduction of cycle times will lead to transient pulses with duration longer than one cycle [2], discarding all currently known techniques that rely on temporal tricks to cope with soft errors, such as [3]. Therefore, new paradigms must be adopted in the design of circuits to be manufactured using those future technologies. In this paper, we propose a low overhead approach to check the correctness of matrix multiplication, which can be applied both to hardware and software implementations. This approach is an extension of a technique proposed by Freivalds [4], that uses randomized computation in order to detect errors in the product with a given probability. Implementations of this approach to harden circuits for matrix multiplication using the proposed technique are simulated, and it is demonstrated that it reaches beyond the limits of Freivalds technique, providing assured detection of any occurrence of faults affecting one element of the product matrix, and triggering the mechanism to correct it. This paper is organized as follows: section 2 discusses related work and shows that current soft error mitigation techniques will no longer be effective to build reliable circuits using future technologies. Section 3 describes the Freivalds technique and its proposed extension. Section 4 presents sample circuits

2 in whose design the proposed approach was applied, together with data that confirms that it allows the detection of transient errors affecting one element of the product with small area and/or performance penalty. In section 5 we comment the obtained results and also describe future work. 2. Related and previous work The reliability of circuits manufactured in future technologies became a major topic of discussion and research in recent years. While some applications cannot accept even small error probabilities, others, such as multimedia and digital signal processing, can stand some small deviations in the results of computations caused by soft errors, which are infrequent. For the latter class of applications, statistical computation can provide enough fault tolerance, and this was the starting point of the research that lead to the technique proposed in this paper. While successful mitigation techniques have already been devised to protect memories against soft errors, the protection of combinational logic, mainly against multiple simultaneous upsets, is a relatively recent concern and still lacks efficient solutions [5]. Due to the variability of their vulnerability periods, the SER of combinational logic is harder to quantify, and so far the mitigation of soft errors in those circuits has been dealt with through redundancy and larger transistors, with obvious costs in area, power and even performance. The authors of [3] propose the use of a duplicated latch to store a delayed sample of the output of the circuit, and the comparison of the two results in order to determine whether, due to a transient fault, the output was wrong or not. This approach assumes that the transient pulse duration is shorter than the delay used to store the duplicated output. Another approach to deal with soft errors is proposed in [6]. In this approach, the critical path of the combinational circuit is hardened through the duplication of gates and transient errors are detected through comparison of the duplicated outputs. Regardless of overhead issues, those approaches are likely to become useless for circuits to be manufactured using future technologies. As shown in [2], the scaling of dimensions and speed of future CMOS devices will not be followed by a proportional scaling in the duration of transient pulses. This will lead to transient pulses longer than the duration of a cycle, thereby precluding the use of the aforementioned proposed solutions. Experiments conducted by our group, in which transient faults were injected in adders and multipliers implemented with the 32 nm CMOS technology, have confirmed that, for some input combinations, transient pulses that last longer than one cycle can produce multiple bit errors. Since the cost of redundant hardware that could correct faults that span through multiple clock cycles would be excessive, the general solution in the future scenario favors detection of the fault occurrence, and re-computation of the required data whenever the fault effect dissipates. However, duplication of components and detection of the errors by comparing the generated outputs is still extremely expensive, and other detection techniques must be developed. When space redundancy is used to implement a fault tolerant solution, there is nothing one can do with less than one extra transistor or gate [7]. Therefore, the use of redundancy at the transistor or gate level, with the intrinsic fine granularity, is not possible when area overhead is a concern. With this is mind, the minimum feasible level for the use of space redundancy would be macro-system level. The use of parallel computation to check the results produced by a processor has been proposed by several authors. In [13], the use of a functional checker that is simpler than the core processor, because it receives the instruction to be executed together with the values of the input operands and of the result produced by the core processor, is proposed. Once the result of the operation is obtained by the checker, it is compared with the result produced by the core processor. If they are equal, the result is forwarded to the commit stage of the processor s pipeline, to be written to the storage. If they differ, the result calculated by the checker is forwarded to the commit stage and the core processor s pipeline is flushed and the processor is restarted at the next instruction using its own speculation recovery mechanism. This technique assumes that the checker never fails, and requires modification of the core processor hardware. The use of larger transistors, triple modular redundancy, and duplication of the checker are proposed by the authors as solutions to harden the checker against faults. However, all those alternatives imply in higher area overheads. In [8], the use of a non intrusive infrastructure IP (I- IP) to harden selected instructions executed by an existing core processor in SoCs used in embedded systems has been proposed. This solution implies the insertion of the I-IP between the code memory and the core processor of the SoC, so that the I-IP is able to monitor the instruction fetch process and decide whether or not to harden a given fetched instruction. Since the I-IP deals only with a small subset of the

3 core processor instructions, it is much smaller than the core processor. Furthermore, the selection of instructions to be hardened at design time allows the scaling of the area overhead according to the specific fault tolerance requirements of the target application. Besides that, the proposed approach is non intrusive, since it does not require access to or modification of the application source code, nor of the SoC s core processor. However, since not every instruction is hardened, this technique does not provide full fault tolerance, falling in the context of solutions based on statistical computation, which are feasible only for specific application fields, such as image and signal processing. All the proposed approaches mentioned before are generic, and therefore not necessarily optimized for matrix multiplication, the target application of this work. 3. Statistical computation and Freivalds technique The concept of processing and checking in parallel the outputs of a system for only a subset of its possible inputs, mentioned in the previous section, is also called fingerprinting [4], and can be applied to the general case of a circuit that must be hardened against soft errors, thus providing for tolerance against transient faults caused by pulses that affect parts of the circuit, even when the duration of the transient pulse is longer than the delay of several gates. Figure 2 illustrates this idea. The random checker circuit performs some of the functions of the main circuit only on a small set of possible inputs, being able to statistically detect errors in the output with a given probability, while being significantly smaller than the main circuit under inspection. inputs main circuit random checker output error Figure 2. Proposed solution - generic scheme The underlying concept presented here is generic, and can be adopted for several different applications or circuits, with the subset of inputs, the operations performed by the checker, the performance, area, and power overheads varying according to the application. In this work, it has been applied to harden a matrix multiplier circuit, as shown in the following subsections Freivalds technique In 1977, Rúsiņš Freivalds [9] proved that probabilistic machines are able to execute some specific computations faster than deterministic ones, and that they can compute approximations of a function in a fraction of the time required to compute the same function deterministically. Also credited to Freivalds, a technique for faster verification of the correctness of matrix multiplication algorithms has been shown in [4]. In summary, Freivalds technique proposes the use of multiplication of matrices by vectors in order to reduce the computation time when verifying the results produced by a given matrix multiplication algorithm, as follows: given n n matrices A and B, and the matrix C, the product of A and B which was computed using the algorithm under test, the following computations are performed: 1. Randomly create a vector r in which the values of the elements are only 0 or Calculate Cr = C r 3. Calculate ABr = A (B r) Freivalds has proven that, whenever A B C, the probability of Cr being equal to ABr is ½. The demonstration is shown in the box below. Demonstration of Freivalds Technique [10] Let p be the probability of error, and ρ a vector whose values are randomly selected from {0,1}. We claim that if A B = C, then p = 0, and if A B C, then p ½. A B = C If A B = C, then A B C = 0, and so, Π = A (Bρ) - Cρ = (A B)ρ - Cρ = (A B C)ρ = Ω, regardless of what our vector ρ was. A B C Let D = A B C = (d ij ), so Π = D ρ = (p 1, p 2,..., p n ) T. Since A B C, we have A B C 0, so some element of D is nonzero. Suppose that the element d ij 0. By the definition of matrix multiplication, and being r i the i th element of ρ, and p i the i th element of Π, we have p i = Σ n k=1 d ik r k = d i1 r d ij r j d in r n = d ij r j +y

4 where y represents the sum of all products d ik r k, except d ij r j. Using Bayes' Theorem, we have Pr[p i =0]=Pr[p i =0 y=0].pr[y=0]+pr[p i =0 y 0].Pr[y 0] Also, note that: Pr[p i = 0 y = 0] = Pr[r j = 0] = ½ Pr[p i = 0 y 0] Pr[r j = 1] = ½ Plugging these in the above equation, we have: Pr[p i = 0] ½. Pr[y = 0] + ½. Pr[y 0] Pr[p i = 0] ½. Pr[y = 0] + ½. (1 - Pr[y = 0]) Pr[p i = 0] ½ Therefore, Pr[Π = 0] Pr[p i = 0] ½ This completes the proof. Furthermore, if the steps 1 to 3 above are performed independently k times (with different values of the vector r), the probability becomes ½ k. Using this technique, the verification of the result can be done in less time than the original multiplication, since matrix multiplication requires O(n 3 ) time to be performed, while multiplication of a matrix by a vector is performed in O(n 2 ) time. However, since this is an statistical technique, there is no assurance that errors will always be detected Going beyond Freivalds The analysis of the technique proposed by Freivalds shows that the probability of detecting one error in C is ½ because the randomly generated elements of the vector r have the same ½ probability of being 0 or 1. Assuming that the element of C which has an erroneous value is C ij, in the calculation of Cr this element is multiplied by a single element r k of the vector, thereby being canceled during the generation of Cr (if r k is equal to 0) or not (when r k is equal to 1). Given that the elements of the vector r can be chosen randomly, if we perform the computation with a second vector, r c, in which each element is the binary complement of the values in r, the elements of C that were cancelled in the first computation will not be canceled in the second one, and vice-versa. Therefore, if C ij has an erroneous value, we will either have A (B r) C r or A (B r c ) C r c, and the probability of detecting an error in a single element of C will be equal to 1, as demonstrated in the following box. Theorem: The use of complementary r and r c vectors allows to detect all single faults with a double execution of Freivalds technique. The computation of the products A (B r) and C r in the Freivalds technique generates two vectors that must be compared. Assuming that matrices A and B have n n elements, the r and r c vectors will have n elements each and the value of an element i of the above products is given by: ABr i = Σ n i=1 ((a 11b 1i + a 12 b 2i a 1n b ni ).r i ) Cr i = c i1 r 1 + c i2 r c in r n It has already been demonstrated that, when no error occurs in the calculation of C, we have ABr = Cr, and regardless of the values of r i the comparison for equality will hold true. However, when ABr Cr there is a probability ½ that the comparison will also hold true. That happens because the values of r i are selected randomly from {0, 1} and, therefore, Pr[r i =0] = Pr[r i =1] = ½. This way, there is a 50% chance that an erroneous value C ij will be masked during the calculation of Cr, and, in this case, ABr is erroneously considered to be equal to Cr. When the r i values are generated randomly, and then the complement of their values are used to set the values of the corresponding elements in vector r c, we have: Pr[r i =1 OR r ci =1] = Pr[r i =1] Pr[ r ci =1] = Pr[r i =1] + Pr[ r ci =1] = ½ + ½ = 1 If the erroneous value is masked in the calculation of ABr or Cr, it is not masked when ABr c or Cr c are calculated, and vice-versa. This property allows the detection of every error in which a single element of C is faulty, with only two executions of the Freivalds technique. This conclusion has been confirmed through exhaustive fault insertion campaigns in simulations using MatLab [11] and the proposed solution provides an efficient method for verification of matrix multiplication algorithms. The next section shows that the same technique can be successfully applied in hardware, thereby hardening a matrix multiplier circuit against transient faults with reduced area penalties, which decrease in the inverse proportion of the size of the matrices. This technique can be used in several applications, such as image filters and image compression, to provide tolerance against transient faults [12].

5 4. Sample implementation In order to demonstrate the low overhead of the proposed approach, we have implemented the scheme depicted in Figure 2 to harden a matrix multiplier circuit through the application of the Freivalds technique, extended as explained in section 3.2. The matrix multiplier and the checker circuits have been implemented using only combinational logic, and array multipliers have been used to implement multiplications, ripple carry adders to implement additions and AND gates to implement multiplications by the elements of the vectors r and r c, since their values are selected from {0, 1} The hardened matrix multiplier Figure 3 shows the operations performed by each circuit for this specific application. The main circuit takes as inputs two matrices A and B and generates their product (matrix C) at the output. inputs (A, B) Figure 3. Matrix multiplier hardening scheme The checker circuit receives the same A and B matrices, and also the resulting C matrix generated by the multiplier circuit, and performs the multiplications by vectors required for the verification of the result. As previously described, the checker calculates the products Cr and ABr twice, and issues the error signal whenever A (B r) C r or A (B r c ) C r c. Table 1. Area comparison (thousands of gates) Matrix width (n) Size of multiplier , ,192.5 Size of checker ,392.8 Checker area overhead (%) C A * B Cr C * r ABr A*(B*r) output (C) error The additional area required by the checker circuit in this implementation, for different sizes of matrices, in terms of thousands of gates per circuit, is shown in Table 1, from which one can confirm that the proposed technique provides reduced area overhead, thus satisfying one of the main requirements of the proposed approach. To calculate the areas, we have considered that the input values (elements of matrices A and B) are 8-bit positive integers and the output (elements of matrix C) are 20-bit positive integers. Despite the use of a hardware implementation as an example, the extension of the Freivalds technique proposed here is also useful for software verification. In order to show the benefits of the proposed technique, we ran a program that uses this technique with different sizes of matrices and counted the number of operations (additions and multiplications) used to calculate the product, and in the verification procedure. The number of operations required for each test case, as well as the percent overhead, are shown in Table 2. Note that this number of operations relates to the CPU time required to execute the algorithm. Table 2. Time comparison (# of operations) Matrix width (n) Multiplication ,936 64,512 Verification ,976 12,096 Overhead (%) Fault injection results The tolerance against transient faults of the proposed hardened matrix multiplier circuit has been tested using a functional description of the circuit in a MatLab program, which allows the injection of independent faults in each of the operators (array multipliers, ripple carry adders, and AND gates) of the multiplication circuit and checks whether the faults propagate to the output (matrix C) or not. The model used for fault injection is an error in a single operator, that can happen when a transient pulse affects a single node of the circuit. However, the effect produced by a single particle that generates two faults in neighbor nodes, as well those produced for transient pulses that last longer than the propagation delay of several gates, thereby affecting more than one bit of the sum or product, are also covered by this model. As foreseen, for all cases in which there was an error in C, the proposed checker has detected the error. Accordingly, for all correct values of C, no error has been flagged. Table 3 shows the results obtained by injecting single faults (only one operator fails in each computation of C) 1,000,000 times, using different A and B matrices and r vectors, with n = 3, all generated randomly. As mentioned before, for each verification, the r c vector is generated by binary complementation of the corresponding elements in r.

6 Table 3. Single fault injection # of repetitions 1,000,000 faults propagated to C 332,142 A*(B*r) = C*r 166,285 A*(B*r) C*r 165,857 A*(B*r c ) = C*r c 165,857 A*(B*r c ) C*r c 166,285 A*(B*r) C*r OR A*(B*r c ) C*r c 332,142 The second line of Table 3 shows the number of faults injected in the multiplier that affected the product matrix (C). Only those propagated faults were considered for the analysis in lines 3 to 7 of the same table. Even though in 332,142 repetitions of the experiment we have got A B C, approximately 50% of the times A (B r) is compared with C r, they are equal and in the other 50% they are different, confirming that the probability of each situation is ½. Those results confirm that the proposed technique detects all propagated faults. 5. Conclusions an future work This work proposes the use of a low cost checker to verify the results produced by a matrix multiplication circuit. The proposed checker uses an extension of the Freivalds technique to check the results of the multiplier and is able to detect all single faults and all multiple faults (faults that affect simultaneously more bits in a single operator, mimicking a fault that lasts longer than a single gate delay) that affect a single element of the product matrix, with reduced area overhead. It has been shown that the overhead scales inversely with the size of the matrices to be multiplied. The bulk of the future research related to this work will concentrate on the development of reduced overhead checkers for other types of extensively used algorithms and their circuit implementation, in order to find low cost redundancy solutions for fault detection and recomputation. Another important research topic, related to this work, is the detailed study and understanding of the transient faults mechanisms in future technologies, taking into account that the scaling in speed will no longer be followed by a corresponding scaling in the duration of transient pulses. 6. References [1] Baumann, R., Soft Errors in Advanced Computer Systems, IEEE Design and Test of Computers, vol. 22, no. 3, IEEE Computer Society, New-York-London, May-June 2005, pp [2] Dodd, P.E., et al., Production and propagation of Single- Event Transients in High-Speed Digital Logic ICs, IEEE Transactions on Nuclear Science, Vol 51, No 6, Part 2, IEEE Computer Society, Los Alamitos, CA, December 2004, pp [3] Anghel, L., Lazzari, C. and Nicolaidis, M., Multiple Defects Tolerant Devices for Unreliable Future Technologies, in Proceedings of the 7 th IEEE Latin- American Test Workshop (LATW 2006), IEEE Computer Society, Los Alamitos, CA, March 2006, pp [4] R. Motwani, and P. Raghavan, Randomized Algorithms, Cambridge University Press, New York, [5] Rossi, D., Omaña, M., Toma, F. and Metra, C., Multiple Transient Faults in Logic: An Issue for Next Generation ICs?, in Proceedings of th 20 th IEEE International Symposium on Defect and Fault Tolerance in VLSI Systems (DFT 2005), IEEE Computer Society, Los Alamitos, CA, October 2005, pp [6] Nieuwland, A. Jasarevic, S. and Jerin, G., Combinational Logic Soft Error Analysis and Protection, in Proceedings of the 12th IEEE International On-Line Test Symposium IOLTS 2006, IEEE Computer Society, Los Alamitos, CA, July 2006, pp [7] Bouajila, A.et al. Organic Computing at the System on Chip Level, available at Last visited in October, 15, [8] Lisbôa, C. A. L., Carro, L., Sonza Reorda, M., and Violante, M. Online Hardening of Programs against SEUs and SETs, in Proceedings of the 21st IEEE International Symposium on Defect and Fault Tolerance in VLSI Systems - DFT 2006, IEEE Computer Society, Los Alamitos, CA, October 2006, pp ,. [9] Freivalds, R. Probabilistic machines can use less running time, in Proceedings of IFIP Congress 77, B. Gilchrist, editor, North-Holland, Toronto, August 1977, pp [10] Last visited October, 15, [11] Last visited in October, 15, [12] Salomon, D. Data Compression: The Complete Reference, 3 rd ed., Springer-Verlag, New York, [13] Austin, T. DIVA: A Reliable Substrate for Deep Submicron Microarchitecture Design. In MICRO32 - Proceedings of the 32 nd ACM/IEEE International Symposium on Microarchitecture, pages , Los Alamitos, CA, November, 1999.