Development and verification of software component level fault injection for safety-critical automotive Ethernet control system

Transcription

1 Development and verification of software component level fault injection for safety-critical automotive Ethernet control system CHENG-YU LIU Department of Electrical Engineering National Taipei University 151, University Rd., San Shia District, New Taipei City TAIWAN Abstract: - Automotive Ethernet, as a communication protocol for next generation automotive control systems, is developed to fulfill the increasing demand on the multimedia and advanced driver assistance systems with higher safety and more comfort. The applications of automotive control system are highly associated with safety, including human life and environment. So they require a severe dependability while the systems are in operation. Therefore, the robustness of software component should be considered carefully, because software component plays the decisive role in the operation of sophisticate safety-critical systems. Consequently, the fault injection methodology must be integrated at early design phase in order to achieve such requirement. In this paper, the research focuses on an effective fault injection framework of software component level for safety-critical automotive Ethernet control system. The proposed framework emphasizes the role of softwareimplemented fault injection theory and associated analyze methodology, which is aimed at the software models verification and validation in order to simulate the system behavior when the software component failure occurs. Thus, the robustness and system behavior analysis with various numbers of fault scenarios will be conducted and the reliability results will be also provided, discussed and compared. Besides, a fault injection experiment environment is designed for automotive Ethernet control system on OMNet++ platform and object file modification tools(e.g. GNU Binutils), in order to generate the software component failures via modify software parameters and package/frame errors in the communication bus. Finally, a simplified advanced driver assistance system is used to verify the functionality and effectiveness of the proposed fault-injection and associate analysis methodology. Key-Words: Automotive Ethernet, Fault Injection, Reliability, Robustness Analysis 1 Introduction According to the growth of high speed network architecture and embedded system, it is essential to focus on system integration. Furthermore, it must be noted that system integration is closely related to human life and environment safety. Thus, the system stability and safety issue must be considered severely. We will begin by considering automotive control system field. ADAS [13] (Advanced Driver Assistance System) is a typical example when combine high speed network architecture with embedded system. However, this will lead us further into a consideration of interference. We should notice that electrical control system is easy to disrupt by electromagnetic waves, charged particle and crosstalk. Consequently, the disruption will cause the temporary system failure. On the other hand, what has to be noticed is that the lifetime of the vehicle become much longer. The question then arises about the system failure caused by the aging of individual component. As a result, we need to figure out an effective solution to deal with the safety and robustness issue of ADAS. We should note that using SoC (System on Chip) as the central unit of ADAS will be essentially in the near future. There are several benefits for the miniaturization of semiconductor manufacture process. For instance, we could raise the density of IC, and increase the function diversity. Moreover, we can expect higher core frequency and lower operating voltage. But, we are confronted by two difficulties. The first is that the internal circuit operation of SoC is prone to be affected by the EMI (Electromagnetic Interference). The second is the temporary soft error. What is more, the failure rate of SoC will be raised [14]. One final point is that the reliability problem will become the major concern of ECU (Electronic Control Unit). ISBN:

2 The question which we must consider is that the vehicle control system design is quite complex. In addition, the complexity will rise if we combine the safety and reliability consideration together. Clearly, we must draw attention to the safety and reliability at the early stage of system design. This will shorten the design cycle and reduce the development cost. Besides, it is vital to execute fault injection at every part of the system in order to evaluate the system reliability at early design phase. The point is that we could examine the system behavior during the failure. We may, therefore, reasonably conclude that we must look more carefully into how to improve the reliability of ADAS by executing fault injection mechanism more effectively. This research topic becomes more and more important in the automotive field [5], [6], [7], [8]. We will begin by considering the control network protocols which are used on the ADAS. There are several types of network protocols used in the field, namely, Local Interconnect Network (LIN), Control Area Network (CAN), Low-Voltage Differential Signaling (LVDS), FlexRay and Media Oriented Systems Transport (MOST). The first point to notice is that the MOST protocol has the best bandwidth performance. According to the specification of the third generation, the MOST can reach the bandwidth to 150M bits/s. By contrast, once integrate all devices into the network, the bandwidth will decrease significantly. In short, the requirement of real time application could not be satisfied. Viewed in this light, the Ethernet protocol can be regarded as the appropriate solution to deal with the fast growing application of vehicular safety and infotainment. In the first place, Ethernet protocol was commercially introduced in 1980, which means the great success in the development process. In the second place, the overall cost of Ethernet platform is quite competitive. Furthermore, the bandwidth of Ethernet protocol is sufficient to implement the real time application. In this paper, we shall concentrate on the fault injection mechanism and system reliability analysis methodology based on the automotive Ethernet. There is one other thing that is important for us to develop the fault injection techniques, which could be used on the software component level of automotive Ethernet. This will lead us further into a consideration of traditional fault injection technique improvements. Thus, we could not only enhance the system reliability by using appropriate fault injection mechanism, but also integrate various fault injection design methodologies on automotive Ethernet control system in more comprehensive way. The paper is organized as follows. We begin in Section 2 by reviewing the previous research results on fault injection mechanism and safety critical system design methodologies for automotive applications. Section 3 provides an overview of the automotive Ethernet control system framework analysis. We next present the software level fault injection mechanism analysis in Section 4. The fault type analysis for automotive Ethernet control system is addressed in Section 5. We present the concept of system quality mapping table in Section 6. Simple case study is discussed in Section 7. Finally we conclude this work in Section 8. 2 Related Work 2.1 Safety standard IEC61058 (Functional safety of electrical, electronic and programmable electronic (E/E/PE) safety-related systems) is an industrial safety standard [1]. It describes in full detail that the functional safety standard of E/E/PE safety-related systems. The important point to note is that the main concept of IEC61508 is preventing danger from system failure. The main reason is that the danger could endanger the safety of human life and environment. At the main time, IEC61058 also provides the methodology for how to establish the safety requirement specification for E/E/PE safetyrelated systems. On the other hand, the question which we must consider is the functional safety issue which related to automotive field. One may notice that the demand on raising the functional safety of automotive control system is growing dramatically. Thus, ISO which based on the IEC was announced at [2] The scope of ISO is focus on the functional safety issue of automotive electric/electronic systems. Besides, ISO formed different safety requirements into various ASIL (Automotive Safety Integrity Level). 2.2 Standardized software design criteria AUTOSAR [3] (AUTomotive Open System Architecture) is the standard which defines the design criteria and architecture that must be followed by automotive software. Besides, AUTOSAR emphasizes the exchangeability and ISBN:

3 Fig.1 The architecture diagram of ADAS control network reuse of software component. That is to say every vehicle manufacture company cooperate on standard, compete on implementation. In addition, AUTOSAR could not only improve the feasibility of automotive software solutions, but also raise the application of the COTS (Commercial Off-The- Shelf). Consequently, the system designer could achieve the expected functional safety level via strict verification mechanism. In [5], the author explored the problem of how to develop the software fault injection mechanism on binary level base on AUTOSAR. Thus, the system robustness could be verified. Furthermore, in the research of [6], the author established the fault injection mechanism on automotive embedded system, and then, the author introduces a case study of brake by wire system in order to examine the performance of fault injection mechanism on distributed architecture. 2.3 Safety-critical system design Let us discuss the subject from safety critical system of view, the application is closely related to human life and environment. So it is vital to establish strict safety requirements [10], [11], [12]. In other words, reliability verification and validation is indispensable during design phase. A close study on how to ensure safety integrity level was made at [11]. It revealed that through safety assessment of subsystems and components, the system designer could make certain that the system reliability meets the goal of IEC61508 and ISO What is more, software tools are essentially for automotive electrical and electronic system development. [4] discusses the question of how to implement the fault tolerant analyze on the operational level model of embedded system. In the research, the author advocates to do characterization for each individual component. Because of the system designer could predict the target system behavior at early design phase. In fact, the goal of characterization for each individual component is not only for reuse the component definition, but also decrease the design period significantly. 3 Automotive Ethernet Control System Framework Analysis As the Fig.1 indicates, the ADAS control network backbone is based on the assumption that automotive Ethernet is successfully implement into the control system. It will be useful, to begin with, to make a distinction between two kinds of ISBN:

4 functional block. The first functional block is responsible for the brake function. As sketched here, the front left (FL), front right (FR), rear left (RL), and rear right (RR) wheel ECU is aimed at receive the brake control signal from brake pedal ECU. Subsequently, the actuator (brake bulbs) and brake action will be acted according to the brake control signal. In the proposed architecture, we should notice that the importance of brake pedal ECU and four wheel ECUs is the same. For one thing, once the failure on brake pedal ECU occurs, the whole brake system functionality will be fail. This is one of the main causes that lead to the accident. What is more, we should not overlook that if one of the wheel ECU failures occurs, it will cause the brake force distribution abnormally. Furthermore, this situation happens when several wheel ECUs fail at the same time. Consequently, the brake force distribution failure will likely result in the vehicle out of control during braking process. What is more, once the turn indictor module (which including software components, sensors and actuators) failure occurs; it will be very difficult for other drivers to predict the direction of the moving vehicle. One final point is that this is also one of the main causes that lead to the accident. The second functional block is responsible for the image processing function. Application examples of this are object detection and surround view. These basic functions could achieve more advance functions such as BSWS (Blind Spot Warning System) and PCS (Pre- Crash System). The question now arises: once the image processing unit failure occurs, it will cause the object detect function abnormally. Moreover, PCS may malfunction due to object detect function failure. In short, the vehicle will not brake automatically once obstacle suddenly comes out. Most of drivers are not able to avoid the obstacle which appears suddenly. This kind of accident could be avoided effectively by proper ADAS intervention. We can conclude with certainty that the fault injection mechanism must integrate into each system module of ADAS during early design phase in order to raise the system reliability. Fig.1 indicates that each SWC (SoftWare Component) is responsible for different function in the ECU. However, the same functionality of SWC could be used in different ECUs. To take a simple example, each wheel ECU must include ABS (Anti-Lock Brake System) software component. Thus, we should categorize the SWCs depend on functionality. Another example is that turn indicator sensor SWC should separate into other category. It seems reasonable to conclude: we should develop appropriate fault injection mechanism for the SWC which the functionality is the same. Added to this, we may consider use existing fault injection mechanism according to the cost and development cycle of the system. This categorization methodology is the main concept of SWC level fault injection mechanism in this paper. Nevertheless, the applications of ECU become more extensive recently. Moreover, the complexity of SWC function rises tremendously. The point is that the integration difficulty between fault injection mechanism and software source code will increase significantly. Besides, the system designer may not be able to get the source code from third party. We may, therefore, reasonably conclude that it is another viable way to execute the fault injection at object file. 4 Software Level Fault Injection Mechanism Analysis In this paper, we have applied the fault injection techniques based on [5]. Moreover, we propose a corresponding analysis methodology. It seems reasonable to consider the proposed fault injection techniques through two types of organization. To start with, we have applied software component original function extraction. Next, let us consider the methodology from wrapping software original function point of view. We could modify the object files via GNU binary utilities. Furthermore, we could disturb the parameter transmission between different SWC. [6] studied the software level fault injection mechanism based on the requirements of brake by wire system. Besides, in the research, the source code of software was generated through AUTOSAR code generator. Thus, the quality of source code is corresponding to the AUTOSAR standard. But [5] fails to account for the generation of test case and fault-scenario; we could not expect the system behavior corresponding to the system output after fault injection. It will be clear from this example that the fault injection framework should be improved in order to apply to the automotive Ethernet control system more properly. ISBN:

5 We may consider the impact factors when designing fault injection mechanism under the following heads: (1) Timing error; (2) Control flow error; (3) Data error; (4) The relationship between SWC signal and memory address of test target; (5) Modeling the effects of errors; (6) Fault coverage Evaluation; (7) System robustness analysis methodology. We will introduce two kinds of fault injection framework in the following sections. 4.1 Fault Injection Framework on Data Level According to the growth of software design complexity, it is necessary to use the SWC and external library from third party during system development. In this paper, we execute fault injection at object file; we should discuss it in detail. Furthermore, we can analyze each section of object file. Fig.4 Object file analysis sample Fig.4 indicates that the object file consists of many sections, for example,.txt is responsible for data processing. Other examples,.data and.bss, are used to represent data content. There is no need to go into details about other sections for linker. On the other hand, we can obtain the assembly code via execute reverse engineering on object file via other functions of GNU binary utilities. In short, we can gain readability in order to analyze the source code. The sample result is represented in Fig.5. Fig.2 Compile flow chart of C/C++ Fig.2 tells us that once absence of source code, the system designer is necessarily to integrate the object files from third party in order to provide software function more completely. We will modify object file via GNU binary utilities in order to simulate the data failure in this situation. Fig.5 Sample assembly code of object file Fig.6 helps to explain the three steps that how to implement fault injection module into object file. As we noted above, we will introduce the implementation methodology specifically, consider the sample source code as shown in Fig3. Fig.4 shows that the object file compiled from sample source code, we can analyze the object file of sample source code via GNU binary utilities. Fig.6 Object file fault injection flow chart Fig.3 Sample source code ISBN:

6 Step 1: We shall extract the original function which we would like to execute fault injection function via parser. Step 2: Select expect fault type from fault library. Subsequently, we could combine the fault type with the original function extract from (1) in order to generate new SWC which include fault injection mechanism. Step 3: We shall compile the new SWC from (2) with the Test target libraries in order to generate the object file which including fault injection mechanism. We shall now look more carefully into how to replace the printf function call via --warp instruction. In the first place, we shall establish the warp module in order to replace the original function block. We can represent warp module as Fig.8 In the second place, we shall compile the warp module and target source code, which is aimed at implement fault injection mechanism. Lastly, the compiled program will redirect to the --warp printf function. Thus, the original printf function will be 4.2 Fault Injection Framework on Functional Level In this paper, we developed the basic SWC based on automotive Ethernet control system application. It must be noted that consider apply only data level fault injection mechanism as mentioned above, methodology speaking, this view is unsatisfactory. The data level fault injection mechanism fails to account for readability issue. The readability of assembly language is not as clear as C/C++ language. From this viewpoint one may say that we could not execute fault injection base on functionality. Take safety-critical system as example, execute fault injection only on data level may not be able to achieve the robustness according to user requirements. Thus, in order to raise the system robustness, the other viable way is implement fault injection function into original function block. Fig.8 Sample of warp module replaced by the --warp printf function. On the other hand, we could call the original printf function via --real_printf instruction. As mentioned above, we could execute the functional test in more advanced way. It can be best The GNU linker (or, GNU ld) provides the -- warp instruction, which could redirect the original function call to another function. A simple source code sample may help see this point more clearly. Fig.7 shows that the simple source code sample which includes function call. Fig.9 Function block source code fault injection flow chart summarized as Fig.9: Step 1: Establish warp module via combine original function block and fault injection module. Step 2: Compile the warp module into object file or execution file. Fig.7 Sample source code of function call 5 Fault Type Analysis for Automotive Ethernet Control System ISBN:

7 The effectiveness of proposed fault injection mechanism in this paper could be evaluated via the fault scenarios. However, it will be inefficiently to simulate and observe the entire fault scenario via experiment. Judging from the above, we shall define the effect of each fault type via mathematical model at early design phase. Moreover, we could take many preventive measures according to the fault type. Consequently, the system design cycle and cost could be reduced significantly. Detailed account of the methodology is given below. 5.1 Fault Type Definition for Automotive Ethernet Control System In this section, we shall concentrate on three fault types that impact the automotive control system and related application mostly. They may be divided into three types. Namely: noise, shift, and spike. Consider the unaffected signal as illustrated in Fig.10: Fig.10 Unaffected signal We may consider the fault types under the following heads: 1. Noise: It was caused by the signal disturbance during transmission. For instance, this may came from the influence of EMC. Another example is that the unaffected signal coupling with the external signal. In other words, it added the randomly amplitude into unaffected signal. In this paper, we modified the parameters and function blocks of SWC randomly to simulate the impact of noise. We can represent the noise diagrammatically as Fig.11: 2. Shift: The signal could not generate at correct time phase due to the time synchronization failure occurs. For example, the shift follows from the defect of oscillator. Another example is operation frequency asynchronous between each system component, which will cause the data transmission problem. In this paper, we disturbed the frames on the communication bus and modified the transfer time between each frames to simulate the shift. We can represent the shift in a simple diagram as Fig.12: 3. Spike: Fig.12 Signal includes shift. The signal fluctuation occurs in the circuit may be due to voltage or current variation. (e.g., switch on/off or the system was struck by lightning.) Usually, this kind of signal fluctuation continues only in 1μs. In this paper, we modified the parameters which transmitted between SWC randomly. Moreover, data lose is a viable way to simulate the effects of spike failure. Spike is an important factor contributing to inaccurate system output signal. We can represent the spike diagrammatically as shown in Fig.13: Fig.13 Signal includes spike. Take the spike for example. It is obviously that the number of possible way to inject spike into different interval will be tremendously: CC nn ssss. (si represents all of the simulation interval, n stands for the number of spike.) Fig.11 signal includes noise. ISBN:

8 5.2 The Mathematical Model for failure subset In this paper, we defined the mathematical model for failure subset. All these efforts are attributable to we could avoid test the same fault scenario repeatedly during experiment. Furthermore, the mathematical models can be divided into three main groups: 1. Functional test-case: It describes the relationship between output signal and specific operation environment. The position of throttle serves as an example: the position shall be 0 degree at initial condition, and then reach 5 degree after 5 seconds. From this viewpoint one may say that the term test case can be defined as: TC SSiiii oooooo ff 0 (tt), ff 1 (tt),, ff nn (tt) (1) Where TC represents test-case, SSiiii oooooo stands for output signal, and ff ii (tt) is signal trace corresponding to time. 2. Fault scenario: It describes the failure situation which must be simulated. The term FS refers to Fault scenario. Moreover, FS stands for the relationship between error signal (ErrSig) and fault type (FF tttttttt ). That is: FS ErrSig FF tttttttt Amp (2) FF tttttttt represents three fault types mentioned above as set, namely: FF tttttttt = {nnnnnnnnnn, sshiiiiii, ssssssssss} (3) Where Amp stands for the amplitude of error signal. For one thing, we used positive real number to account for maximum amplitude of noise. What is more, the real number represents the amplitude of shift. (This may be positive or negative numbers.) One final point is that the positive integer stands for the number of spikes. Judging from above, the fault scenario could be described as: FS sensor1 output = (noise, 5.8) (4) Namely, the sensor-1 detects 5.8 unit of noise. 3. Fault-tolerant requirement specification: It describes the fault-tolerant level must satisfy the system performance specification. The equation can be defined as: ii ΥΥ ii = ΦΦ (5) Where ΥΥ ii represents the fault assumptions before experiment. ΦΦ stands for acceptable system output result. On the other hand, in view of relationship between prediction and result, let us then define the equation as follows: <ErrSig, Fault-Type, Limit>. The following serves as an example: we applied spike (the amplitude is 5 units) to all sensors, it will lead to shift (the amplitude is 5.8 units) generated at actuator-1. The relationship indicates as follows: < Sensor In, spike, 5 > < AAAAAAAAAAttnnAA1 In, shift, 5.8 > (6) 6 System quality mapping table We should notice that in order to analyze the system behavior according to SWC failure, it is necessarily to rely on tremendous simulation. It is important to keep in mind that we shall consider the effective fault model in the sampling space. In short, it is important that we shall develop the methodology in order to reduce the numbers of simulation reasonably. In this paper, we simulated the behavior of SWC, and then characterization according to the system response. Furthermore, we could analyze the advance system behavior via such evaluation. Fig.14 illustrates the system quality mapping table methodology. First, we shall focus on the behavior of individual SWC, or, an operation status defined by user. Subsequently, we shall execute fault injection to do further analysis. In the proposed flow, for one thing, we are aim at record the SWC operation behavior under the considered test case without failure occurs. In this step, we recorded the behavior of each SWC, and then execute characterization. What is more, it must be noted that the importance of characterization is that we could observer the difference between each input signal and output signal, and then record the results into system quality mapping table. One final point is that we could preserve the simulation results systematically via characterization. Clearly, we could apply these analyses to other system design in the future. Fig.14 Flow chart for SWC characterization ISBN:

9 7 Simple case study In this paper, we established SWCs and system nodes in C/C++ via OMNet++ simulator, which could be used to simulate various kinds of fault scenarios. Examples abound, such as: (1) Packages and frames lose during transmission; (2) Communication bus failure (e.g. system disconnection); (3) Modify the data transfer and delay time; (4) Replace the parameters and function blocks of SWC. The proposed simplify ADAS is represented in Fig. 15: Fig.17 SWC structure of camera node The important point to note is the common SWC used in different structure. To take a simple example, scheduler SWC was used in both automotive Ethernet switch and camera node. This figure tells us that the scheduler SWC consists of following parts: Fig.15 The proposed simplify ADAS As sketched here, we used star topology to simulate the network architecture of image process system. Furthermore, we executed fault injection and related analysis. The proposed system was constituted by four camera nodes, one automotive Ethernet switch, and two ECU nodes. We shall now look more carefully into the structure of switch and each camera node. For now, it is not necessary for the purpose of this paper to enter into a detailed discussion of ECU nodes. Here is a figure which shows the internal SWCs of switch: Fig.18 SWC structure of scheduler This drives us to the question how the parameters in these parts affect the system. Table 1 summarizes the parameters used in the scheduler: Table 1: Main parameters of scheduler Name Type Default value Description numperiods int 1 tick double 80ns Length of a tick oscillator.cur Current length double rent_tick of a tick period.cycle_ int 37500tick Number of ticks ticks period.offset _ticks int 0tick for one cycle Number of ticks offset The point to observe is that we disturbed and modified the parameters of scheduler and data on communication bus in order to figure out the effects of data latency. Fig.16 SWC structure of automotive Ethernet switch This is especially noteworthy in the case of object detection. The data latency plays the decisive role for the distance deviation of ADAS. Here is a figure which shows the relationship between latency and distance deviation: On the other hand, we could represent the SWC structure of each camera node diagrammatically as follows: ISBN:

10 Fig.19 The relationship between data latency and distance deviation This is a good illustration of distance deviation for object detection. In the urban, the cameras and sensors shall operate during vehicle moving, usually 40km/h to 50km/h. It must be noted that in this situation, the latency of 1μs could cause the distance deviation over than 1cm. Once the latency time reached 10ms, the distance deviation would larger than 10cm. This is an important factor contributing to the misjudgment of object detection and pre-crash function. Clearly, this is a huge design problem of ADAS. 8 Summary The development and application of automotive Ethernet is growing rapidly. Furthermore, the system safety is closely related to human life and environment. In short, the reliability shall be considered carefully. The proposed methodology in this paper could enhance the effectiveness of fault injection. Moreover, the system designer could collect experiment results systematically via system quality mapping table. Consequently, the analysis and comparison results could provide valuable guidance to system designer. References: [1] CEI International Standard IEC 61508, [2] ISO, International standard ISO Road vehicles Functional safety, ISO, Geneva, [3] AUTOSAR, AUTOSAR Technical Overview v2.2.2, AUTOSAR, Munich, 2011a. [4] D. Das, P. P. Chakrabarti, and P. Sinha, "Robust embedded software design through early analysis of quality faults," in Proc. 4th India Software Engineering Conference, pp , [5] N. M. KARUNAKARAN, "Binary-Level Fault Injection (BLFI) for AUTOSAR-based Systems," Master of Science Thesis. Chalmers University of Technology, Department of Computer Science and Engineering. [6] J. Haraldsson and S. Thorvaldsson, "Software implemented fault injection for AUTOSAR based systems," Master of Science Thesis. Chalmers University of Technology, Department of Computer Science and Engineering. [7] C. Lu, J.-C. Fabre, and M.-O. Killijian, "Robustness of modular multi-layered software in the automotive domain: a wrapping-based approach," IEEE International Conference on Emerging Technologies & Factory Automation, pp. 1-8, [8] D. Cotroneo, A. Lanzaro, R. Natella, and R. Barbosa, "Experimental analysis of binarylevel software fault injection in complex software," Ninth European Dependable Computing Conference (EDCC), pp , [9] MOGENTES project, 2014[Online]. Available at: [10] Y. Papadopoulos, M. Walker, M.-O. Reiser, M. Weber, D. Chen, M. Törngren, et al., "Automatic allocation of safety integrity levels," in Proceedings of the 1st workshop on critical automotive applications: robustness & safety, pp. 7-10, [11] M. Conrad, P. Munier, and F. Rauch, "Qualifying Software Tools According to ISO 26262," MBEES, pp , [12] I. Habli, I. Ibarra, R. S. Rivett, and T. Kelly, "Model-based assurance for justifying automotive functional safety," SAE Technical Paper, [13] J. Fritsch, T. Michalke, A. Gepperth, S. Bone, F. Waibel, M. Kleinehagenbrock, et al., "Towards a human-like vision system for driver assistance," IEEE Intelligent Vehicles Symposium, pp , [14] R. Leveugle et al., "Soft Error Effect and Register Criticality Evaluations: Past, Present and Future," IEEE Workshop on Silicon Errors in Logic System Effects, pp. 1-4, [15] The CoRE research group, 2014[Online]. Available at: ISBN: