Fundamental Technology on Dependable SoC and SiP for Embedded Real-Time Systems

Transcription

1 Fundamental Technology on Dependable SoC and SiP for Embedded Real-Time Systems Nobuyuki Yamasaki (Keio Univ.) Kikuo Wada (NECAT) Masayuki Inaba (Tokyo Univ.)

2 Applications: High-end Embedded Real-Time Systems Robot (Humanoid, etc.) Spacecraft Factory automation Intelligent rooms/buildings Amusement system Car VoD (Video on Demand) Ubiquitous Computing Large systems in which sensors/actuators are spatially distributed Large scale systems that can not be controlled by a single CPU Fault tolerant systems, etc.

3 Distributed Control on Humanoid Robots Kojiro (Tokyo Univ.) Massively distributed control 100-freedom, 60-controller HRP3L-JSK (Tokyo Univ.) Very high power leg 48V: cont.50[a], 15sec 100[A] It is hard to realize a robot system by using a general purpose CPU (x86) and a general purpose OS (Windows/Linux).

4 Requirements in the Field of High-end Robots Requirements for a high power motor driver Real-time processing under high speed communication Motor temperature estimation processing for very high power motor driving such as 20 times overdrive rated at 200W Current control cycle: 10msec 10μsec Reliability, availability, and safety on communication and control under high-stress environment Huge current noise, unusual situation such as cable disconnection, etc Prevention of fatal accidents Requirements for a large scale distributed motor driver Microminiaturization of the controller (size: 36x46x7mm) Area constraint of the digital control part: 20mm square Real-time communication and control under the size constraint Poor processing power of current MPU (H8S/ MHz) Limit of control cycle: 1msec 10μsec External computation servers (Xeon 3.4GHz x 2) are required. High communication traffic 7.2MB/sec Limit of Inter-device synchronization cycle: 8msec (USB) 100μsec (Responsive Link) Reliability of communication under the size limitation Severe noises under the logic servo systems Power saving scheme under the large scale distributed control Static power of a whole logic part: 80W@idle 1W High power leg: HRP3L-JSK (Tokyo Univ.) Continuous current 80[V],100[A],15[sec] Peak current 80[V],200[A],10[msec] Freedom 82 DOFs Driven -muscle 109 # of controllers 60 Kojiro (Tokyo Univ.)

5 Dependability Evaluation Indicators Classification Reliability Availability Safety Maintainability Evaluation items Real-time Power Noise tolerance Heat radiation Footprint Plug-and-Play Network failure Heat control Parts replacement Board replacement Failure analysis Evaluation indicators Hard/soft time constraint (T/F) Time quantum (sec) Jitter (sec) Dynamic power (W) Static power (W) S/N ratio (%) Thermal resistance (deg C/W) Integration to robots (T/F) Correct operating (T/F) Network reconfigure time (sec) Self-monitoring (T/F) Repair and replacement (sec) Repair and replacement (sec) Failure analysis (sec)

6 Real-Time Scheduling Real-time processing/communications are basically controlled by real-time schedulers. EDF (Earliest Deadline First) [1] RM (Rate Monotonic) [1], EDF Deadlines are translated to priority levels. Earlier the deadline, higher the priority. Priority is changed dynamically. Optimal scheduling method Maximum processor utilization: U = 1 [1] Liu, C.and Layland, J, Scheduling algorithms for multiprogramming in a hard real-time Environment, Journal of the ACM, Vol.20, pp.46-61, 1973

7 EDF Sample Schedule Job Release time Deadline Preemption Preemption Time Earlier the deadline, higher the priority.

8 Distributed Real-Time Systems Time constraints (deadline, cycle, etc.) Each controller Its own actuators and sensors Connected via real-time network Responsive Link Almost all real-time scheduling algorithms assume: Preemption Context switching in case of processing Packet overtaking in case of communication Worst case latency WCET (Worst Case Execution Time) in case of processing WCRT (Worst Case Response Time) in case of communication

9 Multi-Level Dependability Support SoC level Real-time processing/communication Processing cores are connected via RT-NoC Redundant processing cores and network links SiP level D-RMTP I SoC and DRAM modules are integrated by FFCSP Real-time DVFS w/ self-monitoring Thermal control w/ self-monitoring Robot level D-RMTP I SiPs are connected via Responsive Link Adaptive ECC for Responsive Link D-RMTP SoC DRAM Modules Network reconfiguration to avoid faulty links Task migration from faulty SiPs D-RMTP I SiP DRMTP SiP DRMTP SiP DRMTP SiP DRMTP SiP DRMTP SiP DRMTP SiP Responsive Link Humanoid Robot

10 SoC for Embedded Real-Time Processing: Responsive Multithreaded Processor (RMTP) 10mm Real-time processing unit: RMT PU Real-time execution mechanism (RMT execution) A context switch is converted to the prioritized SMT execution. 8-thread simultaneous execution in order of priority Thread control bases on priority (256-level) Thread wake-up by an interrupt IPC control (processing speed control of real-time threads): Control of WCET Multimedia processing units (Vector + SIMD) Flexible 2D vector processing units (Integer, FP) Shared vector registers by multiple threads Context cache (32threads): 4-clock context switch Trace buffer Real-time communication : Responsive Link Preemptive communication: Packet overtaking by priority Packet acceleration/deceleration: Packet priority can be replaced with new priority at each node. ISO/IEC Computer I/O peripherals PCI-X, IEEE-1394, Ethernet, etc. Control I/O peripherals SpaceWire (3-ch switch) PWM Generators, Pulse Counters, etc. D-RMTPⅠ SRAM (256kB) Memory bus (256bit) I/O bus (32bit) PCI-X 64bit 66MHz PC SPI 4cs x 2ch (8ch) ADC / DAC DDR SDRAM I/F Gateway IEEE 1394 Digital Camera 128/32bit DDR SDRAM 128/32 bit D-RMTP I RMT PU Real-Time Execution 8way Prioritized SMT IPC Control 32 Context Cache Trace Buffer 2D Vector Units 256/32bit DMAC 1ch UART Digital PWM-in PWM-out Port Encoder 1ch 4ch 8bit In: 3ch Out: 12ch Cnt: 4ch I/O Devices I/O Devices AC/DC Motors 32bit External Bus 2cs 2dreq 2irq ROM / I/O Dev. Ethernet MAC 10mm 32bit DMAC 20ch 1ch Space Wire I/O Internet Devices 32bit DDR SDRAM for Responsive Link Real-Time Network Responsive Link 3ch Data link Event link 4ch RMTP RMTP

11 Real-Time Scheduling Prioritized threads are scheduled and executed in priority order. Release Time Deadline high Priority A time constraint including deadline and cycle is converted to priority. low high Task 0 Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 Task 7 System Task 0 Task 1 Task 2 Task 3 context switch Time Task 4 Task 5 Task 6 Task 7 System low Time A set of prioritized contexts is treated as a task cue of an RT-OS. The contexts are executed in priority order by hardware. Thousands of clock cycles (yellow parts) are needed to switch contexts. Elimination of the overhead the context switches

12 RMT Execution by RMT PU Multiple threads are executed simultaneously in priority order. Implicit context switching: A context switch is converted to RMT execution (prioritized SMT execution). Basically no software context switch exists. High Priority Low Priority Thread 0 Thread 1 Thread 2 Thread 3 Thread 4 Thread 5 Thread 6 Thread 7 System Release Time High throughput real-time processing Deadline Time

13 Requirements for Real-Time Communication Preemption Achieved by packet overtaking Higher priority packets can overtake lower priority packets at each node. WCRT (Worst Case Response Time) Network latency depends on the size of a packet and its blocked time Packet level overtaking Blocking time of a packet becomes constant.

14 Essential Requirement for Real-Time Communication preemption preemption Release time Deadline Preemption capability is required to apply real-time scheduling algorithms to communications.

15 A Real-Time Packet Scheduling Algorithm Virtual Deadline Monotonic [2] node1 Period (=Deadline) connection1 virtual deadline Transfer time connection connection2, connection Link1,2 connection1 0 7 node2 connection3 node3 connection2 Link2,3 connection1 connection2 TX start 5 connection3 TX start connection1 deadline 14 t t priority 2 < 1 3 < 1 [2] S.Kato, Y.Fujita, and N.Yamasaki, Periodic and Aperiodic Communication Techniques for Responsive Link, The 15th IEEE International Conference on Embedded and Real- Time Computing Systems and Applications, pp , 2009.

16 Real-Time Communication Responsive Link Split transmission and independent routing of data and events Full-duplex and differential I/F Virtual cut-through switch with packet overtaking (preemption) function: The packet with higher priority can overtake other packets at each node. Priority replacement: Packet priority can be replaced with new priority at each node to accelerate/decelerate the packet. When the network address is same but the priority is different, the different route can be set to realize an exclusive line or a detour. Fixed packet size for WCRT estimation Data link (64-byte), Event link (16-byte) Powerful adaptive error correction {RS, None}, {BCH, Hamming, None}, {BS+NRZI, 8b10b, 4b10b} Flexible link speed (12.5 to 800 Mbps/link) Point-to-point link for any topology Standardization: ISO/IEC 24740

17 Split Links for Event and Data image sync signal sound sync Shared traffic indefinite latency and throughput sync interrupt status open signal connect Event link image sound voice table text Low Latency Data link High Throughput

18 Dual Physical Communication Links Data Link : like a vessel network Soft real-time communication for bulky data Multimedia data transmission (images, voice, etc.) Relatively large fixed packet size (64B) Total throughput is more important. Event Link : like a nerve network Hard real-time communication for control Control commands, Inter-processor interrupt, synchronization, etc. Relatively small fixed packet size (16B) Low latency is more important.

19 Packet Format Data Packet Format (64B) Source Addr. Destination Addr. Event Packet Format (16B) Source Addr. Destination Addr. Payload Control & Status Payload Control & Status 1 byte 1 bit Control & Status Format (32bits) UD Full Data Length Dirty0 Dirty1 Dirty2 Dirty3 Dirty4 Dirty5 Dirty6 Dirty7 Dirty8 Dirty9 Dirty10 Dirty11 Dirty12 Dirty13 Dirty14 Dirty15 3 Start End Int. Fatal Correct Serial Number (Cnt.) Frame Format (12bits) Data bits Redundancy bits

20 Responsive Link I/F Responsive Link Connector Responsive Link Cable Tx Data+ Tx Data- Rx Data+ Data Link Rx Data- Tx Event+ Tx Event- Rx Event+ Event Link Rx Event-

21 Virtual Cut-through Switch with Packet Overtaking Function

22 Routing Table Priority[7-4] Priority[3-0] EE P[7-4] PE L3 L1 Src Addr(16b) Dtn Addr(16b) DE P[3-0] L4 L2 L0 Reference Referent It is possible to set a different route when the priority is different even if the network address is same. (Default route is priority 0.) It is possible to replace a packet priority with a new priority at each node.

23 Routing based on Priority (1/2) Data (Priority0) Source Data (Priority1) Event (Priority2) Event (Priority0) Destination

24 Routing Based on Priority (2/2) Data with データ priority 3 ( 優先度 3) Source 送信元 ノード Node0 0 ノード 1 Node1 Data with priority 0 データ ( 優先度 0) Node2 ノード 2 ノード Node3 3 ノード Node4 4 ノード Node5 5 ノード Node6 6 Destination 送信先

25 Adaptive Codecs Error correction code Byte (block) error correction RS (1byte error correction) (4B, 6B) None Bit error correction Hamming (1bit error correction) (8b, 12b) BCH (2bit error correction) (8b, 16b) None Line code Bit staffing + NRZI (dynamic, clock embedded, DC balancing) 8b/10b (static, clock embedded, DC balancing) 4b/10b (static, clock embedded, DC balancing, 1bit error correction)

26 Measurement of Real Communication Noise Responsive Link + ECC D-RMTP Ⅰ 30mm sq Eva Kit ト Noise generator (Amplification of real motor noise) D-RMTPⅠ 30mm sq Eva Kit ト Transmitter Pulse Receiver Pulse

27 Performance of Noise Tolerance Combination of Codecs Error correction: {RS, None}, {HAM, BCH, None} Line code: BitStaffing+NRZI, 8b10b, 4b10b Block level ECC Bit level ECC Line code Codec rate BS+NRZI (9, 8) 29.6% BCH (16, 8) 8b10b (10, 8) 26.7% 4b10b (10, 4) 13.3% w/ Reed Solomon BS+NRZI (9, 8) 39.5% (48, 32) Hamming (12, 8) 8b10b (10, 8) 35.6% 4b10b (10, 4) 17.8% ECC None 8b10b (10, 8) 53.3% 4b10b (10, 4) 26.7% BS+NRZI (9, 8) 44.4% BCH (16, 8) 8b10b (10, 8) 40.0% 4b10b (10, 4) 20.0% w/o Reed Solomon BS+NRZI (9, 8) 59.3% Hamming (12, 8) 8b10b (10, 8) 53.3% 4b10b (10, 4) 26.7% ECC None 8b10b (10, 8) 80.0% 4b10b (10, 4) 40.0% Effect of line code: dominant BS+NRZI BER10-3 %: 50% error BER10-2 %: 100% error 8b10b BER10-3 %: 20% error BER10-2 %: 90% error 4b10b BER10-2 %: 0% noise length

28 SiP Level Dependability D-RMTP I SoC and DRAM modules are integrated on a SiP Interposer by FFCSP Real-time DVFS (D-RMTP I ) Low-power while guaranteeing deadline Safety voltage control w/ self-monitoring Prevent D-RMTP I & DRAM from Overheating Thermal control w/ self-monitoring Undershoot 0.80V 1.10V FPGA 20μsec Voltage transition ( V) DRAM DRMTP II Substrate 20μsec Voltage & thermal control (D-RMTP I ) Redundant vertical links (LSI Internal Logic Level) Vertical chip stacking (D-RMTP II )

29 SoC/SiP Co-design for Improvement of Dependability Narrow wiring area High noise tolerance Improvement of dependability 1.Optimizat ion of IP and I/O pin arrangeme nts for SiP design 3. Inside-SoC adjustment scheme of wiring jitters that cannot be adjusted by SiP Conventional design scheme Jitter adjustment by SiP wiring pattern 2. Optimization of bump arrangement for SiP wiring I/O buffers of SoC Our scheme: Codesign of SoC and SiP Jitter adjustment mechanism inside SoC Wide wiring area Low noise tolerance Stabilization of analog characteristic of SiP wiring pattern

30 30mm square D-RMTP SiP RT-DVFS Function ADC for Thermal Sensor (DRAM) Thermal Sensor for DRAM DC/DC Converter for RT-DVFS Thermal Sensor for D-RMTP DRAM DRAM Flash D-RMTP ADC for Supply Voltage FPGA Potentiometer for RT-DVFS ADC for Thermal Sensor (D-RMTP) Almost all functions of PC + Embedded Microcontroller + Real-Time Processing Core + Real-Time Communication

31 Robot Level Dependability D-RMTP I SiPs are connected via Responsive Link Permanent faults (links & boards) Network reconfiguration to avoid faulty links Task migration from faulty D-RMTP I SiPs Transient faults (links) Adaptive ECC & line codes for Responsive Link DRMTP SiP DRMTP SiP ECC code (4Byte) Reed- Solomon (48, 32) ECC code (1Byte) BCH (16, 8) Hamming (12, 8) None Line code BS+NRZI (9, 8) 8b10b (10, 8) 4b10b (10, 4) BS+NRZI (9, 8) 8b10b (10, 8) 4b10b (10, 4) 8b10b (10, 8) 4b10b (10, 4) (1) Permanent faults by link disconnection (2) Transient faults by motor noise DRMTP SiP DRMTP SiP DRMTP SiP DRMTP SiP Responsive Link None BCH (16, 8) BS+NRZI (9, 8) 8b10b (10, 8) 4b10b (10, 4) Received waveform w/ noise Humanoid Robot

32 D-RMTP On-board Motor Driver D-RMTP SiP on board SW/RESET reset OSC CLK bus Flash Altera EPCS Size: 85x60x36 mm GPIO Responsive Link x4 bus bus Gate driver Specification of motor driver Voltage:80V UART JTAG RMTP PWM ENC CLK FPGA Altera EP2C Isolator Current:cont. 50A, max. 200A Vector control Water-cooling 5V 3.3V for I/O 1.2V for FPGA 1.0Vfor RMTP GPIO HOLE ENC RS x 2 ADC ADS7886x4 5 V monitor AD input x 3 Responsive Link D-RMTP SiP Water-cooling

33 D-RMTP I Evaluation Kit FPGA I/O USB (Host & Peripheral) D-RMTP I SoC PWM-IN/OUT Encoder 30mm sq D-RMTP I SiP DC Jack Responsive Link (4ch) RS-232C (2ch) For more information, please contact: Yamasaki lab., Keio Univ.

34 I/O Core SiP Responsive Link Connector Ultra Small Evaluation Kit for Distributed Control via Responsive Link DRAM Responsive Link Connector UART Connector FPGA RS485 I2C Connector SPI Connector Acceleration Sensor Thermal Sensor Flash ROM I/O Core SiP (Bottom) RS232C Analog Input Connector USB Peripheral Connector USB Host Connector A/D Converter Analog I/F Board (Center) I/O UART, GPIO, SPI, I2C, DMAC Responsive Link USB Host Connector PLL I-Cache PCI IO CORE D-Cache JTAG Connector USB I/F I/O Core FPGA USB Board (Top) Flash

35 Programming ISA of RMT Processor : MIPS upper compatible ISA + thread control instructions + vector instructions MIPS instruction C and C++ available RMT Processor own instructions (multithread instructions, vector operations) Assemble language Libraries (boost+, tvnet) OS Linux itron favor, Tflight (our original RT-OS) Cross development tools

36 Simulators Anywhere the D-RMT Processor can be developed. Host OS: Linux, Solaris, Cygwin No real hardware is needed. A laptop PC is available to develop a program. High speed and low functionality: ISS (Instruction Set Simulator) Processor model Major I/O models (Serial, Responsive Link) No timing check Low speed and high functionality: RTLS (RTL Simulator) All functional models of the D-RMTP SiP RMT Processor Responsive Link PCI PWM, etc.

37 Addr Data Addr Data Load/Store Op.(1inst.) 1inst Addr Data Phisical Address(32bit) PC(32bit) Instruction(32bitx8) Operation/Data(4inst.) PC(32bitx8) Priority(8bitx8) Instruction Type(8inst.) Instruction Select(4inst.) Operation(4inst.) Commit Instruction(4inst.) 1inst. 2inst. 1inst. 4inst. 1inst. 2inst. 1inst. 1inst. 1inst. 2inst Addr Data Addr Data Addr Data Load/Store Op.(1inst.) Phisical Addr (32bit) Load Data(64bit) 1inst 4inst 1inst Operation(8inst.) 1inst 1inst Write Back(4inst.) 1inst Thread Op. Conclusion RT-Processor RT-Network Cache Mechanism Instruction Victim Cache Instruction Wait Buffer Data Victim Cache Data Wait Buffer Data MMU Instruction MMU Instruction Cache Data Cache Data Read/Write Buffer Instruction Fetch and Issue Mechanism Fetch Thread Selector Decoder Instruction Analysis Instruction Buffer Register Rename File Register PC Control Unit Thread Control Unit Instruction Issue Selector Context Cache Reorder Buffer Hard/software co-design Dependable SoC QoS RT-OS Reservation Station Reservation Station Reservation Station Reservation Station Reservation Station DMAC Memory Access Branch Unit Integer Divider Integer Unit FP Divider FP Unit 64bit Integer Vector Integer Vector FP IEEE1394 Responsive Link SDRAM IF Common Data Bus Arbitor Instruction Execution Mechanism High power motor driver Context Cache (RF) Context Cache (status) Vector Register (FP) Vector Register (Int.) RMT PU Instruction Cache Data Cache PLL Real-time Noise tolerance Heat radiation SoC/SiP co-design Distributed control RT-DVFS Control board Dependable SiP High power leg 3-D mounted SiP Humanoid robot