SUSE Linux Enterprise Real Time Matthias Nagorni Product Manager SUSE Linux Enterprise 22. März 2007
What is SUSE Linux Enterprise Real Time? An add-on to SUSE Linux Enterprise Server providing deterministic performance in time-critical environments. An industry-standard real-time version of Linux for x86 architectures engineered and supported in cooperation with Concurrent Computer Corp. providing the following key features: CPU Shielding: Dedicated resources (such as CPU, memory, fabric) for high priority processes Kernel Preemption: Fast response to external events, optimized kernel critical sections (aggressive lock breaking) Priority Inheritance: Solves priority inversion problem for mission critical processes 2
Real Time The problem and its solution Problem: Multi-tasking systems with scheduler-driven task execution do not provide deterministic timing. Solution: Exclude processes with real-time requirements from scheduling by running them on an isolated CPU/Core. SLE RT allows to explicitely and exclusively assign a process to a CPU/Core. CPU/Cores can be excluded from ordinary scheduling and interrupts. Together with other measures this results in predictable timing of the shielded process. 3
CPU Shielding: The Processing Highway The way CPU Shielding improves processing can most easily be understood by the analogy of cars running on a highway. Associate: CPU/Core --> Lane, Process/Thread --> Car Interrupt --> Truck, Consumed CPU time --> Distance While this analogy is not strict, it illustrates many effects of Shielding: Shielding of CPUs/Cores from other processes and interrupts is pictured in the upper lane The advantage of shielding is less apparent when there is little load ( = traffic on highway) Even at times of very low traffic, there is no guarantee that no truck will show up. Workloads with few high-priority Processes/Threads benefit most from Shielding. 4
Real Time as a time machine In Real Time applications for Financial Services, SLE RT can give a competitive advantage close to a time machine. Events 5
What's the secret? Standard Linux Kernel Measured 44,759,417 samples Max latency: 92.3 millisecs 44374681 samples < 0.1ms (99.140%) 219672 samples < 0.2ms (99.630%) 93496 samples < 1.0ms (99.843%) 44452 samples < 10.ms (99.939%) 16188 samples < 30.ms (99.975%) 10761 samples < 60.ms (99.999%) 166 samples < 90.ms (99.999%) 1 sample < 100ms (100%) Interrupts fire at 490 us /dev/rtc Kernel with Real-time Extensions Measured 28,800,882 samples Max latency: 27 microsecs Min latency: 11 microsecs Average latency: 11.3 microsecs 28,808,870 samples < 0.02 ms (99.99999%) 12 samples < 0.03 ms (100%) 6 September 08, 2005 Novell Inc, Confidential & Proprietary
Technical Benefits of SUSE Linux Enterprise Real Time Predictable response times with minimal latency High Quality of Service (QoS) even under heavy load High I/O throughput (integration with Infiniband) Fast response to external events 6-11 usec interrupt response time Deterministic and repeatable program execution 7 usec context switch times Optimized interprocess communication Signal delivery 6 usec 7
When Is Real-Time The Appropriate Platform? Standard Kernel: Throughput Sensitive Workloads Non-time sensitive workloads, applications with no process or transaction priority hierarchy, environments with no performance SLAs When fair scheduling algorithms are required across compute resources (time sharing) Web farms serving FIFO based requests Enterprise mail servers Batch processing Real Time Kernel Latency Sensitive Workloads Most improvement applications with a limited number of threads or clear hierarchy with priority; applications requiring faster, predictable data delivery predictable, low data latency Market data feeds (Wombat, Tibco, Reuters) Algorithmic Trading Real Time Enterprise Risk Management High volume Transaction Processing & High Volume Query Processing - Applications, e.g. ATM transactions, credit card authorizations, etc. Use standard kernel for throughput sensitive workloads and real-time kernel for latency sensitive workloads 8
Real Time in Financial Services Applications requiring predictable, low data latency Market Data Delivery > Assure fair access to pricing information. Trading / Algorithmic Trading > Maximize stock trading profit > Minimize time from receipt of new market information to re-calculation of risk and trade execution > Faster trade matching & execution > E.g., Foreign Exchange, Bond, MBS, Equity, and Derivatives trading Other high volume Transaction Processing Applications > E.g., ATM transactions, credit card authorizations, websites, etc. 9
Results in Financial Services Business Results > Large investment bank has said that for every 1 millisecond their program trading applications can beat the market, they would earn an additional $100 Million per year! > A leading Wall Street firm has shown it needs only 20% of its existing market data servers to meet performance SLAs Performance Results > Another Investment Bank is seeing 30%+ improvements for a time-critical program trading application > A large regional Bank has seen performance improvements ranging between 10% to 150%+ under heavy loads for their enterprise Identity Management solution while decreasing their overall physical server footprint by 75% > In database benchmarks, Oracle and Ingres database queries execute 5+ times faster under load 10
Real Time Doubles Wombat Feed Handler capacity to 200,000 messages per second 2 Consumer Applications/Client Handling a total of 200,000 msgs/sec Server 1 Wombat Feed Handler 4 Consumer Application Connections Subscribing to 100,000 Symbols Each 50,000/second per connection Server 2 Wombat Client Switch Server 3 Wombat Client Customer Benefits 2x improvement for Wombat customers running SLERT Partner Components Intel Woodcrest Chips 11
Benchmarks Time (s) 2 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 Maximum Response Time 1 5 10 15 20 User Load Oracle 10g Benchmark 550 500 450 400 350 300 250 200 150 100 50 0 Archipelago Data Using The Flood Test RHEL4 SLERT SLES SLE RT 3 4 5 6 7 8 9 10 11 12 13 14 Market Data Flood Test Time In Seconds 275 250 225 200 175 150 125 100 Standard Linux Kernel: Windows Server 2003: 75 50 25 Real Time Kernel: 0 Ingres Database Performance RHEL4 SLERT 1 1 1 2 2 2 4 4 4 8 8 8 16 16 16 Threads Generating System Load Ingres Benchmark 90 milliseconds 2.3 seconds 22 microseconds Maximum Process Dispatch Latency 12