Supporting Strong Cache Coherency for Active Caches in Multi-Tier Data-Centers over InfiniBand

Transcription

1 Supporting Strong Cache Coherency for Active Caches in Multi-Tier Data-Centers over InfiniBand S. Narravula, P. Balaji, K. Vaidyanathan, S. Krishnamoorthy, J. Wu and D. K. Panda The Ohio State University

2 Presentation Outline Introduction/Motivation Design and Implementation Experimental Results Conclusions

3 Introduction Fast Internet Growth Number of Users Amount of data Types of services Several uses E-Commerce, Online Banking, Online Auctions, etc Types of Content Images, documents, audio clips, video clips, etc - Static Content Stock Quotes, Online Stores (Amazon), Online Banking, etc. - Dynamic Content (Active

4 Presentation Outline Introduction/Motivation Multi-Tier Data-Centers Active Caches InfiniBand Design and Implementation Experimental Results Conclusions

5 Multi-Tier Data-Centers Single Powerful Computers Clusters Low Cost to Performance Ratio Increasingly Popular Multi-Tier Data-Centers Scalability an important issue

6 A Typical Multi-Tier Data-Center Web Servers Apache Clients Tier 0 Proxy Nodes Tier 2 Database Servers WAN Tier 1 Application Servers PHP

7 Tiers of a Typical Multi-Tier Data-Center Proxy Nodes Handle Caching, load balancing, security, etc Web Servers Handle the HTML content Application Servers Handle Dynamic Content, Provide Services Database Servers Handle persistent storage

8 Data-Center Characteristics Front-End Tiers Computation Back-End Tiers The amount of computation required for processing each request increases as we go to the inner tiers of the Data-Center Caching at the front tiers is an important factor for scalability

9 Presentation Outline Introduction/Motivation Introduction Multi-Tier Data-Centers Active Caches InfiniBand Design and Implementation Experimental Results Conclusions

10 Caching Can avoid re-fetching of content Beneficial if requests repeat Static content caching Well studied in the past Widely used Number of Requests Decrease Front-End Tiers Back-End Tiers

11 Active Caching Dynamic Data Stock Quotes, Scores, Personalized Content, etc Simple caching methods not suited Issues Consistency Coherency User Request Proxy Node Cache Back-End Data Update

12 Cache Consistency Non-decreasing views of system state Updates seen by all or none Proxy Nodes Back-End Nodes User Requests Update

13 Cache Coherency Refers to the average staleness of the document served from cache Two models of coherence Bounded staleness (Weak Coherency) Strong or immediate (Strong Coherency)

14 Strong Cache Coherency An absolute necessity for certain kinds of data Online shopping, Travel ticket availability, Stock Quotes, Online auctions Example: Online banking Cannot afford to show different values to different concurrent requests

15 Caching policies No Caching Client Polling Invalidation * TTL/Adaptive TTL Consistency Coherency *D. Li, P. Cao, and M. Dahlin. WCIP: Web Cache Invalidation Protocol. IETF Internet Draft, November 2000.

16 Presentation Outline Introduction/Motivation Introduction Multi-Tier Data-Centers Active Caches InfiniBand Design and Implementation Experimental Results Conclusions

17 InfiniBand High Performance Low latency High Bandwidth Open Industry Standard Provides rich features RDMA, Remote Atomic operations, etc Targeted for Data-Centers Transport Layers VAPI IPoIB SDP

18 Performance 140 Latency 900 Throughput Latency (us) IPoIB SDP VAPI Throughput (MB/s) IPoIB SDP VAPI K 2K 4K 8K 16K Message Size K 4K 16K 64K Message Size Low latencies of less than 5us achieved Bandwidth over 840 MB/s * SDP and IPoIB from Voltaire s Software Stack

19 Performance Throughput (RDMA Read) Throughput (Mbps) K 4K 16K 64K 256K Message Size (bytes) 0 Send CPU Throughput (Poll) Recv CPU Throughput (Event) Receiver side CPU utilization is very low Leveraging the benefits of One sided communication

20 Caching policies No Caching Client Polling Invalidation TTL/Adaptive TTL Consistency Coherency

21 Objective To design an architecture that very efficiently supports strong cache coherency on InfiniBand

22 Presentation Outline Introduction/Motivation Design and Implementation Experimental Results Conclusions

23 Basic Architecture External modules are used Module communication can use any transport Versioning: Application servers version dynamic data Version value of data passed to front end with every request to back-end Version maintained by front end along with cached value of response

24 Mechanism Cache Hit: Back-end Version Check If version current, use cache Invalidate data for failed version check Cache Miss Get data to cache Initialize local versions

25 Architecture Front-End Back-End Request Cache Hit Response Cache Miss

26 Design Every server has an associated module that uses IPoIB, SDP or VAPI to communicate VAPI: When a request arrives at proxy, VAPI module is contacted. Module reads latest version of the data from the back-end using one-sided RDMA Read operation If versions do not match, cached value is invalidated

27 VAPI Architecture Front-End Back-End Request Cache Hit RDMA Read Response Cache Miss

28 Implementation Socket-based Implementation: IPoIB and SDP are used Back-end version check is done using two-sided communication from the module Requests to read and update are mutually excluded at the back-end module to avoid simultaneous readers and writers accessing the same data. Minimal changes to existing software

29 Presentation Outline Introduction/Motivation Design and Implementation Experimental Results Data-Center Throughput Data-Center Response Time Data-Center Break-up Zipf and WC Trace Throughput Conclusions

30 Experimental Test-bed Eight Dual 2.4GHz Xeon processor nodes 64-bit 133MHz PCI-X interfaces 512KB L2-Cache and 400MHz Front Side Bus Mellanox InfiniHost MT23108 Dual Port 4x HCAs MT43132 eight 4x port Switch SDK version Firmware version 1.17

31 Data-Center: Performance DataCenter: Throughput Transactions per second (TPS) Number of Compute Threads No Cache IPoIB VAPI SDP The VAPI module can sustain performance even with heavy load on the back-end servers

32 Data-Center: Performance Datacenter: Response Time Response time (ms) Number of Compute Threads NoCache IPoIB VAPI SDP The VAPI module responds faster even with heavy load on the back-end servers

33 Response Time Breakup Response Time Splitup - 0 Compute Threads Response Time Splitup Compute Threads 8 8 Time (ms) IPoIB SDP VAPI Time (ms) IPoIB SDP VAPI Client Communication Proxy Processing Module Processing Backend version check 0 Client Communication Proxy Processing Module Processing Backend version check Worst case Module Overhead less than 10% of the response time Minimal overhead for VAPI based version check even for 200 compute threads

34 Data-Center: Throughput Throughput: ZipF distribution ThroughPut: World Cup Trace Transactions per Second (TPS) Number of Compute Threads Transactions Per Second (TPS) Number of Compute Threads No Cache IPoIB VAPI SDP NoCache IPoIB VAPI SDP The drop in the throughput of VAPI in World cup trace is due to the higher penalty for cache misses under increased load VAPI implementation does better for real trace too

35 Conclusions An architecture for supporting Strong Cache Coherence External module based design Freedom in choice of transport Minimal changes to existing software Sockets API inherent limitation Two-sided communication High performance Sockets not the solution (SDP) Main benefit One sided nature of RDMA calls

36 Web Pointers NBC home page {narravul, balaji, vaidyana, savitha, wuj,