Membership Library in DPDK Sameh Gobriel & Charlie Tai - Intel DPDK US Summit - San Jose

Transcription

1 Membership Library in DPDK Sameh Gobriel & Charlie Tai - Intel DPDK US Smmit - San Jose

2 Contribtors Yipeng Wang Ren Wang John Mcnamara Pablo De Lara Garch Antonio Fischetti

3 Agenda Membership Library in DPDK Membership Library Usages API Overview Research Proof of Concept: sing Membership Library with OVS

4 Membership Test Usage (example) Bild Set {..} Set of Blacklisted Flows to be Dropped Clients Incoming Flows Blacklisted Flow 1 Blacklisted Flow 2 Blacklisted Flow 3 Legitimate Flows are Forwarded to Backend Server Blacklisted Flow N Membership Test? Check if Flow Belongs to Blacklisted Set S1 S2 SN Membership Set Smmary A Smmary Instead Of Library is a DPDK Library to Provide Users the List Fnctionality Storing Original to Create Different Types of Set-Smmaries

5 Overview of DPDK Membership Library Membership Library 1- Too Mch Storage 2- Slow Lookp Vector Bloom Filter Is X in set? Bloom Filter No Is X in set? Very probable yes Set Smmary Hge Set Get X Smmary of items in Probabilistic data strctre Handle membership test qestions Mch smaller storage Mch faster than hge set lookp [Mlti-Set]: Retrns X is not fond or which set it belongs to (with high probability) [Millions of Entries] Hash Table Set Smmary Cckoo Distribtor

6 Library Usages? Safe Browsing Signatre Matching and packet inspection TCP Connection Tracker Database Semi-join operations Distribted web caching Set intersections and keyword searches P2P Overlay Networks = Object Indexing Network Statistics and smmaries Detecting loops in nicast and mlticast rotes Wildcard Classification Heavy Hitters Flows Detection 100 s of sages for Membership Library in Wide Range of Applications 6

7 Library Usages? 100 s too many to list Roting Loop Detection and/or Network Statistics Distribted Web Cache Encode ID Set Smmary SET SET Web Servers Clients Distribted Web Cache Packet Packet Node ID s encoded in embedded set-smmaries in the packet header Web proxies conslt set smmaries for each http reqest. Element membership in the set-smmary will determine response location. Instead of waiting for slow TTL, node checks membership in setsmmaries. Misses indicate loop-free roting. For element hits, reqests directed to a near cache and misses are forwarded to backend web servers. Idea can be generalized (for e.g. heavy hitters detection,..etc.) to wide range of network stats. 7

8 Library Usages? 100 s too many to list Safe Browsing and/or Signatre Matching TCP Connection Tracker Lookp xyz.com? Flow Key NO xyz.com sspicios? Set Smmary New Flow = New Assignment Old Flow = forward to specific thread Set Smmary Probably Yes URLs membership checked against sspicios set-smmary and misses indicate safe. Flow keys membership tested and misses indicate new flows. Same idea is applied in many signatre matching IDS and deep packet inspection. Hits are forwarded to worker thread for in-order processing. 8

9 Library Usages? 100 s too many to list Mask 1 Mask x Sbtable Sbtable Set Smmary forward to specific sb-table Flow Key ACL & Wild Card Flow Classification Flow keys membership reslts are sed to optimize search for wild card match New? Later Slides: Reslts of applying concept to OvS 9

10 Membership Library API Set-Smmary Create rte_member_create(rte_member_parameters); Parameters: Type, nm_of_keys, key_length, nmber_of_sets, max_fp_rate,..etc. Set-Smmary Element Insertion Set-Smmary Element Lookp rte_member_lookp(*setsm, *key,*set_id) rte_member_lookp_blk(*setsm,*keys, *set_ids) rte_member_lookp_mlti(*setsm,*key, max_match_per_key,..) rte_member_lookp_mlti_blk(*setsm,*keys,..); A single key or a blk of key lookp, retrn the first match or p to max matches per key Set-Smmary Element Delete rte_member_delete(*setsm,*key,set_id) rte_member_add (*set_sm, *key, set_id); Insert a key into a set_smmary data strctre and the vale is pointing to a specific set_id. Reference: member_8h_sorce.html Fnction shown is jst for high-level description & not in accrate Syntax Delete a single key from a given set. Not all modes (e.g. vbf) spport delete in crrent implementation.

11 POC: Open vswitch Flow Lookp Packet Header Flow Mask Rles xxxx 0011 xxxx 1011 xxxx x xxxx 101x xxxx 111x xxxx 011x xxxx Mask L Match 01xx xxxx 10xx xxxx Mask N 1xxx xxxx 0xxx xxxx 1. Set of disjoint sb-table 2. Rle is only inserted into one sb-table (lookp terminates after first match) 3. Lookp is done by seqentially search each sb-table ntil a match is fond 11

12 OVS with Two Layer Lookp sing Membership Library Packet Header Flow Mask Rles xxxx 0011 xxxx 1011 xxxx Set Smmary Provided by Membership Library x xxxx 101x xxxx 111x xxxx 011x xxxx Mask L Match 01xx xxxx 10xx xxxx Mask N Membership library sed to create a 1st level setsmmary indirection Flow Keys are looked p in set-smmaries: Hits: directs to the correct sb-table for searching (correct 97%) Misses: New flow defalt seqential search & pcall if needed 20 Sb-Tables 10K Flows Uniform Traffic 1xxx xxxx 0xxx xxxx Intel(R) Xeon(R) CPU E GHz Hyper-Threading: disabled 6 Max Forwarding Rate (MPPS) 1st Level of Indirection Orig OvS-DPDK OvS-DPDK + ML Library X X-3X Throghpt Improvement for OvS sing DPDK 12 Library Membership

13 Ftre Work Applying Membership Library Optimization to other workloads. Any Partners with hge list of Object?? Crrently Working on ACL Library with high pdate rate based on tplesearch algorithm 13

14 Legal Disclaimers No license (express or implied, by estoppel or otherwise) to any intellectal property rights is granted by this docment. Intel disclaims all express and implied warranties, inclding withot limitation, the implied warranties of merchantability, fitness for a particlar prpose, and non-infringement, as well as any warranty arising from corse of performance, corse of dealing, or sage in trade. This docment contains information on prodcts, services and/or processes in development. All information provided here is sbject to change withot notice. Contact yor Intel representative to obtain the latest forecast, schedle, specifications and roadmaps. Intel technologies featres and benefits depend on system configration and may reqire enabled hardware, software or service activation. Performance varies depending on system configration. No compter system can be absoltely secre. Check with yor system manfactrer or retailer or learn more at intel.com Intel Corporation. Intel, the Intel logo, Intel. Experience What s Inside, and the Intel. Experience What s Inside logo are trademarks of Intel. Corporation in the U.S. and/or other contries. *Other names and brands may be claimed as the property of others.

15 Qestions? Sameh Gobriel Charlie Tai