Xen Network I/O Performance Analysis and Opportunities for Improvement J. Renato Santos G. (John) Janakiraman Yoshio Turner HP Labs Xen Summit April 17-18, 27 23 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice
Motivation CPU cost for TCP connection at 1 Gbps (xen-unstable (3/16/27) ; PV Linux guest; X86-32bit) 14 12 1 CPU utilization 8 6 4 xen xen 2 linux linux RX TX Network I/O has high CPU cost TX: 35% cost of linux RX: 31% cost of linux page 2
Outline Performance Analysis for network I/O RX path (netfront/netback) Network I/O RX optimizations Network I/O TX optimization page 3
Performance Analysis For Network I/O RX Path page 4
Experimental Setup Machines: HP Proliant DL58 (client and server) P4 Xeon 2.8 Ghz, 4 CPU (MT disabled), 64 GB, 512 KB L2, 2MB L3 NIC: Intel E1 (1 Gbps) Network configuration Single switch connecting client and server Server configuration Xen unstable (c.s.14415 March 16, 27) (default xen/xenu configs) Single guest (512MB), dom also with 512MB Benchmark Simple UDP micro benchmark (1gbps, 15 bytes packets) page 5
CPU Profile for network RX path 14 CPU util (%) 12 1 8 6 4 2 app usercopy kernel xenu grantcopy kernel xen linux xen Cost of data copy is significant both in Linux and Xen Xen has the cost of an additional data copy Xen guest kernel alone uses more CPU than linux Most cost for Xen code is in dom page 6
Xen Code Cost 2 CPU util (%) 18 16 14 12 1 8 6 4 2 other mm interrupt event time schedule hypercall domain_page grant_table xen xenu Major Xen overhead is: grant table & dom page map/unmap Copy grant: several expensive atomic operations (lock instr.prefix): Atomic cmpxchg operation for updating status field (grant in use) Increment/decrement grant usage counters (multiple spinlock op) page 7
Dom Kernel Cost CPU util. (%) 35 3 25 2 15 1 5 linux dom other dma+swiotlb hypercall interrupt syscall schedule mm tcp bridge network netback driver Bridge/network is large component in dom cost (Xen summit 26) Can be reduced if netfilter bridge config option is disabled Xen new code: Netback, hypercall, swiotlb Higher interrupt overhead in Xen: extra code in evtchn.c Additional high cost functions in Xen (accounted in other ) spin_unlock_irqrestore(), spin_trylock() page 8
Bridge netfilter cost CPU util. (%) 35 3 25 2 15 1 5 other dma+swiotlb hypercall interrupt syscall schedule mm tcp bridge network netback driver linux nf_br no nf_br What do we need to do to disable bridge netfilter by default? Should we add a netfilter hook in netback? page 9
Overhead in guest CPU util. (%) 3 25 2 15 1 5 linux domu other grant_table hypercall interrupt syscall schedule mm tcp network driver Netfront: 5 times more expensive than e1 driver in Linux Memory op (mm): 2 times more expensive in Xen (?) grant table: high cost of atomic cmpxchg operation to revoke grant access Other : spin_unlock_irqrestore(), spin_trylock() (same as dom) page 1
Source of Netfront Cost CPU util. (%) 3 25 2 15 1 5 linux domu other grant_table hypercall interrupt syscall schedule mm tcp network headcopy driver Netback copy packet data into netfront page fragments Netfront copies first 2 bytes of packet from fragment into main socket buffer data area Large netfront cost is due to this extra data copy page 11
Opportunities for Improvement on RX path page 12
Reduce RX head copy size 3 CPU util. (%) 25 2 15 1 5 linux headcopy 2 bytes headcopy 14 bytes other grant_table hypercall interrupt syscall schedule mm tcp network headcopy driver No need to have all headers in main SKB data area Copy only Ethernet header (14 bytes) Network stack copies more data as needed page 13
Move grant data copy into guest CPU 14 CPU util. (%) 12 1 8 6 4 2 app usercopy kernel xenu grantcopy kernel xen copy in dom copy in guest Dom grant access to data and guest copy it using copy grant Cost of 2 nd copy to user buffer is reduced as data is already in the guest CPU cache (assumes cache is not evicted due to user process delaying read) Additional benefit: Improves dom (driver domain) scalability as more work is done at the guest side Data copy is more expensive in guest (alignment problem) page 14
Grant copy align problem Packet starts at half word copy in guest netback SKB nettfront SKB grant copy 16 bytes 2 packet netback SKB copy in dom netfront fragment page grant copy copy is expensive when destination start is not at word boundary Fix: Copy also 2 prefix bytes source and destination now aligned page 15
Fixing grant copy alignment 14 cpu UTIL. (%) 12 1 8 6 4 2 app usercopy kernel xenu grantcopy kernel xen copy in dom copy in guest align Grant copy in guest becomes more efficient than in current Xen Grant copy in dom: destination is word aligned but Source is not word aligned Can also be improved by copying additional prefix data Either 2 or (2+16) bytes page 16
Fixing alignment in current Xen 14 CPU util. (%) 12 1 8 6 4 2 app usercopy kernelu xenu grantcopy kernel xen original word align buffer align copy in guest Source alignment reduces copy cost Source and dest. at same buffer offset has better performance Reason (?): maybe because same offset in cache? Copy cost in dom is still more expensive than copy in guest. Different cache behavior page 17
Copy in guest has better cache locality grant copy in guest grant copy in dom 12 12 1 8 6 4 2 app usercopy kernelu xenu grantcopy kernel xen CPU util (%) 1 8 6 4 2 app usercopy kernelu xenu grantcopy kernel xen cycles L3 misses L2 misses cycles L3 misses L2 misses Dom copy has more L2 cache misses than guest copy Dom copy has lower cache locality Guest post multiple pages on IO ring. All pages in ring must be used before the same page can be reused For guest copy, pages are allocated on demand and reused more often improving cache locality page 18
Possible grant optimizations Define new simple copy grant: Allow only one copy operation at a time No need to keep grant usage counters (remove lock) avoid cost of atomic cmpxchg operations Separate fields used for enabling grant and usage status Avoid incrementing/decrementing page ref counters Use an RCU scheme for page deallocation (lazy deallocation) page 19
Potential savings in grant modifications 16 14 12 1 8 6 4 2 other mm time schedule hypercall domain_page grant_table original no status update no src page pin no dst page pin Results are optimistic Still need to implement grant modifications Results are based on eliminating current operations page 2
Coalescing netfront RX interrupts 8 CPU util. (%) 7 6 5 4 3 2 1 app usercopy kernelu xenu grantcopy kernel xen no batch 8 pkts 16 pkts 32 pkts 64 pkts NIC (e1) already coalescing HW interrupts (~1 packets/int) Batching packets can provide additional benefit 1% for 32 packets But adds extra latency Dynamic coalescing scheme should be beneficial page 21
Coalescing effect on Xen cost 12 Xen in guest context CPU util. (%) 1 8 6 4 2 other mm time schedule hypercall domain_page grant_table no batch 8 pkts 16 pkts 32 pkts 64 pkts Except grant and domain page map/unmap, all other Xen costs are amortized by larger batches An additional reason for optimizing grant page 22
Combining all RX optimizations 14 CPU util (%) 12 1 8 6 4 2 app usercopy kernelu xenu grantcopy kernel xen linux current optimized Cost of network I/O for RX can be significantly reduced From ~25% to ~7% ovehead (compared to linux) Largest improvement comes for moving grant copy to guest CPU page 23
Optimization for TX path page 24
Lazy page mapping on TX Dom only needs to access packet headers No need to map guest pages with packet payload NIC device access memory directly through DMA Avoid mapping guest page on packet TX Copy packet headers using I/O ring Modified grant operation returns machine address (for DMA) but does not map page in dom. Provide page fault handler to deal with cases in which dom needs to access payload Packet to dom/domu; netfilter rules page 25
Benefit of lazy TX page mapping 1 Gbps TCP TX (64KB msg) 1 Gbps TCP RX 9 18 CPU util. (%) 8 7 6 5 4 3 2 1 app usercopy kernelu xenu grantcopy kernel xen CPU util (%) 16 14 12 1 8 6 4 2 app usercopy kernelu xenu grantcopy kernel xen original TX optimization original tx optimiation Performance improvement for TX optimization ~1% for large TX ~8% for TCP RX due to ACKs Some additional improvement may be possible with grant optimizations page 26
Questions? page 27