HTTP/WebDAV synchronization protocol optimizations. Piotr Mrowczynski

Transcription

1 HTTP/WebDAV synchronization protocol optimizations. Piotr Mrowczynski

2 HTTP/WebDAV synchronization protocol optimizations. - HTTP2 ( - Bundling ( Scope of this talk - Request Scheduling ( - Dynamic Chunking for new chunking algorithm ( - Prioritize by modification time (

3 Current owncloud WebDAV / HTTP1.1 implementation

4 - HTTP/1.1 without pipelining head of line blocking

5 - HTTP/1.1 without pipelining head of line blocking - max 3-6 parallel connections (as in web) Time - each file is single PUT / GET / DELETE / MKDIR / MOVE request within a single persistent (Keep-Alive Header) connection... COL K M Con. 1 PUT Con. 2 PUT Con. 6

6 Server can handle 100 requests in parallel at specific moment client will use anyways max 6

7 - server can handle 100 requests in parallel at specific moment client will bind to max 6 - server with concurrent syncs is overloaded with x6 opened connections (usually SSL)

8 - server can handle 100 requests in parallel at specific moment client will bind to max 6 - server with many concurrent syncs is overloaded with opened connections - Latency: Each file in separate connection has to waste time on latency (usually ms) x 53 ms = 9s x 320 ms = 53s Time Laten cy nsfer a r t d n a Latency Server-side Operations 1000 files / 6 parallel = 167 lines of blocking

9 HTTP1, HTTP2 and BUNDLING

10 Time HTTP/ COL K M Con. 1 PUT Con. 2 PUT Con. 6

11 BUNDLING Time Reduced latency gain... COL K M Con. 1 DLE N BU DLE N BU Con. 2 Con. 6 Time HTTP/ COL K M Con. 1 PUT Con. 2 PUT Con. 6

12 HTTP 2 with owncloud's requests limitation BUNDLING Time Reduced latency gain... Time COL K M Con. 1 Bandwidth/Time Max Parallel Gap DLE N BU DLE N BU Con. 2 Con. 6 HTTP/1.1 Max Parallel { Con. 1 Optimization target request scheduling (will talk later) Time PUT... COL K M Con. 1 PUT Con. 2 PUT Con. 6

13 HTTP2 with owncloud request limitation If optimized by pumping more requests or binary data transfers: - might hide request-response latency as in bundling - might utilize bandwidth - only limited by server/database capability of accepting parallel files

14 HTTP2 - possible benefits for owncloud - The parallel multiplexed requests and response do not block each other.

15 HTTP2 - possible benefits for owncloud - The parallel multiplexed requests and response do not block each other. - Optimized and faster encryption

16 HTTP2 - possible benefits for owncloud - The parallel multiplexed requests and response do not block each other. - Optimized and faster encryption - Binary framing less errors, overhead and more - Header compression

17 HTTP2 - possible benefits for owncloud - The parallel multiplexed requests and response do not block each other. - Optimized and faster encryption - Binary framing less errors, overhead and more - Header compression - Flow control (separate from TCP flow control)

18 BUNDLING - possible benefits for owncloud - Files packed in group of requests, send over the network and single response is returned - Above results in latency reduction and possible better network utilization

19 BUNDLING - possible benefits for owncloud - Files packed in group of requests, send over the network and single response is returned - Above results in latency reduction and possible better network utilization - Reduces PHP overhead (script is fired up once for whole the group instead of per file) - PHP overhead can be also reduced by optimizing server side for single requests

20 HTTP1 vs HTTP2 tests

21 HTTP1 vs HTTP2 tests Files in parallel limitation 1000 files 1kB total 1MB of data Measurement repeated 10 times using Smashbox Benchmarking Tool CERNBOX Geneva, Switzerland WiF Berlin, Germany 79 Mbit w o D, /s bit, Upl 76M y c n te la s i, 53 m Ethernet, 32 0 ms latency,u /s SSD, 8GB RAM, 4x2,4GHz, Melbourne, Australia pl 220 Mbit /s, Dow 1448 Mbit/s Openstack, 12GB RAM, 4x2.5Ghz

22 HTTP1 vs HTTP2 tests Files in parallel limitation 1000 files 1kB total 1MB of data Measurement repeated 10 times using Smashbox Benchmarking Tool Synchronization to CERNBox (EOS), Geneva, Switzerland Protocol HTTP1 Parallel Limit Location 6 53 ms, DE HTTP2 HTTP ms, AU HTTP / / / / / / / (?) / / / / 9.2 (?) -43s ms, AU Download Time [s] / s HTTP2 HTTP1 Upload Time [s] Latency and WAN impact

23 HTTP1 vs HTTP2 tests Files in parallel limitation 1000 files 1kB total 1MB of data Measurement repeated 10 times using Smashbox Benchmarking Tool Synchronization to CERNBox (EOS), Geneva, Switzerland Protocol Parallel Limit Location 6 53 ms, DE HTTP1 HTTP2 HTTP ms, AU HTTP2 Upload Time [s] Download Time [s] / / / / / / / / (?) -170s HTTP1 HTTP2 Client still allows only max 6 connections ms, AU / / / / 9.2 (?) 1. Head of line blocking 2. Server bookkeeping blocks upload HTTP2 Pipelining

24 HTTP1 vs HTTP2 tests Files in parallel limitation 1000 files 1kB total 1MB of data Measurement repeated 10 times using Smashbox Benchmarking Tool Synchronization to CERNBox (EOS), Geneva, Switzerland Protocol HTTP1 Parallel Limit Location 6 53 ms, DE HTTP2 HTTP ms, AU HTTP2 HTTP1 HTTP ms, AU Upload Time [s] Download Time [s] / / / / / / / / (?) / / / / 9.2 (?) On latency 50ms from Berlin to Geneva, we got even 20s 50 Hz (files per second)

25 HTTP1 vs HTTP2 tests SSL Overhead 12 files 1kB total 12kB of data Measurement repeated 10 times using Smashbox Benchmarking Tool Synchronization to CERNBox (EOS), Geneva, Switzerland Protocol Files Synced Location HTTP1 HTTP ms, AU Upload Time [s] Download Time [s] 6.7 +/ / / / s 3-way-handshake and SSL optimization gain, connection reuse limit (only?)

26 HTTP1 vs BUNDLING tests

27 HTTP1 vs BUNDLING tests 1000 files 1kB total 1MB of data Measurement repeated 10 times using Smashbox Benchmarking Tool DAMKEN CLOUD Nuremberg, Germany WiF Berlin, Germany 79 Mbit w o D, /s bit, Upl 76M y c n te la s i, 37 m Ethernet, 27 9 ms latency,u /s SSD, 8GB RAM, 4x2,4GHz, Melbourne, Australia pl 220 Mbit /s, Dow 1448 Mbit/s Openstack, 12GB RAM, 4x2.5Ghz

28 HTTP1 vs BUNDLING tests Files in parallel limitation 1000 files 1kB total 1MB of data Measurement repeated 3 times using Smashbox Benchmarking Tool Synchronization to Damken Cloud, Nuremberg, Germany Protocol Bundled Files HTTP1 - Bundling 100 HTTP1 - Bundling 100 Bundling 10 Location Upload Time [s] / ms, DE -6s / / / / / / / / / s 279 ms, AU Download Time [s] Latency Influence on HTTP1 Bundling in this prototype works only for upload

29 HTTP1 vs BUNDLING tests Files in parallel limitation 100 files 1kB total 100kB of data 1000 files to bundle 10 requests needed 100 files to bundle 6 requests needed (as number of connections) It occurred that for 100 files sync time reduced: 20s 16s in upload for 37ms latency

30 Requests Scheduling

31 Wide and Narrow Pipe problem with max 3-6 connections 2 MB/s available

32 Wide and Narrow Pipe problem with max 3-6 connections 5 MB 5 MB 2 MB/s available 5 MB

33 Better solution utilizing 6 connections 5 MB 5 MB 2 MB/s available

34 Even better solution using HTTP/2 for fast and idle servers Boosted using http2 prioritization? (E. Bocchi, Politecnico di Torino) 5 MB 5 MB 2 MB/s available

35 TU Berlin TU Berlin Test Measurement repeated 10 times using Smashbox Benchmarking Tool Folder A Folder B 50 x 100kB 20 x 5MB

36 TU Berlin TU Berlin Test Old implementation Folder-wise, first folder with small, then folder with big

37 TU Berlin TU Berlin Test New implementation Cross-folder, big files and small files at the same time

38 TU Berlin TU Berlin Test Measurement repeated 10 times using Smashbox Benchmarking Tool 20 x 5MB 50 x 100kB - Big files don't block smaller ones - Small files don't block bigger ones Upload: ~23s ~20s Download: ~16s ~13s

39 Future using HTTP2, Dynamic Chunking and Scheduling? - Max Parallel Negotiation Sending you 3 files Ok, but I can handle 50 files at this moment My dynamic chunk is now 20 MB. Sending you 30 files which fit into it - Using HTTP2 PUSH in Discovery phase? I have more users now Reduce to 15 files (E. Bocchi, Politecnico di Torino)

40 Take Away Message - Changing the way the requests are being send may significantly reduce the sync time (HTTP2, Bundling, Scheduling) HTTP2 Bundling Number of Connections 1 Max 6 Optimized transfer Optimized encryption, Binary stream, header compression, 1 connection PHP overhead reduced, files buffered in bundles on server Number of files processed in parallel Limited only by server capability and client OS/hardware probably max 100 Max 6 Observed improvement CERNBox + EOS (1000 files) 115s 20s on 50ms lat. 210s 40s on 320ms lat. 6.7s 5.9s for 10 files on 320ms lat DamkenCloud (1000 files) 155s 149s on 37ms lat 188s 158s on 279ms lat 20s 16s for 100 files on 37ms lat - If you want to test HTTP2 or any of the features, please contact me piotr@owncloud.com