Caching and tuning fun for high scalability. Wim Godden Cu.be Solutions

Transcription

1 Caching and tuning fun for high scalability Wim Godden Cu.be Solutions

2 Who am I? Wim Godden Owner of Cu.be Solutions ( Open Source developer since 1997 Developer of OpenX Zend Certified Engineer Zend Framework Certified Engineer MySQL Certified Developer

3 Who are you? Developers? System/network engineers? Managers? Caching experience?

4 Goals of this tutorial Everything about caching and tuning A few techniques How-to How-NOT-to Increase reliability, performance and scalability 5 visitors/day 5 million visitors/day (Don't expect miracle cure!)

5 LAMP

6 Architecture

7 Test page 3 DB-queries select firstname, lastname, from user where user_id = 5; select title, createddate, body from article order by createddate desc limit 5; select title, createddate, body from article order by score desc limit 5; Page just outputs result

8 Our base benchmark Apachebench = useful enough Result? Single webserver Apache + PHP Proxy Static PHP Static PHP Limit : CPU, network or disk Limit : database

9 Caching

10 What is caching? CACHE CACHE

11 What is caching? x = 5, y = 2 n = 50 Same result CACHE CACHE select * from article join user on article.user_id = user.id order by created desc limit 10 Doesn't change all the time

12 Theory of caching DB le tab lt rom resu f d ta da turne t c e re sel ta = $da if ($data == false) Page GET /page set( 'key ', $da fal $data) ta = se get( 'key ') Cache

13 Theory of caching DB HIT Cache

14 Caching techniques #1 : Store entire pages #2 : Store part of a page (block) #3 : Store data retrieval (SQL?) #4 : Store complex processing result #? : Your call! When you have data, think : Creating time? Modification frequency? Retrieval frequency?

15 How to find cacheable data New projects : start from 'cache everything' Existing projects : Look at MySQL slow query log Make a complete query log (don't forget to turn it off!) Check page loading times

16 Caching storage - Disk Data with few updates : good Caching SQL queries : preferably not DON'T use NFS or other network file systems especially for sessions locking issues! high latency

17 Caching storage - Disk / ramdisk Local 5 Webservers 5 local caches How will you keep them synchronized? Don't say NFS or rsync!

18 Caching storage - Memcache(d) Facebook, Twitter, YouTube, need we say more? Distributed memory caching system Multiple machines 1 big memory-based hash-table Key-value storage system Keys - max. 250bytes Values - max. 1Mbyte

19 Caching storage - Memcache(d) Facebook, Twitter, YouTube, need we say more? Distributed memory caching system Key-value storage system Keys - max. 250bytes Values - max. 1Mbyte Extremely fast... non-blocking, UDP (!)

20 Memcache - where to install

21 Memcache - where to install

22 Memcache - installation & running it Installation Distribution package PECL Windows : binaries Running No config-files memcached -d -m <mem> -l <ip> -p <port> ex. : memcached -d -m l p 11211

23 Caching storage - Memcache - some notes Not fault-tolerant It's a cache! Lose session data Lose shopping cart data...

24 Caching storage - Memcache - some notes Not fault-tolerant It's a cache! Lose session data Lose shopping cart data Firewall your Memcache port!

25 Memcache in code <?php $memcache = new Memcache(); $memcache->addserver(' ', 11211); $memcache->addserver(' ', 11211); $mydata = $memcache->get('mykey'); if ($mydata === false) { $mydata = GetMyDataFromDB(); // Put it in Memcache as 'mykey', without compression, with no expiration $memcache->set('mykey', $mydata, false, 0); } echo $mydata;

26 Benchmark with Memcache Single webserver Proxy Static PHP Static PHP Apache + PHP Apache + PHP + MC

27 Memcache slabs (or why Memcache says it's full when it's not) Multiple slabs of different sizes : Slab 1 : 400 bytes Slab 2 : 480 bytes (400 * 1.2) Slab 3 : 576 bytes (480 * 1.2) (and so on...) Multiplier (1.2 here) can be configured Store a lot of very large objects Large slabs : full Rest : free Eviction of data!

28 Memcache - Is it working? Connect to it using telnet "stats" command Use Cacti or other monitoring tools STAT pid 2941 STAT uptime STAT time STAT version STAT pointer_size 64 STAT rusage_user STAT rusage_system STAT curr_connections 16 STAT total_connections STAT connection_structures 96 STAT cmd_get STAT cmd_set STAT cmd_flush 0 STAT get_hits STAT get_misses STAT delete_misses 101 STAT delete_hits STAT incr_misses 0 STAT incr_hits 0 STAT decr_misses 0 STAT decr_hits 0 STAT cas_misses 0 STAT cas_hits 0 STAT cas_badval 0 STAT auth_cmds 0 STAT auth_errors 0 STAT bytes_read STAT bytes_written STAT limit_maxbytes STAT accepting_conns 1 STAT listen_disabled_num 0 STAT threads 4 STAT conn_yields 0 STAT bytes STAT curr_items STAT total_items STAT evictions 0 STAT reclaimed 0

29 Memcache - backing up

30 Memcache - tip Page with multiple blocks? use Memcached::getMulti() getmulti($array) Hashing algorithm But : what if you get some hits and some misses?

31 Updating data

32 Updating data LCD_Popular_Product_List

33 Adding/updating data $memcache->delete('lcd_popular_product_list');

34 Adding/updating data

35 Adding/updating data - Why it crashed

36 Adding/updating data - Why it crashed

37 Adding/updating data - Why it crashed

38 Cache stampeding

39 Cache stampeding

40 Memcache code? Memcache code Visitor interface Admin interface DB

41 Cache warmup scripts Used to fill your cache when it's empty Run it before starting Webserver! 2 ways : Visit all URLs Error-prone Hard to maintain Call all cache-updating methods Make sure you have a warmup script!

42 Cache stampeding - what about locking? Seems like a nice idea, but... While lock in place What if the process that created the lock fails?

43 LAMP... LAMMP LNMMP

44 Nginx Web server Reverse proxy Lightweight, fast 12.2% of all Websites

45 Nginx No threads, event-driven Uses epoll / kqueue Low memory footprint active connections = normal

46 Nginx - Configuration server { listen server_name index root 80; *.domain.ext; index.html; /home/domain.ext/www; } server { listen server_name index root } 80; photo.domain.ext; index.html; /home/domain.ext/photo;

47 Nginx with PHP-FPM Since PHP Runs on port 9000 Nginx connects using fastcgi method location / { fastcgi_pass :9000; fastcgi_index index.php; include fastcgi_params; fastcgi_param SCRIPT_NAME $fastcgi_script_name; fastcgi_param SCRIPT_FILENAME /home/ fastcgi_param SERVER_NAME $host; fastcgi_intercept_errors on; }

48 Nginx + PHP-FPM features Graceful upgrade Spawn new processes under high load Chroot Slow request log!

49 Nginx + PHP-FPM features Graceful upgrade Spawn new processes under high load Chroot Slow request log! fastcgi_finish_request() offline processing

50 Nginx + PHP-FPM - performance? Single webserver Proxy Static PHP Static PHP Apache + PHP Apache + PHP + MC Nginx + PHP-FPM + MC Limit : single-threaded Apachebench

51 Reverse proxy time...

52 Varnish Not just a load balancer Reverse proxy cache / http accelerator / Caches (parts of) pages in memory Careful : uses threads (like Apache) Nginx usually scales better (but doesn't have VCL)

53 Varnish - backends + load balancing backend server1 {.host = " "; } backend server2 {.host = " "; } director example_director round-robin { {.backend = server1; } {.backend = server2; } }

54 Varnish - VCL Varnish Configuration Language DSL (Domain Specific Language) compiled to C Hooks into each request Defines : Backends (web servers) ACLs Load balancing strategy Can be reloaded while running

55 Varnish - whatever you want Real-time statistics (varnishtop, varnishhist,...) ESI

56 Varnish - ESI Perfect for caching pages Header (TTL : 60 min) /top Latest news (TTL : 2 min) /news Navigation (TTL : 60 min) /nav Article content page Article content (TTL : 15 min) /article/732 In your article page output : <esi:include src="/top"/> <esi:include src="/nav"/> <esi:include src="/news"/> <esi:include src="/article/732"/> In your Varnish config : sub vcl_fetch { if (req.url == "/news") { esi; /* Do ESI processing */ set obj.ttl = 2m; } elseif (req.url == "/nav") { esi; set obj.ttl = 1m; } elseif.. }

57 Varnish with ESI - hold on tight! Single webserver Proxy Static PHP Static PHP Apache + PHP Apache + PHP + MC Nginx + PHP-FPM + MC Varnish

58 Varnish - what can/can't be cached? Can : Static pages Images, js, css Pages or parts of pages that don't change often (ESI) Can't : POST requests Very large files (it's not a file server!) Requests with Set-Cookie User-specific content

59 ESI no caching on user-specific content? Logged in as : Wim Godden TTL = 0s? TTL=1h 5 messages TTL = 5min

60 Coming soon... Based on Nginx Reduces load by 50 95% Requires code changes! Well-built project few changes Effect on webservers and database servers

61 What's the result?

62 What's the result?

63 Figures First customer : No. of web servers : 18 4 No. of db servers : 6 2 Total : 24 6 (75% reduction!) Second customer (already using Nginx + Memcache) : No. of web servers : 72 8 No. of db servers : 15 4 Total : (86% reduction!)

64 Availability Stable at 2 customers Still under heavy development Beta : July 2012 Final : Sep 2012

65 PHP speed - some tips Upgrade PHP - every minor release has 5-15% speed gain! Use an opcode cache (APC, eaccelerator, XCache)

66 DB speed - some tips Use same types for joins i.e. don't join decimal with int RAND() is evil! count(*) is evil in InnoDB without a where clause! Persistent connect is sort-of evil

67 Caching & frontend

68 Frontend tuning 1. You optimize backend 2. Frontend engineers messes up havoc on backend 3. Don't forget : frontend sends requests to backend! SO... Care about frontend Test frontend Check what requests frontend sends to backend

69 Tuning frontend Minimize requests Combine CSS/JavaScript files

70 Tuning frontend Minimize requests Combine CSS/JavaScript files Use CSS Sprites

71 CSS Sprites

72 Tuning content - CSS sprites

73 Tuning content - CSS sprites 11 images 11 HTTP requests 24KByte 1 image 1 HTTP requests 14KByte

74 Tuning frontend Minimize requests Combine CSS/JavaScript files Use CSS Sprites (horizontally if possible) Put CSS at top Put JavaScript at bottom Max. no connections Especially if JavaScript does Ajax (advertising-scripts, )! Avoid iframes Again : max no. of connections Don't scale images in HTML Have a favicon.ico (don't 404 it!) see my blog

75 What else can kill your site? Redirect loops Multiple requests More load on Webserver More PHP to process Additional latency for visitor Try to avoid redirects anyway In ZF : use $this->_forward instead of $this->_redirect Watch your logs, but equally important... Watch the logging process Logging = disk I/O can kill your server!

76 Above all else... be prepared! Have a monitoring system Use a cache abstraction layer (disk Memcache) Don't install for the worst prepare for the worst Have a test-setup Have fallbacks Turn off non-critical functionality

77 So... Cache But : never delete, always push! Have a warmup script Monitor your cache Have an abstraction layer Apache = fine, Nginx = better Static pages? Use Varnish Tune your frontend impact on backend!

78 Questions?

79 Questions?

80 We're hiring! Lots of challenges Work with cutting-edge technology Varied : PHP development System / network architecture Scalability services Build our own services Work on Open Source mail us : info@cu.be

81 Contact Twitter Web Slides Please... Rate my talk :

82 Thanks! Please... Rate my talk :

83