DTrace for Web2.0 JavaScript, PHP and Coolstack (SAMP) Philip Torchinsky Solaris Evangelist Sun Microsystems

Transcription

1 DTrace for Web2.0 JavaScript, PHP and Coolstack (SAMP) Philip Torchinsky Solaris Evangelist Sun Microsystems 1

2 Agenda Why should you care?? Introduction to DTrace DTrace and JavaScript DTrace on the BackEnd > Java, PHP, Python, Ruby etc. Summary and Resources 2

3 Why should you care Isn't DTrace a Solaris thing, kernel nerds, etc.? Nope, DTrace now have providers for a large number of languages, such as: > JavaScript > Java > PHP > Python > Ruby As well as for some databases such as PostgreSQL and MySQL 3

4 Introduction to DTrace DTrace was introduced in Solaris 10 Allows for dynamic instrumentation of the OS and applications > A typical Solaris 10 system has around 50K probe points plus any probes that can be dynamically inserted in to ANY application running on the system Available on any Solaris 10 system and OpenSolaris distributions Comes with a new dynamic language, D > D is used to script instrumentation whether it's for your application or you are looking at the system it self 4

5 Introduction to DTrace, (Cont.) Designed explicitly for use on production systems Zero performance impact when not in use Designed with safety in mind, includes safeguards against panics, crashes, data corruption or pathological performance degradation Powerful data management primitives eliminate need for most postprocessing -> aggregations Unwanted data is pruned as close to the source as possible 5

6 The DTrace Revolution DTrace tightens the diagnosis loop: hypothesis instrumentation data gathering analysis hypothesis... Tightened loop effects a revolution in the way we diagnose transient failure Focus can shift from instrumentation stage to hypothesis stage (forget multiple printf): > Much less labor intensive, less error prone > Much more brain intensive > Much more effective! (And a lot more fun) 6

7 In a nutshell : DTrace architecture script.d DTrace(1M) lockstat(1m) plockstat(1m) DTrace consumers libdtrace(3lib) DTrace(7D) DTrace userland kernel DTrace providers sysinfo vminfo fasttrap proc syscall sdt fbt 7

8 The D Language D is a C-like language specific to DTrace with some constructs similar to awk(1) Global, thread-local and probe-local variables Built-in variables like execname and timestamp Predicates can use arbitrary expressions to select which data is traced and which is discarded Actions to trace data, record stack backtraces, stop processes at points of interest, etc. 8

9 Providers A provider allows for instrumentation of a particular area of the system Providers make probes available to the virtual machine Providers transfer control to the DTrace virtual machine when an enabled probe is hit DTrace has several providers, e.g.: > The pid provider for C and C++ applications > The hotspot and dvm provider for Java applications > The syscall provider for system calls > The io provider for system I/O > The profile provider for cyclical events 9

10 D Language - Format. When a probe fires then action is executed if predicate evaluates true Example, Print all the system calls executed by bash probe description / predicate / { action statements } #!/usr/sbin/dtrace -s syscall:::entry /execname== bash / { printf( %s called\n,probefunc); } 10

11 Aggregations Often the patterns are more interesting than each individual datum Want to aggregate data to look for larger trends DTrace supports the aggregation of data as a first class operation An aggregation is the result of an aggregating function > count(), min(), max(), avg(), quantize() May be keyed by any arbitrary tuple 11

12 Aggregation - = aggfunc(args); '@' - key to show that name is an aggregation. keys comma separated list of D expressions. aggfunc could be one of... > sum(expr) total value of specified expression > count() number of times called. > avg(expr) average of expression > min(expr)/max(expr) min and max of expressions > quantize()/lquantize() - power of two & linear distribution 12

13 Aggregation Example #!/usr/sbin/dtrace -s pid$target:libc:malloc:entry Distribution"]=quantize(arg0); } $ aggr_malloc.d -c who dtrace: script './aggr2.d' matched 1 probe... dtrace: pid 1401 has exited Malloc Distribution value Distribution count

14 Calculating time spent One of the most common request is to find time spent in a given function Here is how this can be done #!/usr/sbin/dtrace -s syscall::open*:entry, syscall::close*:entry { ts=timestamp; } Whats wrong with this?? syscall::open*:return, syscall::close*:return { timespent = timestamp - ts; printf("threadid %d spent %d nsecs in %s", tid, timespent, probefunc); ts=0; /*allow DTrace to reclaim the storage */ timespent = 0; } 14

15 Thread Local Variable self->variable = expression; > self keyword to indicate that the variable is thread local > A boon to multi-threaded debugging > As name indicates this is specific to the thread. See code re-written #!/usr/sbin/dtrace -s syscall::open*:entry, syscall::close*:entry { self->ts=timestamp; } syscall::open*:return, syscall::close*:return { timespent = timestamp - self->ts; printf("threadid %d spent %d nsecs in %s", tid, timespent, probefunc); self->ts=0; /*allow DTrace to reclaim the storage */ timespent = 0; } 15

16 Built-in Variable Here are a few built-in variables. arg0... arg9 Arguments represented in int64_t format args[ ] - Arguments represented in correct type based on function. cpu current cpu id. cwd current working directory errno error code from last system call gid, uid real group id, user id pid, ppid, tid process id, parent proc id & thread id probeprov, probemod, probefunc, probename - probe info. timestamp, walltimestamp, vtimestamp time stamp nano sec from an arbitary point and nano sec from epoc. 16

17 DTrace for scripting languages Dynamic Tracing for Dynamic development 17

18 DTrace and Scripting Language DTrace has been integrated into many scripting languages. We will see two samples. > PHP > JavaScript For this exercise we will use Coolstack version of PHP. > Get coolstack from Please note the DTrace providers are experimental and not official 18

19 DTrace for PHP D tracing for P Hyper Processor 19

20 DTrace and PHP DTrace provider for PHP is a PECL/PEAR module To get the provider # pear install dtrace or # pecl install dtrace After coolstack, just download CSKamp and pkgadd it! then add extension=dtrace.so to the php.ini file then restart PHP 20

21 DTrace and PHP Couple links for more information Wez Furlong who created the provider has details in Bryan Cantrill has a nice demo script in his blog. 21

22 DTrace and PHP There are two probes in the provider. > function-return > function-entry You can use the 5 args in the action. > arg0 = the function name > arg1 = the filename > arg2 = the line number > arg3 = classname (or an empty string) > arg4 = object/class operator (::, ->, or an empty string) 22

23 DTrace and PHP Here is a PHP example prime.php <?php $numprimes=10000; $parray[0]=3; $test=5; $num=0; function doesdevide($x,$y) { return($x % $y); } function isprime($x) { global $parray; global $num; $index=0; $check=1; while($check==1 && $index <= $num && $x >= ($parray[$index] * $parray[$index]) ) { if( doesdevide($x, $parray[$index]) == 0) { $check=0; } else $index++; } return($check); } 23

24 DTrace and PHP Here is a PHP example(cont) while($num<$numprimes) { if(isprime($test)==1){ $num++; $parray[$num]=$test; if($num%1000==0){ printf("progress done...%d\n",$num); } } $test+=2; }?> 24

25 DTrace and PHP Here is a simple D-Script #!/usr/sbin/dtrace -Zqs php*:::function-entry } Note: -Z will allow probes that have zero match. Run this script in one window while you run the php script in another window. 25

26 DTrace and PHP Example run:./php.d ^C printf 10 isprime doesdevide

27 DTrace for Javascript Get look in to the fire, Firefox that is... 27

28 DTrace and Javascript DTrace probes have been added to Mozilla to help observe Javascript application. Here is the list of the probes > js_execute-start and js_execute-done > js_function-entry & js_function-return > js_object-create, js_object-create-start, js_objectcreate-done & js_object-finalize > layout-start & layout-end BUT get the latest firefox from - > > The probe name changes! 28

29 DTrace and MySQL With PID provider and c++filt to manually go through MySQL code, see: > > (in Russian, coming soon) DTrace support will be integrated in MySQL > messageid=

30 DTrace and Apache Open sourced version > Will be integrated in nv85 > > Detail is in > 30

31 DTrace for Web2.0 JavaScript, PHP and CoolStack (SAMP) Philip Torchinsky 31