Red Hat Enterprise Linux C++ Toolchains: 10 Tips to Drive Your Development

Transcription

1 attacks much harder

2 Red Hat Enterprise Linux C++ Toolchains: 10 Tips to Drive Your Development Matt Newsome Senior Engineering Manager, Tools v1.0

3 % bit.ly/dtstalks

4 bit.ly/dtstalks

5 RED HAT ENTERPRISE LINUX TOOLCHAINS RED HAT DEVELOPER TOOLSET YOUR QUESTIONS bit.ly/dtstalks

6 TIP #1 USE THE MOST RECENT TOOLS YOU CAN

7 Toolchain Support RHEL 5 7 RHEL 6 10 YEARS OF SUPPORT RHEL 7 Fedora VERY HIGH STABILITY LIMITED NEW FEATURES bit.ly/dtstalks

8 Building with RHEL tools Application Sources BUILD Branched Sources BUILD bit.ly/dtstalks gcc x gcc y DEPLOY DEPLOY RHEL 6 RHEL 7

9 Building with RHEL tools (2) Application Sources x.1 BUILD Sources x.2 gcc x.2 Sources y.2 Sources y.3 bit.ly/dtstalks BUILD gcc y.1 gcc y.2 gcc y.3 RHEL 6.y RHEL 6.n+1 gcc x.3 Sources x.3 Application Sources y.1 gcc x.1 DEPLOY RHEL 6.n+2 DEPLOY RHEL 7.y RHEL 7.n+1 RHEL 7.n+2

10 Ideal Scenario RHEL 6 Sources Build Latest Tools Test & Deploy RHEL 7 bit.ly/dtstalks

11 RED HAT DEVELOPER TOOLSET

12 What is Red Hat Developer Toolset (DTS)? The latest stable tools for [typically C/C++] developers (Compiler, IDE, performance and other developer tools) An extra set - does not replace your existing RHEL tools RHEL 6 RHEL 7 DTS 3 and later tools run on RHEL 6 & 7 Applications built with DTS run on the same and following major release of RHEL, i.e....build with DTS 3.1 on RHEL 6, run on 6 & 7 Available with supported Red Hat Enterprise Linux Server and Workstation subscriptions v3.1 GA available today bit.ly/dtstalks

13 Developer Toolset (DTS) EL RHEL RHEL 6 & 7 x86-64 C, C++ & Fortran SOFTWARE COLLECTIONS bit.ly/dtstalks

14 SOFTWARE COLLECTIONS Structure for delivering software outside OS /opt (per Filesystem Hierarchy Standard) Different filesystem root per application version Activated via script Allows multiple versions installed in parallel Independent lifecycle from RHEL bit.ly/dtstalks

15 Developer Toolset is a Software Collection / usr gcc bin lib RHEL x.y System Compiler etc. libc.so Developer Toolset Compiler etc. opt rh dts-3.x root usr Special Invocation Separate tools, not default Lifecycle independent from RHEL root bit.ly/dtstalks enable gcc etc. enable dts-2.x bin... Internal script invoked via scl utility

16 WHAT'S INCLUDED?

17 What s in Developer Toolset v3.1? CORE TOOLS GCC 4.9 ECLIPSE IDE GDB 7.8 BINUTILS LUNA DEBUG & PERFORMANCE TOOLS SYSTEMTAP 2.6 OPROFILE VALGRIND DYNINST STRACE 4.8 MEMSTOMP DWZ-0.11 ELFUTILS LTRACE bit.ly/dtstalks SOFTWARE COLLECTIONS RUNTIME

18 What s in Developer Toolset v3.1? CORE TOOLS GCC 4.9 ECLIPSE IDE GDB 7.8 BINUTILS 2.24 SOFTWARE COLLECTIONS RUNTIME LUNA DEBUG & PERFORMANCE TOOLS SYSTEMTAP 2.6 OPROFILE VALGRIND DYNINST STRACE 4.8 MEMSTOMP DWZ-0.11 ELFUTILS LTRACE bit.ly/dtstalks KEY: NEW IN DTS3 UPDATED IN 3.1 UNCHANGED IN 3.1

19 Updated Compiler: GCC 4.9 ISO C/C++ Stds OpenMP4 / Cilk+ IPA / LTO CPU bit.ly/dtstalks C++11 library improved, experimental C++14 support Generic lambdas, var-length arrays & digit separators C11: Atomics, generic selections, thread-local storage C and C++ support for OpenMP 4.0 C/C++ language extensions for parallel programming Interprocedural analysis (IPA) improvements Link-Time Optimization (LTO) faster, smaller Intel AVX-512 target architecture support added Advanced Vector Extensions supported in binutils

20 Eclipse IDE v4.4.0 ( Luna ) [RHEL 6 and RHEL 7] bit.ly/dtstalks

21 Developer Toolset Life Cycle ANNUAL RELEASE DRIVEN BY GCC SPRING bit.ly/dtstalks SUMMER TOOLSET

22 Developer Toolset Life Cycle v2.0 MID-YEAR v2.1 2.x CRITICAL FIXES AND SECURITY UPDATES 2.x END OF SUPPORT TOOLSET v2.x FIRST YEAR SECOND YEAR bit.ly/dtstalks v3.0 MID-YEAR Annual major Mid-year minor Async updates v3.1 3.x CRITICAL FIXES & SECURITY UPDATES 3.x END OF SUPPORT TOOLSET v3.x

23 Usage EITHER Subscription # rhn-channel --add --channel=rhel-x86_64-workstation-dts-6 # yum-config-manager --enable rhel-server-dts-6-rpms { Installation # yum install devtoolset-3 Usage EITHER } { scl enable devtoolset-3 'gcc...' Click the Developer Toolset Eclipse 3.x Eclipse icon Advanced scl enable devtoolset-3 'bash...' bit.ly/dtstalks }

24 TIP #1 DEVELOPER TOOLSET

25 TIP #2 SANITIZERS

26 AddressSanitizer ( asan ) int main(int argc, char **argv) { int stack_array[100]; stack_array[1] = 0; return stack_array[argc + 100]; } // out of bounds $ gcc fsanitize=address outofbounds.c o outofbounds $./outofbounds ==3126== ERROR: AddressSanitizer: stack buffer overflow on address 0x7fff2d3afbc4 at pc 0x bp 0x7fff2d3af9f0 sp 0x7fff2d3af9e0

27 AddressSanitizer ( asan ) (2) $ gcc fsanitize=address outofbounds.c o dts outofbounds /opt/rh/devtoolset 3/root/usr/libexec/gcc/x86_64 redhat linux/4.9.2/ld: cannot find libasan_preinit.o: No such file or directory /opt/rh/devtoolset 3/root/usr/libexec/gcc/x86_64 redhat linux/4.9.2/ld: cannot find lasan collect2: error: ld returned 1 exit status For DTS you'll need to yum install devtoolset-3-libasan-devel Collision between libasan in RHEL and Developer Toolset Resolve by yum remove libasan and then yum install either libasan for RHEL 7 or libasan for DTS3 We're looking at a fix for this in a future DTS release

28 ThreadSanitizer ( tsan ) #include <pthread.h> #include <stdio.h> int Global; // global variable without any mutex, etc. void *Thread1(void *x) { Global++; return NULL; } void *Thread2(void *x) { Global ; return NULL; } int main() { pthread_t t[2]; pthread_create(&t[0], NULL, Thread1, NULL); pthread_create(&t[1], NULL, Thread2, NULL); pthread_join(t[0], NULL); pthread_join(t[1], NULL); }

29 ThreadSanitizer ( tsan ) (2) $ gcc fsanitize=thread race.cc o race $./race WARNING: ThreadSanitizer: data race (pid=12639) For DTS you'll need to yum install devtoolset-3-libtsan-devel Collision between libtsan in RHEL and Developer Toolset Resolve by yum remove libtsan and then yum install either libtsan for RHEL 7 or libtsan for DTS3 We're looking at a fix for this in a future DTS release

30 UndefinedBehaviorSanitizer ( ubsan ) #include <stdio.h> #include <limits.h> volatile int i = 23, j, *n=null, iarray[2]; main () { i <<= 32; // shift equal to promoted left operand i = INT_MIN; j = i; // overflow iarray[5] = 123; // out of bounds access *n = i; // store to a NULL pointer } $ gcc fsanitize=undefined ub.cc o ub $./ub ub.c:5:5: shift exponent 32 is too large for 32 bit 'int' ub.c:6:20: negation of cannot be represented in type 'int [2]'; cast to an unsigned type to negate this value to itself ub.c:7:9: index 5 out of bounds for type 'int [2]' ub.c:8:6: store to null pointer of type 'volatile int'

31 TIP #3 New language standards: C++11 and C++14

32 C++11 #include <iostream> int main() { // lambda function auto sum = [](int x, int y) { return x + y; }; std::cout << sum(12,5) << std::endl; std::cout << sum(13,2) << std::endl; } $ g++ std=c++11./lambda.cc o lambda $./lambda 17 15

33 C++11 and C++14 C++11 feature complete in gcc-4.9 (DTS-3.1) C++14 experimental support since same release C++11 and RHEL/DTS C++98 objects can be mixed C++11 objects need to be rebuilt across major releases of gcc (and hence DTS)

34 TIPS #4-#6 SECURITY, SECURITY, SECURITY

35 TIP #4 SECURITY: ASLR/PIE

36 Address Space Layout Randomization (ASLR) Security mechanism to counter Return Oriented Programming (ROP) ROP attacks exploit existing executable code fragments to perform unintended actions ASLR randomizes the position of binaries, making it much harder to perform these attacks Review against performance cost

37 Position Independent Executables #include <stdio.h> int main() { printf("hello PIE! (0x%x)\n", &main); } $ gcc./pie.c o pie $./pie ;./pie ;./pie Hello PIE! (0x400536) Hello PIE! (0x400536) Hello PIE! (0x400536) $ gcc fpie pie./pie.c o pie $./pie ;./pie ;./pie Hello PIE! (0x6c5d7790) Hello PIE! (0x663b3790) Hello PIE! (0x1e2e3790)

38 TIP #5 SECURITY: RELRO

39 RELocatable Read-Only (RELRO) Security mechanism to counter attacks against tables of dynamically linked function addresses A couple of variants of RELRO exist (partial and full) Reorders writeable data to follow internal data structures Review against performance cost (moves all dynamic relocation costs to startup)

40 RELRO #include int main size_t p[0] = printf } <stdio.h> (int argc, char *argv[]) { *p = (size_t *)strtol (argv[1], NULL, 16); 0xDEADBEEF; ("RELRO: %p\n", p); $ gcc w g Wl, z,relro, z,now./relro.c o relro $ readelf r./relro grep printf fd R_X86_64_JUMP_SLO... printf + 0 $ echo "r fd8" gdb q./relro Program received signal SIGSEGV, Segmentation fault. 0x c0 in main (argc=2, argv=0x7fffffffddf8)

41 TIP #6 SECURITY: MEMSTOMP

42 memcpy() calls with overlapping arguments #include <string.h> main() { char src[] = "text-to-copy"; char *dest = src + 2; // call memcpy() with overlapping args memcpy((void*)dest,(void*)src,3); return 0; } [mattn@rhel]$ gcc memcpy_bad.c -o memcpy_bad [mattn@rhel]$./memcpy_bad Segmentation fault (core dumped) bit.ly/dtstalks

43 memstomp to the rescue! memstomp./memcpy_bad memstomp: successfully initialized for process memcpy_bad (pid 28195). memcpy(dest=0x7fff103cc1c2, src=0x7fff103cc1c0, bytes=3) overlap for memcpy_bad(28195) /.../libmemstomp.so(+0x10a7) [0x7fdb4760a0a7]./memcpy_bad(main+0x45) [0x400795] /lib64/libc.so.6( libc_start_main+0xfd) [0x357fe1ed1d]./memcpy_bad() [0x400669] Indicates overlapping arguments to memcpy()...but also indicates where that call occurs bit.ly/dtstalks

44 memcpy() calls with overlapping arguments fixed #include <string.h> main() { char src[] = "text-to-copy"; char *dest = src + 5; // no longer overlaps memcpy((void*)dest,(void*)src,3); return 0; } [mattn@rhel]$ gcc memcpy_fixed.c -o memcpy_fixed [mattn@rhel]$./memcpy_fixed [mattn@rhel]$ memstomp./memcpy_fixed memstomp: successfully initialized for process memcpy_fixed (pid 31322). bit.ly/dtstalks

45 Fixing memcpy() calls The best way: fix memcpy() calls Alternative: replace memcpy() with memmove() bit.ly/dtstalks

46 TIP #7 CONTAINERS

47 TIP #8 OVERFLOW BUILTINS

48 Overflow checking builtins // requires gcc5 or later #include <stdio.h> int main() { signed char c1 = 125, res = 0; while (! builtin_add_overflow(c1, (signed char)1, &c1)) { printf("%d: ok\n", c1); } printf("%d: overflow!\n", c1); } $ gcc./overflow.c o overflow $./overflow 126: ok 127: ok 128: overflow!

49 Overflow checking builtins (2) GCC5 only Stay tuned for a future Developer Toolset release... Variants for different arithmetic operators / types No special compilation required Experiment with GCC5 in a Docker container: # docker pull fedora # docker run -i -t fedora 'bash' # enter shell # yum install gcc

50 TIP #9 PERF & DEBUG TOOLS

51 Performance Analysis and Debugging Tools Debugger: underused, powerful tool SystemTap: Live application analysis without rebuilds PAPI: Programmer interface to perf. counter hardware OProfile: Unobtrusive, system-wide code profiler Valgrind: Runtime analysis (particularly memory)

52 TIP #10 CONTRIBUTE BACK

53 Contribute back! Raise bugs Contribute to community discussions Contribute patches and new features Pay it forwards: release software under open source licenses The whole community benefits

54 ACCESSING DEVELOPER TOOLSET

55 Access and Installation 1. Access a subscription that includes Red Hat Software Collections bit.ly/dtstalks

56 How does one access Red Hat Software Collections? Red Hat Developer Toolset (March 2014) STANDARD PREMIUM RED HAT ENTERPRISE LINUX DEVELOPER WORKSTATION RED HAT ENTERPRISE LINUX WORKSTATION RED HAT ENTERPRISE LINUX SERVER RED HAT ENTERPRISE LINUX DEVELOPER SUBSCRIPTIONS asfa RED HAT ENTERPRISE LINUX ACADEMIC SITE SUBSCRIPTIONS Available Only Self Support RED HAT ENTERPRISE LINUX DEVELOPER SUITE Available Only Self Support bit.ly/dtstalks

57 Access and Installation 1. Access a subscription that includes Red Hat Software Collections 2. If using Satellite, generate a new certificate 3. Register your RHEL system 4. Attach a subscription 5. Add the channel [refer to release notes: red.ht/devtoolset] 6. yum install devtoolset 3 bit.ly/dtstalks

58 Red Hat Software Collections Delivers the latest stable versions of dynamic languages, web servers and open source databases asfa KEY BENEFITS Choose the runtime versions best suited for your projects Preserve application stability with side-by-side versioning Red Hat Support for 2 or 3 years bit.ly/dtstalks More up-to-date versions of languages and databases than offered with Red Hat Enterprise Linux

59 Links Developer Program Developer Toolset Documentation This talk: bit.ly/dtstalks

60 Contacts General questions, thoughts, etc. Red Hat Developer Toolset / Software Collections Product Manager Brian Gollaher (bgollahe@redhat.com) Engineering Leads DTS: Martha Benitez (mbenitez@redhat.com) RHSCL: Joe Orton (jorton@redhat.com) bit.ly/dtstalks

61 YOUR QUESTIONS bit.ly/dtstalks

62 THANK-YOU bit.ly/dtstalks