Shared Libraries: The Dynamic Linker

Transcription

1 Building and Using Shared Libraries on Linux Shared Libraries: The Dynamic Linker Michael Kerrisk, man7.org c 2017 mtk@man7.org November 2017 Outline 3 Operation of the Dynamic Linker Rpath: specifying library search paths in an object An aside: shared libraries and the static linker Handling secondary dependencies Finding shared libraries at run time Symbol resolution, library load order, and link maps Debugging the operation of the dynamic linker 3-35

2 Outline 3 Operation of the Dynamic Linker Rpath: specifying library search paths in an object An aside: shared libraries and the static linker Handling secondary dependencies Finding shared libraries at run time Symbol resolution, library load order, and link maps Debugging the operation of the dynamic linker 3-35 Specifying library search paths in an object So far, we have two methods of informing DL of location of a shared library: LD_LIBRARY_PATH Installing library in one of the standard directories Third method: during static linking, we can insert a list of directories into the executable So-called run-time library path (rpath) list At run time, DL will search listed directories to resolve dynamic dependencies Useful if libraries will reside in locations that are fixed, but not in standard list [TLPI 41.10] Shared Libraries on Linux c 2017, Michael Kerrisk Operation of the Dynamic Linker

3 Defining an rpath list when linking To embed an rpath list in an executable, use the -rpath linker option (gcc -Wl,-rpath,path-to-lib-dir) Multiple -rpath options can be specified ordered list Alternatively, multiple directories can be specified as a colon-separated list in a single -rpath option Example: $ cc -g - Wall -Wl,- rpath,$pwd -o prog prog. c \ libdemo.so $ objdump -p prog grep RPATH RPATH / home / mtk / tlpi / code / shlibs $./ prog Called mod1 - x1 Called mod2 - x2 Embeds current working directory in rpath list objdump command allows us to inspect rpath list Executable now tells DL where to find shared library Shared Libraries on Linux c 2017, Michael Kerrisk Operation of the Dynamic Linker Shared libraries can have rpath lists Shared libraries can themselves have dependencies can use -rpath linker option to embed rpath lists when building shared libraries Shared Libraries on Linux c 2017, Michael Kerrisk Operation of the Dynamic Linker

4 An rpath improvement: DT_RUNPATH There are two types of rpath list: Differ in precedence relative to LD_LIBRARY_PATH Original (and default) rpath list has higher precedence DT_RPATH ELF entry The original rpath behavior was a design error We want user to have full control when using LD_LIBRARY_PATH Shared Libraries on Linux c 2017, Michael Kerrisk Operation of the Dynamic Linker An rpath improvement: DT_RUNPATH Newer rpath type has lower precedence Gives user possibility to override rpath at runtime using LD_LIBRARY_PATH (usually what we want) DT_RUNPATH ELF entry Supported in DL since 1999 Use: cc -Wl,-rpath,some-dir-path -Wl,--enable-new-dtags binutils 2.24 (2013) and later: inserts only DT_RUNPATH entry Before binutils 2.24, inserted DT_RUNPATH and DT_RPATH (to allow for old DLs that didn t understand DT_RUNPATH) If both types of rpath list are embedded in an object, DT_RUNPATH has precedence I.e., DL ignores DT_RPATH list Shared Libraries on Linux c 2017, Michael Kerrisk Operation of the Dynamic Linker

5 Magic names in rpath lists DL understands certain special names in rpath list Written as $NAME or ${NAME} $ORIGIN: expands to directory containing program or library Write turn-key applications! Installer unpacks tarball containing application with library in (say) a subdirectory; application can be linked with: cc -Wl,- rpath, $ORIGIN / lib Note use of quotes to prevent interpretation of $ by shell! Shared Libraries on Linux c 2017, Michael Kerrisk Operation of the Dynamic Linker Magic names in rpath lists $LIB: expands to lib or lib64, depending on architecture E.g., useful on multi-arch platforms to supply 32-bit or 64-bit library, as appropriate $PLATFORM: expands to string corresponding to processor type (e.g., x86_64) Rpath entry can include arch-specific directory component DL also understands these names in some other contexts See discussion of LD_LIBRARY_PATH, LD_PRELOAD, and LD_AUDIT in ld.so(8) Shared Libraries on Linux c 2017, Michael Kerrisk Operation of the Dynamic Linker

6 Outline 3 Operation of the Dynamic Linker Rpath: specifying library search paths in an object An aside: shared libraries and the static linker Handling secondary dependencies Finding shared libraries at run time Symbol resolution, library load order, and link maps Debugging the operation of the dynamic linker 3-35 Recap of a linker rule When creating an executable, linker (ld) requires that there must be a resolution available for all symbol references All symbols in executable and all shared libraries it uses Goal: prevent (frequent) occurrence of run-time link failures --allow-shlib-undefined linker option permits undefined references in shared libraries used by executable But see preceding point Note: the resolution seen at static link time may not be the resolution used at run time... ld does not enforce this requirement when linking to produce a shared library Unless --no-undefined linker option is used Shared Libraries on Linux c 2017, Michael Kerrisk Operation of the Dynamic Linker

7 Finding shared libraries at static link time The static linker (ld(1)) looks for shared libraries in: 1 Directories specified by -rpath-link options 2 Directories specified by -rpath options Directories specified by -rpath are included in executable and used at runtime By contrast, -rpath-link option is effective only at link time 3 If the -rpath and -rpath-link were not used, the directories specified by LD_RUN_PATH 4 The directories specified by LD_LIBRARY_PATH 5 The directories in DT_RUNPATH or DT_RPATH of a shared library are searched for shared libraries it needs DT_RPATH entries are ignored if DT_RUNPATH entries exist 6 /lib and /usr/lib (or /lib64 and /usr/lib64) 7 The directories specified in /etc/ld.so.conf Shared Libraries on Linux c 2017, Michael Kerrisk Operation of the Dynamic Linker

8 Outline 3 Operation of the Dynamic Linker Rpath: specifying library search paths in an object An aside: shared libraries and the static linker Handling secondary dependencies Finding shared libraries at run time Symbol resolution, library load order, and link maps Debugging the operation of the dynamic linker 3-35 Handling secondary dependencies Suppose that: We have a main program that depends on lib_x1.so which in turn depends on lib_x2.so lib_x1.so was not built with an rpath pointing to location of lib_x2.so Because we did not want to fix run-time location of lib_x2.so at link time Shared Libraries on Linux c 2017, Michael Kerrisk Operation of the Dynamic Linker

9 Handling secondary dependencies Suppose we try to build the libraries and link the program as follows: $ cc -o libx2. so - shared - fpic libx2. c $ cc -o libx1.so - shared -fpic libx1.c libx2.so $ cc -o main main. c libx1. so / usr / bin /ld: warning : libx2.so, needed by libx1.so, not found ( try using - rpath or - rpath - link ) The linker has detected that lib_x1.so depends on lib_x2.so, but does not know where to find it If lib_x1.so had rpath pointing to location of lib_x2.so, ld would have found lib_x2.so There are wrong ways we might try to solve this problem... Shared Libraries on Linux c 2017, Michael Kerrisk Operation of the Dynamic Linker Handling secondary dependencies (the wrong way) Using the -rpath linker option: cc -o main main.o -Wl,- rpath,< dirpath > libx1.so Embeds rpath in main for dependency of libx1.so :-( main should not have to care about this The rpath may cause other libraries to be loaded from wrong location at run time Shared Libraries on Linux c 2017, Michael Kerrisk Operation of the Dynamic Linker

10 Handling secondary dependencies (the wrong way) Specifying libx2.so when linking main: cc -o main main.o libx1.so libx2.so Embeds a direct dependency on libx2.so in main :-( $ objdump -p main grep NEEDED NEEDED libx1. so NEEDED libx2. so Such overlinking can cause various problems May cause unnecessary work in build systems E.g., rebuild of main when libx2.so changes Changes load order of libraries, which may affect symbol resolution (see later) Might build main and libx1.so that depend on different major versions of libx2.so (both loaded at run time?) Shared Libraries on Linux c 2017, Michael Kerrisk Operation of the Dynamic Linker Handling secondary dependencies correctly (-rpath-link) We can correctly handle secondary dependencies with the -rpath-link linker option: $ cc -o libx2. so - shared - fpic libx2. c $ cc -o libx1.so - shared -fpic libx1.c libx2.so $ cc -o main main. c \ -Wl,- rpath -link,< dirpath > libx1.so <dirpath> is path where libx2.so can be found Tells linker where to look for needed libraries Linker can verify that all symbol references have a resolution Does not create an rpath in main Does not embed a dependency on libx2.so in main See ld(1) man page Shared Libraries on Linux c 2017, Michael Kerrisk Operation of the Dynamic Linker

11 Outline 3 Operation of the Dynamic Linker Rpath: specifying library search paths in an object An aside: shared libraries and the static linker Handling secondary dependencies Finding shared libraries at run time Symbol resolution, library load order, and link maps Debugging the operation of the dynamic linker 3-35 Finding shared libraries at run time When resolving dependencies in dynamic dependency list, DL deals with each dependency string as follows: If the string contains a slash interpret dependency as a relative or absolute pathname Otherwise, search for shared library using these rules 1 If object has DT_RPATH list and does not have DT_RUNPATH list, search directories in DT_RPATH list 2 If LD_LIBRARY_PATH defined, search directories it specifies For security reasons, LD_LIBRARY_PATH is ignored in secure mode (set-uid and set-gid programs, etc.) 3 If object has DT_RUNPATH list, search directories in that list 4 Check /etc/ld.so.cache for a corresponding entry 5 Search /lib and /usr/lib (in that order) Or /lib64 and /usr/lib64 [TLPI 41.11] Shared Libraries on Linux c 2017, Michael Kerrisk Operation of the Dynamic Linker

12 Exercises 1 The directory shlibs/mysleep contains two files: mysleep.c: implements a function, mysleep(nsecs), which prints a message and calls sleep() to sleep for nsecs seconds. mysleep_main.c: takes one argument that is an integer string. The program calls mysleep() with the numeric value specified in the command-line argument. Using these files, perform the following steps: Build a shared library from mysleep.c. Compile and link mysleep_main.c against the shared library so that it embeds an rpath list containing the (relative) run-time location of the shared library. Verify that the program executes successfully without the use of LD_LIBRARY_PATH. Shared Libraries on Linux c 2017, Michael Kerrisk Operation of the Dynamic Linker Exercises 2 Modify the previous example to use an rpath list containing the string $ORIGIN/sub. Copy the executable to some directory, and place the shared library in the subdirectory sub under that directory. Verify that the program runs successfully. When employing the rpath linker option, remember to use single quotes around $ORIGIN! Shared Libraries on Linux c 2017, Michael Kerrisk Operation of the Dynamic Linker