Linkers and Loaders. CS 167 VI 1 Copyright 2008 Thomas W. Doeppner. All rights reserved.

Does Location Matter? int main(int argc, char *[ ]) { return(argc); } main: pushl %ebp ; push frame pointer movl %esp, %ebp ; set frame pointer to point to new frame movl 8(%ebp), %eax ; put argc into return register (eax) movl %ebp, %esp ; restore stack pointer popl %ebp ; pop stack into frame pointer ret ; return: pops end of stack into eip CS 167 VI 2 Copyright 2008 Thomas W. Doeppner. All rights reserved. The material in this slide through slide 14 is taken directly from the textbook.

Location Matters int X=6; int *ax = &X; int main( ) { void subr(int); int y=x; subr(y); return(0); } void subr(int i) { printf("i = %d\n", i); } CS 167 VI 3 Copyright 2008 Thomas W. Doeppner. All rights reserved. We don t need to look at the assembler code to see what s different about this program: the machine code produced for it can t simply be copied to an arbitrary location in our computer s memory and executed. The location identified by the name ax should contain the address of the location containing X. But since the address of X will not be known until the program is copied into memory, neither the compiler nor the assembler can initialize ax correctly. Similarly, the addresses of subr and printf are not known until the program is copied into memory again, neither the compiler nor the assembler would know what addresses to use.

A Slight Revision extern int X; int *ax = &X; #include <stdio.h> int X; int main( ) { void subr(int); int y = *ax; subr(y); return(0); } main.c void subr(int i) { printf("i = %d\n", i); } subr.c gcc o prog main.c subr.c CS 167 VI 4 Copyright 2008 Thomas W. Doeppner. All rights reserved.

main.s 0:.data ; what follows is initialized data 0:.globl ax ; ax is global: it may be used by others 0: ax: 0:.long X 4: 0:.text ; offset restarts; what follows is text (read-only code) 0:.globl main 0: main: 0: pushl %ebp ; save the frame pointer 1: movl %esp,%ebp ; point to current frame 3: subl $4,%esp ; make space for y on stack 6: movl ax,%eax ; put contents of X into eax 11: movl (%eax),%eax ; put *X into %eax 13: movl %eax,-4(%ebp) ; store *ax into y 16: pushl -4(%ebp) ; push y onto stack 19: call subr 24: addl $4,%esp ; remove y from stack 27: movl $0,%eax ; set return value to 0 31: movl %ebp, %esp ; restore stack pointer 33: popl %ebp ; pop frame pointer 35: ret CS 167 VI 5 Copyright 2008 Thomas W. Doeppner. All rights reserved.

subr.s 0:.data ; what follows is initialized data 0: printfarg: 0:.string "i = %d\n" 8: 0:.comm X,4 ; 4 bytes in BSS is required for global X 4: 0:.text ; offset restarts; what follows is text (read-only code) 0:.globl subr 0: subr: 0: pushl %ebp ; save the frame pointer 1: movl %esp, %ebp ; point to current frame 3: pushl 8(%ebp) ; push i onto stack 6: pushl $printfarg ; push address of string onto stack 11: call printf 16: addl $8, %esp ; pop arguments from stack 19: movl %ebp, %esp ; restore stack pointer 21: popl %ebp ; pop frame pointer 23: ret CS 167 VI 6 Copyright 2008 Thomas W. Doeppner. All rights reserved.

main.o Data: Size: 4 Global: ax, offset 0 Undefined: X Relocation: offset 0, size 4, value: address of X Contents: 0x00000000 bss: Size: 0 Text: Size: 36 Global: main, offset 0 Undefined: subr Relocation: offset 7, size 4, value: address of ax offset 20, size 4, value: PC-relative address of subr Contents: [machine instructions] CS 167 VI 7 Copyright 2008 Thomas W. Doeppner. All rights reserved.

subr.o Data: Size: 8 Contents: "i = %d\n" bss: Size: 4 Global: X, offset 0 Text: Size: 44 Global: subr, offset 0 Undefined: printf Relocation: offset 7, size 4, value: address of printfarg offset 12, size 4, value: PC-relative address of printf Contents: [machine instructions] CS 167 VI 8 Copyright 2008 Thomas W. Doeppner. All rights reserved.

printf.o Data: Size: 1024 Global: StandardFiles Contents: bss: Size: 256 Text: Size: 12000 Global: printf, offset 100 Undefined: write Relocation: offset 211, value: address of StandardFiles offset 723, value: PC-relative address of printf Contents: [machine instructions] CS 167 VI 9 Copyright 2008 Thomas W. Doeppner. All rights reserved.

prog Text Data bss main 4096 subr 4132 printf 4156 write 16156 startup 16172 ax 16384 printfargs 16388 StandardFiles 16396 X 17420 errno 17680 CS 167 VI 11 Copyright 2008 Thomas W. Doeppner. All rights reserved.

Shared Libraries Process A Process B printf( ) stdio printf( ) printf( ) CS 167 VI 12 Copyright 2008 Thomas W. Doeppner. All rights reserved. Consider the situation shown in the slide: we have two processes, each containing a program that calls printf. Up to this point in our discussion, the two processes have no means for sharing a single copy of printf each must have its own. If you consider that pretty much every C program calls printf, a huge amount of disk space in the world could be wasted because of all the copies of printf. Furthermore, when each program is loaded into primary memory, large amount of such memory is wasted because of multiple copies of printf. What is needed is a means for programs to share a single copy of printf (as well as other routines). However, sharing of code is not trivial to implement. A big problem is relocation. The code for printf might well contain relocatable addreses, such as references to global data and other procedures. What makes things difficult is that the code for printf might be mapped into the two processes at different virtual locations.

Relocation and Shared Libraries 1) Prerelocation: relocate libraries ahead of time 2) Limited sharing: relocate separately for each process 3) Position-Independent Code: no need for relocation CS 167 VI 13 Copyright 2008 Thomas W. Doeppner. All rights reserved. If all users of printf agree to load it and everything it references into the same locations in their address spaces, we would have no relocation problem. But such agreement is, in general, hard to achieve. It is, however, the approach used in Windows. A possibility might be for the users of printf to share a single on-disk copy, but for this copy to be relocated separately in each process when loaded. This would allow sharing of disk space, but not of primary storage. Another possibility is for printf to be written in such a way that relocation is not necessary. Code written in this fashion is known as position-independent code (PIC).

Position-Independent Code ld r2, r1[printf] call r2 0 printf( ) { ld r2, r1[doprint] call r2... } ld r2, r1[printf] call r2 r1 printf 10000 doprint 11000 1000 doprint( ) {... } r1 printf 20000 doprint 21000 CS 167 VI 14 Copyright 2008 Thomas W. Doeppner. All rights reserved. Here is an example of the use of position-independent code (PIC). Processes A and B are sharing the library containing printf (note that printf contains a call to another shared routine, doprint), though each has it mapped into a different location. Each process maintains a private table, pointed to by register r1. In the table are the addresses of shared routines, as mapped into the process. Thus, rather than call a routine directly (via an address embedded in the code), a position-independent call is made: the address of the desired routine is stored at some fixed offset within the table. The contents of the table at this offset are loaded into register r2, and then the call is made via r2.

Linking and Loading on Linux with ELF Substitution Shared libraries Versioning Dynamic linking Interpositioning CS 167 VI 15 Copyright 2008 Thomas W. Doeppner. All rights reserved. ELF stands for executable and linking format and is used on most Unix systems, including Linux, Solaris, FreeBSD, NetBSD, and OpenBSD, but not MacOS X.

Creating a Library % gcc -c sub1.c sub2.c sub3.c % ls sub1.c sub2.c sub3.c sub1.o sub2.o sub3.o % ar cr libpriv1.a sub1.o sub2.o sub3.o % ar t libpriv1.a sub1.o sub2.o sub3.o % CS 167 VI 16 Copyright 2008 Thomas W. Doeppner. All rights reserved.

Using a Library % cat prog.c int main() { sub1(); sub2(); sub3(); } % cat sub1.c void sub1() { puts("sub1"); } % gcc -o prog prog.c -L. -lpriv1 Where does puts come from? %gcc o prog prog.c L. \ -lpriv1 L/lib -lc CS 167 VI 17 Copyright 2008 Thomas W. Doeppner. All rights reserved.

Substitution % cat myputs.c int puts(char *s) { write(1, "My puts: ", 9); write(1, s, strlen(s)); write(1, "\n", 1); return 1; } % gcc c myputs.c % ar cr libmyputs.a myputs.o % gcc -o prog prog.c -L. -lpriv1 -lmyputs % CS 167 VI 18 Copyright 2008 Thomas W. Doeppner. All rights reserved.

Shared Libraries 1 Compile program 2 Track down linkages with ld archives (containing relocatable objects) in.a files are statically linked shared objects in.so files are dynamically linked 3 Run program ld.so is invoked to complete the linking and relocation steps, if necessary CS 167 VI 19 Copyright 2008 Thomas W. Doeppner. All rights reserved. Linux supports two kinds of libraries static libraries, contained in archives, whose names end with.a (e.g. libc.a) and shared objects, whose names end with.so (e.g. libc.so). When ld is invoked to handle the linking of object code, it is normally given a list of libraries in which to find unresolved references. If it resolves a reference within a.a file, it copies the code from the file and statically links it into the object code. However, if it resolves the reference within a.so file, it records the name of the shared object (not the complete path, just the final component) and postpones actual linking until the program is executed. If the program is fully bound and relocated, then it is ready for direct execution. However, if it is not fully bound and relocated, then ld arranges things so that when the program is executed, rather than starting with the program s main routine, a runtime version of ld, called ld.so, is called first. ld.so maps all the required libraries into the address space, completes the linkages, and then calls the main routine.

Creating a Shared Library (1) % gcc -fpic -c myputs.c % ld -shared -o libmyputs.so myputs.o % gcc -o prog prog.c -L. -lpriv1 lmyputs %./prog./prog: error while loading shared libraries: libmyputs.so: cannot open shared object file: No such file or directory % ldd prog libmyputs.so => not found libc.so.6 => /lib/tls/i686/cmoc/libc.so.6 /lib/ld-linux.so.2 => /lib/ld-linux.so.2 CS 167 VI 20 Copyright 2008 Thomas W. Doeppner. All rights reserved.

Creating a Shared Library (2) % gcc -o prog prog.c -L. -lpriv1 lmyputs \ Wl,-rpath. % ldd prog libmyputs.so =>./libmyputs.so libc.so.6 => /lib/tls/i686/cmoc/libc.so.6 /lib/ld-linux.so.2 => /lib/ld-linux.so.2 %./prog My puts: sub1 My puts: sub2 My puts: sub3 CS 167 VI 21 Copyright 2008 Thomas W. Doeppner. All rights reserved.

Versioning % gcc -fpic -c myputs.c % ld -shared soname libmyputs.so.1 \ -o libmyputs.so.1 myputs.o % ln s libmyputs.so.1 libmyputs.so % gcc -o prog1 prog1.c -L. -lpriv1 lmyputs \ Wl,-rpath. % vi myputs.c % ld -shared soname libmyputs.so.2 \ -o libmyputs.so.2 myputs.o % rm f libmyputs.so % ln s libmyputs.so.2 libmyputs.so % gcc -o prog2 prog2.c -L. -lpriv1 lmyputs \ Wl,-rpath. CS 167 VI 22 Copyright 2008 Thomas W. Doeppner. All rights reserved.

How To? #include <dlfcn.h> int puts(const char *s) { int (*pptr)(const char *); pptr = (int(*)())dlsym(rtld_next, "puts"); } write(2, "calling myputs: ", 16); return (*pptr)(s); CS 167 VI 24 Copyright 2008 Thomas W. Doeppner. All rights reserved.

Compiling/Linking It % gcc -fpic -c myputs.c -D_GNU_SOURCE % ld -shared -soname libmyputs.so.1 \ -o libmyputs.so.1 myputs.o -ldl % gcc -o tputs tputs.c./libmyputs.so.1 \ -Wl,-rpath. CS 167 VI 25 Copyright 2008 Thomas W. Doeppner. All rights reserved.

What s Going On gcc/ld compiles code does static linking - searches list of libraries - adds references to shared objects runtime program invokes ld.so to finish linking - maps in shared objects - does relocation and procedure linking as required dlsym invokes ld.so to do more linking CS 167 VI 26 Copyright 2008 Thomas W. Doeppner. All rights reserved.

Delayed Wrapping LD_PRELOAD environment variable checked by ld.so specifies additional shared objects to search (first) when program is started CS 167 VI 27 Copyright 2008 Thomas W. Doeppner. All rights reserved.

Example % gcc o tputs tputs.c %./tputs This is a boring message. % setenv LD_PRELOAD./libmyputs.so.1 %./tputs calling myputs: This is a boring message. % CS 167 VI 28 Copyright 2008 Thomas W. Doeppner. All rights reserved.

Position-Independent Code Processor-dependent; x86 32-bit version: each dynamic executable and share object has: - procedure-linkage table shared, read-only executable code essentially stubs for calling subroutines - global-offset table private, read-write data relocated dynamically for each process - dynamic structure shared, read-only data contains relocation info and symbol table CS 167 VI 29 Copyright 2008 Thomas W. Doeppner. All rights reserved. To provide position-independent code on a 32-bit x86, ELF requires three data structures for each dynamic executable (i.e., the program binary loaded by exec) and shared object: the procedure-linkage table, the global-offset table, and the dynamic structure. To simplify discussion, we refer to dynamic executables and shared objects as modules. The procedure linkage table contains the code that s actually called when control is to be transferred to an externally defined routine. It is shared by all processes using the associated executable or object, and makes use of data in the global-object table to link the caller to the called program. Each process has its own private copy of each global-object table. It contains the relocated addresses of all externally defined symbols. Finally, the dynamic structure contains much information about each module. What is used for linking is relocation information and the symbol table, as we explain in the next few slides. How things work is similar for other architectures, but definitely not the same. Detailed (but confusing) information can be found at http://dlc.sun.com/pdf/817-1984/817-1984.pdf.

Global-Offset Table: Data References Global Offset Table errno errno address myglob myglob address CS 167 VI 30 Copyright 2008 Thomas W. Doeppner. All rights reserved. To establish position-independent references to global variables, the compiler produces, for each module, a global-offset table. Modules refer to global variables indirectly by looking up their addresses in the table. The way this works is that when code in the module refers to something in the table, a register is set to point to it. The item needed is at some fixed offset from the beginning of the table. When the module is loaded into memory, ld.so is responsible for putting into it the actual addresses of all the needed global variables.

Before Calling Name1.PLT0: pushl 4(%ebx) jmp 8(%ebx) nop; nop nop; nop.plt1: jmp name1(%ebx).plt1next: pushl $name1reloffset jmp.plt0.plt2: jmp name2(%ebx).plt2next pushl $name2reloffset jmp.plt0 Procedure-Linkage Table ebx Relocation info: _GLOBAL_OFFSET_TABLE:.long _DYNAMIC.long identification.long ld.so name1:.long.plt1next name2:.long.plt2next GOT_offset(name1), symx(name1) GOT_offset(name2), symx(name2) _DYNAMIC CS 167 VI 32 Copyright 2008 Thomas W. Doeppner. All rights reserved. Dealing with references to external procedures is considerably more complicated than dealing with references to external data. This slide shows the procedure linkage table, global offset table, and relocation information for a module that contains references to external procedures name1 and name2. Let s follow a call to procedure name1. The general idea is before the first call to name1, the actual address of the name1 procedure is not recorded in the global-offset table, Instead, the first call to name1 actually invokes ld.so, which is passed parameters indicating what is really wanted. It then finds name1 and updates the globaloffset table so that things are more direct on subsequent calls. To make this happen, references from the module to name1 are statically linked to entry.plt1 in the procedurelinkage table. This entry contains an unconditional jump to the address contained in the name1 offset of the global-offset table (pointed to by register ebx). Initially this address is of the instruction following the jump instruction, which contains code that pushes onto the stack the offset of the name1 entry in the relocation table. The next instruction is an unconditional jump to the beginning of the procedure-linkage table, entry.plt0. Here there s code that pushes onto the stack the second 32-bit word of the global-offset table, which contains a value identifying this module. The following instruction is an unconditional jump to the address in the third word of the global-offset table, which is conveniently the address of ld.so. Thus control finally passes to ld.so, which looks back on the stack and determines which module has called it and what that module really wants to call. It figures this out based on the module-identification word and the relocation table entry, which contains the offset of the name1 entry in the global-offset table (which is what must be updated) and the index of name1 in the symbol table (so it knows the name of what it must locate).

After Calling Name1.PLT0: pushl 4(%ebx) jmp 8(%ebx) nop; nop nop; nop.plt1: jmp name1(%ebx).plt1next: pushl $name1reloffset jmp.plt0.plt2: jmp name2(%ebx).plt2next pushl $name2reloffset jmp.plt0 Procedure-Linkage Table ebx Relocation info: _GLOBAL_OFFSET_TABLE:.long _DYNAMIC.long identification.long ld.so name1:.long name1 name2:.long.plt2next GOT_offset(name1), symx(name1) GOT_offset(name2), symx(name2) _DYNAMIC CS 167 VI 33 Copyright 2008 Thomas W. Doeppner. All rights reserved. Finally, ld.so writes the actual address of the name1 procedure into the name1 entry of the global-offset table, and, after unwinding the stack a bit, passes control to name1. On subsequent calls by the module to name1, since the global-offset table now contains name1 s address, control goes to it more directly, without an invocation of ld.so.