March 10, 2015
Section 1
Why Goal: apply a binary patch to kernel on-line. is done without shutdown quick response to a small but critical issue the goal is not to avoid downtime Limitations: simple changes changes in data structure layout
Section 2
Similar projects kexec CRIU
Live patching project (Suse) kpatch (Red Hat) ksplice (Oracle)
Section 3
Ftrace Insert a jmp at the beginning of all the functions gcc -pg -mfentry (add 5 bytes, that call mcount) mcount = 13% overhead Store each trampoline location in a section " mcount_loc" scripts/recordmcount use the linker to merge them all into vmlinux At boot: foreach entries in " mcount_loc", replace jmp with 5 NOP Set the call to mcount only if needed insert INT3 and its handler at NOP[0] insert addr at NOP[1-4] insert jmp at NOP[0] send NMI IPI to flush the instruction decoders ftrace_calller invokes the hooks
Ftrace Insert a jmp at the beginning of all the functions gcc -pg -mfentry (add 5 bytes, that call mcount) mcount = 13% overhead Store each trampoline location in a section " mcount_loc" scripts/recordmcount use the linker to merge them all into vmlinux At boot: foreach entries in " mcount_loc", replace jmp with 5 NOP Set the call to mcount only if needed insert INT3 and its handler at NOP[0] insert addr at NOP[1-4] insert jmp at NOP[0] send NMI IPI to flush the instruction decoders ftrace_calller invokes the hooks
Ftrace Insert a jmp at the beginning of all the functions gcc -pg -mfentry (add 5 bytes, that call mcount) mcount = 13% overhead Store each trampoline location in a section " mcount_loc" scripts/recordmcount use the linker to merge them all into vmlinux At boot: foreach entries in " mcount_loc", replace jmp with 5 NOP Set the call to mcount only if needed insert INT3 and its handler at NOP[0] insert addr at NOP[1-4] insert jmp at NOP[0] send NMI IPI to flush the instruction decoders ftrace_calller invokes the hooks
Ftrace Insert a jmp at the beginning of all the functions gcc -pg -mfentry (add 5 bytes, that call mcount) mcount = 13% overhead Store each trampoline location in a section " mcount_loc" scripts/recordmcount use the linker to merge them all into vmlinux At boot: foreach entries in " mcount_loc", replace jmp with 5 NOP Set the call to mcount only if needed insert INT3 and its handler at NOP[0] insert addr at NOP[1-4] insert jmp at NOP[0] send NMI IPI to flush the instruction decoders ftrace_calller invokes the hooks
Section 4
Subsection 1
new functions must be applied at once old function must not be executed after switching to the new one no threads runs on old functions no threads sleeps on them
Subsection 2
ksplice and kpatch_v1 add ftrace entry stop_machine() stop running processes disable interrupts Safeness check walk through the threads and check the stack enable the hook
ksplice and kpatch_v1 + safe + simple - stop_machine() stops all processes a while ( 40ms) - fail to upgrade non-quiescient kernel function (schedule)
Subsection 3
Functions can be called while patching. atomic reference counter inc at function entry dec_if_pos at function exit Active safeness check at context switch check stack entries safely sleeping tasks can be checked safely
+ get rid of stop_machine() - kretprobe has no error notification - not incremental (big patch has many functions)
Why relying on the stack may be hazardous How Linux retrieve the stack entries (return addresses): get an address on the stack (local variable) while (valid_stack_ptr(addr)) ++addr check if pointer is in.text section use frame pointer if available ret address lies just above a frame pointer the frame pointer chain broke in some rarely used assembler code
Subsection 4
Reality check trampoline: a per-thread flag is set on each kernel entry/exit set a trampoline which monitor kernel entry/exit and redirect to the old/new function remove the trampoline and set the new function when all the processes have changed their universe at least once
- all processes must wake up or execute a syscall - sometimes requires a signal to be sent - kernel thread never leave the kernel + does not rely on the stack entries
Section 5
Dazed and confused Questions?