CS510 Concurrent Systems Class 1a. Linux Kernel Locking Techniques

Transcription

1 CS510 Cncurrent Systems Class 1a Linux Kernel Lcking Techniques

2 Intr t kernel lcking techniques (Linux) Why d we need lcking in the kernel? Which prblems are we trying t slve? What implementatin chices d we have? Is there a ne-size-fits-all slutin? CS510 - Cncurrent Systems 2

3 Hw des cncurrency arise? True cncurrency Multiple prcessrs execute instructins simultaneusly Pseud cncurrency Instructins f multiple executin sequences are interleaved CS510 - Cncurrent Systems 3

4 Surces f pseud cncurrency Sftware-based preemptin Vluntary preemptin (sleep/yield) Invluntary preemptin (preemptible kernel) Scheduler switches threads regardless f whether they are running in user r kernel mde The instructins f multiple threads running in kernel mde are interleaved Hardware preemptin Interrupt/trap/fault/exceptin handlers can start executing at any time Reentrancy A functin calls itself CS510 - Cncurrent Systems 4

5 Critical sectins Sectins f cde that are subject t cncurrent executin in which at least ne executin path mdifies shared data Lcking can be used t prvide mutually exclusive access t critical sectins Varius lcking primitives exist in Linux Linux is a symmetric multiprcessing (SMP) preemptible kernel CS510 - Cncurrent Systems 5

6 Atmic peratrs Simplest synchrnizatin primitives Primitive peratins that are indivisible Tw types methds that perate n integers methds that perate n bits Implementatin Assembly language sequences that use the atmic readmdify-write instructins f the underlying CPU architecture CS510 - Cncurrent Systems 6

7 Atmic integer peratrs atmic_t v; atmic_set(&v, 5); /* v = 5 (atmically) */ atmic_add(3, &v); /* v = v + 3 (atmically) */ atmic_dec(&v); /* v = v - 1 (atmically) */ printf("this will print 7: %d\n", atmic_read(&v)); Beware: Can nly pass atmic_t t an atmic peratr atmic_add(3,&v); and { atmic_add(1,&v); atmic_add1(2,&v); } are nt same! CS510 - Cncurrent Systems 7

8 Spin lcks Mutual exclusin fr larger (than ne peratr) critical sectins requires additinal supprt Spin lcks Single hlder lcks When lck is unavailable, acquiring prcess keeps trying CS510 - Cncurrent Systems 8

9 Basic use f spin lcks spinlck_t mr_lck = SPIN_LOCK_UNLOCKED; spin_lck(&mr_lck); /* critical sectin... */ spin_unlck(&mr_lck); spin_lck() Acquires the spinlck using atmic instructins required fr SMP spin_unlck() Releases the spinlck CS510 - Cncurrent Systems 9

10 What if the spin lck hlder is interrupted? spinlck_t mr_lck = SPIN_LOCK_UNLOCKED; unsigned lng flags; spin_lck_irqsave(&mr_lck, flags); /* critical sectin... */ spin_unlck_irqrestre(&mr_lck, flags); spin_lck_irqsave() disables interrupts lcally acquires the spinlck using instructins required fr SMP spin_unlck_irqrestre() Restres interrupts t the state they were in when the lck was acquired CS510 - Cncurrent Systems 10

11 What if we re n a uniprcessr? Previus cde cmpiles t: unsigned lng flags; save_flags(flags); /* save previus CPU state */ cli(); /* disable interrupts */ /* critical sectin... */ restre_flags(flags); /* restre previus CPU state */ Hmm, why nt just use: cli(); /* disable interrupts */ sti(); /* enable interrupts */ CS510 - Cncurrent Systems 11

12 Dealing with interrupt cntext Need t knw if data being prtected is accessed in interrupt cntext r just nrmal kernel cntext Interrupt handlers shuld nt blck! Interrupted spin lck hlders shuld nt be delayed Use apprpriate primitives t manage hardware r sftware preemptin Need t knw if interrupts were already enabled r disabled Use apprpriate primitives t save and restre CPU state CS510 - Cncurrent Systems 12

13 Bttm halves, sftirqs and tasklets Sftirqs, tasklets and BHs are deferrable functins think f them as delayed interrupt handling that is scheduled Sftirqs the basic building blck Statically allcated can nt be interrupted by anther sftirq n the same CPU nn-preemptive scheduling f sftirqs sftirqs f the same type can run cncurrently n different CPUs synchrnize with each ther using spin-lcks Tasklets built n sftirqs dynamically allcated can nt be interrupted by anther tasklet n the same CPU tasklets f the same type can nt run cncurrently n different CPUs BHs built n sftirqs (static, nt cncurrent) CS510 - Cncurrent Systems 13

14 Spin lcks and deferred functins spin_lck_bh() implements the standard spinlck disables sftirqs needed fr cde utside a sftirq that manipulates data als used inside a sftirq spin_unlck_bh() Releases the spinlck Enables sftirqs CS510 - Cncurrent Systems 14

15 Spin lck rules D nt try t re-acquire a spinlck yu already hld! it leads t self deadlck! Spinlcks shuld nt be held fr a lng time Excessive spinning wastes CPU cycles! What is a lng time? D nt sleep while hlding a spinlck! Smene spinning waiting fr yu will waste a lt f CPU never call any functin that tuches user memry, allcates memry, calls a semaphre functin r any f the schedule functins while hlding a spinlck! All these can blck. CS510 - Cncurrent Systems 15

16 Semaphres Semaphres are lcks that are safe t hld fr lnger perids f time cntentin fr semaphres causes blcking nt spinning shuld nt be used fr shrt duratin critical sectins! Why? safe t sleep with! Can be used t synchrnize with user cntexts that might blck r be preempted Semaphres can allw cncurrency fr mre than ne prcess at a time, if necessary CS510 - Cncurrent Systems 16

17 Semaphre implementatin Implemented as a wait queue and a usage cunt wait queue: list f prcesses blcking n the semaphre usage cunt: number f cncurrently allwed hlders if negative, the semaphre is unavailable, and abslute value f usage cunt is the number f prcesses currently n the wait queue if initialized t 1, the semaphre is a mutex CS510 - Cncurrent Systems 17

18 Semaphre peratins Dwn() Up() attempts t acquire the semaphre by decrementing the usage cunt and testing if its negative blcks if usage cunt is negative releases the semaphre by incrementing the usage cunt and waking up ne r mre tasks blcked n it CS510 - Cncurrent Systems 18

19 Can yu be interrupted when blcked? dwn_interruptible() Returns EINTR if signal received while blcked Returns 0 n success dwn_trylck() attempts t acquire the semaphre n failure it returns nnzer instead f blcking CS510 - Cncurrent Systems 19

20 Reader/writer Lcks N need t synchrnize cncurrent readers unless a writer is present reader/writer lcks allw multiple cncurrent readers but nly a single writer (with n cncurrent readers) Bth spin lcks and semaphres have reader/writer variants CS510 - Cncurrent Systems 20

21 Reader/writer spin lcks (rwlck) rwlck_t mr_rwlck = RW_LOCK_UNLOCKED; read_lck(&mr_rwlck); /* critical sectin (read nly)... */ read_unlck(&mr_rwlck); write_lck(&mr_rwlck); /* critical sectin (read and write)... */ write_unlck(&mr_rwlck); CS510 - Cncurrent Systems 21

22 Reader/writer semaphres (rw_semaphre) struct rw_semaphre mr_rwsem; init_rwsem(&mr_rwsem); dwn_read(&mr_rwsem); /* critical regin (read nly)... */ up_read(&mr_rwsem); dwn_write(&mr_rwsem); /* critical regin (read and write)... */ up_write(&mr_rwsem); CS510 - Cncurrent Systems 22

23 Reader/writer lck warnings reader lcks cannt be autmatically upgraded t the writer variant attempting t acquire exclusive access while hlding reader access will deadlck! if yu knw yu will need t write eventually btain the writer variant f the lck frm the beginning r, release the reader lck and re-acquire the lck as a writer CS510 - Cncurrent Systems 23

24 Big reader lcks (br_lck) Specialized frm f reader/writer lck very fast t acquire fr reading very slw t acquire fr writing gd fr read-mstly scenaris Implemented using per-cpu lcks readers acquire their wn CPU s lck writers must acquire all CPUs lcks CS510 - Cncurrent Systems 24

25 Big kernel lck (BKL) A glbal kernel lck - kernel_flag used t be the nly SMP lck mstly replaced with fine-grain lcalized lcks Implemented as a recursive spin lck Reacquiring it when held will nt deadlck Usage but dn t ;) lck_kernel(); /* critical regin... */ unlck_kernel(); CS510 - Cncurrent Systems 25

26 Preemptible kernel issues Have t be careful f legacy cde that assumes per- CPU data is implicitly prtected frm preemptin May need t use new preempt_disable() and preempt_enable() calls Calls are nestable fr each n preempt_disable() calls, preemptin will nt be reenabled until the nth preempt_enable() call CS510 - Cncurrent Systems 26

27 Cnclusins Ww! Why des ne system need s many different ways f ding synchrnizatin? Actually, there are mre ways t d synchrnizatin in Linux, this is just lcking CS510 - Cncurrent Systems 27

28 Cnclusins One size des nt fit all: need t be aware f different cntexts in which cde executes (user, kernel, interrupt etc) and the implicatins this has fr whether hardware r sftware preemptin r blcking can ccur the cst f synchrnizatin is imprtant, particularly its impact n scalability Yu nly use mre than ne CPU because yu hpe t execute faster! Each synchrnizatin technique makes a different perfrmance vs. cmplexity trade-ff Real-time characteristics and I/O are becming mre and mre imprtant in general purpse systems CS510 - Cncurrent Systems 28