CPython cannot into threads

Transcription

1 GIL

2 CPython cannot into threads

3 1992

4 PyPy Jython CPython IronPython Brython

5 PyPy Jython CPython IronPython Brython

6 PyPy Jython CPython IronPython Brython

7 CPython cannot into threads

8 CPython cannot into threads CPython can run only on single core

9 CPython cannot into threads CPython can run only on single core

10 CPython cannot into threads CPython can run only on single core "CPython process can execute Python bytecode in one thread at the time"

11

12 Staleness

13 Misconceptions

14 CC-SA-BY,

15 Evolution

16 GIL

17 Threads (system threads) (not user-space threads)

18 Thread 1

19 Thread 1 Thread 2 Process

20 1. running 2. runnable 3. blocked

21 System calls

22 Thread scheduling

23 Signals

24 $ kill [PID]

25 $ kill [PID] SIGTERM

26 Concurrency vs. parallelism

27

28

29

30

31

32

33 1. Tracing garbage collection 2. Reference counting 3....

34 1. Tracing garbage collection

35 1. Tracing garbage collection

36 1. Tracing garbage collection

37 1. Tracing garbage collection

38 1. Tracing garbage collection 2. Reference counting int

39 1. Tracing garbage collection 2. Reference counting 3

40 1. Tracing garbage collection 2. Reference counting 0

41

42

43 load ++ save 0 load ++ time save

44 load ++ save 2 load ++ time save

45 load load save time save

46 load load save time save

47 GIL finally

48 Protector

49 Reference counters

50 Internal global state

51 C-mutable data structures dict, list,

52 C-mutable data structures dict, list, str, tuple, int

53 Binary modules and extensions

54 Cost

55 In single-threaded code? It doesn t matter (duh)

56 On single core? It doesn t matter

57 In multi-threaded I/O-bound code? It may not matter

58 With binary extensions? It may not matter

59 >= 3.2

60 1

61 1 2

62 1 2 Can I has GIL? Nope.

63 1 2 switch interval

64 1 2 Can I has GIL?

65 1 2 Can I has GIL? Yes!

66 1 2 Can I has GIL? Nope.

67 1 2 gil_drop_request = 1

68 1 2 gil_drop_request?

69 1 2

70 1 2 switch_cond? gil_last_holder?

71 1 2

72 <=

73 ticks

74 >>> print( foo ) >>> -1 in range( )

75 >>> print( foo ) >>> -1 in range( ) >>> n -= 1

76 N ticks N ticks N = 100 check check check

77

78 100s ms few ms

79 1 2 Can I has GIL? And now? Nope. And now? Nope. Nope.

80 1 2 Can I has GIL? And now? Nope. And now? Nope. Aw yiss Nope.

81 PyPy Jython CPython IronPython Brython

82 PyPy Jython CPython IronPython Brython

83 An alternative PyPy which works without a GIL, while at the same time continuing to give the Python programmer the complete illusion of having one.

84 PyPy Jython CPython IronPython Brython

85 Ruby Python Go

86 Ruby Python Go

87 Ruby Python Goroutines

88 Ruby Python Goroutines channels

89 def fib(n):

90

91 More!

92 import multiprocessing

93 multiprocessing binary extensions cdef int foo = 0

94 multiprocessing binary extensions cdef int foo = 0 with nogil:...

95 multiprocessing binary extensions Py_BEGIN_ALLOW_THREADS Py_END_ALLOW_THREADS

96 multiprocessing binary extensions asynchronous I/O

97 Revolution?

98 C

99 Requirements

100 1. Single-threaded and multi-threaded I/O-bound performance 2. C extensions compatibility 3. Complexity

101 1. Single-threaded and multi-threaded I/O-bound performance 2. C extensions compatibility 3. Complexity

102 1. Single-threaded and multi-threaded I/O-bound performance 2. C extensions compatibility 3. Complexity

103 1996

104 Free-threading patch

105 Free-threading patch subinterpreters

106 Free-threading patch subinterpreters GILectomy

107 Free-threading patch subinterpreters bu re ff GILectomy fe re er nc ed e co un t in g

108 Free-threading patch subinterpreters bu re ff GILectomy fe re er e t nc ed a v i e r co p d un a g e n i t in r k h g c t lo

109 Free-threading patch i mm o rt subinterpretersob al jec bu re ts ff GILectomy fe re er e t nc ed a v i e r co p d un a g e n i t in r k h g c t lo

110 --without-gil Py_LOCK(o) Py_UNLOCK(o)

111 Free-threading patch subinterpreters GILectomy

112 What to do?

113 What to do? nothing

114 What to do? nothing migrate to 3.5 (3.6 even!)

115 What to do? nothing migrate to 3.5 (3.6 even!) assert the baseline, then act

116 Thank you! :) kolodziejj.info/talks/gil/

117 What to do? nothing migrate to 3.5 (3.6 even!) assert the baseline, then act contribute! kolodziejj.info/talks/gil/