An Implementation of Python for Racket. Pedro Palma Ramos António Menezes Leitão

Transcription

1 An Implementation of Python for Racket Pedro Palma Ramos António Menezes Leitão

2 Contents Motivation Goals Related Work Solution Performance Benchmarks Future Work 2

3 Racket + DrRacket 3

4 Racket + DrRacket 4

5 Racket + DrRacket 5

6 Our goal... 6

7 Our goal... 7

8 Why Python? Python is replacing Scheme in introductory programming courses 8

9 Rosetta IDE 9

10 Rosetta IDE Front ends: Back ends: AUTOLISP Rosetta (Racket) 10

11 Borrows influences from Lisp High level, dynamically typed, GC d Multiple paradigms Huge standard library + third-party libraries 11

12 Goals Correctness + Completeness Performance DrRacket Integration Interoperability with Racket 12

13 Related implementations Language(s) written Platform(s) targetted Speedup (vs. CPython) Std. library support CPython C CPython s VM 1x Full 13

14 Related implementations Language(s) written Platform(s) targetted Speedup (vs. CPython) Std. library support CPython C CPython s VM 1x Full Jython Java JVM ~1x Most IronPython C# CLI ~1.8x Most CLPython Common Lisp Common Lisp ~0.5x Most 14

15 Related implementations Language(s) written Platform(s) targetted Speedup (vs. CPython) Std. library support CPython C CPython s VM 1x Full Jython Java JVM ~1x Most IronPython C# CLI ~1.8x Most CLPython Common Lisp Common Lisp ~0.5x Most PLT Spy Scheme, C Scheme ~0.001x Full 15

16 Our solution... 16

17 Pipeline.py.rkt Python Racket (source-to-source compiler) Racket Bytecode (Racket compiler + JIT) 17

18 Architecture 18

19 Racket Modules reader module (for compilation) read: input-port (listof s-expression) read-syntax: input-port (listof syntax-object)? python module (for runtime behaviour) Provides functions/macros used in compiled code 19

20 Syntax-objects S-expression Source location information File, line number, column number, span Lexical-binding information 20

21 Syntax-objects (py-print (py-get-index arr 6)) line: 3, cols: 0-12 py-print (py-get-index arr 6) line: 3, cols: 6-12 py-get-index arr line: 3, cols: line: 3, cols:

22 Architecture 22

23 How to implement Python s behaviour? 23

24 Runtime implementation Two alternatives: Mapping to Python/C API (via Racket Foreign Function Interface) Racket reimplementation 24

25 Architecture 25

26 FFI Approach libpython module Racket CPython VM Racket FFI (Foreign Function Interface) foreign calls on C pointers Python/C API 26

27 FFI Runtime - Example x + y (define (py-add x y) (PyObject_CallObject (PyObject_GetAttrString x " add ") (make-py-tuple y))) (define (make-py-tuple. elems) (let ([py-tuple (PyTuple_New (length elems))]) (for ([i (range (length elems))] [elem elems]) (PyTuple_SetItem py-tuple i elem)) py-tuple)) 27

28 FFI Runtime - Disadvantages Bad Performance Expensive type conversions + FFI calls Finalizers for GC Clumsy Interoperability with Racket Wrappers/Unwrappers 28

29 What about implementing it over Racket data types? We must first understand Python s data model 29

30 Python s Data Model Every value is an object Every object has a reference to its type-object Type-objects hold hash-table for method dispatching Maps method names to function objects Operator behaviour is mapped to methods 30

31 Optimizations Basic types mapped to Racket types int, long, float, complex, string, dict Avoids wrapping/unwrapping Early method dispatching for operators Avoids expensive method dispatching for common uses 31

32 Racket Runtime - Example x + y (define (py-add x y) (py-method-call x " add " y)) (define (py-add x y) (cond [(and (number? x) (number? y)) (+ x y)] [(and (string? x) (string? y)) (string-append x y)] [else (py-method-call x " add " y)])) 32

33 How are modules imported? 33

34 Python import system import <module> <module> is imported as a module object from <module> import <id> <id> is imported as a new binding from <module> import * module->exports + dynamic-require require All bindings from <module> are imported Special syntax for Racket imports 34

35 Import - Example #lang python import "racket" as rkt def add_cons(c): return rkt.car(c) + rkt.cdr(c) c1 = rkt.cons(2, 3) c2 = rkt.cons("abc", "def") > add_cons(c1) 5 > add_cons(c2) "abcdef" 35

36 Import - Example #lang python from "racket" import cons, car, cdr def add_cons(c): return car(c) + cdr(c) c1 = cons(2, 3) c2 = cons("abc", "def") > add_cons(c1) 5 > add_cons(c2) "abcdef" 36

37 Import - Example (Macros) #lang python from "racket/trace" import trace def factorial(n): if n == 0: return 1 else: return n * factorial(n-1) trace(factorial) > factorial(5) >(factorial 5) > (factorial 4) > >(factorial 3) > > (factorial 2) > > >(factorial 1) > > > (factorial 0) < < < 1 < < <1 < < 2 < <6 < 24 <

38 Other Features Class definitions class statement new type object Exception handling raise, try...except statements raise, with-handlers forms Flow control statements return, break, continue, yield escape continuations 38

39 Benchmarks Ackermann computing the Ackermann function Mandelbrot computing if a complex sequence diverges after a limited number of iterations 39

40 Ackermann (define (ackermann m n) (cond [(= m 0) (+ n 1)] [(and (> m 0) (= n 0)) (ackermann (- m 1) 1)] [else (ackermann (- m 1) (ackermann m (- n 1)))])) (ackermann 3 9) def ackermann(m,n): if m == 0: return n+1 elif m > 0 and n == 0: return ackermann(m-1,1) else: return ackermann(m-1, ackermann(m,n-1)) print ackermann(3,9) 40

41 Mandelbrot (define (mandelbrot limit c) (let loop ([i 0] [z 0+0i]) (cond [(> i limit) i] [(> (magnitude z) 2) i] [else (loop (add1 i) (+ (* z z) c))]))) (mandelbrot i) def mandelbrot(limit, c): z = 0+0j for i in range(limit+1): if abs(z) > 2: return i z = z*z + c return i+1 print mandelbrot( ,.2+.3j) 41

42 Miliseconds Ackermann - Results (a) Racket on Racket (b) (c.1) Python on CPython Python on Racket (FFI) (c.2) (d.1) Python on Racket (FFI + finalizers) Python on Racket (native) (d.2) Python on Racket (native + early dispatch) (a) (b) (c.1) (c.2) (d.1) (d.2) 42

43 Miliseconds Mandelbrot - Results (a) (b) (c.1) (c.2) (d.1) (d.2) Racket on Racket Python on CPython Python on Racket (FFI) Python on Racket (FFI + finalizers) Python on Racket (native) Python on Racket (native + early dispatch) (a) (b) (c.1) (c.2) (d.1) (d.2) 43

44 Future Work Fully implement compilation process Implement behaviour for built-in types Integrate FFI calls with current data model Formal testing for correctness 44

45 Thank you for listening! Questions? Comments? 45