Python VSIP API: A first draft

Transcription

1 Python VSIP API: A first draft Stefan Seefeld HPEC WG meeting, December 9, 2014

2 Goals Use cases: Promote VSIP standard to a wider audience (SciPy users) Add more hardware acceleration to SciPy Allow VSIP application prototyping using Python Requirements: Find the right balance between Pythonic (expected) syntax and semantics and adherence to VSIP conventions Support easy integration with existing Python SciPy platform (notably: NumPy's ndarrayapi) Allow easy integration of Python and C / C++ VSIP runtimes to pass (block and view) objects across language boundaries for visualization and verification

3 Pythonic API that is intuitive to use NumPy examples creating arrays: import numpy as np > a = np.array([1, 2, 3, 4]) # Create array from sequence > a = np.arange(10) # 0.. n-1 > a = np.arange(1, 9, 2) # start, end (exclusive), step > a = np.zeros((2, 2), dtype=float) # zero 2x2 matrix of floats > a = np.random.randn(4) # Gaussian NumPy indexing and slicing examples: > b = a[2:9:3] # [start:end:step] > b = a[0,3:5] # access subvector of 0th row > a[2:9:2] = 0 # write-access works the same way Note that the above corresponds directly to VSIPL++ operator()(domain)! auto b = a(domain<1>(2,9,3)); // Use C++11 syntax for compactness auto b = a(0,domain<1>(3,1,5)); a(domain<1>(2,9,2)) = 0;

4 Be as close to VSIP Data model as possible As with NumPy, VSIP uses logical element addresses instead of physical addresses As with NumPy, the physical data layout may be specified at construction time In contrast to NumPy, VSIP views have constant (immutable) types and shapes: > from vsip import vector > from vsip import random > v = vector(float, 8) # Create vector from scratch > a = np.arange(8, dtype=float) > v = vector(array=a) # Create vector from numpy array > rng = random.rand(float) # Create a RNG for floats > v = rng.randn(4) # Create a vector filled with four random numbers View indexing and slicing works just like NumPy array indexing and slicing: > v[0] = 1 # assign to first element

5 > v[::2] = 1 # assign to every other element > repr(v[-1::-1]) # print in reverse order vector(dtype=float, shape=(8), [7, 6, 5, 4, 3, 2, 1, 0])

6 Views vs. Blocks Three view types are provided: vector, matrix, tensor Views contain / reference (polymorphic) blocks Blocks are reference counted, and may form (acyclic) graphs to support sub blocks, expressions, etc. Example: OpenVSIP Views are pure Python objects Blocks are C++ objects exposed via Boost.Python Depending on the required type, the appropriate extension module is loaded Increase modularity, allow to only implement minimal subset (e.g., for single precision float)

7 Sharing block objects across Python and C++ runtimes allows for copy free data transfer in hybrid (Python / C++) code. Call optimized C++ kernels from Python Embedd Python code into C++ applications (e.g., to visualize data, or compare test results against Gold Standard)

8 Operations on Views Operations may be dispatched depending on the view / block type: def dot(a, b): # a and b are views b1 = a.block b2 = b.block if b1.dtype == b2.dtype: mod = _import_module(b1.dtype) # determine and load the appropriate operation kernel return mod.dot(b1, b2) # call backend on block references else:... # fall back to un-optimized implementation Structure the functionality into modules and packages similar to VSIPL & VSIPL++

9 Misc: Python Embedding Support visualization Construct objects in C++, then pass them to Python runtime to display or validate them Vector<float> filter = blackman(1024); bpl::object main = bpl::import(" main "); bpl::object global(main.attr(" dict ")); global["vsip"] = bpl::import("vsip"); global["filter"] = filter; // pass VSIPL++ vector into the Python runtime bpl::object result = bpl::exec("from matplotlib.pyplot import *\n" "plot(filter)\n" "show()\n", global);

10 Misc: OpenCL & CUDA prototyping Support prototyping of OpenCL & CUDA kernels in Python Use block's own storage management to lock data to specific device storage. from vsip.opencl import cl, dda from vsip import vector cxt = cl.default_context() src = """ kernel void copy( global char const *in, global char *out) { int num = get_global_id(0); out[num] = in[num]; }""" program = cl.program.create_with_source(cxt, src) program.build(cxt.devices()) kernel = program.create_kernel("copy") v1 = vector(array=np.arange(16, dtype=np.float32)) v2 = vector(np.float32, 16) kernel(queue, 16, v1, v2)