Renderscript. Lecture May Android Native Development Kit. NDK Renderscript, Lecture 10 1/41

Transcription

1 Renderscript Lecture 10 Android Native Development Kit 6 May 2014 NDK Renderscript, Lecture 10 1/41

2 RenderScript RenderScript Compute Scripts RenderScript Runtime Layer Reflected Layer Memory Allocation Bibliography Keywords NDK Renderscript, Lecture 10 2/41

3 Outline RenderScript RenderScript Compute Scripts RenderScript Runtime Layer Reflected Layer Memory Allocation Bibliography Keywords NDK Renderscript, Lecture 10 3/41

4 Android NDK NDK - perform fast rendering and data-processing Lack of portability Native lib that runs on ARM won t run on x86 Lack of performance Hard to identify (CPU/GPU/DSP) cores and run on them Deal with synchronization Lack of usability JNI calls may introduce bugs RenderScript developed to overcome these problems NDK Renderscript, Lecture 10 4/41

5 RenderScript Native app speed with SDK app portability No JNI needed Language based on C99 - modern dialect of C Pair of compilers and runtime Graphics engine for fast 2D/3D rendering Deprecated from Android 4.1 Developers prefer OpenGL Compute engine for fast data processing Running threads on CPU/GPU/DSP cores Compute engine only on CPU cores NDK Renderscript, Lecture 10 5/41

6 RenderScript Architecture Android apps running in the Dalvik VM Graphics/compute scripts run in the RenderScript runtime Communication between them through instances of reflected layer classes Classes are wrappers around the scripts Generated using the Android build tools Eliminate the need of JNI Memory management App allocated memory Memory bound to the RenderScript runtime - memory accessed from the script Script may have additional fields to store data NDK Renderscript, Lecture 10 6/41

7 RenderScript Architecture NDK Renderscript, Lecture 10 7/41

8 RenderScript Architecture Asynchronous call to RenderScript runtime to start script Through the reflected layer classes Low-Level Virtual Machine (LLVM) front-end compiler When building the apk Compiles code into device-independent bytecode stored in the apk The reflected layer class is created LLVM back-end compiler on the device App is launched Compiles bytecode into device-specific code Caches the code on the device Achieves portability NDK Renderscript, Lecture 10 8/41

9 Outline RenderScript RenderScript Compute Scripts RenderScript Runtime Layer Reflected Layer Memory Allocation Bibliography Keywords NDK Renderscript, Lecture 10 9/41

10 Compute Engine-based App Architecture Java code and.rs file (compute script) API in android.renderscript package Class RenderScript Defines context Static RenderScript.create() returns class instance Class Allocation Input and output allocations Sending data to, receiving data from script Reflected layer class Name: ScriptC_ + script name If script = computation.rs => class = ScriptC computation.rs script Placed in src/ Contains kernels, functions and variables NDK Renderscript, Lecture 10 10/41

11 Script Example #pragma v e r s i o n ( 1 ) #pragma r s j a v a p a c k a g e n a m e ( com. example. h e l l o c o m p u t e ) // m u l t i p l i e r s to c o n v e r t a RGB c o l o r s to b l a c k and w h i t e c o n s t s t a t i c f l o a t 3 gmonomult = { f, f, f } ; uchar4 a t t r i b u t e ( ( k e r n e l ) ) c o n v e r t ( uchar4 i n ){ // unpack a c o l o r to a f l o a t 4 f l o a t 4 f 4 = r s U n p a c k C o l o r ( i n ) ; // t a k e t h e dot p r o d u c t o f t h e c o l o r and t h e m u l t i p l i e r f l o a t 3 mono = dot ( f 4. rgb, gmonomult ) ; // r e p a c k t h e f l o a t to a c o l o r r e t u r n rspackcolorto8888 ( mono ) ; } NDK Renderscript, Lecture 10 11/41

12 Java Code Example p r i v a t e v o i d u s e S c r i p t ( ) { R e n d e r S c r i p t mrs = R e n d e r S c r i p t. c r e a t e ( t h i s ) ; A l l o c a t i o n i n p u t = A l l o c a t i o n. createfrombitmap (mrs, inputbitmap ) ; A l l o c a t i o n o u t p u t = A l l o c a t i o n. createfrombitmap (mrs, outputbitmap ) ; ScriptC mono s c r i p t = new ScriptC mono (mrs ) ; s c r i p t. f o r E a c h c o n v e r t ( i n p u t, o u t p u t ) ; o u t p u t. copyto ( outputbitmap ) ; } NDK Renderscript, Lecture 10 12/41

13 Script Contents #pragma to specify RenderScript version #pragma version(1) Version 1 is the only one available #pragma to specify Java package name #pragma rs java_package_name(com.example.app) Global variables We may set values from Java - used for parameter passing Invokable functions Single-threaded function Called from Java with arbitrary arguments For initial setup or serial computations Optional init() function Special type of invokable function Runs when the script is first instantiated NDK Renderscript, Lecture 10 13/41

14 Script Contents Static global variables and functions Cannot be set/called from Java Compute kernels Parallel functions Executed for every Element within an Allocation attribute ((kernel)) -> RenderScript kernel in -> one Element from the input Allocation Returned value put in one Element from the output Allocation Multiple input/output -> declared global with rs_allocation Default root function A special kernel function used in older versions Returns void attribute ((kernel)) not needed NDK Renderscript, Lecture 10 14/41

15 Using RenderScript from Java Code Create RenderScript context Using create() May take a long time Create at least one Allocation Provides storage for a fixed amount of data Input and output for the kernels Created with createtyped(renderscript, Type) or createfrombitmap(renderscript, Bitmap) Create script User-defined kernels Instantiate ScriptC_filename class ScriptIntrinsic - built-in kernels for common operations NDK Renderscript, Lecture 10 15/41

16 Using RenderScript from Java Code Put data in Allocations Use copy functions from Allocation class Set script globals Using set_globalname from ScriptC_filename class set_elements(int) Execute kernels Using foreach_kernelname() from ScriptC_filename class Takes one or two Allocations as arguments Executed over the input entire Allocation by default Default root function Invoked using foreach_root NDK Renderscript, Lecture 10 16/41

17 Using RenderScript from Java Code Launch invoked functions Using invoke_functionname from ScriptC_filename class Obtain data from Allocations Copy data into Java buffers using copy methods from Allocation class Synchronizes with asynchronous kernel Destroy RenderScript context Using destroy() function Or let it be garbage collected Further use causes an exception NDK Renderscript, Lecture 10 17/41

18 RenderScript Intrinsics Pre-defined scripts ScriptIntrinsicBlend Kernels for blending two buffers ScriptIntrinsicBlur Gaussian blur filter Apply blur of a specified radius to the elements of an allocation ScriptIntrinsicColorMatrix Apply color matrix to allocation Hue filter Each element is multiplied with a 4x4 color matrix ScriptIntrinsicConvolve3x3 Embossing filter Apply 3x3 convolve to allocation NDK Renderscript, Lecture 10 18/41

19 Intrinsics Example p r i v a t e v o i d u s e B l u r S c r i p t ( ) { R e n d e r S c r i p t mrs = R e n d e r S c r i p t. c r e a t e ( t h i s ) ; A l l o c a t i o n i n p u t = A l l o c a t i o n. createfrombitmap (mrs, inputbitmap ) ; A l l o c a t i o n o u t p u t = A l l o c a t i o n. createfrombitmap (mrs, outputbitmap ) ; S c r i p t I n t r i n s i c B l u r s c r i p t = S c r i p t I n t r i n s i c B l u r. c r e a t e (mrs, Element. U8 4 (mrs ) ) ; s c r i p t. s e t R a d i u s ( ( f l o a t ) ) ; s c r i p t. s e t I n p u t ( i n p u t ) ; s c r i p t. f o r E a c h ( o u t p u t ) ; o u t p u t. copyto ( outputbitmap ) ; } NDK Renderscript, Lecture 10 19/41

20 Outline RenderScript RenderScript Compute Scripts RenderScript Runtime Layer Reflected Layer Memory Allocation Bibliography Keywords NDK Renderscript, Lecture 10 20/41

21 RenderScript Runtime Layer The code is executed in a RenderScript runtime layer Runtime API - computation portable and scalable to the number of cores Code compiled into intermediate bytecode using LLVM compiler part of the Android build system Bytecode compiled just-in-time to machine code by another LLVM compiler on the device Machine code is optimized for that platform and cached NDK Renderscript, Lecture 10 21/41

22 RenderScript Runtime Libraries Manage memory allocation requests Large number of math functions Scalar and vector typed overloaded versions of common functions Adding, multiplying, dot product, cross product Atomic arithmetic and comparison functions Conversion functions for primitives, vectors, matrix, date and time Vector types for defining two-, three- and four-vectors Logging functions NDK Renderscript, Lecture 10 22/41

23 Outline RenderScript RenderScript Compute Scripts RenderScript Runtime Layer Reflected Layer Memory Allocation Bibliography Keywords NDK Renderscript, Lecture 10 23/41

24 Reflected Layer Set of classes generated by the Android build tools Allow access to RenderScript runtime from the Android framework Methods and constructors for allocating memory for the RenderScript code Reflected components: Class ScriptC_filename for each script Non-static functions Non-static global variables Get/set methods for each variable For a const the set method is not generated Global pointers Class ScriptField_structname for each structure An array of the struct Allocate memory for one or more instances of the struct NDK Renderscript, Lecture 10 24/41

25 Functions are reflected into ScriptC_filename class Asynchronous -> cannot have return values Reflected functions When function is called, the call is queued and executed when possible To send a value back to Java use rssendtoclient() For function void touch(float x, float y, float pressure, int id) it generates code: p u b l i c v o i d i n v o k e t o u c h ( f l o a t x, f l o a t y, f l o a t p r e s s u r e, i n t i d ) { F i e l d P a c k e r t o u c h f p = new F i e l d P a c k e r ( 1 6 ) ; t o u c h f p. addf32 ( x ) ; t o u c h f p. addf32 ( y ) ; t o u c h f p. addf32 ( p r e s s u r e ) ; t o u c h f p. addi32 ( i d ) ; i n v o k e ( mexportfuncidx touch, t o u c h f p ) ; } NDK Renderscript, Lecture 10 25/41

26 Variables Variables are reflected into ScriptC_filename class Set/get methods are generated For uint32_t index are generated: p r i v a t e l o n g m E x p o r t V a r i n d e x ; p u b l i c v o i d s e t i n d e x ( l o n g v ){ m E x p o r t V a r i n d e x = v ; s e t V a r ( m E x p o r t V a r I d x i n d e x, v ) ; } p u b l i c l o n g g e t i n d e x ( ) { r e t u r n m E x p o r t V a r i n d e x ; } NDK Renderscript, Lecture 10 26/41

27 Structs Structs are reflected into ScriptField_structname classes Class extends android.renderscript.script.fieldbase Class includes: A static nested class Item that includes the fields of struct An Item array Get/set methods for each field in the struct index parameter to specify exact Item in the array Setter has copynow parameter - immediately sync memory to RenderScript runtime Get/set methods for a certain Item in the array Overloaded ScriptField_structname(RenderScript rs, int count) count - number of elements in the array to allocate resize() - expand allocated memory (array dimension) copyall() - sync memory to the RenderScript runtime NDK Renderscript, Lecture 10 27/41

28 Pointers Pointers reflected into ScriptC_filename class Pointer to struct or any RenderScript type Struct cannot contain pointers or nested arrays For int32_t *index is generated: p r i v a t e A l l o c a t i o n m E x p o r t V a r i n d e x ; p u b l i c v o i d b i n d i n d e x ( A l l o c a t i o n v ) { m E x p o r t V a r i n d ex = v ; i f ( v == n u l l ) b i n d A l l o c a t i o n ( n u l l, m E x p o r t V a r I d x i n d e x ) ; e l s e b i n d A l l o c a t i o n ( v, m E x p o r t V a r I d x i n d e x ) ; } p u b l i c A l l o c a t i o n g e t i n d e x ( ) { r e t u r n m E x p o r t V a r i n d e x ; Bind function - bind allocated memory in the VM to RenderScript runtime Cannot allocate memory in the script NDK Renderscript, Lecture 10 28/41

29 Outline RenderScript RenderScript Compute Scripts RenderScript Runtime Layer Reflected Layer Memory Allocation Bibliography Keywords NDK Renderscript, Lecture 10 29/41

30 Binding Apps run in the Android VM RenderScript code runs natively and needs to access the memory allocated in the VM Binding Memory allocated in the VM is attached to the RenderScript runtime Needed for dynamically allocated memory Scripts cannot allocate memory explicitly Statically allocated memory is created at compile time NDK Renderscript, Lecture 10 30/41

31 Memory Allocation Element class One cell of memory allocation Basic element - any RenderScript data type (float, float4, etc) Complex element - list of basic elements, created from structs Type class Encapsulates the Element and a number of dimensions Layout of memory - usually an array of Elements Allocation class Performs actual allocation based on the Type Sync is needed when memory is modified NDK Renderscript, Lecture 10 31/41

32 Non-static global variables and pointers Non-static global variables Allocated statically at compile time No allocation in Java Initialized by the RenderScript layer Access them from Java using get/set methods in the reflected class Global pointers Allocate memory dynamically in Java through the reflected class Bind memory to the RenderScript runtime Access memory from Java or from script NDK Renderscript, Lecture 10 32/41

33 Allocate and Bind Dynamic Memory For pointers to structs call ScriptField_structname class constructor Call reflected bind method - bind memory to RenderScript runtime S c r i p t F i e l d P o i n t p o i n t s = new S c r i p t F i e l d P o i n t (mrs, 1 0 ) ; m S c r i p t. b i n d p o i n t s ( p o i n t s ) ; For primitive pointers - manually create Allocation A l l o c a t i o n e l e m e n t s = A l l o c a t i o n. c r e a t e S i z e d (mrs, Element. I 3 2 (mrs), 1 0 ) ; m S c r i p t. b i n d e l e m e n t s ( e l e m e n t s ) ; NDK Renderscript, Lecture 10 33/41

34 Access global variables Statically allocated memory Get/set methods in Java, access directly in script Changes in script are not propagated in Java Access in script: t y p e d e f s t r u c t P o i n t { i n t x ; i n t y ; } P o i n t t ; P o i n t t p o i n t ; [.. ] p o i n t. x = 1 ; p o i n t. y = 1 ; rsdebug ( P o i n t :, p o i n t. x, p o i n t. y ) ; NDK Renderscript, Lecture 10 34/41

35 Access global variables If you modify in Java, values are propagated to the RenderScript runtime Cannot get modifications from script Access in Java: S c r i p t C e x a m p l e m S c r i p t ; [.. ] Item p = new S c r i p t F i e l d P o i n t. Item ( ) ; p. x = 1 ; p. y = 1 ; m S c r i p t. s e t p o i n t ( p ) ; Log. i ( TAG, P o i n t : + m S c r i p t. g e t p o i n t ( ). x + + m S c r i p t. g e t p o i n t ( ). y ) ; NDK Renderscript, Lecture 10 35/41

36 Access pointers Dynamically allocated memory Allocate memory in Java and bind to the RenderScript runtime Use get/set methods to access from Java Access pointers directly from script Changes are automatically propagated to Java From script: t y p e d e f s t r u c t P o i n t { i n t x ; i n t y ; } P o i n t t ; P o i n t t p o i n t ; p o i n t [ i n d e x ]. x = 1 ; p o i n t [ i n d e x ]. y = 1 ; NDK Renderscript, Lecture 10 36/41

37 Access pointers From Java: S c r i p t F i e l d P o i n t p o i n t s = new S c r i p t F i e l d P o i n t (mrs, 1 0 ) ; Item p = new S c r i p t F i e l d P o i n t. Item ( ) ; p. x = 2 5 ; p. y = 7 0 ; p o i n t s. s e t ( p, 0, t r u e ) ; m S c r i p t. b i n d p o i n t ( p o i n t s ) ; p o i n t s. g e t x ( 0 ) ; p o i n t s. g e t y ( 0 ) ; Call bind just once No need to re-bind every time a change is made NDK Renderscript, Lecture 10 37/41

38 Outline RenderScript RenderScript Compute Scripts RenderScript Runtime Layer Reflected Layer Memory Allocation Bibliography Keywords NDK Renderscript, Lecture 10 38/41

39 Bibliography renderscript/compute.html renderscript/advanced.html Android Recipes A Problem-Solution Approach, Chapter 9 NDK Renderscript, Lecture 10 39/41

40 Outline RenderScript RenderScript Compute Scripts RenderScript Runtime Layer Reflected Layer Memory Allocation Bibliography Keywords NDK Renderscript, Lecture 10 40/41

41 RenderScript C99 LLVM Allocation Compute kernels Invokable functions Reflected layer Pointers Structs Memory binding Element Type NDK Renderscript, Lecture 10 41/41