Master Class: Large Scale Data Analysis and Visualization Daniel Armyr Application Engineer

Transcription

1 Master Class: Large Scale Data Analysis and Visualization Daniel Armyr Application Engineer 2014 The MathWorks, Inc. 1

2 Parsing complex text data Using disk to work with huge data Writing clean code Visualization in 3d 2

3 Parsing complex text data Using disk to work with huge data Writing clean code Visualization in 3d 3

4 Regular Expressions Wikipedia: A sequence of characters that forms a search pattern for pattern matching with strings, or "find and replace"-like operations A very powerful, but not very beginner-friendly, way of working with text data. Want more? 4

5 Parsing complex text data Using disk to work with huge data Writing clean code Visualization in 3d 5

6 Memory mapped files Use the virtual memory system of the operating system to treat a file like a normal variable in MATLAB 6

7 Memory mapped files 'Format', {... int16, [2 2], 'model';... 'uint32, [1 1], 'serialno';... 'single, [1 3], 'expenses'}); 7

8 Memory mapped files <Header> A 1 - [1000 1] B 1 - [1000 1] C 1 - [1000 1] A 2 - [1000 1] B 2 - [1000 1] C 2 - [1000 1]... 8

9 Memory mapped files Use the virtual memory system of the operating system to treat a file like a normal variable in MATLAB Want more? 9

10 Parsing complex text data Using disk to work with huge data Writing clean code Visualization 3D data 10

11 Writing clean code Runs faster Easier to debug Easier to change Easier to read Can be converted to C-code* Want more? * Using MATLAB Coder 11

12 Parsing complex text data Using disk to work with huge data Writing clean code Visualization 3D data 12

13 Visualization 3D data 13

14 Plotting scalar data Plotting vector data A unifying example 14

15 [x, y, z] = meshgrid([-1 0 1]); v = x.* exp(-x.^2 - y.^2 - z.^2); scatter3(x(:), y(:), z(:), [], v(:)); 15

16 [x, y, z] = meshgrid(-3:0.25:3); v = x.* exp(-x.^2 - y.^2 - z.^2); scatter3(x(:), y(:), z(:), [], v(:)); 16

17 slice(x, y, z, v, [], [], 0) 17

18 slice(x, y, z, v, [-1.5, 1.5], 0, 0) 18

19 [xs,ys] = meshgrid(-3:0.1:3); zs = sin(xs) - cos(ys); slice(x, y, z, v, xs, ys, zs); 19

20 20

21 21

22 22

23 isosurface(x, y, z, v, 1e-5); 23

24 fv = isosurface(x,y,z,v,1e-5); h = patch(fv); 24

25 view(3); % shortcut for view(-37.5, 30); 25

26 set(h, 'FaceColor', [ ]) 26

27 set(h, 'EdgeColor', 'none') 27

28 camlight; 28

29 camproj perspective 29

30 daspect([1 1 1]); 30

31 alpha(0.75); 31

32 fv2 = isosurface(x, y, z, v, 1e-3); patch(fv2, 'FaceColor', [ ], 'EdgeColor', 'none'); 32

33 Plotting scalar data Plotting vector data A unifying example 33

34 [x, y, z] = meshgrid([-1 0 1]); u = x + cos(4*x) + 3; v = sin(4*x) - sin(2*y); w = -z; quiver3(x, y, z, u, v, w); 34

35 [x, y, z] = meshgrid(-1.5:0.1:1.5) u = x + cos(4*x) + 3; v = sin(4*x) - sin(2*y); w = -z; quiver3(x, y, z, u, v, w); 35

36 36

37 37

38 38

39 39

40 40

41 [sx,sy,sz] = meshgrid(-1:1); streamline(x,y,z,u,v,w,sx,sy,sz); 41

42 42

43 43

44 44

45 45

46 46

47 47

48 48

49 streamslice(x,y,z,u,v,w, [ ], [], []); 49

50 streamslice(x,y,z,u,v,w, 1.5, 1.5, -1.5); 50

51 51

52 52

53 53

54 54

55 55

56 56

57 [sx,sy,sz] = meshgrid(-1.5, -1:1, -1:1); streamribbon(x, y, z, u, v, w, sx, sy, sz); 57

58 Plotting low-dimensional data Plotting scalar data Plotting vector data A unifying example 58

59 Visualizing 3D Data Just because you can, doesn t mean that you should. But please do try this at home. Want more? 59

60 Parsing complex text data Using disk to work with huge data Writing clean code Visualization in 3d 60

61 61