Analysis and Visualization with yt. Matthew Turk UIUC School of Information Sciences UIUC Astronomy

Transcription

1 Analysis and Visualization with yt Matthew Turk UIUC School of Information Sciences UIUC Astronomy

2 yt-project.org hub.yt

3 Layers of Representation Bit

4 Layers of Representation Data Bit

5 Layers of Representation Model Data Bit

6 yt-project.org Volumetric analysis and visualization

7 Jill Naiman, AJ Christensen, Kalina Borkiewicz ytini.com

8 the yt project Python-based (C, Cython, etc) Community developed NumFOCUS FSA Code of Conduct Governance structure 100+ contributors Volumetric and non-spatial data Used in nearly 300 papers Grids, particles, octrees, and unstructured meshes Arbitrary geometric representations Minimize time to inquiry

9 Ingestion

10 Ingestion Representation

11 Ingestion Representation Analysis

12 Ingestion Representation Analysis Visualization

13 Community Ingestion Representation Analysis Visualization

14 Project Members Kenza Arraki Corentin Cadiou Brian Crosby Bili Dong Hilary Egan Nathan Goldbaum Cameron Hummels Suoqing Ji Allyson Julian Ben Keller Kacper Kowalik Sam Leitner Alex Lindsay Chris Malone Andrew Myers Jill Naiman Jeff Oishi Brian O'Shea Douglas Rudd Anthony Scopatz Sam Skillman Stephen Skory Britton Smith Casey Stark Matthew Turk John Wise Michael Zingale John ZuHone About 100 contributors...

15 D

16 yt D

17 Some Simulation

18 Layers of Representation Model Data Bit

19

20

21

22

23

24

25

26

27 ART ARTIO Athena Carpet Castro Chombo Eagle Enzo ExodusII FITS FLASH Gadget Gadget-FOF Gasoline GDF Maestro MOAB Nyx OWLS OWLS-Subfind PKDGrav PLUTO RAMSES Rockstar SDF Stream HTTP Stream Grids Stream Octree Stream Particles Stream Unstructured

28 ART ARTIO Athena Carpet Castro Chombo Eagle Enzo ExodusII FITS FLASH Gadget Gadget-FOF Gasoline GDF Maestro MOAB Nyx OWLS OWLS-Subfind PKDGrav PLUTO RAMSES Rockstar SDF Stream HTTP Stream Grids Stream Octree Stream Particles Stream Unstructured

29 ART ARTIO Athena Carpet Castro Chombo Eagle Enzo ExodusII FITS FLASH Gadget Gadget-FOF Gasoline GDF Maestro MOAB Nyx OWLS OWLS-Subfind PKDGrav PLUTO RAMSES Rockstar SDF Stream HTTP Stream Grids Stream Octree Stream Particles Stream Unstructured

30 ART ARTIO Athena Carpet Castro Chombo Eagle Enzo ExodusII FITS FLASH Gadget Gadget-FOF Gasoline GDF Maestro MOAB Nyx OWLS OWLS-Subfind PKDGrav PLUTO RAMSES Rockstar SDF Stream HTTP Stream Grids Stream Octree Stream Particles Stream Unstructured

31 ART ARTIO Athena Carpet Castro Chombo Eagle Enzo ExodusII FITS FLASH Gadget Gadget-FOF Gasoline GDF Maestro MOAB Nyx OWLS OWLS-Subfind PKDGrav PLUTO RAMSES Rockstar SDF Stream HTTP Stream Grids Stream Octree Stream Particles Stream Unstructured

32 Some Simulation

33 Some Selector Simulation

34 Selected

35 Selected D

36 Layers of Representation Model Data Bit

37 Low-Level Operations

38 Low-Level Operations

39 Low-Level Operations Chunk Chunk Chunk Chunk Chunk Chunk Chunk Chunk Chunk

40 Selected unk Chunk Chunk Chunk Chunk Chunk Chunk Chunk C

41 Low-Level Operations Chunk

42 Low-Level Operations Chunk

43 Low-Level Operations Chunk Chunk Chunk Chunk Chunk Chunk Chunk Chunk Chunk Chunk

44 Low-Level Operations Cells Chunk Chunk Chunk Particles Chunk Chunk Chunk Chunk Chunk Chunk Elements

45 Layers of Representation Model Data Bit

46 Data Representation Coordinate Handling Cartesian Cylindrical Spherical (geographic, tomographic) Symbolic Units Derived fields Dependency calculation Arithmetic and spatial

47 Fields Representation of state.

48 Fields Representation of state. Name Units Context Prescription

49 Fields Representation of state. Primitive

50 Fields Representation of state. Primitive Derived

51 Fields Arithmetic Operations E = mv2 2

52 Fields Arithmetic units="erg") def energy(field, data): E = 0.5 * (data["mass"] * data["velocity_magnitude"]**2) return E

53 Fields Spatial Operations div V = δv x + δv y + δv z δx δy δz

54 Fields Spatial units="1/s", validators=[validatespatial(1)]) def divergence(field, data): dx = data["index", "dx"] dy = data["index", "dy"] dz = data["index", "dz"] f = data["velocity_x"][sl_r,1:-1,1:-1]/ds f -= data["velocity_x"][sl_l,1:-1,1:-1]/ds f += data["velocity_y"][1:-1,sl_r,1:-1]/ds f -= data["velocity_y"][1:-1,sl_l,1:-1]/ds f += data["velocity_z"][1:-1,1:-1,sl_r]/ds f -= data["velocity_z"][1:-1,1:-1,sl_l ]/ds return f

55 Mid-Level Operations x =

56 Mid-Level Operations x = m s -1 m/s

57 Mid-Level Operations m x s -1 = m/s Symbolic manipulation and pragmatic ontologies

58 Semantically-meaningful Data x = m s -1 m/s Symbolic manipulation and pragmatic ontologies

59 Semantically-meaningful Data m x s -1 = m/s Symbolic manipulation and pragmatic ontologies

60

61 43 primitive fields 363 derived fields 35 distinct units 2.5 primitive fields per derived

62 High-Level Operations Chunk

63 High-Level Operations Chunk

64 High-Level Operations Chunk

65 High-Level Operations Chunk Chunk Chunk Chunk Chunk Chunk Chunk Chunk Chunk

66 Imaging and volume tools Volumetric segmentation Parallel Irregular resolution data Marching cubes Ray-tracing Radiative transfer Volume rendering Rasterization / pixelization Coordinate systems Discretization

67

68

69 Time Location

70 Time Location

71 Time Location

72 Time Location

73 Temperature Density Turk et al 2009

74

75 Disk Simulation

76 Simulation libyt Disk Analysis Hsi-Yu Schive

77 f(x,y,z) First Order Alex Lindsay, Andrew Myers

78 Second Order Alex Lindsay, Andrew Myers

79 Approximate Second Order Alex Lindsay, Andrew Myers

80 Approximate Second Order Alex Lindsay, Andrew Myers

81 Compressed bitmap indices

82 Compressed bitmap indices

83 Compressed bitmap indices Spatial Tree

84 Compressed bitmap indices Lang & Turk, 2016

85 Community Not the biggest, not the smallest, but active 375 on the users mailing list 115 on the dev mailing list Code Peer review Mentorship Continuous testing system

86 Community How can we increase diversity? How can we foster careers? How can we lower barriers?

87 What are our core values?

88 Technically Easy Socially Challenging Socially Easy Technically Challenging

89 Functional Detrimental Humanitarian Problematic

90 product versus project

91 product versus project

92 product versus project

93 product versus project the thing

94 product versus project the people

95 Thank you

96 Three Options: $ docker pull xarthisius/ythub-jupyter $ docker run --rm -ti -p 8888:8888 xarthisius/ythub-jupyter $ conda install -c conda-forge yt yt-project.org and click on Get yt