Database Engineering. Percona Live, Amsterdam, September, 2015

Transcription

1 Database Engineering Percona Live, Amsterdam, 2015 September, 2015

2 engineering, not administration 2

3 yesterday s DBA gatekeeper master builder superhero siloed specialized 3

4 engineering quantitative interdisciplinary results focused repeatable and code-driven 4

5 systems engineering designing complex systems for complete life-cycle translation from business to system, with focus on communications, resiliency, flexibility and interoperability ensuring the system generates necessary data for measuring its performance against KPIs 5

6 operations engineering allocate resources to balance competing objectives of cost, service level and productivity (results focused) designing and building processes, tooling and ecosystem to maintain peak flexibility and quality to serve the business (interdisciplinary, repeatable and code driven) gathering and utilizing data to effectively provide continuous improvement to systems (quantitative) 6

7 database engineering participates in systems and operations engineering processes by providing expertise to facilitate all work understands and teaches dataset nuances to ensure all KPIs are met anchors teams with expertise for troubleshooting, recovery and other tasks requiring depth, not breadth 7

8 database engineering shared services for: cross-functional development teams ops teams architecture 8

9 reliability engineering focuses on the glue common to all services, platformcentric deployment, efficiency, scale, performance, observability often done by systems and operations engineering, rather than being their own function 9

10 paradigm shifts 10

11 11

12 polyglot persistence relational is not the end of the line data exists as part of a pipeline function dictates form we cannot predict all uses 12

13

14 virtualization and cloud forces horizontal scaling forces designing for resilience elasticity drives new data storage management by API enhances continuous deployment 14

15 infrastructure as code forces standardization enables immutability forces us to learn to code we start building platforms changes become deployments 15

16 continuous delivery brings us to the source code control paradigm we must be teachers, not gatekeepers testing and compliance become top priorities 2014 Pythian 16

17 devops cultures lean manufacturing defines our workflows theory of constraints tighter feedback loops require organizational shifts experimentation and controlled failure shift architecture and process design integration drives empathy 17

18 database engineers: O.G. devops shared goals, tools and processes DEV OPS DBE 18

19 today s database engineer 19

20 db engineer s manifesto it s all about the mission protect the data eliminate waste data-driven decision making databases are not special eliminate the barriers between software and ops 20

21 specialty = depth analytical thinking problem solving edge case knowledge exhibited in at least one system MySQL/Solaris Cassandra/RHEL PostgreSQL/Windows 21

22 breadth, crossing disciplines understands core concepts can communicate with experts apply knowledge to intersection with core disciplines database engineers need: TCP/IP networking concepts scripting virtualization, automation and orchestration 22

23 the craft 23

24 maslow s hierarchy of needs physiological safety love and belonging esteem self-actualization with loving credit and glory to Charity F. Majors 24

25 physiological physiological does it run and service your application? is it stable? can data be backed up and restored? can basic levels of service be monitored? 25

26 safety physiological safety do I have n+1 available copies? is failover built, documented and tested? is my data distributed to multiple physical locations? can I detect, and recover from corruption? is access managed via roles, separation of duties and appropriate authorization and encryption? is appropriate data encrypted in transport and at rest? 26

27 love and belonging physiological safety love and belonging are database changes part of integration and deployment processes? is continuous testing done with proper data sets? is infrastructure managed via configuration management, automation and orchestration? is all database interaction done via scripted, repeatable processes? is consistency of configuration monitored and managed? 27

28 esteem physiological safety love and belonging esteem are all relevant metrics being captured at appropriate granularity? can problems be detected preemptively? are capacity correlations understand and monitored? are developers trained and knowledgeable on the datastore? can the system automatically remediate frequent and simple faults? 28

29 self-actualization physiological safety love and belonging esteem are you using the right datastore for the right job? are your developers in tight feedback loops with the datastore? is experimentation and data driven decision making supported by the datastore? is the mission fully supported? self-actualization 29

30 design 30

31 what do we design for? mission objectives aka. the business self-actualization function, not form self-actualization manageability by operations love and belonging always remember the hierarchy of database needs 31

32 a day in the life... selecting datastores to add to production platform catalogs dbms and feature education to software, systems and operations engineers integrating with company-wide services (reporting, monitoring, config mgmt etc ) 32

33 a day in the life... validating acceptable configurations testing and benchmarking new versions, features and configurations documenting and sharing standards 33

34 deploy 2014 Pythian 34

35 deploying infrastructure configuration management orchestration test automation self-service to the team 35

36 deploying software agile participation with engineers using version control for schemas and metadata more flexible data model approaches teach your engineering teams how to assess risk, performance and impact support test planning, building and execution 36

37 defining done standards and documentation of acceptable use integration into operational visibility platforms playbooks and game days for operations staff repeatable tests, manual and automated to verify quality all users can find the results of their efforts from the data 37

38 a day in the life... attending scrums, grooming and planning committing deployment scripts to the codebase building, running and automating test cases monitoring commits for key areas that need your attention 38

39 a day in the life (cont) providing new or modified recipes for CM and automation pairing with, and teaching engineers, iterating on the schema 39

40 maintain 40

41 maintaining the datastores maintaining the pipeline backup and recovery change management incident management capacity planning 41

42 the data must flow data brokers pub/sub systems real-time analytics batch processing cache population, use, and flushing search stores 42

43 backup and recovery this has not changed, we live and die by the safety of our data borrow ideas of continuous deployment, for continuous recovery testing build backup and recovery into every possible process 43

44 change management business velocity often demands that developers and DBEs work together to assess risk, rather than gate keep where possible, we are pushing changes through code deployments and automated scripting immutable architectures will force us to create change at the template layer and redeploy exploit the constraint! 44

45 incident management where possible, we are identifying items in need of work before they become problems we sometimes share pagers with ops and dev groups, often as tier 2/3 escalation 45

46 a day in the life... reviewing current workloads and tuning managing escalations on DB issues continuous improvement writing, testing and performing change plans as part of the deployment process 46

47 how? 47

48 software engineering bring DBEs into your agile workflows learn the software versioning system learn the testing frameworks study the language, the framework, the drivers and the ORMs teach, teach teach! 48

49 systems engineering design your data pipelines together collaborate on data standards together learn the entire stack to moderate proficiency understand the entire flow teach, teach, teach! 49

50 operations engineering build your scripting chops learn the build and deploy frameworks learn the configuration management, automation and orchestration layers and patterns learn operational visibility layers and patterns plan, execute and learn from failures 50

51 further deep dives understand the statistics and math around distributions, anomaly detection and correlation write and push code! answer the customer service phones dive into your network layers teach everyone about the data 51