Teaching Elephants to Dance (and Fly!) A Developer s Journey to Digital Transformation

Transcription

1 Teaching Elephants to Dance (and Fly!) A Developer s Journey to Digital Transformation

2

3 6 Blind Men (and an elephant)

4

5 Perspective & Assumptions What we see depends on Who we are (worldview). What we re looking for. Where we re standing. What we know (or don t).

6

7 Burdened and lumbering?

8 Or remover of obstacles?

9 A Developer s Reality What most of us are up against.

10

11

12

13

14 &

15 The average life expectancy of a Fortune 500 company has declined from around 75 years half a century ago to less than 15 years today. Deloitte Shift Index

16 By 2020, more than 75% of the S&P 500 will be companies that we have not heard of yet. Professor Richard Foster, Yale University, in Lean Enterprise

17 By 2020 every business will become a digital predator or digital prey. Nigel Fenwick, Forrester Research, 2015

18 87% of surveyed executives believe digital technologies will disrupt their industries MIT Sloan Management Review, 2016

19 yet only 44% indicated their organizations were taking appropriate measures to avert disruption. MIT Sloan Management Review, 2016

20 Business & IT: 80s to Future ChUI GUI Web Mobile Things 80s 90s 00s 10s

21 Transformers On Black Friday (2013) the Walmart servers didn t go over 1% CPU utilization and the team deployed with 200,000,000 users online.

22 High-performing organizations are decisively outperforming their lower performing peers

23 200X More Deployments

24 24X Faster Recovery from Failure

25 3X Lower Change Failure Rate

26 2555X Shorter Lead Times

27 Your code has no business value until it s deployed.

28 Every business is a technology business.

29

30

31 More Developer Reality Your deployments likely take 6 to 9 months today (super stars might deploy every 3 to 4 weeks but that is not the norm). What if competitive pressures change? What if regulatory requirements change? What if a major CVE descends on your OS, App Server, JVM, framework?

32 Digital Darwinism The Developer s Journey Re-Org to DevOps Self-Service, On-Demand, Elastic Infrastructure Automation Puppet, Chef, Ansible, Kubernetes CI & CD Deployment Pipeline Advanced Deployment Techniques Microservices (and flying elephants!)

33 1 Re-Org to DevOps

34 An Ugly Reality? If this looks familiar, consider changing your ways.

35

36

37

38

39 But wait there s more! The Extended DevOps Reality

40

41

42

43

44 Any organization that designs a system will inevitably produce a design whose structure is a copy of the organization's communication structure. Conway s Law

45 Chinese Whispers

46

47

48 Upgrade Java bugs and lembas Upgrade Java? bugs and lembas Upgrade Java bugs and lambaz WTF? Java upgrade bugs and lambs? Upgrade to Java 1.8, bunch of bug fixes, Lambdas Damn Devs want Java upgrade! something about buggy sheep? Upgrade to JRE 1.8, bug fixes, Lambdas I think of heard of that sheep bug

49

50

51 When restructuring teams yields better results, it illustrates once again that

52 software development is still mostly a communication problem. ThoughtWorks Radar

53 Building cross-functional teams increases the beneficial surface area of communication across traditionally segregated job roles, which in turn removes friction imposed by artificial structures like silos.

54 Only 11% of organizations across industries indicated their existing in-house talent has the competitive skills necessary for success in the digital economy. MIT Sloan Management Review, 2016

55

56

57

58

59 Part of a clever shop that files tickets? How many tickets does it take?

60 2 Self-Service, On-Demand, Elastic Infrastructure

61 How many weeks do you wait for a new VM to be provisioned? Why do expensive resources like developers wait so long for inexpensive resources like VMs?

62 Self-Service Means No Tickets

63 3 Automation, Puppet, Chef, Ansible, Kubernetes

64 Phoenix Servers vs. Snowflakes ( Programmable Infrastructure as Code Containers move developers closer a production environment, even on a laptop. Address the But it works on my machine issue!

65 Datasource Version of the JDBC driver Configuration of the db connection pool JVM settings JMS Queues Default User/Passwords / vs \

66 4 CI & CD Deployment Pipeline

67 Jez Humble Continuous Integration Software (trunk) is always deployable Everyone is checking into trunk daily (at least) not feature branches If the build breaks, it s fixed in 10 minutes (all hands on deck) A new engineer can be on-boarded in 1 day with a production-like environment on the developer workstation Deployment is a low-risk, push-button affair

68 The job of a deployment pipeline is to prove that the release candidate is unreleasable. Jez Humble

69 5 Advanced Deployment Techniques

70 Speed Kills! vs. Go Fast, Go Safe

71 Shellshock

72

73 JDK Vulnerabilities It was discovered that the Hotspot component of OpenJDK did not properly check arguments of the System.arraycopy() function in certain cases. An untrusted Java application or applet could use this flaw to corrupt virtual machine's memory and completely bypass Java sandbox restrictions. (CVE )

74 Actually, Slow Kills! Apache Struts 2 zero-day vulnerability Struts html

75

76 118K JMX Consoles

77

78 Blue/Green Deployment SCM BUILD DEVELOPMENT QA STAGING PRODUCTION ROUTER USERS Starts with a git commit and git push

79 Blue/Green Deployment SCM BUILD DEVELOPMENT QA STAGING PRODUCTION ROUTER USERS

80 Blue/Green Deployment SCM BUILD DEVELOPMENT QA STAGING PRODUCTION ROUTER USERS

81 Blue/Green Deployment SCM BUILD DEVELOPMENT QA STAGING PRODUCTION ROUTER USERS

82 Blue/Green Deployment SCM BUILD DEVELOPMENT QA STAGING PRODUCTION ROUTER USERS

83 Blue/Green Deployment SCM DEVELOPMENT QA STAGING PRODUCTION ROUTER USERS

84 Blue/Green Deployment SCM DEVELOPMENT QA STAGING PRODUCTION ROUTER USERS

85 6 Microservices (and flying elephants!)

86 THE QUICK SHALL INHERIT THE EARTH. Fast is better than slow. Facebook s Little Red Book

87 Why Pipeline? Code offers no value until it deploys.

88 Why Canary Deployments?

89

90 Canary Deployment SCM DEVELOPMENT QA STAGING PRODUCTION ROUTER USERS

91 Canary Deployment SCM DEVELOPMENT QA STAGING PRODUCTION ROUTER USERS

92 Canary Deployment SCM DEVELOPMENT QA STAGING PRODUCTION ROUTER USERS

93 Canary Deployment SCM DEVELOPMENT QA STAGING PRODUCTION ROUTER USERS

94 Canary Deployment SCM DEVELOPMENT QA STAGING PRODUCTION ROUTER USERS

95 Canary Deployment SCM DEVELOPMENT QA STAGING PRODUCTION ROUTER USERS

96 Canary Deployment SCM DEVELOPMENT QA STAGING PRODUCTION ROUTER USERS

97 Canary Deployment SCM DEVELOPMENT QA STAGING PRODUCTION ROUTER USERS

98 Canary Deployment SCM DEVELOPMENT QA STAGING PRODUCTION ROUTER USERS

99 Canary Deployment SCM DEVELOPMENT QA STAGING PRODUCTION ROUTER USERS

100 THE BEST CODE

101 Our experience at Microsoft is no different only about 1/3 of ideas improve the metrics they were designed to improve. Ronny Kohavi, Microsoft (Amazon)

102 If you must code, first do no harm! Developers Hippocratic Oath

103 So what s your hypothesis? In search of the Holy Grail (A/B Testing)

104 A/B Testing SCM DEVELOPMENT QA STAGING PRODUCTION ROUTER USERS Hypothesis & Experiment (two variants): Check-out button placement increase sales conversions for ios users?

105 ACME Laptop 128GB SSD, 8GB RAM $ ACME Laptop 128GB SSD, 8GB RAM $ A Touchscreen 128GB SSD 8GB RAM Core i3 Windows 10 Add to Cart B Add to Cart Touchscreen 128GB SSD 8GB RAM Core i3 Windows 10 In-Store Pickup (15 available) Raleigh, Central Ave, Store #1123 In-Store Pickup (15 available) Raleigh, Central Ave, Store #1123 Recommendations Recommendations

106 ACME Laptop 128GB SSD, 8GB RAM $ ACME Laptop 128GB SSD, 8GB RAM $ Touchscreen 128GB SSD 8GB RAM Core i3 Windows 10 Touchscreen 128GB SSD 8GB RAM Core i3 Windows 10 A Add to Cart B Add to Cart In-Store Pickup (15 available) Raleigh, Central Ave, Store #1123 In-Store Pickup (15 available) Raleigh, Central Ave, Store #1123 Recommendations Recommendations

107 ACME Laptop 128GB SSD, 8GB RAM $ ACME Laptop 128GB SSD, 8GB RAM $ Touchscreen 128GB SSD 8GB RAM Core i3 Windows 10 Touchscreen 128GB SSD 8GB RAM Core i3 Windows 10 A Add to Cart B Add to Cart In-Store Pickup (15 available) Raleigh, Central Ave, Store #1123 Recommendations In-Store Pickup (15 available) Raleigh, Central Ave, Store #1123

108 At 1-week deployment intervals? Assess whether this meets your needs! Majestic Monolith

109 Not yet fast enough? Shrink your elephant!

110 Instead of getting a bigger office, why don t we get a smaller elephant?

111 Fallacies of Distributed Computing The network is reliable. Latency is zero. Bandwidth is infinite. The network is secure. Topology doesn't change. There is one administrator. Transport cost is zero. The network is homogeneous. The first rule of distributed computing is DO NOT!

112 Strangle the Monolith

113

114

115

116

117

118

119

120

121

122 There s an Elephant in the Room Your Monolithic Database (Your DBA will kill you.) Download a copy of Edson s book at developers.redhat.com

123 java -jar myapp.jar DropWizard Spring Boot WildFly Swarm Vert.x JAX-RS API First to market DropWizard Metrics Embeddable servers: Jetty projects.spring.io/spring -boot Spring API Starter POMs: start.spring.io Embeddable servers: Tomcat, Jetty, Undertow wildfly-swarm.io Java EE 7 APIs Starter POMs: wildflyswarm.io/generator Embeddable servers: WildFly (Undertow) vertx.io Reactive Async/non-blocking vertx run myhttp.java HTTP, HTTP/2, TCP, UDP, Websockets, etc out-of-the-box

124 Failure Happens

125 CHAPTER 2: The Exception that Grounded an Airline The delays were shown on Good Morning America (complete with video of pathetically stranded single moms and their babies)

126 VERIZON OUTAGE CRASHES JETBLUE AIRLINES ELECTRONICS SYSTEMS, DELAYING FLIGHTS AND BRINGING DOWN ONLINE SITES FOR BOOKING AND CHECK-IN January 14, A Verizon data center outage Thursday morning brought down JetBlue s electronic systems, causing flight delays and shutting down the airline s website, along with its online booking and check-in systems.

127 SOUTHWEST AIRLINES SYSTEM OUTAGE BRINGS FLIGHTLINE TO FULL GROUND STOP, DELAYS AND MADDENS PASSENGERS July 21, Southwest Airlines has cancelled 1,150 flights since the airline's full ground stop for on Wednesday. The trouble started with a "system outage," and the ground stop lasted for just over an hour.

128 DELTA AIRLINES SAYS THE TOTAL BILL FOR ITS DEVASTATING COMPUTER OUTAGE WILL COME TO $150 MILLION, AUGUST 8, 2016 COMPUTER OUTAGE GROUNDS DELTA FLIGHTS IN U.S., JANUARY 17, 2017

129 Digital Darwinism The Developer s Journey Re-Org to DevOps Self-Service, On-Demand, Elastic Infrastructure Automation Puppet, Chef, Ansible, Kubernetes CI & CD Deployment Pipeline Advanced Deployment Techniques Microservices (and flying elephants!)

130 If we don t create the thing that kills Facebook, someone else will. Embracing change isn t enough. It has to be so hardwired into who we are that even talking about it seems redundant. Facebook s Little Red Book

131 Some Practical Advice 1. Learn Linux, AWS/GCP/Azure, Docker & Kubernetes 2. Start an List 3. Every other week Demo Day 4. Opposite every other week Book Club 5. Quarterly Mini-Conference internal and external presenters 6. Document your Value Stream Map

132 BUT remember

133 Organization First

134 Focus on Team

135

136 Continuous Improvement

137 Life-long Learning

138 Always Accountable

139 When we are no longer able to change a situation, we are challenged to change ourselves. Viktor Frankl

140

141 First they ignore you, then they laugh at you, then they fight you, then you win. Gandhi

142

143 THANK YOU!