What I ll be talking about. About me & bol.com The CI/CD bol.com Current setup. The future in the cloud

Transcription

1 bol.com

2 What I ll be talking about About me & bol.com The CI/CD bol.com Current setup Mayfly The future in the cloud

3 About me Maarten on Twitter In IT since 2007 (5 bol.com) Java developer -> CI/CD engineer Bitten by the container bug in 2014 Hobby: local politics

4 About bol.com Largest online retailer in the Netherlands and Belgium (5.8 million customers, 10+ million products) 55 (and growing) multi-disciplinary teams of 5-8 people Strong Scrum culture (introduced in 2009) 200+ services and apps (SOA, mostly Java + DB backend) Mix of fixed sprint rhythm and CD

5 The Developer Freedom index

6 Once upon a build-time...

7 The situation ca week release cycle Big-bang release Scrum -> 200 stories per cycle to production Jenkins -> DeployIT -> Schuberg Phillis Shop went offline! Software was thrown over the wall to ops Every team had admin rights on their jenkins jobs

8 Freedom index: not free Developers couldn t do their own releases at the time of their choosing Releases had to be coordinated with SBP Even some property changes Stack on which apps ran was tightly locked down Developers could go crazy on TST, but could do almost nothing on PRO Endless requests for SSH access to servers which were inevitably denied 2 levels of gatekeeping: ops and SBP On the plus side, they could configure their Jenkins jobs...

9 Current pipeline

10 our CD story Man on the Moon to give teams autonomy

11 How things get to production Build Store Orchestrate Run Deploy

12 Key aspects TAXP system: custom abstraction over Jenkins jobs No more job admin rights for teams Teams can deploy to PRO at will (have to send notification) TST, ACC environments (ACC is production-like, used for performance tests) No change management process SRT gatekeeper of deploy functionality and new services

13 Mayfly

14 r> Genesis of Mayfly te <m as as te <m r> Test <m as r> te as <m Acc Pro te r>

15 Mayfly idea <branch> <branch> <branch> <branch> <branch> <branch> <branch> <branch> <branch> <branch> <branch> <branch> Pro

16 Mayfly provides per user story: Feature branch in SCM (currently git via Stash) Continuous integration jobs (Jenkins) Isolated, production-like runtime environment (Docker cluster) Automated Definition of Done check Logs & metrics (Logstash, Graphite, Prometheus) Optional user story-specific database (Oracle, PostgreSQL, Mongo)

17 30% of all commits done via Mayfly

18 Freedom index: partially free Developers control their releases Developers don t control CI or CD Mayfly offers lots of freedom, until TST

19 Building and deploying in the cloud

20 The challenge Build a container-centric, cloud-native CI/CD pipeline that: Is easy to use and get started with Makes it easy to deploy small changes Is fully customizable Can scale to thousands of deploys per day

21 What about the current stack? Will the CI/CD stack that we use at the moment suffice in the cloud?

22 The current stack Maintenance nightmare Very expensive docker registry *Not* a deployment tool Jenkins Artifactory Rundeck Puppet Do builds Store artifacts Orchestration of RPM builds and rolling out of artifacts Actually install the new artifact on an existing machine Want immutable infrastructure

23 The current stack would work, but we can do better by using cloudnative tools

24 The new stack More developer control & less DPI maintenance Gitlab CI Build docker images Google s concern Google Container Registry Store docker images Actual deployment tool Facilitates immutable infrastructure Spinnaker Kubernetes Deploy dockerized apps Run docker containers

25 Convention over configuration Convincing over compulsion Opt-out is an option! CI/CD is a product that needs to appeal Iterate on a vision, don t crowdsource the design

26 Freedom index: free Developers have full control over CI Developers have full control over CD Developers have full control over the stack Well, at least from the kernel up Constraints that do exist are, as much as possible, handled transparently And if opt-out is an option, but comes with many responsibilities

27 Thanks! Maarten Dirkse