There and Back Again The BOSH Windows Story

Transcription

1 There and Back Again The BOSH Windows Story

2 Introductions Natalie Arellano Software Engineer, Pivotal natalieparellano 2 Matthew Horan Manager, Software Engineering, Pivotal mhoran

3 Windows on Cloud Foundry Motivation 3 cf push enables a seamless deployment experience for Linux applications We wanted to bring this experience to.net developers

4 What we ll cover 4 The early days of Windows Diego cell installation What we learned when porting the BOSH agent to Windows The choices we made when designing the interface for Windows BOSH releases The benefits we found in moving to BOSH

5 The MSI Workflow aka the MVP Linux Diego Cells Windows cells are manually deployed and configured. Diego cf push -s windows2012r2 5 CAPI

6 The MSI workflow aka the MVP 6 Validate that.net developers are willing to replatform their apps to run on Cloud Foundry Operators manually spin up VMs, install Cloud Foundry components (Diego rep, Consul, Metron, and garden server) on the cell Focused on the developer experience, not the operator experience

7 Operator Experience On AWS, run our Cloud Formation template setup.ps1 to configure VM settings -- install Windows features, change DNS settings, set up Windows firewall, enable disk quotas 7 Requires knowledge of other CF components & their configuration When injecting VMs into the IaaS that aren t controlled by bosh, you re gonna have a bad time If you re not on AWS, good luck If you ran the Cloud Formation template, this happened for you. If not, you inevitably forgot to run it (hence, hakim.exe)

8 Operator Experience What is running on the cell? Garden-Windows Diego-Windows (consul, rep, metron) Maintained by other teams. We have no idea what s going on with this software Hakim 8 Written and maintained by the Greenhouse team, built upon IronFrame which was extracted from IronFoundry We built this because support requests would come in and we didn t even know if people followed basic instructions (e.g., enable interactive logon for IIS_IUSRS)

9 Operator Experience, contd. MSI takes a bunch of configuration parameters Windows operators already have processes for maintaining Windows servers 9 generate.exe automates the creation of an install script that includes those parameters The generator had to authenticate with the BOSH Director to obtain properties for Diego release Typically required servers to be domain joined

10 Limitations of this approach Manual Complicated & Fragile 10 If BOSH properties are changed in the BOSH manifest, the MSI must be re-installed (and install script re-generated) Rolling updates are performed by hand Group policies can interfere with CF components Hostnames must be unique (consul used this as its identifier) See above: manual Did you run the setup.ps1? Did you provide all the proper options to the install script generator?

11

12 Outcome 12 Operators were underwhelmed. But developers loved it! This is the validation of the MVP we were looking for

13 The Fellowship of BOSH 13 Deploy Windows cells like the rest of Cloud Foundry No snowflakes Easy to rebuild, automatable

14 Porting the BOSH Agent to Windows Agent is written in Go We can switch code paths in a single code base based on the operating What needed to be changed for Windows? We started with the platform interface Implemented one function at a time How did we test this? 14 system. Shared functionality doesn t have to be rewritten for Windows. Reverse engineered the director protocol and wrote a lightweight test suite for the agent.

15 What s different about BOSH Agent for Windows? Process Monitoring Architecture Agent Agent winsw monit your-exe Linux 15 Windows Service API your.exe Windows

16 What s different about BOSH Agent for Windows? monitfile 16

17 what else? Processes are stopped with SIGKILL instead of SIGINT/Control-C Syslog forwarding specified in component releases (for now ) Packaging script is a PowerShell script 17 Needed a way to skip Windows packaging scripts on Linux. source exiter.ps1

18 Windows BOSH Jobs rep_windows, consul_agent_windows, metron_agent_windows all integrated into their respective component releases garden-windows 18 We wanted upstream components to be maintained and packaged by their owners. This required cross-team collab. and enablement. Most components were already cross-compiled for Windows in the MSI world. People were afraid of Windows! Now that we have bosh, they can easily deploy Windows Concourse workers! No more scary AppVeyor... This is now maintained by a team dedicated to Windows containerization.

19 Operator Experience cf-deployment ops-file! No domain join. Deploying applications, not managing servers. 19 Doing things the CF way Antivirus add-on!

20 Operator Experience Stemcells 20 We build stemcells for public IaaSes, but operators have to build their own on-prem stemcells due to Windows licensing Stemcell building experience is a bit challenging at the moment, but we re working on a better solution We know this is one of the most difficult things about BOSH Windows

21

22 Other BOSH Windows Limitations BOSH agent on Windows doesn't support all the features of Linux, including: 22 Stop scripts Persistent disks More than one ephemeral disk BOSH SSH Currently no Windows compiled releases

23 What s next for BOSH Windows? Implement missing features 23 Stop scripts, SSH, persistence Windows Server 2016 stemcell Better on-prem stemcell experience? Windows Event Log in syslog

24 Get In Touch 24 #bosh on slack.cloudfoundry.org A. William Martin & Colin Jackson are our Product Managers We re listening on

25