CONTINUOUS INTEGRATION; TIPS & TRICKS

Transcription

1 CONTINUOUS INTEGRATION; TIPS & TRICKS

2 BIO

3 I DO TECH THINGS

4 I DO THINGS

5 I DO THINGS

6 BLUE OCEAN

7 BEEP BEEP

8 REFACTOR PEOPLE S HOUSES

9 MY TIPS & TRICKS FOR CI - CI Infrastructure - CI Architecture - Pipeline Architecture - CI Security - CI Debugging - Docker Tips

10

11

12

13 CI SERVER ARCHIECTURE - Dockerized workers? docker in docker? Can t be good for stability or speed - Availability: Set your executors to 4 per machine (odd numbers are for weirdos) - Speed: If you want pipeline speed, set your executor to 1 per build

14 PICK YOUR CI TOOL - Pick a CI Product - Jenkins: gives you everything you want - gotta know how to tame the beast - GoCD / DroneCD - for hipsters - it ll die out and you ll be back to Jenkins again anyway - CircleCI / Bitbucket / Gitlab - they re alright

15 OWN YOUR OWN SLAVE INFRA - Always use your own infra - don t use shared infra - it s slow - it s insecure - you don t control its speed or execution times - you can t SSH into the boxes and do stuff

16 CLOUD DOCKER BUILD - AVOID - docker hub - quay.io - google s one - slow execution time - queued - slow servers

17 CONTROL YOUR DOCKER BUILD - Give yourself a 32 core machine - it s cheap - AWS spot instance is like $2 / hour - Run all your docker builds in parallel - easy peasy

18 CONTROL YOUR DOCKER BUILD

19 CI INFRASTRUCTURE - Use AWS EC2 Plugin (Best) - Use terraform plugin (AWS Slaves) - Write your own bash to register your slave

20 CREATE YOUR OWN AMI - put things in it like: - java, docker, docker-compose, - pre-download tools/libraries you need at runtime

21 CHOOSE YOUR INSTANCE SIZE JENKINS MASTER (dockerized) BIG SLAVE (x1) BIG SLAVE (x1) SMALL SLAVE (x4) BIG SLAVE (x1) BIG SLAVE (x1) SMALL SLAVE (x4) BIG SLAVE (x1) BIG SLAVE (x1) SMALL SLAVE (x4)

22 WARMING YOUR SLAVES - docker pull commonly used images - docker pull base images (nginx, upstream?) - put some creds on the box, if you really need to

23 PIPELINE ARCHITECTURE

24

25 FAST BUILD INITIALISATION - Skip fetching tags (git) (defer until the end) - Shallow Clone (fast) - Jenkins Optimized Mode

26 SETUP A BUILD TAG - ASAP

27 ABSTRACT YOUR PIPELINE STEPS

28 ABSTRACT YOUR PIPELINE - Use bash - You can re-run the same commands on local or on CI box when debugging

29 TURN OFF PARTS OF YOUR PIPELINE

30 CHECKING FOR ON/OFF

31 SETUP COMMIT HOOKS

32

33

34 INIT

35 NOTIFY SLACK

36 INIT

37 INIT

38 LINTING

39 BENEFITS OF LINTING - you get early feedback on your code before even thinking about a docker build or npm install - pre-built environments where linting tools exist (docker images) - put your code into the container, run the linter, publish the results

40 THINGS YOU CAN LINT - terraform validate - spellcheck (check bash scripts) - cloudformation: github.com/awslabs/cfn-python-lint - docker-compose -f docker-compose.yml config - SCSS: github.com/brigade/scss-lint - TypeScript - TSLint - github.com/palantir/tslint - github.com/jakubonderka/php-parallel-lint - Dockerfile: github.com/projectatomic/dockerfile_lint

41 NODE - ASSET COMPILATION

42 USING NODE_MODULES?

43 USING NODE_MODULES?

44 PARALLEL NODE ING - don t run npm install in a Dockerfile unless you re shipping npm into prod - extract that out to bash scripts that you mount into the container and run for you

45 COMPILING ASSETS IN PARALLEL

46 PARALLEL NODE ING

47 DOCKER BUILDING

48 DOCKER BUILDING - docker build --pull -t ${BACKEND_IMAGE_FULL}. - always push these to the registry, if all docker builds steps pass, even if test suites fail - you can then pull down the images onto localhost and debug them

49 UNIT TESTING

50 UNIT TESTING - don t just run only unit tests - run prep steps for next stage

51 INTEGRATION / E2E TESTING

52 PARALLELISATION - parallelise the running of test suites - parallelise the tests within each test suite - show selenium browser scaling - show protractor settings for parallel - show VNC watcher for test runs

53 PARALLELISATION - parallelise the running of test suites - parallelise the tests within each test suite - show selenium browser scaling - show protractor settings for parallel - show VNC watcher for test runs

54 POST-PIPELINE RUNS - push docker images to registry - push git tag - push other artifacts - with some logic in place

55 CI DEBUGGING

56 POST-PIPELINE RUNS

57 POST-PIPELINE - DOCKER PUSH

58 POST-PIPELINE - DOCKER PUSH

59 CI DEBUGGING - CLEANUP - Don t wipe the workspace at the end of the pipeline - leave the HDD intact - Beginning of pipeline

60 CI SECURITY

61

62 USING SECURE CREDENTIALS

63 CHEERS CONSULTING / TRAINING PAUL@TEAM-NEEV.COM 63

64