Who Moved My Module? 1

Transcription

1 Who Moved My Module? 1

2 About Me Yoav Landman - JFrog s CTO and Co-Founder - Creator of the Artifactory Project - 13 years experience in commercial enterprise build and development environments 2

3 Agenda A binary repository - why manage modules? Modules and release management - Release strategies and models - Common release issues 3

4 A Binary Repository A shared place for binaries - 3rd party and local artifacts Much more than a passive storage Critical for CI and ALM 4

5 What s wrong with my... Subversion Apache file server/webdav Shared file system DVD Disk-on-key, tape, drawer... 5

6 Can your shared file system/vcs Download and cache remote artifacts? Promote release artifacts? Control licenses used in your software? Track and preserve artifacts used by CI builds? Enforce module CRUD security easily? Track changes made to artifacts? Automatically clean up integration garbage? Manage artifacts with powerful searches? Expose management REST API? 6

7 The Artifact Repository Ecosystem replication Modules/Artifacts/Libraries content-filtering security mgmt. promotion Repository continuous builds searches build isolation user extensions Remote Repositories VCS build tracking REST access host & provision license control proxy & cache metadata and tagging 7

8 Demo Binary Repository 8

9 Artifact Build Integration Platform 9

10 Artifact Build Integration Platform #1 #1-1 #1-2 #3-2 #3-1 #2 #2-1 #2-2 #3-3 #3 10

11 Integrates with build context 11

12 Demo Build integration 12

13 The release pipeline Multiple steps towards declaring a release Check points for advancing the release flow modules modules modules modules Source: Agile ALM, Michael Hüttermann, Manning Publications Co. 13

14 The release pipeline model A B C GM Linear A B Non-linear C GM 14

15 The release pipeline - YMMV 15

16 Typical release issues Avoiding artifact rebuilding on release Picking a source/release strategy Integration build Isolation Avoiding license/arbitrary artifact creep 16

17 Avoiding artifact rebuilding on release Product Binaries v1.1-snap v1.1-snap v1.1-snap v1.1-snap Qualified v1.1-snap v1.1-snap v1.1-snap v1.1-snap A collection of release-qualified integration packages Lack of release status - No way to express that without changing pom/ivy descriptors Resolution for packaging coupled to descriptors 17

18 Resolution with integration versions Ivy/Gradle - Integration revisions same as any release revision ( ) - By default all dynamic versions are resolved and replaced in delivered modules - Can be used in a release package Maven - Published artifacts can contain dynamic dependencies (SNAPSHOT or ranges) + unresolved properties - Very hard to keep reproducibility 18

19 Source/release strategies Release-to-branch M2 Release plugin Artifactory build-integration release 19

20 Merge-to-branch job#1: build/test trunk release branch tag merge to branch job#2: build/test release tag tag a release 20

21 Jenkins M2 release plugin 1. Check out latest VCS revision 2. Compile & test 3. Change POMs to next release version 4. Compile & test 5. Commit new POMs (!) 6. Create a VCS tag from WC 7. Change POMs to next dev version (++-SNAPSHOT) 8. Commit new POMs 9. Check out previously created VCS tag 10. Compile & test 11. Publish binaries to repository 21

22 Jenkins M2 release plugin Attempts to reuse a user-operated plugin in a CI environment - Uses separate external Maven process - Uses separate external VCS (svn auth) No good rollback - End users may like CI s VCS permissions Maven only 22

23 Artifactory plugin (SVN) 1. Change descriptors to next release version 2. Compile & test 3. Create a VCS tag from WC 4. Change descriptors to next dev version (e.g. version++-snapshot) 5. Commit new descriptors 6. Publish binaries to repository 23

24 Artifactory plugin (git) 1. Create a local release branch 2. Change descriptors to next release version 3. Local commit + tag the release branch 4. Compile & test 5. Checkout master branch 6. Change descriptors to next dev version 7. Commit + push all changes 8. Publish binaries to repository 24

25 Releasing with the Artifactory plugin Fast Easy automatic rollback - E.g. tag removal Closely integrated with CI server - Reuses build tool and VCS definitions Gradle support Bamboo and TeamCity support Support for Ivy in the works 25

26 Demo Release & promotion 26

27 Release build isolation Cascading jobs - tests run longer A downstream job may take wrong artifacts put by an upstream job No isolation - the build integration silo is broken 27

28 Keeping integration private time job-a #12 job-b #4 job-a #13 job-c X A#12 B#4 C#2??? A#13 28

29 Isolating resolution Deploy as many integration revs in parallel Each build resolves only the artifacts in the build chain it is part of Achieved with CI artifact painting and matrix params for resolution: 29

30 License creep Verify no unwanted 3rd party is packaged into the release Discover and apply license information continuously! - Information is there - Discover and act early 30

31 License violation detection 31

32 Demo Time Detecting license violations 32

33 Artifact creep A generic way to control artifacts Audit or gate-keep what is being used download { altresponse {request, responserepopath ->} altremotepath {repopath ->} altremotecontent {repopath ->} beforeremotedownload {request, repopath ->} afterremotedownload {request, repopath ->} beforedownload {request, responserepopath ->} } storage { beforecreate { item -> } aftercreate {item ->} beforedelete {item ->} afterdelete {item ->} } jobs {} executions {} realms {} 33

34 Demo Time Artifact filtering 34

35 Summary The need for using a binary repository Module management plays key roles in - Continuous integration and delivery - Release management Changes the way we use and think about binaries Newly evolving domain 35

36 Thank you! Questions? 36