TOP100 从代码修改到持续交付给客户 快速自动化软件集成及交付实践

Transcription

1 TOP100 从代码修改到持续交付给客户 快速自动化软件集成及交付实践 演讲者 : 赵延涛 (Tony Zhao) 职位 : Principal SW CM Engineer 来自公司 : Ericsson Sweden

2 Build Improvement 摘要 1. 概述 2. 实践 3. 展望 CI & CD 设计 - 工具的选择与环境的搭建 CI & CD - all in Git 代码库 编译加速 EA 和 RAMDISK Jenkins Builds Promotion and Lock Code Change Auto-Review & Sanity Test in Gerrit Setup Gerrit in Jenkins Git 代码修改自动关联 User Story 或者 Bug Fix Git Submodules 自动提升 Artifactory: 包文件管理与发布 使用 Openstack 和 Docker 快速搭建 Build & Test 环境 Integrated Tools Build CI Deploy Test Automation Stable Environment Report Branch Strategy Test Test Strategy Competent Team

3 案例标题 a 案例简述 云计算平台和 IP 路由器 b 开始时遇到的问题 从代码的变化到最终产品反馈 速度慢 周期长 c 解决思路 从提交代码修改开始 除了必要的人工代码评审 快速自动化完 成持续集成和持续交付的流水线作业

4 Background and self introduction Tony Zhao ( 赵延涛 ) ============================================ Principal SCM Engineer CI team member in PDU Cloud System Website: Oracle Certificated Professional Sun Certificated Java Programmer Master in Computer Software and Theory Hobbies: Tennis, Badminton, Table Tennis, Billiard, Climbing, So ccer, Karting, Hiking, Movie, Outdoor Travel ============================================ 04/2014- till now: Ericsson AB, SCM Engineer (Level 7). 05/ /2014: Ericsson China R&D, Principal SCM Engineer, tech lead. 05/ /2006: Siemens Com, Senior CM Engineer. 06/ /2004: Lenovo Software Design Center, Senior SW Designer. 07/ /2002: Vice lead of R&D Center at a state-owned company.

5 怎么做到的 搭建前 制定目标与量化指标 易于维护 过程透明 白盒化 开发人员易于使用 简单化 架构易于扩充与修改配置 便于支持与维护 出现错误时可以及时回溯到历史版本 不同任务之间要有清晰的接口 可移植性 促进而不是阻碍开发 尽量使用工具的原生命令 日志记录便于分析改进 搭建过程中 采用敏捷方式 分期迭代部署

6 1. 概述 产品开发反馈模型 Design Test Function Test System Test Designer Team Project Customer Lab Live Network

7 1. 概述 持续化集成 -> 交付 -> 发布 -> 实施 Continuous Integration Continuous Delivery Automated process of secure and frequent integration of source code into source baselines, and binaries into system baselines. Automated process of secure and frequent internal provisioning of ready-to-install software product versions of integrated software. Continuous integration can be thought of as frequently assembling a product on the factory floor, checking that it is not obviously broken and ensuring that the production line is in working order for the next unit to be produced. Continuous delivery can be thought of as frequently taking an assembled product from the factory floor, taking it through quality assurance, verifying compliance to relevant regulations, putting it in a box and placing it in inventory. Continuous Release Continuous Deployment Automated process of secure and frequent provisioning of delivered software product to external customers and clients. Automated process of secure and frequent production, testing and/or monitoring, and deployment of software products to customer equipment in a live environment. Continuous release can be thought of as frequently taking a boxed and ready-to-go product from inventory and placing it in the storefront window. Continuous deployment can be thought of as frequently taking a boxed and ready-to-go product, either from inventory or from the storefront window, home to the customer, installing it and verifying that it works as expected in that environment. In this document, Cx should be read as continuous integration, delivery, release and deployment.

8 1. 概述 CI & CD 关注领域 Customer Zone Production Zone Requirement Analyzation Feature Decision Customer Business Need Feature Opportunity SW Design Design Function System Test Test Test Test Analysis NIV SW Package Ready PRIM Product Management Software Development Test & Verification Deployment Pipeline GASK Customer Software Network Customer Disk Lab Test Gateway Feature Activated in Customer Network

9 实践 2.1 集成的 CI & CD 环境 tuleap

10 实践 2.1 tuleap 的 3 种安装方式 下载虚拟机文件 打开即可

11 实践 2.1 或者有不同的工具组合 Tuleap: Project Doc, WiKi, Tracker Bugzilla: Bug Github.com/gerrithub.io: Code/Review Artifactory: Binary Repository OpenStack: Build/Test server virtualization EA/ramdisk: Build Acceleration Ansible: CI environment configuration OBS: Open Build Service, packing

12 实践 2.1 设计 CI- 分阶段分层次 以 Mirantis (OpenStack Contributor and Service Company) 为例

13 实践 2.1 以 Docker Images 的方式发布

14 实践 2.2 ALL in Git version control: Server Env Configurations Jenkins Job Configurations Source Code Jenkins Job Builder or Jenkins DSL Plugin: Jenkins Job Configurations 对比 Two Instances: Staging Production 不要直接在 Production Jenkins 的界面修改 Job 配置 Server Env Configurations Jenkins Job Configurations Jenkins Job Inside Scripts

15 实践 2.2 Server Env Configurations in Ansible

16 实践 2.2 Jenkins Job Configura tions into DSL script

17 实践 2.2 用 JobDSL 脚本生成 Jenkins Job In Jenkins, create a DSL seed job, copy *.groovy into the workspace. Then build the seed job.

18 实践 2.3 编译加速 EA: One Pager Report Significant improvements have been observed Up to 12 times faster builds for full clean build (200Mins vs. 17Mins) Total Investment: 240,000 SEK per year 50 EA Agents 5 Agent Servers RIO Payback Period: 0.4 months make emake

19 实践 2.3 EA Architecture Network electricinsight Gerrit/Git Repository Automate Accelerate Analyze Production Machines Build with ElectricMak e instead of GNU Make Cluster Manager electriccommand er electricaccelerator Agent Developer Machines Agent User-level Kernel Build Cluster ( CPUs) Electric File System Node

20 实践 2.3 Build Acceleration: Y-Axis is Agents Build History File Each block is a job running on an agent for some duration White space is the enemy Cluster is idle while blocking job runs (serialization or "stall") X-Axis is Time 20

21 实践 2.3 EA Conclusion 6 Hours to 90 Minutes by build avoidance. Build out 3rd party platform from nightly build. Reuse 3rd party platform in a new workspace. 90 Minutes to 10 Minutes by build acceleration. Research on EA and deploy it in build environment. EA requires specific configuration for a given project. The tuning of EA is a continuous activity. Other considerations. Create workspace on local disk will reduce build time by 50%. Revise makefile to invoke local compiler first.

22 实践 2.3 编译加速 - 虚拟内存盘 RAMDISK 原理和用途 通过软件将一部分内存 RAM 模拟为硬盘来使用的一种技术 实现及软件编辑 DOS 系统 XMSDSK Windows 系统 VSuite Ramdisk Linux 系统 直接格式化并挂载 /dev/ramx 即可 效果 提速一倍

23 实践 2.4 Jenkins Builds Promotion and Lock Build Keeper Plugin Build Promotion Plugin To Keep a build for ever

24 添加内容 实践 2.5 Code Change Auto-Review in Gerrit

25 实践 2.5 Run Gerrit Sanity Test on All Code Changes

26 实践 2.5 如果任何一项检查没有通过

27 实践 2.6 把你的代码存入 Github

28 实践 2.6 通过 gerrithub 来 review

29 实践 2.6 Setup Gerrit in Jenkins Jenkins System Jenkins Job

30 实践 2.7 关联代码修改与 User Story 和 Bug MINGW64 ~/src/iptnms-p/.git/hooks (GIT_DIR!) applypatch-msg.sample* pre-applypatch.sample* commit-msg* commit-msg.sample* post-update.sample* pre-commit.sample* prepare-commit-msg* prepare-commit-msg.sample* pre-push.sample* pre-rebase.sample* update.sample*

31 @git commit 实践 2.7 关联代码修改与 User Story 和 Bug Fix

32 实践 2.8 Git Submodules Auto Uplift $ cat.gitmodules [submodule "fuel/library"] path = fuel/library url =../../ecs/mirantis/openstack/fuel-library branch = cee/ecs-mos [submodule "fuel/ostf"] path = fuel/ostf url =../../ecs/mirantis/openstack/fuel-ostf branch = cee/ecs-mos

33 实践 2.9 Artifactory+Docker: 包文件管理与发布 Artifactory will empower your experience, especially with the following integrations. Working with Docker: Working with Git: Working with Jenkins: rtifactory+plug-in Ant+Ivy Maven 和 Gradle 这些构建工具都可以自动下载 Artifactory 里的构件 artifact 此外 Jenkins Bamboo 等 CI 工具也可以通过构建工具将生成的 构件 artifact 部署到 Artifactory 上 Or Nexus

34 实践 2.9 Uploading using Jenkins - Artifactory All deliveries from components to Release Management shall be done by uploading the binary packages into Artifactory. This includes installation packages, installation scripts, change logs and/or release notes, etc. Any file type is supported (zip, gz, tgz, etc.)

35 实践 2.9 Uploading using CLI - Artifactory Get ARM-Uploader and use it in your CLI. To get usage description, run: java -jar artifactoryuploader.jar -h Usage example: Suppose you want to upload a file named package.tar as version of CEE delivery to ARM. You would execute: java -jar artifactoryuploader.jar -repo proj-ecs-dev-local -p cee -r usr <username> -pwd <password> -file /path/to/my/package/package.tar NOTE! Be sure to use your <code>encrypted</code> password ([[#Artifactory see this section]])

36 实践 2.9 Uploading using Web GUI - Artifactory 1. Log in to ARM with your account 2. Click Deploy and choose the file that you want to deploy 3. Enter the details as show in the picture below. 4. Refer to Naming your binaries for more details on file naming

37 实践 2.10 采用容器和虚拟机

38 实践 2.10 VMs and Container As mentioned by BraveNewCurrency, the only relationship between the host OS and the container is the Kernel. It is one of the main difference between docker and 'regular' virtual machines, there is no overhead, everything takes place directly within the host's kernel. This is why you can run only Linux based distribution/binaries within the container. If you want to run something else, it is not impossible, but you would need some kind of virtualization within the container (qemu, kvm, etc.) Contianer vs VM Jenkins and Docker

39 实践 2.10 Install OpenStack Mirantis Openstack deployment Install DevStack (with Neutron) on Ubuntu Virtual Machine OpenStack Architecture OpenStack All-In-One Single Machine 节省硬件资源 需要时才搭建一台构建或者测试服务器 管理使用OpenStack

40 实践 2.10 使用 VirtualBox 代替 OpenStack

41 3. 展望 CI & CD Developing a continuous integration and delivery system that helps streamline the software release process, to make the tools and process used by software engineers as elegant as the shipping products.

42