Integra(ng OS Package Management and the Erlang VM

Transcription

1 Integra(ng OS Package Management and the Erlang VM

2 An Erlang Startup Write awesome Erlang code.??? Profit.

3 An Erlang Startup Write awesome Erlang code.??? Deploy to EC2.??? Profit.

4 An Erlang Startup Write awesome Erlang code.??? Deploy to EC2.??? Profit.

5 How do I launch my sofware

6 How do I launch my sofware to EC2

7 Erlang/OTP sofware structure

8 Erlang/OTP sofware structure

9 Erlang/OTP sofware structure

10 Erlang/OTP sofware structure

11 Erlang/OTP sofware structure

12 Erlang/OTP sofware structure

13 Erlang/OTP sofware structure

14 Erlang/OTP sofware structure

15 Erlang/OTP sofware structure

16 Erlang/OTP sofware structure

17 Erlang/OTP sofware structure

18 Erlang/OTP sofware structure

19 Erlang/OTP sofware structure

20 Erlang/OTP sofware structure

21 Erlang/OTP sofware structure

22 Erlang/OTP sofware structure

23 Erlang/OTP sofware structure

24 Erlang/OTP sofware structure

25 Erlang/OTP sofware structure

26 Erlang/OTP sofware structure

27 Erlang/OTP sofware structure

28 Erlang/OTP sofware structure

29 Erlang/OTP sofware structure

30 Erlang/OTP sofware structure

31 Erlang/OTP sofware structure cool.erl

32 Erlang/OTP sofware structure cool.erl

33 Erlang/OTP sofware structure cool.erl

34 Erlang/OTP sofware structure cool.erl

35 Erlang/OTP sofware structure cool.erl

36 Erlang/OTP sofware structure cool.erl mega.erl

37 Erlang/OTP sofware structure src cool.erl mega.erl sweet.erl

38 src Erlang/OTP sofware structure

39 Erlang/OTP sofware structure src

40 Erlang/OTP sofware structure src

41 Erlang/OTP sofware structure src

42 Erlang/OTP sofware structure foo src ebin

43 Erlang/OTP sofware structure foo src ebin foo.app foo.appup

44 Erlang/OTP sofware structure foo src ebin {applica(on, foo, [{descrip(on, My App"}, {vsn, 0.1.0"}, {modules, [cool,mega,sweet]}, {registered, [foobar]}, {applica(ons, [kernel,stdlib,sasl,bar]}, {mod, {cool,[]}} ]}. foo.app foo.appup

45 Erlang/OTP sofware structure foo src ebin {applica(on, foo, [{descrip(on, My App"}, {vsn, 0.1.0"}, {modules, [cool,mega,sweet]}, {registered, [foobar]}, {applica(ons, [kernel,stdlib,sasl,bar]}, {mod, {cool,[]}} ]}. foo.app Which modules? foo.appup

46 Erlang/OTP sofware structure foo src ebin {applica(on, foo, [{descrip(on, My App"}, {vsn, 0.1.0"}, {modules, [cool,mega,sweet]}, {registered, [foobar]}, {applica(ons, [kernel,stdlib,sasl,bar]}, {mod, {cool,[]}} ]}. foo.app foo.appup Which modules? Prerequisites

47 Erlang/OTP sofware structure foo src ebin { 0.1.0", [{ 0.0.0", [{load_module, mega, [ch3]}]}], [{ 0.0.0", [{load_module, mega, [ch3]}]}] }. foo.app foo.appup

48 Erlang/OTP sofware structure foo src ebin { 0.1.0", [{ 0.0.0", [{load_module, mega, [ch3]}]}], [{ 0.0.0", [{load_module, mega, [ch3]}]}] }. foo.app foo.appup DSL Upgrade Commands

49 Erlang/OTP sofware structure foo src ebin foo.app foo.appup

50 foo Erlang/OTP sofware structure

51 Erlang/OTP sofware structure foo 0.1.0

52 Erlang/OTP sofware structure foo 0.1.0

53 Erlang/OTP sofware structure foo 0.1.0

54 Erlang/OTP sofware structure lib foo bar 0.2.1

55 Erlang/OTP sofware structure lib foo bar baz 1.3.4

56 lib Erlang/OTP sofware structure

57 lib Erlang/OTP sofware structure

58 lib Erlang/OTP sofware structure

59 Erlang/OTP sofware structure lib

60 Erlang/OTP sofware structure lib releases

61 Erlang/OTP sofware structure lib releases zzz.rel

62 Erlang/OTP sofware structure lib releases {release,{ zzz","0.4.5"}, {erts,"5.5.5"}, [{kernel,"2.11.4"}, {stdlib,"1.14.4"}, {foo,"0.1.0"}, {bar, 0.2.1"}]} zzz.rel

63 Erlang/OTP sofware structure lib releases What applica(ons to start The order to start them zzz.rel zzz.script

64 Erlang/OTP SoFware Structure A module is a unit of code. An applica(on is a collec(on of modules. Defines how to start and stop. Defines how to upgrade/downgrade. A release is a collec(on of applica(ons. Defines what cons(tutes the system. Defines how to start the system. Also can define upgrade/downgrade.

65 Erlang/OTP Launch Strategy Collect versioned applica(ons together. Defines the new state of the system. Complete, not incremental. Create the release using OTP tools. Make the release accessible. The deployment problem. Upgrade using release handler tools.

66 Large Companies Infrequent complete releases. Heavy integra(ve QA.

67 Large Companies Infrequent complete releases Heavy integra(ve QA

68 Large Companies Infrequent complete releases Heavy integra(ve QA

69 Large Companies Infrequent complete releases Heavy integra(ve QA

70 Startup Agile launch processes

71 Large Companies Infrequent complete releases Heavy integra(ve QA

72 Use the OS Package Manager Uniformity. Dependency Specifica(on. Between Erlang applica(ons. Between Erlang and non Erlang components. Installa(on hooks. Hot code management. Provisioning. Deployment.

73 1 Xen Instance = 1 Erlang VM

74 1 Xen Instance = 1 Erlang VM

75 1 Xen Instance = 1 Erlang VM

76 1 Xen Instance = 1 Erlang VM

77 1 Xen Instance = 1 Erlang VM

78 1 Xen Instance = 1 Erlang VM

79 1 Xen Instance = 1 Erlang VM

80 1 Xen Instance = 1 Erlang VM

81 1 Xen Instance = 1 Erlang VM

82 1 Xen Instance = 1 Erlang VM

83 1 Xen Instance = 1 Erlang VM

84 1 Xen Instance = 1 Erlang VM

85 1 Xen Instance = 1 Erlang VM

86 1 Xen Instance = 1 Erlang VM

87 1 Xen Instance = 1 Erlang VM

88 1 Xen Instance = 1 Erlang VM EC2 Instance Erlang VM s3fs Package Archive

89 1 Package = 1 Applica(on Small enough to launch frequently. No need to assemble all applica(on changes. Release = set of packages. Large enough to be interes(ng. Supervision hierarchy. Upgrade seman(cs.

90 Our Launch Process Create applica(on OS package. Province of the build system. Upload to package archive. Standard tools available. Install the package on server. Automa(cally starts, stops, upgrades, downgrades applica(ons as appropriate.

91 Our Launch Process Create applica(on OS package. Province of the build system. Upload to package archive. Standard tools available. Install the package on server. Automa(cally starts, stops, upgrades, downgrades applica(ons as appropriate.

92 Erlrc: The Vision 199% sudo apt get install egerl Reading package lists... Done Building dependency tree Reading state informa(on... Done The following packages will be upgraded: egerl 1 upgraded, 0 newly installed, 0 to remove and 34 not upgraded. Need to get 0B/113kB of archives. AFer unpacking 0B of addi(onal disk space will be used. WARNING: The following packages cannot be authen(cated! egerl Install these packages without verifica(on [y/n]? y (Reading database files and directories currently installed.) Preparing to replace egerl (using.../archives/egerl_4.1.0_all.deb)... Unpacking replacement egerl... erlrc upgrade: Upgrading 'egerl': (cb8eec1a1b85ec017517d3e51c5aee7b) upgraded Seqng up egerl (4.1.0)... erlrc start: Star(ng 'egerl': (cb8eec1a1b85ec017517d3e51c5aee7b) already_running

93 Erlrc: The Vision 199% sudo apt get install egerl Reading package lists... Done Building dependency tree Reading state informa(on... Done The following packages will be upgraded: egerl 1 upgraded, 0 newly installed, 0 to remove and 34 not upgraded. Need to get 0B/113kB of archives. AFer unpacking 0B of addi(onal disk space will be used. WARNING: The following packages cannot be authen(cated! egerl Install these packages without verifica(on [y/n]? y (Reading database files and directories currently installed.) Preparing to replace egerl (using.../archives/egerl_4.1.0_all.deb)... Unpacking replacement egerl... erlrc upgrade: Upgrading 'egerl': (cb8eec1a1b85ec017517d3e51c5aee7b) upgraded Seqng up egerl (4.1.0)... erlrc start: Star(ng 'egerl': (cb8eec1a1b85ec017517d3e51c5aee7b) already_running

94 Problems to solve Sta(c: boo(ng the VM properly Dynamic: hot install/upgrade/remove

95 Erlang VM Boot lib releases What applica(ons to start The order to start them zzz.rel zzz.script

96 Config File vs. Config Directory apache.conf

97 Config File vs. Config Directory apache.conf

98 Config File vs. Config Directory apache.conf

99 Config File vs. Config Directory apache.conf

100 Config File vs. Config Directory apache.conf

101 Config File vs. Config Directory apache.conf

102 Config File vs. Config Directory apache.conf apache.conf.d

103 Config File vs. Config Directory apache.conf apache.conf.d

104 Config File vs. Config Directory apache.conf apache.conf.d

105 Config File vs. Config Directory apache.conf apache.conf.d

106 Config File vs. Config Directory zzz.script /etc/erlrc.d/applica(ons

107 Actual Applica(ons Directory % ls /etc/erlrc.d/applications appinspect egerl fragmentron genherd nodefinder zfile combonodefinder erlrc fuserl loggins schemafinder ec2nodefinder erlsom gencron n54etsbugfix virtuerl

108 Actual Applica(ons Directory % ls /etc/erlrc.d/applications appinspect egerl fragmentron genherd nodefinder zfile combonodefinder erlrc fuserl loggins schemafinder ec2nodefinder erlsom gencron n54etsbugfix virtuerl Filename = Applica(on Name Files are Empty (for Now)

109 Actual Applica(ons Directory % ls /etc/erlrc.d/applications appinspect egerl fragmentron genherd nodefinder zfile combonodefinder erlrc fuserl loggins schemafinder ec2nodefinder erlsom gencron n54etsbugfix virtuerl Filename = Applica(on Name Files are Empty (for Now) Start Up Order

110 Actual Applica(ons Directory % ls /etc/erlrc.d/applications appinspect egerl fragmentron genherd nodefinder zfile combonodefinder erlrc fuserl loggins schemafinder ec2nodefinder erlsom gencron n54etsbugfix virtuerl Filename = Applica(on Name Files are Empty (for Now) Start Up Order?

111 Determining the Startup Order foo src ebin foo.app foo.appup

112 Determining the Startup Order foo src ebin {applica(on, foo, [{descrip(on, My App"}, {vsn, 0.1.0"}, {modules, [cool,mega,sweet]}, {registered, [foobar]}, {applica(ons, [kernel,stdlib,sasl,bar]}, {mod, {cool,[]}} ]}. foo.app foo.appup

113 Determining the Startup Order foo src ebin {applica(on, foo, [{descrip(on, My App"}, {vsn, 0.1.0"}, {modules, [cool,mega,sweet]}, {registered, [foobar]}, {applica(ons, [kernel,stdlib,sasl,bar]}, {mod, {cool,[]}} ]}. foo.app Prerequisites define a par(al order Choose a consistent start order foo.appup

114 Regular Applica(on foo

115 Regular Applica(on foo

116 Regular Applica(on foo applica(on controller

117 Included Applica(on foo

118 Included Applica(on foo

119 Included Applica(on foo

120 Included Applica(on bar foo

121 Included Applica(on bar Star(ng bar starts foo. applica(on controller foo

122 Included Applica(on Logic If bar includes foo, and both bar and foo are to be started, then only start bar. If baz depends upon foo, and bar is to be started, then baz depends upon bar.

123 Boot Procedure Packages contain addi(onal empty file. /etc/erlrc/applica(ons/<appname>.app prerequisites are correct. Start using any boot script, e.g., start_sasl. Pass control to erlrc_boot:boot/0. erl boot start_sasl s erlrc_boot boot

124 Problems to solve Sta(c: boo(ng the VM properly Dynamic: hot install/upgrade/remove

125 Erlrc upgrade components erlrcdynamic module methods. erlrcdynamic:downgrade/3 shell scripts erlrc downgrade rendezvous at /etc/erlrc.d/nodes % ls /etc/erlrc.d/nodes cb8eec1a1b85ec017517d3e51c5aee7b

126 Package hook correspondence Shell script erlrcdynamic method Descrip3on When (debian) erlrc start start/2 Idempotent start pos(nst configure ; pos(nst abortremove erlrc stop stop/2 Idempotent stop prerm remove erlrc upgrade upgrade/3 Upgrade erlrc downgrade downgrade/3 applica(on Downgrade applica(on pos(nst upgrade pos(nst abort_upgrade

127 Example: prerm #! /bin/sh package_name="example" package_version="0.0.0 opera(on="$1 case "$opera(on" in remove) which erlrc stop >/dev/null 2>/dev/null test $? ne 0 erlrc stop "${package_name}" "${package_version}" exit 1 ;; *) ;; esac exit 0

128 Example: prerm #! /bin/sh package_name="example" package_version="0.0.0 opera(on="$1 case "$opera(on" in remove) which erlrc stop >/dev/null 2>/dev/null test $? ne 0 erlrc stop "${package_name}" "${package_version}" exit 1 ;; *) ;; esac exit 0

129 Best prac(ces for build system Automa(cally generate erlrc_boot file. /etc/erlrc.d/applications/<appname> Generate both dependencies from single specifica(on..app prerequisites OS package dependencies Automa(cally generate package hooks.

130 Upgrade Logic If there is an appup file in the newer applica(on version, use it; otherwise, generate one automa(cally. Ensure all modules listed in the current applica(on specifica(on are loaded. Find any added and removed included applica(ons by examining the current and target version OTP app files. Stop any added included applica(ons. Execute release_handler:eval_appup_script/4. Start any removed included applica(ons, if they are listed in $ERLRC_ROOT/applica(ons.

131 Upgrade Logic If there is an appup file in the newer applica(on version, use it; otherwise, generate one automa(cally. Ensure all modules listed in the current applica(on specifica(on are loaded. Find any added and removed included applica(ons by examining the current and target version OTP app files. Stop any added included applica(ons. Execute release_handler:eval_appup_script/4. Start any removed included applica(ons, if they are listed in /etc/erlrc.d/applica(ons.

132 Upgrade Logic If there is an appup file in the newer applica(on version, use it; otherwise, generate one automa(cally. Ensure all modules listed in the current applica(on specifica(on are loaded. Find any added and removed included applica(ons by examining the current and target version OTP app files. Stop any added included applica(ons. Execute release_handler:eval_appup_script/4. Start any removed included applica(ons, if they are listed in /etc/erlrc.d/applica(ons.

133 Upgrade Logic If there is an appup file in the newer applica(on version, use it; otherwise, generate one automa(cally. Ensure all modules listed in the current applica(on specifica(on are loaded. Find any added and removed included applica(ons by examining the current and target version OTP app files. Stop any added included applica(ons. Execute release_handler:eval_appup_script/4. Start any removed included applica(ons, if they are listed in /etc/erlrc.d/applica(ons.

134 Included Applica(on bar Star(ng bar starts foo. applica(on controller foo

135 Included Applica(on bar does not start Star(ng bar starts foo. applica(on controller

136 Included Applica(on Invariants If an applica(on is listed in the /etc/erlrc.d/applications directory, then it should be running. When an applica(on is running, all of its included applica(ons are considered running. An applica(on cannot both run itself and be included.

137 Included Applica(on Logic If X is included by Y, then before star(ng Y, X is stopped. If X is included by Y, then afer stopping Y, X is started. If Y is upgraded or downgraded such that X becomes included when it was not previously, before Y is upgraded or downgraded, X is stopped. If Y is upgraded or downgraded such that X becomes no longer included when it was previously, afer Y is upgraded or downgraded, X is started.

138 Included Applica(on Logic If X is included by Y, then before star(ng Y, X is stopped. If X is included by Y, then afer stopping Y, X is started. If Y is upgraded or downgraded such that X becomes included when it was not previously, before Y is upgraded or downgraded, X is stopped. If Y is upgraded or downgraded such that X becomes no longer included when it was previously, afer Y is upgraded or downgraded, X is started.

139 .appup files Manual genera(on: tedious, error prone. Typically can be automa(cally generated. Strategy: automa(on with override. Use.appup file if provided (rare). Otherwise automa(cally generate (common). Key tool: beam_lib:chunks/2.

140 Automa(c.appup genera(on foo src ebin foo.app cool.beam

141 Automa(c.appup genera(on foo src foo src pos$nst upgrade ebin ebin foo.app foo.app cool.beam cool.beam

142 Automa(c.appup genera(on foo src foo src pos$nst upgrade ebin ebin foo.app Compare.app files foo.app cool.beam cool.beam

143 Automa(c.appup genera(on foo src foo src pos$nst update ebin ebin foo.app foo.app cool.beam Compare.beam files cool.beam

144 Example: load_module {applica(on, foo, [{descrip(on, My App"}, {vsn, 0.1.0"}, {modules, [cool,mega,sweet]}, {registered, [foobar]}, {applica(ons, [kernel,stdlib,sasl,bar]}, {mod, {cool,[]}} ]}. {applica(on, foo, [{descrip(on, My App"}, {vsn, 0.1.1"}, {modules, [cool,mega,sweet,ultra]}, {registered, [foobar]}, {applica(ons, [kernel,stdlib,sasl,bar]}, {mod, {cool,[]}} ]}.

145 Example: load_module {applica(on, foo, [{descrip(on, My App"}, {vsn, 0.1.0"}, {modules, [cool,mega,sweet]}, {registered, [foobar]}, {applica(ons, [kernel,stdlib,sasl,bar]}, {mod, {cool,[]}} ]}. {applica(on, foo, [{descrip(on, My App"}, {vsn, 0.1.1"}, {modules, [cool,mega,sweet,ultra]}, {registered, [foobar]}, {applica(ons, [kernel,stdlib,sasl,bar]}, {mod, {cool,[]}} ]}. ultra is a new module emit load_module direc(ve

146 Example: update update direc(ve is used to upgrade the code and the state of an ac(ve process, e.g., gen_server. All of the gen_xxx family export code_change/3 in their behaviour.

147 Example: update foo ebin foo ebin pos$nst update cool.beam cool.beam Exports code_change/3? If yes, issue update direc(ve.

148 .appup file genera(on Compute added and removed modules by looking at the old and new OTP app files. Emit a load_module direc(ve for every added module. For each module which is in both versions of the applica(on: If the module implements the supervisor behaviour Emit an "upgrade for supervisors" instruc(on. If the module exports a sup_upgrade_no(fy/2 func(on, emit an instruc(on to call it. Else if the module exports a code_change/3 func(on, emit a update direc(ve for that module. Otherwise, emit a load_module direc(ve for that module. If there is a start module for the applica(on: Determine if the new beam for the start module exports a version_change/2 func(on. If so, emit a direc(ve to call it. Emit a delete_module direc(ve for every removed module.

149 .appup file genera(on Compute added and removed modules by looking at the old and new OTP app files. Emit a load_module direc(ve for every added module. For each module which is in both versions of the applica(on: If the module implements the supervisor behaviour Emit an "upgrade for supervisors" instruc(on. If the module exports a sup_upgrade_no(fy/2 func(on, emit an instruc(on to call it. Else if the module exports a code_change/3 func(on, emit a update direc(ve for that module. Otherwise, emit a load_module direc(ve for that module. If there is a start module for the applica(on: Determine if the new beam for the start module exports a version_change/2 func(on. If so, emit a direc(ve to call it. Emit a delete_module direc(ve for every removed module.

150 Custom.appup hooks <supervisor>:sup_upgrade_notify/2 Used to start or stop newly added or removed children. <application>:version_change/2 Used to no(fy the applica(on start module that the applica(on version has changed.

151 Distribu(on and provisioning ec2 do escript for rolling window execu(on across an ec2 group. % ec2-do g production m 3 apt-get install egerl apt snapshot provisioning tool based upon.deb packages % apt-snapshot create <snapshot> % apt-snapshot restore host:<snapshot>

152 An Erlang Startup Write awesome Erlang code.??? Deploy to EC2.??? Profit.

153 An Erlang Startup Write awesome Erlang code. Use erlrc and make OS packages. Deploy to EC2.??? Profit.

154 More Informa(on h~p://dukesoferl.blogspot.com/ h~p://code.google.com/p/erlrc/ h~p://n54.com/