Erlang: distributed May 11, 2012 1 / 21
Fault tolerance in Erlang links, exit signals, system process in Erlang OTP Open Telecom Platform 2 / 21
General idea Links Exit signals System processes Summary of exit signals Idioms for trapping exits Advanced 3 / 21
General idea General idea Links Exit signals System processes Summary of exit signals Idioms for trapping exits Advanced system process handles exit signals from linked processes traps exit messages { EXIT,Pid,Why} default: usual process dies if Why normal link connection between 2 processes symmetric, must be set explicitely exit signal sent to the set of linked processes when process dies { EXIT,B,Why} when process finishes { EXIT,B,normal} A 4 / 21
General idea General idea Links Exit signals System processes Summary of exit signals Idioms for trapping exits Advanced system process handles exit signals from linked processes traps exit messages { EXIT,Pid,Why} default: usual process dies if Why normal link connection between 2 processes symmetric, must be set explicitely exit signal sent to the set of linked processes when process dies { EXIT,B,Why} when process finishes { EXIT,B,normal} A B 4 / 21
General idea General idea Links Exit signals System processes Summary of exit signals Idioms for trapping exits Advanced system process handles exit signals from linked processes traps exit messages { EXIT,Pid,Why} default: usual process dies if Why normal link connection between 2 processes symmetric, must be set explicitely exit signal sent to the set of linked processes when process dies { EXIT,B,Why} when process finishes { EXIT,B,normal} A B 4 / 21
General idea General idea Links Exit signals System processes Summary of exit signals Idioms for trapping exits Advanced system process handles exit signals from linked processes traps exit messages { EXIT,Pid,Why} default: usual process dies if Why normal link connection between 2 processes symmetric, must be set explicitely exit signal sent to the set of linked processes when process dies { EXIT,B,Why} when process finishes { EXIT,B,normal} A { EXIT,B,Why} B 4 / 21
Links General idea Links Exit signals System processes Summary of exit signals Idioms for trapping exits Advanced link defines error propagation path between 2 processes if one dies then another gets exit signal links are established with link(b) or spawn_link(fun) 5 / 21
Exit signals General idea Links Exit signals System processes Summary of exit signals Idioms for trapping exits Advanced exit signal generated when process dies or finishes { EXIT,Pid,Why} Why=normal if a process just finishes (i.e. recursion ends) Why=<exception desc> if there was a problem exit(why) may be called to stop itself sent to all linked processes faking death: exit(pid2, Why) sends { EXIT,Pid,Why} to process Pid2 continues exection 6 / 21
System processes General idea Links Exit signals System processes Summary of exit signals Idioms for trapping exits Advanced usual process dies if receives exit signal from any linked process where Why normal system process set with process_flag(trap_exit,true) traps exit signals from linked processes messages { EXIT,Pid,Why} are added to its mailbox exit signals with Why=kill are not caught at all! process is killed, even system process { EXIT,Pid,killed} broadcasted to all linked processes (notice that kill is propagated as killed) 7 / 21
General idea Links Exit signals System processes Summary of exit signals Idioms for trapping exits Advanced on_exit(pid, Fun) -> spawn(fun() -> process_flag(trap_exit, true), link(pid), receive { EXIT,Pid,Why} -> Fun(Why) end end). 8 / 21
General idea Links Exit signals System processes Summary of exit signals Idioms for trapping exits Advanced F = fun() -> receive X -> list_to_atom(x) end end. Pid = spawn(f). on_exit(pid, fun(why) -> io:format("~p died with ~p~n",[pid,why]) end). create a process that transforms lists to atoms add error handler with on_exit Now sending Pid! hello. results in <0.61.0> died with:{badarg,[{ 9 / 21
Summary of exit signals trap_exit Exit signal Action true kill Die: broadcast the exit signal killed to the link set true X Add { EXIT,Pid,X} to the mailbox false normal Continue: Do nothing, signal vanishes false kill Die: broadcast the exit signal killed to the link set false X Die: broadcast the exit signal X to the link set 10 / 21
Idioms for trapping exits General idea Links Exit signals System processes Summary of exit signals Idioms for trapping exits Advanced 1. Don t care about new process Pid=spawn(fun() ->... end) 2. Want to die if new process dies Pid=spawn_link(fun() ->... end) 3. Want to handle errors if new process dies... process_flag(trap_exit, true), Pid=spawn_link(fun() ->... end), loop(...). loop(state) -> receive { EXIT, SomePid, Reason} -> %% do something with the error loop(state1)... end. 11 / 21
Advanced General idea Links Exit signals System processes Summary of exit signals Idioms for trapping exits Advanced one fails everyone up to system process dies P1 P2 P3 P4 one fails supervisor process restarts one or all Sup P1 P2 P3 12 / 21
Create servers Distribution primitives 13 / 21
Create servers Distribution primitives 1. Erlang spawn process on any Erlang node Erlang VM instance message passing and error handling works without changes run any module function on any node unsafe: should be used in trusted environments, e.g. dedicated LAN 2. Socket based distribution handle and abstract TCP socket communication lib_chan module in examples safe: use in untrusted environments, e.g. internet 14 / 21
Create servers Create servers Distribution primitives $ erl -sname <name[@server]> short name, usable only on the same host $ erl -name <name[@fqdn]> server name, DNS must be accessible, at least /etc/hosts olegus@hebe:~$ erl -name myerlangnode Erlang R15B (erts-5.9) [source] [64-bit] [smp: Eshell V5.9 (abort with ^G) (myerlangnode@hebe.domenis.ut.ee)1> different machines must have the same cookie ~/.erlang.cookie -setcookie parameter 15 / 21
Distribution primitives Create servers Distribution primitives built in rpc module rpc:call(node,mod,function,args) -> Result {badrpc,reason} built in global module extended spawn functions, etc spawn(node,fun) -> Pid spawn(node,mod,func,arglist) -> Pid spawn_link(node,fun) -> Pid spawn_link(node,mod,func,arglist) -> Pid disconnect_node(node) -> bool() ignored node() -> Node node(arg) -> Node nodes() -> [Node] Pid! Msg {RegName,Node}! Msg 16 / 21
Create servers Distribution primitives Run 2 servers with names me and you erl -sname me@localhost erl -sname you@localhost Test with (run from you ) (you@localhost)8> rpc:call(me@localhost, io, format, ["hello~n"]). hello (you@localhost)9> rpc:call(me@localhost, erlang, node, []). me@localhost test remote spawn... 17 / 21
Create servers Distribution primitives Write and compile dist.erl -module(dist). -export([start/0]). loop() -> receive X -> io:format("~p: Im running on ~p~n", [X,node()]) end, loop(). test it 13> RP = spawn(me@localhost, fun dist:loop/0). 14> RP! "Hi". "Hi": Im running on me@localhost 18 / 21
Description 19 / 21
Description provides process restart strategies, event and error logging, etc. declare gen_server module behaviour (checks for undefined functions during compilation) -module(myserver). -behaviour(gen_server). specify 6 callbacks init(startargs) -> {ok, InitState}. handle_call(msg,from,state) -> {reply, ok, State} handle_cast(msg,state) -> {noreply, State}. handle_info(info,state) -> {noreply, State}. terminate(reason, State) -> ok. code_change(oldv, State, Extra) -> {ok, State}. start server with gen_server:start_link(name, CallBackMod, StartArgs, Opts) 20 / 21
Description Description handle_call is a state transform function that is called for each incoming message (synchronous) handle_cast is the same but does not return value (asynchronous) handle_info is for spontaneous messages, e.g. exit signals from linked processes terminate is executed when agent stops code_change is code hot swapping 21 / 21