porcelain Documentation Release <author> August 20, 2014
Contents 1 Overview 3 2 Installation 5 3 Usage 7 3.1 Launching one-off programs....................................... 7 3.2 Passing input and getting output..................................... 7 3.3 Streams.................................................. 8 3.4 Messages................................................. 8 4 Configuring the Goon driver 11 4.1 Setting the driver............................................. 11 4.2 Goon options............................................... 11 5 Going deeper 13 6 Known issues and roadmap 15 7 Acknowledgements 17 8 License 19 9 API Reference 21 9.1 Porcelain................................................. 21 9.2 Porcelain.Driver.Basic.......................................... 24 9.3 Porcelain.Driver.Goon.......................................... 25 9.4 Porcelain.Process............................................. 25 9.5 Porcelain.Result............................................. 26 9.6 Porcelain.UsageError........................................... 27 Elixir Module Index 29 i
ii
Porcelain implements a saner approach to launching and communicating with external OS processes from Elixir. Built on top of Erlang s ports, it provides richer functionality and simpler API. Simply put, Porcelain removes the pain of dealing with ports and substitutes it with happiness and peace of mind. Contents 1
2 Contents
CHAPTER 1 Overview Having some 20 odd options, the Erlang port API can be unwieldy and cumbersome to use. Porcelain replaces it with a simpler approach and provides defaults for the common cases. User-level features include: sane API ability to launch external programs in a synchronous or asynchronous manner multiple ways of passing input to the program and getting back its output (including working directly with files and Elixir streams) being able to work with programs that try to read the whole input until EOF before producing output ability to send OS signals to external processes (requires goon v2.0) To read background story on the library s design and possible future extensions, please refer to the wiki. 3
4 Chapter 1. Overview
CHAPTER 2 Installation Add Porcelain as a dependency to your Mix project: def application do [applications: [:porcelain]] end defp deps do [{:porcelain, "~> 2.0"}] end Now, some of the advanced functionality is provided by the external program called goon. See which particular features it implements in the reference docs here. Go to goon s project page to find out how to install it. 5
6 Chapter 2. Installation
CHAPTER 3 Usage Examples below show some of the common use cases. See also this demo app. Refer to the API docs to familiarize yourself with the complete set of provided functions and options. 3.1 Launching one-off programs If you need to launch an external program, feed it some input and capture its output and maybe also exit status, use exec() or shell(): alias Porcelain.Result %Result{out: output, status: status} = Porcelain.shell("date") IO.inspect status #=> 0 IO.inspect output #=> "Fri Jun 6 14:12:02 EEST 2014\n" result = Porcelain.shell("date cut -b 1-3") IO.inspect result.out #=> "Fri\n" # Use exec() when you want launch a program directly without using a shell File.write!("input.txt", "lines\nread\nfrom\nfile\n") result = Porcelain.exec("sort", ["input.txt"]) IO.inspect result.out #=> "file\nfrom\nlines\nread\n" 3.2 Passing input and getting output Porcelain gives you many options when it comes to interacting with external processes. It is possible to feed input from a file or a stream, same for output: File.write!("input.txt", """ This file contains some patterns >like this< interspersed with other text... >like this< the end. """) Porcelain.exec("grep", [">like this<", "-m", "2"], in: {:path, "input.txt"}, out: {:append, "output.txt"}) IO.inspect File.read!("output.txt") #=> ">like this<\n... >like this< the end.\n" 7
3.3 Streams Programs can be spawned asynchronously (using spawn() and spawn_shell()) allowing for continuously exchanging data between Elixir and the external process. In the next example we will use streams for both input and output. alias Porcelain.Process, as: Proc instream = SocketStream.new( example.com, 80) opts = [in: instream, out: :stream] proc = %Proc{out: outstream} = Porcelain.spawn("grep", ["div", "-m", "4"], opts) Enum.into(outstream, IO.stream(:stdio, :line)) # div { # div { # <div> # </div> Proc.alive?(proc) #=> false Alternatively, we could pass the output stream directly to the call to spawn(): opts = [ in: SocketStream.new( example.com, 80), out: IO.stream(:stderr, :line), ] Porcelain.exec("grep", ["div", "-m", "4"], opts) #=> this will be printed to stderr of the running Elixir process: # div { # div { # <div> # </div> The SocketStream module used above wraps a tcp socket in a stream. Its implementation can be found in the test/util/socket_stream.exs file. 3.4 Messages If you prefer to exchange messages with the external process, you can do that: alias Porcelain.Process, as: Proc alias Porcelain.Result proc = %Proc{pid: pid} = Porcelain.spawn_shell("grep ohai -m 2 --line-buffered", in: :receive, out: {:send, self()}) Proc.send_input(proc, "ohai proc\n") receive do {^pid, :data, data} -> IO.inspect data end #=> "ohai proc\n" Proc.send_input(proc, "this won t match\n") Proc.send_input(proc, "ohai") Proc.send_input(proc, "\n") receive do 8 Chapter 3. Usage
{^pid, :data, data} -> IO.inspect data #=> "ohai\n" end receive do {^pid, :result, %Result{status: status}} -> IO.inspect status #=> 0 end 3.4. Messages 9
10 Chapter 3. Usage
CHAPTER 4 Configuring the Goon driver There are a number of options you can tweak to customize the way goon is used. All of the options described below should put into your config.exs file. 4.1 Setting the driver config :porcelain, :driver, <driver> This option allows you to set a particular driver to be used at all times. By default, Porcelain will try to detect the goon executable. If it can find one, it will use Porcelain.Driver.Goon. Otherwise, it will print a warning to stderr and fall back to Porcelain.Driver.Basic. By setting Porcelain.Driver.Basic above you can force Porcelain to always use the basic driver. If you set Porcelain.Driver.Goon, Porcelain will always use the Goon driver and will fail to start if the goon executable can t be found. 4.2 Goon options config :porcelain, :goon_driver_path, <path> Set an absolute path to the goon executable. If this is not set, Porcelain will search your system s PATH by default. 11
12 Chapter 4. Configuring the Goon driver
CHAPTER 5 Going deeper Take a look at the reference docs for the full description of all provided functions and supported options. 13
14 Chapter 5. Going deeper
CHAPTER 6 Known issues and roadmap there are known crashes happening when using Porcelain across two nodes error handling when using the Goon driver is not completely shaped out 15
16 Chapter 6. Known issues and roadmap
CHAPTER 7 Acknowledgements Huge thanks to all who have been test-driving the library in production, in particular to Josh Adams Tim Ruffles 17
18 Chapter 7. Acknowledgements
CHAPTER 8 License This software is licensed under the MIT license. 19
20 Chapter 8. License
CHAPTER 9 API Reference 9.1 Porcelain 9.1.1 Overview The main module exposing the public API of Porcelain. Basic concepts Functions in this module can either spawn external programs directly (exec/3 and spawn/3) or using a system shell (shell/2 and spawn_shell/2). Functions exec/3 and shell/2 are synchronous (or blocking), meaning they don t return until the external program terminates. Functions spawn/3 and spawn_shell/2 are non-blocking: they immediately return a Porcelain.Process struct and use one of the available ways to exchange input and output with the external process asynchronously. Error handling Using undefined options, passing invalid values to options or any function arguments will fail with a function clause error or Porcelain.UsageError exception. Those are programmer errors and have to be fixed. Any other kinds of runtime errors are reported by returning an error tuple: <reason> is a string explaining the error. {:error, <reason>} where 9.1.2 Summary exec/3 Execute a program synchronously reinit/1 Reruns the initialization and updates application env shell/2 Execute a shell invocation synchronously spawn/3 Spawn an external process and return a Porcelain.Process struct to be able to communicate with it spawn_shell/2 Spawn a system shell and execute the command in it 21
9.1.3 Functions exec(prog, args, options \\ [])(function) Specs: exec(binary, [binary], Keyword.t/0) :: Porcelain.Result.t Execute a program synchronously. Porcelain will look for the program in PATH and launch it directly, passing the args list as command-line arguments to it. Feeds all input into the program (synchronously or concurrently with reading output; see :async_in option below) and waits for it to terminate. Returns a Porcelain.Result struct containing program s output and exit status code. When no options are passed, the following defaults will be used: [in: "", out: :string, err: nil] This will run the program with no input and will capture its standard output. Available options: :in specify the way input will be passed to the program. Possible values: <iodata> the data is fed into stdin as the sole input for the program <stream> interprets <stream> as a stream of iodata to be fed into the program {:path, <string>} path to a file to be fed into stdin {:file, <file>} <file> is a file descriptor obtained from e.g. File.open; the file will be read from the current position until EOF :async_in can be true or false (default). When enabled, an additional process will be spawned to feed input to the program concurrently with receiving output. :out specify the way output will be passed back to Elixir. Possible values: nil discard the output :string (default) the whole output will be accumulated in memory and returned as one string to the caller :iodata the whole output will be accumulated in memory and returned as iodata to the caller {:path, <string>} the file at path will be created (or truncated) and the output will be written to it {:append, <string>} the output will be appended to the the file at path (it will be created first if needed) {:file, <file>} <file> is a file descriptor obtained from e.g. File.open; the file will be written to starting at the current position <coll> feeds program output (as iodata) into the collectable <coll>. Useful for outputting directly to the console, for example: 22 Chapter 9. API Reference
stream = IO.binstream(:standard_io, :line) exec("echo", ["hello", "world"], out: stream) #=> prints "hello\nworld\n" to stdout :err specify the way stderr will be passed back to Elixir. Possible values are the same as for :out. In addition, it accepts the atom :out which denotes redirecting stderr to stdout. Caveat: when using Porcelain.Driver.Basic, the only supported values are nil (stderr will be printed to the terminal) and :out. :dir takes a path that will be used as the directory in which the program will be launched. :env set additional environment variables for the program. The value should be an enumerable with elements of the shape {<key>, <val>} where <key> is an atom or a binary and <val> is a binary or false (meaning removing the corresponding variable from the environment). Basically, it accepts any kind of dict, including keyword lists. reinit(driver \\ nil)(function) Reruns the initialization and updates application env. This function is useful in the following cases: 1.The currently used driver is Goon and the location of the goon executable has changed. 2.You want to change the driver being used. shell(cmd, options \\ [])(function) Specs: shell(binary, Keyword.t/0) :: Porcelain.Result.t Execute a shell invocation synchronously. This function will launch a system shell and pass the invocation to it. This allows using shell features like haining multiple programs with pipes. The downside is that those advanced features may be unavailable on some platforms. It is similar to the exec/3 function in all other respects. spawn(prog, args, options \\ [])(function) Specs: spawn(binary, [binary], Keyword.t/0) :: Porcelain.Process.t Spawn an external process and return a Porcelain.Process struct to be able to communicate with it. You have to explicitly stop the process after reading its output or when it is no longer needed. Use the Porcelain.Process.await/2 function to wait for the process to terminate. Supports all options defined for exec/3 plus some additional ones: in: :receive input is expected to be sent to the process in chunks using the Porcelain.Process.send_input/2 function. :out and :err can choose from a few more values (with the familiar caveat that Porcelain.Driver.Basic does not support them for :err): :stream the corresponding field of the returned Process struct will contain a stream of iodata. Note that the underlying port implementation is message based. This means that the external program will be able to send all of its output to an Elixir process and terminate. The data will be kept in the Elixir process s memory until the stream is consumed. 9.1. Porcelain 23
{:send, <pid>} send the output to the process denoted by <pid>. Will send zero or more data messages and will always send one result message in the end. The data messages have the following shape: {<from>, :data, :out :err, <iodata>} where <from> will be the same pid as the one contained in the Process struct returned by this function. The result message has the following shape: {<from>, :result, %Porcelain.Result{} nil} The result will be nil if the :result option that is passed to this function is set to :discard. Note: if both :out and :err are set up to send to the same pid, only one result message will be sent to that pid in the end. :result specify how the result of the external program should be returned after it has terminated. This option has a smart default value. If either :out or :err option is set to :string or :iodata, :result will be set to :keep. Otherwise, it will be set to :discard. Possible values: :keep the result will be kept in memory until requested by calling Porcelain.Process.await/2 or discarded by calling Porcelain.Process.stop/1. :discard discards the result and automatically closes the port after program termination. Useful in combination with out: :stream and err: :stream. spawn_shell(cmd, options \\ [])(function) Specs: spawn_shell(binary, Keyword.t/0) :: Porcelain.Process.t Spawn a system shell and execute the command in it. Works similar to spawn/3. 9.2 Porcelain.Driver.Basic 9.2.1 Overview Porcelain driver that offers basic functionality for interacting with external programs. Users are not supposed to call functions in this module directly. Use functions in Porcelain instead. This driver has two major limitations compared to Porcelain.Driver.Goon: the exec function does not work with programs that read all input until EOF before producing any output. Such programs will hang since Erlang ports don t provide any mechanism to indicate the end of input. If a program is continuously consuming input and producing output, it could work with the spawn function, but you ll also have to explicitly close the connection with the external program when you re done with it. sending OS signals to external processes is not supported 24 Chapter 9. API Reference
9.3 Porcelain.Driver.Goon 9.3.1 Overview Porcelain driver that offers additional features over the basic one. Users are not supposed to call functions in this module directly. Use functions in Porcelain instead. This driver will be used by default if it can locate the external program named goon in the executable path. If goon is not found, Porcelain will fall back to the basic driver. The additional functionality provided by this driver is as follows: ability to signal EOF to the external program send an OS signal to the program (to be implemented) more efficient piping of multiple programs 9.4 Porcelain.Process 9.4.1 Overview Module for working with external processes launched with Porcelain.spawn/3 or Porcelain.spawn_shell/2. 9.4.2 Summary struct /0A struct representing a wrapped OS processes which provides the ability to exchange data with it alive?/1 Check if the process is still running await/2 Wait for the external process to terminate send_input/2send iodata to the process s stdin signal/2 Send an OS signal to the processes stop/1 Stops the process created with Porcelain.spawn/3 or Porcelain.spawn_shell/2. Also closes the underlying port 9.4.3 Types t t :: %Porcelain.Process{err: term, out: term, pid: term} signal signal :: :int :kill non_neg_integer 9.4.4 Functions struct ()(function) Specs: struct :: %Porcelain.Process{err: term, out: term, pid: term} A struct representing a wrapped OS processes which provides the ability to exchange data with it. 9.3. Porcelain.Driver.Goon 25
alive?(process)(function) Specs: alive?(t) :: true false Check if the process is still running. await(process, timeout \\ :infinity)(function) Specs: await(t, non_neg_integer :infinity) :: {:ok, Porcelain.Result.t} {:error, :noproc :timeout} Wait for the external process to terminate. Returns Porcelain.Result struct with the process s exit status and output. Automatically closes the underlying port in this case. If timeout value is specified and the external process fails to terminate before it runs out, atom :timeout is returned. send_input(process, data)(function) Specs: send_input(t, iodata) :: iodata Send iodata to the process s stdin. End of input is indicated by sending an empty message. Caveat: when using Porcelain.Driver.Basic, it is not possible to indicate the end of input. You should stop the process explicitly using stop/1. signal(process, sig)(function) Specs: signal(t, signal) :: signal Send an OS signal to the processes. No further communication with the process is possible after sending it a signal. stop(process)(function) Specs: stop(t) :: true Stops the process created with Porcelain.spawn/3 or Porcelain.spawn_shell/2. Also closes the underlying port. May cause broken pipe message to be written to stderr. 9.5 Porcelain.Result 9.5.1 Overview A struct containing the result of running a program after it has terminated. 9.5.2 Types t t :: %Porcelain.Result{err: term, out: term, status: term} 26 Chapter 9. API Reference
9.6 Porcelain.UsageError 9.6.1 Overview This exception is meant to indicate programmer errors (misuses of the library API) that have to be fixed prior to release. 9.6.2 Summary exception/1 message/1 Callback implementation of Exception.exception/1 Callback implementation of Exception.message/1 9.6.3 Functions exception(args)(function) Specs: exception(keyword.t/0) :: Exception.t/0 Callback implementation of Exception.exception/1. message(exception)(function) Specs: message(exception.t/0) :: String.t/0 Callback implementation of Exception.message/1. 9.6. Porcelain.UsageError 27
28 Chapter 9. API Reference
Elixir Module Index p Porcelain, 21 Porcelain.Driver.Basic, 24 Porcelain.Driver.Goon, 25 Porcelain.Process, 25 Porcelain.Result, 26 Porcelain.UsageError, 27 29