porcelain Documentation

Size: px
Start display at page:

Download "porcelain Documentation"

Transcription

1 porcelain Documentation Release <author> August 20, 2014

2

3 Contents 1 Overview 3 2 Installation 5 3 Usage Launching one-off programs Passing input and getting output Streams Messages Configuring the Goon driver Setting the driver Goon options Going deeper 13 6 Known issues and roadmap 15 7 Acknowledgements 17 8 License 19 9 API Reference Porcelain Porcelain.Driver.Basic Porcelain.Driver.Goon Porcelain.Process Porcelain.Result Porcelain.UsageError Elixir Module Index 29 i

4 ii

5 Porcelain implements a saner approach to launching and communicating with external OS processes from Elixir. Built on top of Erlang s ports, it provides richer functionality and simpler API. Simply put, Porcelain removes the pain of dealing with ports and substitutes it with happiness and peace of mind. Contents 1

6 2 Contents

7 CHAPTER 1 Overview Having some 20 odd options, the Erlang port API can be unwieldy and cumbersome to use. Porcelain replaces it with a simpler approach and provides defaults for the common cases. User-level features include: sane API ability to launch external programs in a synchronous or asynchronous manner multiple ways of passing input to the program and getting back its output (including working directly with files and Elixir streams) being able to work with programs that try to read the whole input until EOF before producing output ability to send OS signals to external processes (requires goon v2.0) To read background story on the library s design and possible future extensions, please refer to the wiki. 3

8 4 Chapter 1. Overview

9 CHAPTER 2 Installation Add Porcelain as a dependency to your Mix project: def application do [applications: [:porcelain]] end defp deps do [{:porcelain, "~> 2.0"}] end Now, some of the advanced functionality is provided by the external program called goon. See which particular features it implements in the reference docs here. Go to goon s project page to find out how to install it. 5

10 6 Chapter 2. Installation

11 CHAPTER 3 Usage Examples below show some of the common use cases. See also this demo app. Refer to the API docs to familiarize yourself with the complete set of provided functions and options. 3.1 Launching one-off programs If you need to launch an external program, feed it some input and capture its output and maybe also exit status, use exec() or shell(): alias Porcelain.Result %Result{out: output, status: status} = Porcelain.shell("date") IO.inspect status #=> 0 IO.inspect output #=> "Fri Jun 6 14:12:02 EEST 2014\n" result = Porcelain.shell("date cut -b 1-3") IO.inspect result.out #=> "Fri\n" # Use exec() when you want launch a program directly without using a shell File.write!("input.txt", "lines\nread\nfrom\nfile\n") result = Porcelain.exec("sort", ["input.txt"]) IO.inspect result.out #=> "file\nfrom\nlines\nread\n" 3.2 Passing input and getting output Porcelain gives you many options when it comes to interacting with external processes. It is possible to feed input from a file or a stream, same for output: File.write!("input.txt", """ This file contains some patterns >like this< interspersed with other text... >like this< the end. """) Porcelain.exec("grep", [">like this<", "-m", "2"], in: {:path, "input.txt"}, out: {:append, "output.txt"}) IO.inspect File.read!("output.txt") #=> ">like this<\n... >like this< the end.\n" 7

12 3.3 Streams Programs can be spawned asynchronously (using spawn() and spawn_shell()) allowing for continuously exchanging data between Elixir and the external process. In the next example we will use streams for both input and output. alias Porcelain.Process, as: Proc instream = SocketStream.new( example.com, 80) opts = [in: instream, out: :stream] proc = %Proc{out: outstream} = Porcelain.spawn("grep", ["div", "-m", "4"], opts) Enum.into(outstream, IO.stream(:stdio, :line)) # div { # div { # <div> # </div> Proc.alive?(proc) #=> false Alternatively, we could pass the output stream directly to the call to spawn(): opts = [ in: SocketStream.new( example.com, 80), out: IO.stream(:stderr, :line), ] Porcelain.exec("grep", ["div", "-m", "4"], opts) #=> this will be printed to stderr of the running Elixir process: # div { # div { # <div> # </div> The SocketStream module used above wraps a tcp socket in a stream. Its implementation can be found in the test/util/socket_stream.exs file. 3.4 Messages If you prefer to exchange messages with the external process, you can do that: alias Porcelain.Process, as: Proc alias Porcelain.Result proc = %Proc{pid: pid} = Porcelain.spawn_shell("grep ohai -m 2 --line-buffered", in: :receive, out: {:send, self()}) Proc.send_input(proc, "ohai proc\n") receive do {^pid, :data, data} -> IO.inspect data end #=> "ohai proc\n" Proc.send_input(proc, "this won t match\n") Proc.send_input(proc, "ohai") Proc.send_input(proc, "\n") receive do 8 Chapter 3. Usage

13 {^pid, :data, data} -> IO.inspect data #=> "ohai\n" end receive do {^pid, :result, %Result{status: status}} -> IO.inspect status #=> 0 end 3.4. Messages 9

14 10 Chapter 3. Usage

15 CHAPTER 4 Configuring the Goon driver There are a number of options you can tweak to customize the way goon is used. All of the options described below should put into your config.exs file. 4.1 Setting the driver config :porcelain, :driver, <driver> This option allows you to set a particular driver to be used at all times. By default, Porcelain will try to detect the goon executable. If it can find one, it will use Porcelain.Driver.Goon. Otherwise, it will print a warning to stderr and fall back to Porcelain.Driver.Basic. By setting Porcelain.Driver.Basic above you can force Porcelain to always use the basic driver. If you set Porcelain.Driver.Goon, Porcelain will always use the Goon driver and will fail to start if the goon executable can t be found. 4.2 Goon options config :porcelain, :goon_driver_path, <path> Set an absolute path to the goon executable. If this is not set, Porcelain will search your system s PATH by default. 11

16 12 Chapter 4. Configuring the Goon driver

17 CHAPTER 5 Going deeper Take a look at the reference docs for the full description of all provided functions and supported options. 13

18 14 Chapter 5. Going deeper

19 CHAPTER 6 Known issues and roadmap there are known crashes happening when using Porcelain across two nodes error handling when using the Goon driver is not completely shaped out 15

20 16 Chapter 6. Known issues and roadmap

21 CHAPTER 7 Acknowledgements Huge thanks to all who have been test-driving the library in production, in particular to Josh Adams Tim Ruffles 17

22 18 Chapter 7. Acknowledgements

23 CHAPTER 8 License This software is licensed under the MIT license. 19

24 20 Chapter 8. License

25 CHAPTER 9 API Reference 9.1 Porcelain Overview The main module exposing the public API of Porcelain. Basic concepts Functions in this module can either spawn external programs directly (exec/3 and spawn/3) or using a system shell (shell/2 and spawn_shell/2). Functions exec/3 and shell/2 are synchronous (or blocking), meaning they don t return until the external program terminates. Functions spawn/3 and spawn_shell/2 are non-blocking: they immediately return a Porcelain.Process struct and use one of the available ways to exchange input and output with the external process asynchronously. Error handling Using undefined options, passing invalid values to options or any function arguments will fail with a function clause error or Porcelain.UsageError exception. Those are programmer errors and have to be fixed. Any other kinds of runtime errors are reported by returning an error tuple: <reason> is a string explaining the error. {:error, <reason>} where Summary exec/3 Execute a program synchronously reinit/1 Reruns the initialization and updates application env shell/2 Execute a shell invocation synchronously spawn/3 Spawn an external process and return a Porcelain.Process struct to be able to communicate with it spawn_shell/2 Spawn a system shell and execute the command in it 21

26 9.1.3 Functions exec(prog, args, options \\ [])(function) Specs: exec(binary, [binary], Keyword.t/0) :: Porcelain.Result.t Execute a program synchronously. Porcelain will look for the program in PATH and launch it directly, passing the args list as command-line arguments to it. Feeds all input into the program (synchronously or concurrently with reading output; see :async_in option below) and waits for it to terminate. Returns a Porcelain.Result struct containing program s output and exit status code. When no options are passed, the following defaults will be used: [in: "", out: :string, err: nil] This will run the program with no input and will capture its standard output. Available options: :in specify the way input will be passed to the program. Possible values: <iodata> the data is fed into stdin as the sole input for the program <stream> interprets <stream> as a stream of iodata to be fed into the program {:path, <string>} path to a file to be fed into stdin {:file, <file>} <file> is a file descriptor obtained from e.g. File.open; the file will be read from the current position until EOF :async_in can be true or false (default). When enabled, an additional process will be spawned to feed input to the program concurrently with receiving output. :out specify the way output will be passed back to Elixir. Possible values: nil discard the output :string (default) the whole output will be accumulated in memory and returned as one string to the caller :iodata the whole output will be accumulated in memory and returned as iodata to the caller {:path, <string>} the file at path will be created (or truncated) and the output will be written to it {:append, <string>} the output will be appended to the the file at path (it will be created first if needed) {:file, <file>} <file> is a file descriptor obtained from e.g. File.open; the file will be written to starting at the current position <coll> feeds program output (as iodata) into the collectable <coll>. Useful for outputting directly to the console, for example: 22 Chapter 9. API Reference

27 stream = IO.binstream(:standard_io, :line) exec("echo", ["hello", "world"], out: stream) #=> prints "hello\nworld\n" to stdout :err specify the way stderr will be passed back to Elixir. Possible values are the same as for :out. In addition, it accepts the atom :out which denotes redirecting stderr to stdout. Caveat: when using Porcelain.Driver.Basic, the only supported values are nil (stderr will be printed to the terminal) and :out. :dir takes a path that will be used as the directory in which the program will be launched. :env set additional environment variables for the program. The value should be an enumerable with elements of the shape {<key>, <val>} where <key> is an atom or a binary and <val> is a binary or false (meaning removing the corresponding variable from the environment). Basically, it accepts any kind of dict, including keyword lists. reinit(driver \\ nil)(function) Reruns the initialization and updates application env. This function is useful in the following cases: 1.The currently used driver is Goon and the location of the goon executable has changed. 2.You want to change the driver being used. shell(cmd, options \\ [])(function) Specs: shell(binary, Keyword.t/0) :: Porcelain.Result.t Execute a shell invocation synchronously. This function will launch a system shell and pass the invocation to it. This allows using shell features like haining multiple programs with pipes. The downside is that those advanced features may be unavailable on some platforms. It is similar to the exec/3 function in all other respects. spawn(prog, args, options \\ [])(function) Specs: spawn(binary, [binary], Keyword.t/0) :: Porcelain.Process.t Spawn an external process and return a Porcelain.Process struct to be able to communicate with it. You have to explicitly stop the process after reading its output or when it is no longer needed. Use the Porcelain.Process.await/2 function to wait for the process to terminate. Supports all options defined for exec/3 plus some additional ones: in: :receive input is expected to be sent to the process in chunks using the Porcelain.Process.send_input/2 function. :out and :err can choose from a few more values (with the familiar caveat that Porcelain.Driver.Basic does not support them for :err): :stream the corresponding field of the returned Process struct will contain a stream of iodata. Note that the underlying port implementation is message based. This means that the external program will be able to send all of its output to an Elixir process and terminate. The data will be kept in the Elixir process s memory until the stream is consumed Porcelain 23

28 {:send, <pid>} send the output to the process denoted by <pid>. Will send zero or more data messages and will always send one result message in the end. The data messages have the following shape: {<from>, :data, :out :err, <iodata>} where <from> will be the same pid as the one contained in the Process struct returned by this function. The result message has the following shape: {<from>, :result, %Porcelain.Result{} nil} The result will be nil if the :result option that is passed to this function is set to :discard. Note: if both :out and :err are set up to send to the same pid, only one result message will be sent to that pid in the end. :result specify how the result of the external program should be returned after it has terminated. This option has a smart default value. If either :out or :err option is set to :string or :iodata, :result will be set to :keep. Otherwise, it will be set to :discard. Possible values: :keep the result will be kept in memory until requested by calling Porcelain.Process.await/2 or discarded by calling Porcelain.Process.stop/1. :discard discards the result and automatically closes the port after program termination. Useful in combination with out: :stream and err: :stream. spawn_shell(cmd, options \\ [])(function) Specs: spawn_shell(binary, Keyword.t/0) :: Porcelain.Process.t Spawn a system shell and execute the command in it. Works similar to spawn/ Porcelain.Driver.Basic Overview Porcelain driver that offers basic functionality for interacting with external programs. Users are not supposed to call functions in this module directly. Use functions in Porcelain instead. This driver has two major limitations compared to Porcelain.Driver.Goon: the exec function does not work with programs that read all input until EOF before producing any output. Such programs will hang since Erlang ports don t provide any mechanism to indicate the end of input. If a program is continuously consuming input and producing output, it could work with the spawn function, but you ll also have to explicitly close the connection with the external program when you re done with it. sending OS signals to external processes is not supported 24 Chapter 9. API Reference

29 9.3 Porcelain.Driver.Goon Overview Porcelain driver that offers additional features over the basic one. Users are not supposed to call functions in this module directly. Use functions in Porcelain instead. This driver will be used by default if it can locate the external program named goon in the executable path. If goon is not found, Porcelain will fall back to the basic driver. The additional functionality provided by this driver is as follows: ability to signal EOF to the external program send an OS signal to the program (to be implemented) more efficient piping of multiple programs 9.4 Porcelain.Process Overview Module for working with external processes launched with Porcelain.spawn/3 or Porcelain.spawn_shell/ Summary struct /0A struct representing a wrapped OS processes which provides the ability to exchange data with it alive?/1 Check if the process is still running await/2 Wait for the external process to terminate send_input/2send iodata to the process s stdin signal/2 Send an OS signal to the processes stop/1 Stops the process created with Porcelain.spawn/3 or Porcelain.spawn_shell/2. Also closes the underlying port Types t t :: %Porcelain.Process{err: term, out: term, pid: term} signal signal :: :int :kill non_neg_integer Functions struct ()(function) Specs: struct :: %Porcelain.Process{err: term, out: term, pid: term} A struct representing a wrapped OS processes which provides the ability to exchange data with it Porcelain.Driver.Goon 25

30 alive?(process)(function) Specs: alive?(t) :: true false Check if the process is still running. await(process, timeout \\ :infinity)(function) Specs: await(t, non_neg_integer :infinity) :: {:ok, Porcelain.Result.t} {:error, :noproc :timeout} Wait for the external process to terminate. Returns Porcelain.Result struct with the process s exit status and output. Automatically closes the underlying port in this case. If timeout value is specified and the external process fails to terminate before it runs out, atom :timeout is returned. send_input(process, data)(function) Specs: send_input(t, iodata) :: iodata Send iodata to the process s stdin. End of input is indicated by sending an empty message. Caveat: when using Porcelain.Driver.Basic, it is not possible to indicate the end of input. You should stop the process explicitly using stop/1. signal(process, sig)(function) Specs: signal(t, signal) :: signal Send an OS signal to the processes. No further communication with the process is possible after sending it a signal. stop(process)(function) Specs: stop(t) :: true Stops the process created with Porcelain.spawn/3 or Porcelain.spawn_shell/2. Also closes the underlying port. May cause broken pipe message to be written to stderr. 9.5 Porcelain.Result Overview A struct containing the result of running a program after it has terminated Types t t :: %Porcelain.Result{err: term, out: term, status: term} 26 Chapter 9. API Reference

31 9.6 Porcelain.UsageError Overview This exception is meant to indicate programmer errors (misuses of the library API) that have to be fixed prior to release Summary exception/1 message/1 Callback implementation of Exception.exception/1 Callback implementation of Exception.message/ Functions exception(args)(function) Specs: exception(keyword.t/0) :: Exception.t/0 Callback implementation of Exception.exception/1. message(exception)(function) Specs: message(exception.t/0) :: String.t/0 Callback implementation of Exception.message/ Porcelain.UsageError 27

32 28 Chapter 9. API Reference

33 Elixir Module Index p Porcelain, 21 Porcelain.Driver.Basic, 24 Porcelain.Driver.Goon, 25 Porcelain.Process, 25 Porcelain.Result, 26 Porcelain.UsageError, 27 29

CSC209 Review. Yeah! We made it!

CSC209 Review. Yeah! We made it! CSC209 Review Yeah! We made it! 1 CSC209: Software tools Unix files and directories permissions utilities/commands Shell programming quoting wild cards files 2 ... and C programming... C basic syntax functions

More information

System Workers. 1 of 11

System Workers. 1 of 11 System Workers System workers allow JavaScript code to call any external process (a shell command, PHP, etc.) on the same machine. By using callbacks, Wakanda makes it possible to communicate both ways.

More information

CSC209: Software tools. Unix files and directories permissions utilities/commands Shell programming quoting wild cards files

CSC209: Software tools. Unix files and directories permissions utilities/commands Shell programming quoting wild cards files CSC209 Review CSC209: Software tools Unix files and directories permissions utilities/commands Shell programming quoting wild cards files ... and systems programming C basic syntax functions arrays structs

More information

CSC209: Software tools. Unix files and directories permissions utilities/commands Shell programming quoting wild cards files. Compiler vs.

CSC209: Software tools. Unix files and directories permissions utilities/commands Shell programming quoting wild cards files. Compiler vs. CSC209 Review CSC209: Software tools Unix files and directories permissions utilities/commands Shell programming quoting wild cards files... and systems programming C basic syntax functions arrays structs

More information

Unix Processes. What is a Process?

Unix Processes. What is a Process? Unix Processes Process -- program in execution shell spawns a process for each command and terminates it when the command completes Many processes all multiplexed to a single processor (or a small number

More information

SystemWorker.exec( )

SystemWorker.exec( ) 1 28/06/2012 11:18 System workers allow JavaScript code to call any external process (a shell command, PHP, etc.) on the same machine. By using callbacks, Wakanda makes it possible to communicate both

More information

Creating a Shell or Command Interperter Program CSCI411 Lab

Creating a Shell or Command Interperter Program CSCI411 Lab Creating a Shell or Command Interperter Program CSCI411 Lab Adapted from Linux Kernel Projects by Gary Nutt and Operating Systems by Tannenbaum Exercise Goal: You will learn how to write a LINUX shell

More information

Introduction Variables Helper commands Control Flow Constructs Basic Plumbing. Bash Scripting. Alessandro Barenghi

Introduction Variables Helper commands Control Flow Constructs Basic Plumbing. Bash Scripting. Alessandro Barenghi Bash Scripting Alessandro Barenghi Dipartimento di Elettronica, Informazione e Bioingegneria Politecnico di Milano alessandro.barenghi - at - polimi.it April 28, 2015 Introduction The bash command shell

More information

Contents: 1 Basic socket interfaces 3. 2 Servers 7. 3 Launching and Controlling Processes 9. 4 Daemonizing Command Line Programs 11

Contents: 1 Basic socket interfaces 3. 2 Servers 7. 3 Launching and Controlling Processes 9. 4 Daemonizing Command Line Programs 11 nclib Documentation Release 0.7.0 rhelmot Apr 19, 2018 Contents: 1 Basic socket interfaces 3 2 Servers 7 3 Launching and Controlling Processes 9 4 Daemonizing Command Line Programs 11 5 Indices and tables

More information

OS COMPONENTS OVERVIEW OF UNIX FILE I/O. CS124 Operating Systems Fall , Lecture 2

OS COMPONENTS OVERVIEW OF UNIX FILE I/O. CS124 Operating Systems Fall , Lecture 2 OS COMPONENTS OVERVIEW OF UNIX FILE I/O CS124 Operating Systems Fall 2017-2018, Lecture 2 2 Operating System Components (1) Common components of operating systems: Users: Want to solve problems by using

More information

CS246 Spring14 Programming Paradigm Files, Pipes and Redirection

CS246 Spring14 Programming Paradigm Files, Pipes and Redirection 1 Files 1.1 File functions Opening Files : The function fopen opens a file and returns a FILE pointer. FILE *fopen( const char * filename, const char * mode ); The allowed modes for fopen are as follows

More information

Review of Fundamentals

Review of Fundamentals Review of Fundamentals 1 The shell vi General shell review 2 http://teaching.idallen.com/cst8207/14f/notes/120_shell_basics.html The shell is a program that is executed for us automatically when we log

More information

Implementation of a simple shell, xssh

Implementation of a simple shell, xssh Implementation of a simple shell, xssh What is a shell? A process that does command line interpretation Reads a command from standard input (stdin) Executes command corresponding to input line In the simple

More information

Shell Scripting. Todd Kelley CST8207 Todd Kelley 1

Shell Scripting. Todd Kelley CST8207 Todd Kelley 1 Shell Scripting Todd Kelley kelleyt@algonquincollege.com CST8207 Todd Kelley 1 If we have a set of commands that we want to run on a regular basis, we could write a script A script acts as a Linux command,

More information

Cross-platform daemonization tools.

Cross-platform daemonization tools. Cross-platform daemonization tools. Release 0.1.0 Muterra, Inc Sep 14, 2017 Contents 1 What is Daemoniker? 1 1.1 Installing................................................. 1 1.2 Example usage..............................................

More information

Processes. What s s a process? process? A dynamically executing instance of a program. David Morgan

Processes. What s s a process? process? A dynamically executing instance of a program. David Morgan Processes David Morgan What s s a process? process? A dynamically executing instance of a program 1 Constituents of a process its code data various attributes OS needs to manage it OS keeps track of all

More information

Processes in linux. What s s a process? process? A dynamically executing instance of a program. David Morgan. David Morgan

Processes in linux. What s s a process? process? A dynamically executing instance of a program. David Morgan. David Morgan Processes in linux David Morgan What s s a process? process? A dynamically executing instance of a program 1 Constituents of a process its code data various attributes OS needs to manage it OS keeps track

More information

Implementation of a simple shell, xssh

Implementation of a simple shell, xssh Implementation of a simple shell, xssh What is a shell? A process that does command line interpretation Reads a command from standard input (stdin) Executes command corresponding to input line In simple

More information

Review of Fundamentals. Todd Kelley CST8207 Todd Kelley 1

Review of Fundamentals. Todd Kelley CST8207 Todd Kelley 1 Review of Fundamentals Todd Kelley kelleyt@algonquincollege.com CST8207 Todd Kelley 1 GPL the shell SSH (secure shell) the Course Linux Server RTFM vi general shell review 2 These notes are available on

More information

A Small Web Server. Programming II - Elixir Version. Johan Montelius. Spring Term 2018

A Small Web Server. Programming II - Elixir Version. Johan Montelius. Spring Term 2018 A Small Web Server Programming II - Elixir Version Johan Montelius Spring Term 2018 Introduction Your task is to implement a small web server in Elixir. exercise is that you should be able to: The aim

More information

A shell can be used in one of two ways:

A shell can be used in one of two ways: Shell Scripting 1 A shell can be used in one of two ways: A command interpreter, used interactively A programming language, to write shell scripts (your own custom commands) 2 If we have a set of commands

More information

The Actor Model. CSCI 5828: Foundations of Software Engineering Lecture 13 10/04/2016

The Actor Model. CSCI 5828: Foundations of Software Engineering Lecture 13 10/04/2016 The Actor Model CSCI 5828: Foundations of Software Engineering Lecture 13 10/04/2016 1 Goals Introduce the Actor Model of Concurrency isolation, message passing, message patterns Present examples from

More information

Overview of the Ruby Language. By Ron Haley

Overview of the Ruby Language. By Ron Haley Overview of the Ruby Language By Ron Haley Outline Ruby About Ruby Installation Basics Ruby Conventions Arrays and Hashes Symbols Control Structures Regular Expressions Class vs. Module Blocks, Procs,

More information

Process Management! Goals of this Lecture!

Process Management! Goals of this Lecture! Process Management! 1 Goals of this Lecture! Help you learn about:" Creating new processes" Programmatically redirecting stdin, stdout, and stderr" (Appendix) communication between processes via pipes"

More information

CS Programming Languages: Python

CS Programming Languages: Python CS 3101-1 - Programming Languages: Python Lecture 5: Exceptions / Daniel Bauer (bauer@cs.columbia.edu) October 08 2014 Daniel Bauer CS3101-1 Python - 05 - Exceptions / 1/35 Contents Exceptions Daniel Bauer

More information

Standard File Pointers

Standard File Pointers 1 Programming in C Standard File Pointers Assigned to console unless redirected Standard input = stdin Used by scan function Can be redirected: cmd < input-file Standard output = stdout Used by printf

More information

PROGRAMMING PROJECT ONE DEVELOPING A SHELL

PROGRAMMING PROJECT ONE DEVELOPING A SHELL PROGRAMMING PROJECT ONE DEVELOPING A SHELL William Stallings Copyright 2011 Supplement to Operating Systems, Seventh Edition Prentice Hall 2011 ISBN: 013230998X http://williamstallings.com/os/os7e.html

More information

Processes COMPSCI 386

Processes COMPSCI 386 Processes COMPSCI 386 Elements of a Process A process is a program in execution. Distinct processes may be created from the same program, but they are separate execution sequences. call stack heap STACK

More information

Command-line interpreters

Command-line interpreters Command-line interpreters shell Wiki: A command-line interface (CLI) is a means of interaction with a computer program where the user (or client) issues commands to the program in the form of successive

More information

my $full_path = can_run('wget') or warn 'wget is not installed!';

my $full_path = can_run('wget') or warn 'wget is not installed!'; NAME IPC::Cmd - finding and running system commands made easy SYNOPSIS use IPC::Cmd qw[can_run run run_forked]; my $full_path = can_run('wget') or warn 'wget is not installed!'; ### commands can be arrayrefs

More information

Processes, Context Switching, and Scheduling. Kevin Webb Swarthmore College January 30, 2018

Processes, Context Switching, and Scheduling. Kevin Webb Swarthmore College January 30, 2018 Processes, Context Switching, and Scheduling Kevin Webb Swarthmore College January 30, 2018 Today s Goals What is a process to the OS? What are a process s resources and how does it get them? In particular:

More information

Starting the System & Basic Erlang Exercises

Starting the System & Basic Erlang Exercises Starting the System & Basic Erlang Exercises These exercises will help you get accustomed with the Erlang development and run time environments. Once you have set up the Erlang mode for emacs, you will

More information

Bashed One Too Many Times. Features of the Bash Shell St. Louis Unix Users Group Jeff Muse, Jan 14, 2009

Bashed One Too Many Times. Features of the Bash Shell St. Louis Unix Users Group Jeff Muse, Jan 14, 2009 Bashed One Too Many Times Features of the Bash Shell St. Louis Unix Users Group Jeff Muse, Jan 14, 2009 What is a Shell? The shell interprets commands and executes them It provides you with an environment

More information

Process Management 1

Process Management 1 Process Management 1 Goals of this Lecture Help you learn about: Creating new processes Programmatically redirecting stdin, stdout, and stderr (Appendix) communication between processes via pipes Why?

More information

Operating Systems and Networks Project 1: Reliable Transport

Operating Systems and Networks Project 1: Reliable Transport Spring Term 2016 Operating Systems and Networks Project 1: Reliable Transport Assigned on: 22 April 2016 Due by: 13 May 2016 1 Introduction In this project, your task is to implement a reliable sliding

More information

sottotitolo A.A. 2016/17 Federico Reghenzani, Alessandro Barenghi

sottotitolo A.A. 2016/17 Federico Reghenzani, Alessandro Barenghi Titolo presentazione Piattaforme Software per la Rete sottotitolo BASH Scripting Milano, XX mese 20XX A.A. 2016/17, Alessandro Barenghi Outline 1) Introduction to BASH 2) Helper commands 3) Control Flow

More information

SOFTWARE ARCHITECTURE 3. SHELL

SOFTWARE ARCHITECTURE 3. SHELL 1 SOFTWARE ARCHITECTURE 3. SHELL Tatsuya Hagino hagino@sfc.keio.ac.jp slides URL https://vu5.sfc.keio.ac.jp/sa/login.php 2 Software Layer Application Shell Library MIddleware Shell Operating System Hardware

More information

Introduction to the Shell

Introduction to the Shell [Software Development] Introduction to the Shell Davide Balzarotti Eurecom Sophia Antipolis, France What a Linux Desktop Installation looks like What you need Few Words about the Graphic Interface Unlike

More information

System Administration

System Administration Süsteemihaldus MTAT.08.021 System Administration UNIX shell basics Name service DNS 1/69 Command Line Read detailed manual for specific command using UNIX online documentation or so called manual (man)

More information

Chapter 1 - Introduction. September 8, 2016

Chapter 1 - Introduction. September 8, 2016 Chapter 1 - Introduction September 8, 2016 Introduction Overview of Linux/Unix Shells Commands: built-in, aliases, program invocations, alternation and iteration Finding more information: man, info Help

More information

Process Management! Goals of this Lecture!

Process Management! Goals of this Lecture! Process Management! 1 Goals of this Lecture! Help you learn about:" Creating new processes" Programmatically redirecting stdin, stdout, and stderr" Unix system-level functions for I/O" The Unix stream

More information

Process. Program Vs. process. During execution, the process may be in one of the following states

Process. Program Vs. process. During execution, the process may be in one of the following states What is a process? What is process scheduling? What are the common operations on processes? How to conduct process-level communication? How to conduct client-server communication? Process is a program

More information

HW 1: Shell. Contents CS 162. Due: September 18, Getting started 2. 2 Add support for cd and pwd 2. 3 Program execution 2. 4 Path resolution 3

HW 1: Shell. Contents CS 162. Due: September 18, Getting started 2. 2 Add support for cd and pwd 2. 3 Program execution 2. 4 Path resolution 3 CS 162 Due: September 18, 2017 Contents 1 Getting started 2 2 Add support for cd and pwd 2 3 Program execution 2 4 Path resolution 3 5 Input/Output Redirection 3 6 Signal Handling and Terminal Control

More information

NAME SYNOPSIS DESCRIPTION. Behavior of other Perl features in forked pseudo-processes. Perl version documentation - perlfork

NAME SYNOPSIS DESCRIPTION. Behavior of other Perl features in forked pseudo-processes. Perl version documentation - perlfork NAME SYNOPSIS DESCRIPTION perlfork - Perl's fork() emulation NOTE: As of the 5.8.0 release, fork() emulation has considerably matured. However, there are still a few known bugs and differences from real

More information

3. Process Management in xv6

3. Process Management in xv6 Lecture Notes for CS347: Operating Systems Mythili Vutukuru, Department of Computer Science and Engineering, IIT Bombay 3. Process Management in xv6 We begin understanding xv6 process management by looking

More information

Introduction to Asynchronous Programming Fall 2014

Introduction to Asynchronous Programming Fall 2014 CS168 Computer Networks Fonseca Introduction to Asynchronous Programming Fall 2014 Contents 1 Introduction 1 2 The Models 1 3 The Motivation 3 4 Event-Driven Programming 4 5 select() to the rescue 5 1

More information

Distributed Places. Version 6.3. Kevin Tew. November 20, (require racket/place/distributed)

Distributed Places. Version 6.3. Kevin Tew. November 20, (require racket/place/distributed) Distributed Places Version 6.3 Kevin Tew November 20, 2015 (require racket/place/distributed) package: distributed-places-lib See also 20.3 Distributed Places in The Racket Guide. Distributed places support

More information

Pipes and FIFOs. Woo-Yeong Jeong Computer Systems Laboratory Sungkyunkwan University

Pipes and FIFOs. Woo-Yeong Jeong Computer Systems Laboratory Sungkyunkwan University Pipes and FIFOs Woo-Yeong Jeong (wooyeong@csl.skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Open Files in Kernel How the Unix kernel represents open files? Two descriptors

More information

Pebbles Kernel Specification September 26, 2004

Pebbles Kernel Specification September 26, 2004 15-410, Operating System Design & Implementation Pebbles Kernel Specification September 26, 2004 Contents 1 Introduction 2 1.1 Overview...................................... 2 2 User Execution Environment

More information

CS 326: Operating Systems. Process Execution. Lecture 5

CS 326: Operating Systems. Process Execution. Lecture 5 CS 326: Operating Systems Process Execution Lecture 5 Today s Schedule Process Creation Threads Limited Direct Execution Basic Scheduling 2/5/18 CS 326: Operating Systems 2 Today s Schedule Process Creation

More information

Command Interpreters. command-line (e.g. Unix shell) On Unix/Linux, bash has become defacto standard shell.

Command Interpreters. command-line (e.g. Unix shell) On Unix/Linux, bash has become defacto standard shell. Command Interpreters A command interpreter is a program that executes other programs. Aim: allow users to execute the commands provided on a computer system. Command interpreters come in two flavours:

More information

OSE Copyright Ericsson AB. All Rights Reserved. OSE 1.0 June 23, 2014

OSE Copyright Ericsson AB. All Rights Reserved. OSE 1.0 June 23, 2014 OSE Copyright 2014-2014 Ericsson AB. All Rights Reserved. OSE 1.0 June 23, 2014 Copyright 2014-2014 Ericsson AB. All Rights Reserved. The contents of this file are subject to the Erlang Public License,

More information

Assignment 1. Teaching Assistant: Michalis Pachilakis (

Assignment 1. Teaching Assistant: Michalis Pachilakis ( Assignment 1 Teaching Assistant: Michalis Pachilakis ( mipach@csd.uoc.gr) System Calls If a process is running a user program in user mode and needs a system service, such as reading data from a file,

More information

UNIX input and output

UNIX input and output UNIX input and output Disk files In UNIX a disk file is a finite sequence of bytes, usually stored on some nonvolatile medium. Disk files have names, which are called paths. We won t discuss file naming

More information

Error num: 1 Meaning: Not owner Error num: 2 Meaning: No such file or directory Error num: 3 Meaning: No such process Error num: 4 Meaning:

Error num: 1 Meaning: Not owner Error num: 2 Meaning: No such file or directory Error num: 3 Meaning: No such process Error num: 4 Meaning: Error num: 1 Meaning: Not owner Error num: 2 Meaning: No such file or directory Error num: 3 Meaning: No such process Error num: 4 Meaning: Interrupted system call Error num: 5 Meaning: I/O error Error

More information

CSci 4061 Introduction to Operating Systems. Input/Output: High-level

CSci 4061 Introduction to Operating Systems. Input/Output: High-level CSci 4061 Introduction to Operating Systems Input/Output: High-level I/O Topics First, cover high-level I/O Next, talk about low-level device I/O I/O not part of the C language! High-level I/O Hide device

More information

Introduction to Erlang. Franck Petit / Sebastien Tixeuil

Introduction to Erlang. Franck Petit / Sebastien Tixeuil Introduction to Erlang Franck Petit / Sebastien Tixeuil Firstname.Lastname@lip6.fr Hello World % starts a comment. ends a declaration Every function must be in a module one module per source file source

More information

Configuring CSM Scripts

Configuring CSM Scripts CHAPTER 10 This chapter describes how to configure content switching and contains these sections: Configuring TCL Scripts, page 10-1 Configuring Scripts for Health Monitoring Probes, page 10-10 Configuring

More information

Today. Introduction to Computer Systems /18 243, Fall th Lecture. Control Flow. Altering the Control Flow.

Today. Introduction to Computer Systems /18 243, Fall th Lecture. Control Flow. Altering the Control Flow. Today Introduction to Computer Systems 15 213/18 243, Fall 2009 11 th Lecture Exceptional Control Flow Processes Instructors: Greg Ganger and Roger Dannenberg Control Flow Processors do only one thing:

More information

Processes. Johan Montelius KTH

Processes. Johan Montelius KTH Processes Johan Montelius KTH 2017 1 / 47 A process What is a process?... a computation a program i.e. a sequence of operations a set of data structures a set of registers means to interact with other

More information

Table of contents. Our goal. Notes. Notes. Notes. Summer June 29, Our goal is to see how we can use Unix as a tool for developing programs

Table of contents. Our goal. Notes. Notes. Notes. Summer June 29, Our goal is to see how we can use Unix as a tool for developing programs Summer 2010 Department of Computer Science and Engineering York University Toronto June 29, 2010 1 / 36 Table of contents 1 2 3 4 2 / 36 Our goal Our goal is to see how we can use Unix as a tool for developing

More information

Maria Hybinette, UGA. ! One easy way to communicate is to use files. ! File descriptors. 3 Maria Hybinette, UGA. ! Simple example: who sort

Maria Hybinette, UGA. ! One easy way to communicate is to use files. ! File descriptors. 3 Maria Hybinette, UGA. ! Simple example: who sort Two Communicating Processes Hello Gunnar CSCI 6730/ 4730 Operating Systems Process Chat Maria A Hi Nice to Hear from you Process Chat Gunnar B Dup & Concept that we want to implement 2 On the path to communication

More information

Standard C Library Functions

Standard C Library Functions Demo lecture slides Although I will not usually give slides for demo lectures, the first two demo lectures involve practice with things which you should really know from G51PRG Since I covered much of

More information

A process. the stack

A process. the stack A process Processes Johan Montelius What is a process?... a computation KTH 2017 a program i.e. a sequence of operations a set of data structures a set of registers means to interact with other processes

More information

Project 1: Implementing a Shell

Project 1: Implementing a Shell Assigned: August 28, 2015, 12:20am Due: September 21, 2015, 11:59:59pm Project 1: Implementing a Shell Purpose The purpose of this project is to familiarize you with the mechanics of process control through

More information

Inter-Process Communications (IPC)

Inter-Process Communications (IPC) ICS332 Operating Systems Fall 2017 Communicating Processes? So far we have seen independent processes Each process runs code independently Parents and aware of their children, and children are aware of

More information

ffmpy3 Documentation Release Eric Ahn

ffmpy3 Documentation Release Eric Ahn ffmpy3 Documentation Release 0.2.3 Eric Ahn February 03, 2017 Contents 1 Installation 3 2 Quickstart 5 3 Documentation 7 3.1 ffmpy3.................................................. 7 3.2 Examples.................................................

More information

Altering the Control Flow

Altering the Control Flow Altering the Control Flow Up to Now: two mechanisms for changing control flow: Jumps and branches Call and return using the stack discipline. Both react to changes in program state. Insufficient for a

More information

Lecture 8: Structs & File I/O

Lecture 8: Structs & File I/O ....... \ \ \ / / / / \ \ \ \ / \ / \ \ \ V /,----' / ^ \ \.--..--. / ^ \ `--- ----` / ^ \. ` > < / /_\ \. ` / /_\ \ / /_\ \ `--' \ /. \ `----. / \ \ '--' '--' / \ / \ \ / \ / / \ \ (_ ) \ (_ ) / / \ \

More information

Project 2: User Programs

Project 2: User Programs Project 2: User Programs CS140 - Winter 2010 Slides by Andrew He, adapted from previous CS140 offerings Overview Project 2 is due Thursday, February 4 This project requires an understanding of: How user

More information

2.3 Unix Streaming and Piping

2.3 Unix Streaming and Piping 2.3 Unix Streaming and Piping In addition to streams explicitly opened by applications, the Unix system provides you with 3 special streams: stdin (standard input): This stream is usually connected to

More information

Piqi-RPC. Exposing Erlang services via JSON, XML and Google Protocol Buffers over HTTP. Friday, March 25, 2011

Piqi-RPC. Exposing Erlang services via JSON, XML and Google Protocol Buffers over HTTP. Friday, March 25, 2011 Piqi-RPC Exposing Erlang services via JSON, XML and Google Protocol Buffers over HTTP 1 Anton Lavrik http://piqi.org http://www.alertlogic.com 2 Overview Call Erlang functions using HTTP POST : Server

More information

Consider the following program.

Consider the following program. Consider the following program. #include int do_sth (char *s); main(){ char arr [] = "We are the World"; printf ("%d\n", do_sth(arr)); } int do_sth(char *s) { char *p = s; while ( *s++!= \0 )

More information

Debugging (Part 1) The material for this lecture is drawn, in part, from The Practice of Programming (Kernighan & Pike) Chapter 5

Debugging (Part 1) The material for this lecture is drawn, in part, from The Practice of Programming (Kernighan & Pike) Chapter 5 Debugging (Part 1) The material for this lecture is drawn, in part, from The Practice of Programming (Kernighan & Pike) Chapter 5 1 For Your Amusement When debugging, novices insert corrective code; experts

More information

File and Console I/O. CS449 Fall 2017

File and Console I/O. CS449 Fall 2017 File and Console I/O CS449 Fall 2017 What is a Unix(or Linux) File? Narrow sense: a resource provided by OS for storing informalon based on some kind of durable storage (e.g. HDD, SSD, DVD, ) Wide (UNIX)

More information

Remote Invocation. Today. Next time. l Overlay networks and P2P. l Request-reply, RPC, RMI

Remote Invocation. Today. Next time. l Overlay networks and P2P. l Request-reply, RPC, RMI Remote Invocation Today l Request-reply, RPC, RMI Next time l Overlay networks and P2P Types of communication " Persistent or transient Persistent A submitted message is stored until delivered Transient

More information

The Process Abstraction. CMPU 334 Operating Systems Jason Waterman

The Process Abstraction. CMPU 334 Operating Systems Jason Waterman The Process Abstraction CMPU 334 Operating Systems Jason Waterman How to Provide the Illusion of Many CPUs? Goal: run N processes at once even though there are M CPUs N >> M CPU virtualizing The OS can

More information

Python in 10 (50) minutes

Python in 10 (50) minutes Python in 10 (50) minutes https://www.stavros.io/tutorials/python/ Python for Microcontrollers Getting started with MicroPython Donald Norris, McGrawHill (2017) Python is strongly typed (i.e. types are

More information

File and Console I/O. CS449 Spring 2016

File and Console I/O. CS449 Spring 2016 File and Console I/O CS449 Spring 2016 What is a Unix(or Linux) File? File: a resource for storing information [sic] based on some kind of durable storage (Wikipedia) Wider sense: In Unix, everything is

More information

Altering the Control Flow

Altering the Control Flow Altering the Control Flow Up to Now: two mechanisms for changing control flow: Jumps and branches Call and return using the stack discipline. Both react to changes in program state. Insufficient for a

More information

NAME SYNOPSIS DESCRIPTION. Behavior of other Perl features in forked pseudo-processes. Perl version documentation - perlfork

NAME SYNOPSIS DESCRIPTION. Behavior of other Perl features in forked pseudo-processes. Perl version documentation - perlfork NAME SYNOPSIS DESCRIPTION perlfork - Perl's fork() emulation NOTE: As of the 5.8.0 release, fork() emulation has considerably matured. However, there are still a few known bugs and differences from real

More information

Useful Unix Commands Cheat Sheet

Useful Unix Commands Cheat Sheet Useful Unix Commands Cheat Sheet The Chinese University of Hong Kong SIGSC Training (Fall 2016) FILE AND DIRECTORY pwd Return path to current directory. ls List directories and files here. ls dir List

More information

Lab 2: Linux/Unix shell

Lab 2: Linux/Unix shell Lab 2: Linux/Unix shell Comp Sci 1585 Data Structures Lab: Tools for Computer Scientists Outline 1 2 3 4 5 6 7 What is a shell? What is a shell? login is a program that logs users in to a computer. When

More information

Process management. What s in a process? What is a process? The OS s process namespace. A process s address space (idealized)

Process management. What s in a process? What is a process? The OS s process namespace. A process s address space (idealized) Process management CSE 451: Operating Systems Spring 2012 Module 4 Processes Ed Lazowska lazowska@cs.washington.edu Allen Center 570 This module begins a series of topics on processes, threads, and synchronization

More information

bistro Documentation Release dev Philippe Veber

bistro Documentation Release dev Philippe Veber bistro Documentation Release dev Philippe Veber Oct 10, 2018 Contents 1 Getting started 1 1.1 Installation................................................ 1 1.2 A simple example............................................

More information

Rudy: a small web server. Johan Montelius. October 2, 2016

Rudy: a small web server. Johan Montelius. October 2, 2016 Rudy: a small web server Johan Montelius October 2, 2016 Introduction Your task is to implement a small web server in Erlang. The aim of this exercise is that you should be able to: describe the procedures

More information

CSE 451: Operating Systems Winter Module 4 Processes. Mark Zbikowski Allen Center 476

CSE 451: Operating Systems Winter Module 4 Processes. Mark Zbikowski Allen Center 476 CSE 451: Operating Systems Winter 2015 Module 4 Processes Mark Zbikowski mzbik@cs.washington.edu Allen Center 476 2013 Gribble, Lazowska, Levy, Zahorjan Process management This module begins a series of

More information

System Programming. Introduction to Unix

System Programming. Introduction to Unix Content : by Dr. B. Boufama School of Computer Science University of Windsor Instructor: Dr. A. Habed adlane@cs.uwindsor.ca http://cs.uwindsor.ca/ adlane/60-256 Content Content 1 Introduction 2 3 Introduction

More information

The Classical OS Model in Unix

The Classical OS Model in Unix The Classical OS Model in Unix Nachos Exec/Exit/Join Example Exec parent Join Exec child Exit SpaceID pid = Exec( myprogram, 0); Create a new process running the program myprogram. int status = Join(pid);

More information

CSE 390a Lecture 2. Exploring Shell Commands, Streams, Redirection, and Processes

CSE 390a Lecture 2. Exploring Shell Commands, Streams, Redirection, and Processes CSE 390a Lecture 2 Exploring Shell Commands, Streams, Redirection, and Processes slides created by Marty Stepp, modified by Jessica Miller & Ruth Anderson http://www.cs.washington.edu/390a/ 1 2 Lecture

More information

Windows architecture. user. mode. Env. subsystems. Executive. Device drivers Kernel. kernel. mode HAL. Hardware. Process B. Process C.

Windows architecture. user. mode. Env. subsystems. Executive. Device drivers Kernel. kernel. mode HAL. Hardware. Process B. Process C. Structure Unix architecture users Functions of the System tools (shell, editors, compilers, ) standard library System call Standard library (printf, fork, ) OS kernel: processes, memory management, file

More information

OstrichLib Documentation

OstrichLib Documentation OstrichLib Documentation Release 0.0.0 Itamar Ostricher May 10, 2016 Contents 1 utils package 3 1.1 collections utils module......................................... 3 1.2 path utils module.............................................

More information

Last class: Today: Thread Background. Thread Systems

Last class: Today: Thread Background. Thread Systems 1 Last class: Thread Background Today: Thread Systems 2 Threading Systems 3 What kind of problems would you solve with threads? Imagine you are building a web server You could allocate a pool of threads,

More information

Review of Fundamentals. Todd Kelley CST8207 Todd Kelley 1

Review of Fundamentals. Todd Kelley CST8207 Todd Kelley 1 Review of Fundamentals Todd Kelley kelleyt@algonquincollege.com CST8207 Todd Kelley 1 The CST8207 course notes GPL the shell SSH (secure shell) the Course Linux Server RTFM vi general shell review 2 Linux

More information

CSE 410: Computer Systems Spring Processes. John Zahorjan Allen Center 534

CSE 410: Computer Systems Spring Processes. John Zahorjan Allen Center 534 CSE 410: Computer Systems Spring 2018 Processes John Zahorjan zahorjan@cs.washington.edu Allen Center 534 1. What is a process? Processes 2. What's the process namespace? 3. How are processes represented

More information

Essential Unix and Linux! Perl for Bioinformatics, ! F. Pineda

Essential Unix and Linux! Perl for Bioinformatics, ! F. Pineda Essential Unix and Linux! Perl for Bioinformatics, 140.636! F. Pineda Generic computer architecture Memory Storage Fig. 1.2 From Designing Embedded Hardware, 2 nd Ed. by John Catsoulis OS concepts Shell

More information

Linux shell scripting Getting started *

Linux shell scripting Getting started * Linux shell scripting Getting started * David Morgan *based on chapter by the same name in Classic Shell Scripting by Robbins and Beebe What s s a script? text file containing commands executed as a unit

More information

IP address. Subnetting. Ping command in detail. Threads in Python. Sub process in Python

IP address. Subnetting. Ping command in detail. Threads in Python. Sub process in Python IP address Subnetting Ping command in detail Threads in Python Sub process in Python IPv4 Addressing- Introduction An IP address is a 32-bit address that uniquely and universally defines the connection

More information

Systems Programming. 08. Standard I/O Library. Alexander Holupirek

Systems Programming. 08. Standard I/O Library. Alexander Holupirek Systems Programming 08. Standard I/O Library Alexander Holupirek Database and Information Systems Group Department of Computer & Information Science University of Konstanz Summer Term 2008 Last lecture:

More information

Parallelism, Concurrency, and Asynchrony in Perl 6. Jonathan Worthington

Parallelism, Concurrency, and Asynchrony in Perl 6. Jonathan Worthington Parallelism, Concurrency, and Asynchrony in Perl 6 Jonathan Worthington Hi! I'm Jonathan. Lead developer of Rakudo Perl 6 Founder and architect of MoarVM Work as a software architect and teacher Live in

More information