matross Release SNAPSHOT

Transcription

1 matross Release SNAPSHOT June 01, 2014

2

3 Contents 1 Getting Started What is matross? Who is matross for? How does matross compare to? How do I use matross? Design simple automation simple to extend reusable and composable flexible Task Plugins command script template file temp-file stream-to-file stream-from-file Connection Plugins Local SSH Vagrant Contributing Writing Task Plugins Writing Connection Plugins i

4 ii

5 remote process gun Contents 1

6 2 Contents

7 CHAPTER 1 Getting Started 1.1 What is matross? Matross is a process automation utility that makes it easy to do work on local or remote machines. 1.2 Who is matross for? People who need to run processes in an automated way. 1.3 How does matross compare to? Matross loves you and cares about you. Matross wants you to relax and get some sleep. Matross wants you to snuggle up with a good book and a glass of scotch and enjoy life. Matross wants to see you go outside and enjoy the sun and not worry about computers. Other process automation utilities are sadistic. They want to see you cry. They lure you in with a sense of simplicity and ease of use only to turn around and force their psychotic tendencies on you. They draw you in and then bind your hands and try to shove you into some multi-layered abstraction that leaks everywhere all over your nice new clothes. Matross just wants to help. 1.4 How do I use matross? Matross is currently pre-alpha software. If you are stubborn and want to use it in spite of this see the contribution guide. 3

8 4 Chapter 1. Getting Started

9 CHAPTER 2 Design Quick and dirty high level architecture & design goals, please patch this!! Make something that is actually useful. Don t compromise simplicity for convenience. Make automation natural and complex things possible. Simple means single responsiblity, without interleaving or overlapping parts. 2.1 simple automation reference based template system immutable, always available data scalable configuration (1 machine or 1k machines) 2.2 simple to extend plugin system designed around protocols 2.3 reusable and composable plugins configurations operations 2.4 flexible with or without sudo able to work on any type of connection agented or agentless Keep in mind this is not a feature list but instead a general direction. Change as you see fit. 5

10 6 Chapter 2. Design

11 CHAPTER 3 Task Plugins Tasks are matross main unit of coordination across connections. Using tasks you can execute specific commands over a remote connection and ensure the system is in a desired state. The core set of task plugins include: 3.1 command Execute a shell command on the target machine in a normalized environment. For example, {:command "echo $VAR" :env {:VAR "herp"}} is equivalent to running /usr/bin/env -i VAR=herp /bin/sh -c echo $VAR on the target machine Example {:command "echo $message", :env {:message "hello, world!"}} Configuration :command command to run :env map containing shell environment (optional) - default: {} :shell path to shell to use (optional) - default: /bin/sh Resources Task Source 3.2 script Execute a script :file or :inline content on the target machine, optionally (by default) rendering it as a mustache template prior to execution. See also matross.tasks.command 7

12 3.2.1 Example {:file "hello.sh.mustache", :type :script, :vars {:cool "script, yo"}} Configuration :file a file to render :template whether or not to template the script - default: true :env environment variables to expose to the script - default: {} :inline a string to render (optional, instead of :file) :vars map of vars for templating (optional) - default: {} Resources Task Source 3.3 template Render the :file or :inline content as a mustache template and write the result to the :dest file on the target machine Example {:type :template, :dest "~/wow", :inline "{{ example }} is cool!", :vars {:example "matross"}} Configuration :file a local template file to render :dest where to put the templated content :inline string to render (optional, instead of file) :vars for template substitution Resources Task Source 3.4 file Coerces files on the remote filesystem 8 Chapter 3. Task Plugins

13 3.4.1 Example {:path "/tmp/sensitive-file", :type :file, :state :absent} {:path "/etc/db/required-directory", :group "db", :mode 750, :type :file, :state :directory, :owner " Configuration :path remote file path :group file group :mode unix file permissions :state one of: :file, :directory, :absent - default: :file :owner file owner Resources Task Source 3.5 temp-file Creates a temporary file on the target machine and returns the file path in :out Example {:temp-dir "/my/other/tmp", :type :temp-file} ;; you can also use the with-temp-files conveneince macro (with-temp-files [tmp-a tmp-b] (use-temp-files tmp-a tmp-b)) Configuration :temp-dir Remote temporary directory (must already exist) - default: /tmp Resources Task Source 3.6 stream-to-file Write the contents of the given source input to the desired file on the target machine temp-file 9

14 3.6.1 Example {:type :stream-to-file, :src "alias git=wow", :dest "~/.bash_aliases"} Configuration :src source input, string or stream :dest remote filepath Resources Task Source 3.7 stream-from-file Low level task to stream the contents of a file on the target machine to the given writer locally Example {:type :stream-from-file, :src "~/.bashrc", :dest my-input-stream} Configuration :src remote file path :dest destination writer Resources Task Source 10 Chapter 3. Task Plugins

15 CHAPTER 4 Connection Plugins Connections are what matross uses to control remote machines. A connection can be any sort of system you can connect to and execute commands. This could be local, ssh, a virtual container, or anything else you might think to implement. The core set of connection plugins include: 4.1 Local The local connection allows you to run tasks against the local machine Example {:type :local} Configuration None Resources Connection Source 4.2 SSH The SSH connection lets you connect to remote systems over the secure shell protocol Example {:type :ssh :hostname "example.com" :port 2222} 11

16 4.2.2 Configuration :hostname The hostname of the remote machine to connect to. :port Defaults to 22. :username Defaults to current username. :private-key-path By default will use ~/.ssh/id_rsa :public-key-path By default will use ~/.ssh/id_rsa.pub In theory this also supports all options that can be passed to ssh -o, but this has not been tested Resources Connection Source 4.3 Vagrant The vagrant connection allows you to run tasks against a vagrant box over SSH. Uses vagrant ssh-config <box-name> to query information about the vagrant box Example {:type :vagrant :name "ubuntu-14.04"} Configuration :name Name of the vagrant box. Defaults to default (same as vagrant) Resources Connection Source 12 Chapter 4. Connection Plugins

17 CHAPTER 5 Contributing Matross is free software, everyone is encouraged to help make it better. Matross should be simple and pleasant to use, anything else is unexpected behavior and should be reported and fixed. Here are a few specific ways that you could help: use early or unreleased versions report bugs or unexpected behavior suggest new features write or edit documentation write or refactor code (no patch is too small) review patches file or resolve issues Matross is designed to be extended. 5.1 Writing Task Plugins Tasks are configurable operations that execute on machines. They are provided a remote connection and a usersupplied config. With these they can chain together any number of other internal tasks and remote processes Defining tasks The deftask macro is a convenient way to implement the required protocols and multimethods required to hook into the matross plugin system. (deftask :my-task [conn config] ;; (run conn commands...) (task-result task-successful? changed? result-map)) This will register the task :my-task for use with matross. Your task may take any configuration and return any result-map. 13

18 5.1.2 Using your task {:type :my-task :my-arg "hello, world"} This map will be provided instances of :my-task as config. External Resources Check out the source code for futher information. Task Interface Template Task 5.2 Writing Connection Plugins Connections are core to the design and operation of matross. Using a simple process interface matross can connect to a remote system and execute commands. Built in connection types include local and ssh connections, but the system is designed to be extended Connection Interface Matross connections must implement two lifecycle functions: ; IConnect (connect [self] "connect to a machine") (disconnect [self] "disconnect from a machine") And implement a single entry point: ; IRun (run [self process] "execute a command on a machine") Process Model The contract of implementing run for a connection is that given a process specification as input, a representation of a running process will be returned. Above process is a map containing the keys :cmd and optionally :in. The command is a sequence of a program to run and any number of positional arguments. ;; list representing a command to be run ["/bin/sh" "-c" "seq 10 tail -n 50"] A processes :in should be either a String or an InputStream. {:in "hello world" :cmd ["cat"]} The result of executing run should be a representation of a process: specifically two streams, stdout & stderr, and a future of the process exit status. { :out InputStream :err InputStream :exit FutureExitCode } 14 Chapter 5. Contributing

19 5.2.3 Registering Connections In order to register with the matross runner a connection must implement the get-connection multimethod. ; matross.connections.core/get-connection (defmethod get-connection :my-type [connection-spec] (build-my-connection connection-spec)) The registered function is expected to return something that satisfies both the IConnect and IRun protocols. The input is a user-provided map representing the connection, the only hard requirement being that the :type of the map matches your dispatch key (:my-type in the example above). ;; example connection spec { :type :docker :image "ubuntu" } The docker connection type doesn t exist, you should implement it! External Resources Check out the source code for futher information. Connection Interface Local Connection 5.2. Writing Connection Plugins 15