Dodo Commands Documentation

Transcription

1 Dodo Commands Documentation Release Maarten Nieber Apr 04, 2019

2

3 Contents 1 Goals of the Dodo Commands system Provide a per-project environment Single entry point to run a variety of command scripts Give access to the configuration of the current project Flexible configurations Show what each command script does Share commands easily, while allowing customization Run commands in a docker container Install dependencies automatically Installation Step 1: Install software Step 2: (Optional) Extend the path Step 3: (Optional) Tweak global configuration Step 4: (Optional) Install extra default commands Step 5: (Optional) Enable autocompletion Step 6: (Optional) Activate the latest project automatically Upgrading The basics of a dodo commands script Default commands and the command path The hello-world command script Explanation Creating and using projects Creating a project Running a command Activating a project Activating the last used project Configuration files Configuration files Using configuration values in scripts How to use layers Including bits of configuration from packages Preserving the configuration history The global configuration file i

4 6 Commands Command path Default commands Excluding directories from the command path Importing symbols from another module Specifying command dependencies in the.meta file Aliases The Dodo singleton The is_main function The get_config function The parse_args function ( confirm and echo) Config arguments The run function Marking a script as unsafe Decorators Constructing a tree Prepending an argument Appending an argument Mapping decorators to commands Printing arguments Sharing projects Objectives Bootstrapping Checking the config version Starting a project from a cookiecutter template Symlinking to a local src directory (useful with monolithic repositories) The diagnose command Jinja2 documentation templates Diagnose dodo configuration The dodo_expand jinja2 filter Indices and tables 33 ii

5 Source code: Contents: Contents 1

6 2 Contents

7 CHAPTER 1 Goals of the Dodo Commands system The main goal is to associate a set of short commands with each project to automate frequent tasks. The following list gives a general overview of how this achieved. We also encourage you to try the tutorial on this page. 1.1 Provide a per-project environment Each Dodo Commands project contains a specific set of commands and configuration values. It s also possible to install default commands that will be available in any project. 1.2 Single entry point to run a variety of command scripts Dodo Commands finds command scripts for you in the project s search path, so there is no need to remember where they are located. This means that commands work from any current working directory. Running dodo help shows the list of available commands. 1.3 Give access to the configuration of the current project Dodo Commands fetches arguments from the current project s configuration file. However, arguments not found in the configuration file can still be obtained from the command line. 1.4 Flexible configurations Configuration files can be layered, and layers can be switched on or off, giving you a lot of flexibility. Just like excel sheets, configuration files may contain cross-references. The power of Dodo Commands is mostly in the configuration file. 3

8 1.5 Show what each command script does In some cases you don t want to run a script blindly. Running dodo foo --bar --confirm makes the foo.py script print each command line call that it executes, and ask for confirmation. This allows you to see exactly what the script does. 1.6 Share commands easily, while allowing customization You can synchronize your local project configuration with a shared one. This works by cherry-picking the parts you need. New colleagues can initialize their project by boot-strapping from the shared configuration. 1.7 Run commands in a docker container You can tell Dodo Commands to prefix particular commands with a Docker invocation. The appropriate volume mappings, environment variables, etc come from the configuration file. 1.8 Install dependencies automatically By specifying the dependencies of a command script in a <script-name>.meta file, missing Python packages can be automatically installed into the virtualenv of the current Dodo Commands project. 4 Chapter 1. Goals of the Dodo Commands system

9 CHAPTER 2 Installation 2.1 Step 1: Install software Dodo Commands depends on the python-virtualenv package. In addition, some of the Dodo commands use git. # install prerequisites sudo apt-get install python-virtualenv git # install dodo commands sudo pip install dodo_commands Tip: On Mac you may need to create a file ~/.distutils.cfg that sets an empty prefix to prevent errors stating must supply either home or prefix/exec-prefix not both. 2.2 Step 2: (Optional) Extend the path Note: This step is only necessary if you installed dodo_commands into a virtual environment. To make sure that the dodo command is always found, call which dodo and add the resulting path (the directory part) to the PATH in in your ~/.bashrc. The result should look something like this: export PATH=$PATH:/home/maarten/projects/dodo_commands_env/bin 5

10 2.3 Step 3: (Optional) Tweak global configuration The first time you call dodo, a global ~/.dodo_commands/config file is created (unless it already exists) with the following settings: projects_dir is the location where your projects are stored (defaults to ~/projects) python is the python interpreter that is used in the virtualenv of your projects (defaults to python). If your OS uses Python 2 by default then you may want to set this to python3 to use the latest python. diff_tool is the diff tool used to show changes to your project configuration files. It s recommended to install and use meld for this option: dodo global-config settings.diff_tool meld 2.4 Step 4: (Optional) Install extra default commands The dodo_standard_commands directory is added by default to the ~/.dodo_commands/ default_commands directory where default commands are installed. To install additional commands into this directory, you can run dodo install-default-commands --pip dodo_webdev_commands --pip dodo_git_commands -- pip dodo_deploy_commands 2.5 Step 5: (Optional) Enable autocompletion The dodo register-autocomplete command prints a line that - when executed - installs an autocompletion script in /etc/bash_completion.d. You can paste this line into a bash-console, or run it directly with $(dodo register-autocomplete). # returns: sudo register-python-argcomplete dodo > /etc/bash_completion.d/dodo_ autocomplete.sh dodo register-autocomplete 2.6 Step 6: (Optional) Activate the latest project automatically To automatically activate the last used Dodo Commands project, add this line to your ~/.bashrc file: $(dodo activate --latest) If you wish to be able to toggle the automatic activation on and off, read about Activating the last used project. 2.7 Upgrading To upgrade Dodo Commands, you need to upgrade its pip package. If you have activated a Dodo Commands project with $(dodo activate foo) then the first step is to deactivate it by typing deactivate. This is necessary because we don t want to use the pip from the virtual environment. 6 Chapter 2. Installation

11 Now, to upgrade, simply run pip install --upgrade dodo_commands. To upgrade a commands package (for example: dodo_git_commands), simply install it again using dodo install-default-commands --pip dodo_git_commands Upgrading 7

12 8 Chapter 2. Installation

13 CHAPTER 3 The basics of a dodo commands script In this section we ll explain how to run default commands, and discuss the basic outline of a Dodo Commands script. This should give you an idea of how to automate any task. Later we will expand on this information, and show how to organize your commands into different projects. 3.1 Default commands and the command path The recommended way of using Dodo Commands is to create a Dodo Commands project and write a configuration file. However, it s possible to run commands without having any project. In this case, Dodo Commands will provide a fallback configuration that gives access to all commands in the default commands directory. If you inspect this fallback configuration by running dodo print-config, you will see that it tells Dodo Commands which commands are available via the /ROOT/command_path setting. To list all available commands, simply run dodo. 3.2 The hello-world command script The example we will use is a hello-world.py script which parses some command line arguments and prints a text: # hello-world.py from argparse import ArgumentParser from dodo_commands.framework import Dodo def _args(): # noqa parser = ArgumentParser(description=('Print hello world.')) args = Dodo.parse_args(parser) # 1 args.project_name = Dodo.get_config('/ROOT/project_name', 'no project') # 2 return args (continues on next page) 9

14 if Dodo.is_main( name ): # 3 args = _args() Dodo.run(['echo', 'Hello world']) # 4 (continued from previous page) Assuming that dodo_commands is installed, you can run this simply as a Python script: python hello-world.py In later sections, we will explain how to extend the command path so that you will be able to run it with dodo hello-world. 3.3 Explanation The lines that require special attention are numbered with a comment: 1. We use Dodo.parse_args(parser) instead of parser.parse_args() to give Dodo Commands the chance to add some extra arguments to your script, such as the --confirm flag (which is explained below). 2. Through the Dodo singleton, every command has access to the configuration values of the current project. The value no project is a fallback in case there is no current project. 3. This line is a variation on the usual if name == ' main '. It allows you to execute the script when invoking it with dodo hello-world. 4. The purpose of Dodo.run is to execute a command while giving Dodo Commands a chance to intercept the call and do something different. Important use cases are the --confirm flag (which will ask for confirmation before proceeding) and the --echo flag (which will print the command without doing anything). Note that it s encouraged to make your Dodo Commands script act like a shell alias. When users see the script as a way to avoid typing, rather than as a magical invocation, they are more likely to use it. 10 Chapter 3. The basics of a dodo commands script

15 CHAPTER 4 Creating and using projects We ve seen previously that the dodo executable can be used to execute any default command. However, the power of this framework comes from using project specific configuration files. This requires creating a project, as explained below. 4.1 Creating a project Projects are created with the dodo activate command: dodo activate FooBar create Running this command creates a new ~/projects/foobar directory with various files, such as the configuration file. The most important file however is ~/projects/foobar/dodo_commands/env/bin/dodo. This copy of the dodo script is specific to the created project. Calling it implies that you are operating on the FooBar project. 4.2 Running a command You run a command on a project by calling its dodo script with the name of the command: ~/projects/foobar/dodo_commands/env/bin/dodo foo --bar This will execute the following steps: 1. the dodo script will read the command_path from the project s configuration file (~/projects/foobar/ dodo_commands/res/config.yaml). The command path is a list of directories with command scripts. To inspect your project s command path, run dodo print-config /ROOT/command_path. 2. the foo.py script is found on the command_path and run with the bar option 11

16 4.3 Activating a project The ~/projects/foobar/dodo_commands/env/bin/dodo script is part of a virtual python environment that was installed when we called dodo activate FooBar --create. To avoid having to type the full path to dodo, you can first activate this python environment with: source ~/projects/foobar/dodo_commands/env/bin/activate Running dodo foo --bar will now use the project-specific dodo script, rather than the system one. The shortcut to activating the python environment is: $(dodo activate FooBar) Note that if you omit the $() part then dodo activate prints the line source ~/projects/foobar/ dodo_commands/env/bin/activate. The $(...) syntax takes care of executing this in the shell. To create a new project and activate it at the same time, call: $(dodo activate FooBar --create) Finally, if you activated a project and you want to go back to the previously active project, run $(dodo activate -). 4.4 Activating the last used project You can activate the last used project with $(dodo activate --latest). To perform this action automatically when you open a terminal, you can call dodo autostart on. This writes a small script file to ~/. dodo_commands_autostart. dodo autostart on Inspecting the result with cat ~/.dodo_commands_autostart will print $(dodo activate --latest) dodo check-config-version dodo check-version Then, add the following lines to your ~/.bashrc to execute this script when a terminal is opened: if [ -f ~/.dodo_commands_autostart ]; then. ~/.dodo_commands_autostart fi If you want to disable the autostart behaviour, call dodo autostart off. This will delete the ~/. dodo_commands_autostart file, and therefore disable the automatic project activation. 12 Chapter 4. Creating and using projects

17 CHAPTER 5 Configuration files After activating a project such as FooBar with $(dodo activate FooBar), the project s configuration file is accessible to any command that you execute. This chapter explains how to store values in the configuration file and use them in a command script. 5.1 Configuration files A new configuration file is created automatically when the project is first created (e.g. ~/projects/foobar/ dodo_commands/res/config.yaml). You can print the configuration file with dodo print-config. The configuration file uses the yaml format, with a few extra rules: 1. environment variables (such as $PATH) in values are expanded automatically 2. values may reference other values in the configuration file: BUILD: nr_of_threads: 2 FOO: bar: ${/BUILD/nr_of_threads} # value will be the number 2 ${/BUILD/nr_of_threads}_foo: baz # key will be the string "2_foo" 3. the following magic values are automatically added: ${/ROOT/project_name}, ${/ROOT/ project_dir}, ${/ROOT/res_dir}. Finally the dodo_system_commands directory is automatically added to ${/ROOT/command_path}. 4. ${/ROOT/layers} lists additional yaml files that are layered on top of the root configuration file. If a key exists both in the root configuration and in the layer, then values replace values, lists are concatenated, and dictionaries are merged. The ${/ROOT/layer_dir} value can be used to specify the sub-directory where layer configuration files are searched. However, layers may also be prefixed with an absolute path. Finally, wildcards are allowed. ROOT: layer_dir: layers (continues on next page) 13

18 layers: # contents of this file are layered on top of this configuration - buildtype.debug.yaml # layer with an absolute path - ~/.dodo_commands/default_layer.yaml # example of using wildcards - ~/.dodo_commands/default_layers/*.yaml (continued from previous page) Layers can be switched on and off with the dodo layer command (except for the ones with absolute paths). In the above example, to replace the layer buildtype.debug.yaml with buildtype.release.yaml call: dodo layer buildtype release 5. if a configuration key ends in _EVAL, then its associated value is evaluated in Python and stored as a string. If the associated value is a list or dictionary, then every value in that list or dictionary is evaluated (but not recursively). Use with caution! For example: BUILD: nr_of_threads_eval: (2 + 4) # evaluates to the string "6" FOO: bar_eval: - (1 + 2) # evaluates to "3" - (3 + 4) # evaluates to "7" foobar: ${/BUILD/nr_of_threads_EVAL} # value will be "6" 5.2 Using configuration values in scripts To use a configuration value in your script, call Dodo.get_config('/SOME/path/to/the/value'). By convention, items in the root of the configuration are capitalized. Though you will rarely need to, you can access array elements by index, e.g. Dodo.get_config('/SOME/array/3/name'). Finally, you can specify a fallback value, e.g. Dodo.get_config('/ROOT/maybe/some/value', 42) 5.3 How to use layers The layering mechanism is simple but powerful. As an example, consider placing the settings for different web servers in server.develop.yaml, server.staging.yaml and server.production.yaml. You can now select a server (staging, for example) with dodo layer server staging. Another common use-case is switching between debug and release builds. Note: Calling dodo layer foo bar makes a small change in your configuration file by adding foo.bar. yaml to the ${/ROOT/layers} node. Make sure that you do not have any unsaved configuration changes before calling this command. If you want to keep your layers in a separate directory, use the ${ROOT/layer_dir} setting. To list all active layers, use dodo which --layers. 14 Chapter 5. Configuration files

19 5.4 Including bits of configuration from packages When you install a package with dodo install-default-commands it may contain more than just command scripts. Some packages contain a so-called drop-in directory with configuration files and other resources such as Dockerfiles. Since the Dodo Commands philosophy is that you own your local configuration, the way to use these files is through copying them: dodo install-default-commands --pip dodo_deploy_commands # copy drop-in directory to ${/ROOT/res_dir}/drops/dodo_deploy_commands dodo drop-in dodo_deploy_commands The dodo drop-in command copies the package s drop-in directory to ${/ROOT/res_dir}/drops/ <package_name>. The default location of the drop-in source directory is in the root of the package. Alternatively, the package root may contain a.drop-in file that holds the relative path to the actual drop-in directory. You can use a copied configuration file by including it as a layer: # enable layer (drop.on.yaml) dodo layer drops/dodo_deploy_commands/drop on # disable layer (drop.off.yaml) dodo layer drops/dodo_deploy_commands/drop off 5.5 Preserving the configuration history Breaking your local configuration can be serious problem, because it stops all Dodo Commands from working. Therefore, it s advisable to store your local configuration in a local git repository so that you can always restore a previous version. The dodo commit-config command makes this easy. It initializes a local git repository (if one doesn t exist already) next to your configuration files, and stages and commits all changes to the configuration. 5.6 The global configuration file The location of the global configuration file can be obtained with dodo which --global-config. From the command line, you can set a global configuration value foo in the bar section using dodo global-config bar.foo somenewvalue Including bits of configuration from packages 15

20 16 Chapter 5. Configuration files

21 CHAPTER 6 Commands 6.1 Command path The search path for commands is determined by the ${/ROOT/command_path} setting in the project s config. yaml. The dodo_system_commands module is added to the command path by default. In the example below, all subdirectories of the default commands directory are used (via the * wildcard), as well as the special_commands directory: ROOT: command_path: - ~/.dodo_commands/default_commands/* - ${/ROOT/src_dir}/special_commands 6.2 Default commands The default Dodo Commands scripts are stored in ~/.dodo_commands/default_commands. As we saw above, the default commands are made available by adding ~/.dodo_commands/default_commands/* to the command path. You can install additional default commands using: # creates a symlink to ``my_commands`` in ``~/.dodo_commands/default_commands/my_ commands`` dodo install-default-commands /path/to/my_commands or # adds ``~/.dodo_commands/default_commands/mycommands`` by installing the # ``my_commands`` package dodo install-default-commands --pip my_commands Tip: Don t use dodo install-default-commands for project specific command scripts. Instead, store them inside the project s source tree, and reference them directly in the /ROOT/command_path node of the configuration 17

22 file. 6.3 Excluding directories from the command path Use ${/ROOT/command_path_exclude} to exclude certain paths from the command path: ROOT: command_path_exclude: - ~/.dodo_commands/default_commands/foo 6.4 Importing symbols from another module In your Dodo command script, you can import a symbol from another module in the command path using (for example) import foo from dodo_my_commands.bar. Note that the names of the modules in the command path must be unique. Also, it is required that dodo_my_commands exists in the active project s command path, otherwise the import will fail. 6.5 Specifying command dependencies in the.meta file Each Dodo command should ideally run out-of-the-box. If your foo command needs additional Python packages, you can describe them in a foo.meta file: requirements: - dominate==2.2.0 In this example, calling the foo command will ask the user for confirmation to automatically install the dominate package into the python virtual environment of the active Dodo Commands project. 6.6 Aliases You can added aliases for any dodo command in the aliases section of The global configuration file, e.g. [alias] wh = which pc = print-config 18 Chapter 6. Commands

23 CHAPTER 7 The Dodo singleton This chapter describes the functions offered by the Dodo singleton class. You can use these functions in any Dodo Command script. 7.1 The is_main function Using if Dodo.is_main( name ) instead of the usual if name == ' main ' allows Dodo Commands to execute your script when its invoked through calling dodo. from dodo_commands.framework import Dodo if Dodo.is_main( name ): print("hello world") 7.2 The get_config function Calling Dodo.get_config('/ROOT/my/key', 'default-value') will retrieve a value from the project s Configuration files. Use Dodo.get_config() to get direct access to the entire configuration dictionary. 7.3 The parse_args function ( confirm and echo) The Dodo.parse_args(parser) function uses parser to parse the arguments in sys.argv. It adds an --echo and --confirm flag to the command line arguments of your script: 1. the --echo flag changes the behaviour of run so that it only prints a command line instead of executing the command. 2. the --confirm flag changes the behaviour of run so that it prints a command line and asks for confirmation before executing the command. 19

24 Note that although the --echo and --confirm flags are included in the return value of Dodo.parse_args, there is no need to pass them to Dodo.run. This is because the Dodo singleton also stores them internally. If you call Dodo.parse_args then you should do so before any calls to Dodo.run so that --echo and --confirm can take effect: from dodo_commands.framework import Dodo from argparse import ArgumentParser def _args(): parser = ArgumentParser() parser.add_argument('--verbose') return Dodo.parse_args(parser) if Dodo.is_main( name ): args = _args() if args.verbose: Dodo.run( ['echo', 'hello world'], cwd=dodo.get_config('/root/src_dir') ) 7.4 Config arguments Although it s possible to use Dodo.get_config directly inside the Dodo.run invocation, doing this work in the _args() helper function yields a better separation of concerns: def _args(): parser = ArgumentParser() parser.add_argument('--verbose') args = Dodo.parse_args(parser) args.src_dir = Dodo.get_config('/ROOT/src_dir') return args if Dodo.is_main( name ): args = _args() # You can now use args.src_dir This approach opens up an interesting possibility: if the requested configuration key is absent then we could still ask the user for a value on the command line. This can be achieved through the ConfigArg helper class: from dodo_commands.framework import Dodo, ConfigArg from argparse import ArgumentParser def _args(): parser = ArgumentParser() parser.add_argument( verbose ) return Dodo.parse_args(parser, config_args=[ ]) /ROOT/src_dir, src_dir, help= Location of the source files The ConfigArg is constructed with the configuration key, followed by any (keyword) arguments that parser. add_argument accepts. If the key is found in the configuration, then the corresponding value will be inserted into the return value of Dodo.parse_args. Otherwise, an extra argument will be added to the command line syntax. This ensures that the value is either read from the configuration or from the command line. 20 Chapter 7. The Dodo singleton

25 7.5 The run function The Dodo.run function takes a list of arguments (and a current working directory) and runs them on the command line. It also adds all variables in ${/ENVIRONMENT/variable_map} to the system environment for the duration of running the command. if Dodo.is_main( name ): Dodo.run(['echo', 'hello'], cwd='/tmp') 7.6 Marking a script as unsafe Since command scripts are written in Python, the script can in principle perform any operation without explicitly asking your permission. In other words, it may choose to ignore the --confirm and --echo options. This sitation should of course be avoided. However, if a Command script does not completely honor the --confirm and --echo flags, it should pass safe=false when it calls Dodo.is_main, as shown in the example below. Unsafe commands will not run with the echo flag, and will pause with a warning when run with the confirm flag. if Dodo.is_main( name, safe=false): # NOTE: setting the _safe flag here # Do destructive things without asking permission. Having this call # is the reason we used safe=false to mark the script as unsafe. # Running the script with ``--echo`` is not possible, because that would # lead to unpleasant surprises. Running with ``--confirm`` will inform # you that unpleasant surprises can be expected if you continue. os.unlink('/tmp/foo.text') # Delete the /tmp directory. Since this time we are using Dodo.run, # the user can use the --confirm flag to inspect and cancel it. # This makes this call *relatively* safe, but if you blindly run this script (without # using ``--confirm``) you may still be unpleasantly surprised. Dodo.run(['rm', '-rf', '/tmp']) 7.5. The run function 21

26 22 Chapter 7. The Dodo singleton

27 CHAPTER 8 Decorators A Decorator is a class that alters the workings of a Dodo Command script by extending or modifying the arguments that are passed to Dodo.run. For this purpose, the arguments for Dodo.run are constructed as a tree. Initially, this tree only has a root node that holds all arguments. Each decorator may restructure the argument tree by adding new tree nodes and changing existing nodes. Dodo Commands obtains the final list of arguments by flattening the tree in a pre-order fashion. This allows each decorator to prepend or append arguments, or completely rewrite the tree. 8.1 Constructing a tree Each node in the tree is of type ArgsTreeNode. A node has attributes args (these are the command line arguments) and is_horizontal (this flag determines how the arguments are printed when the --echo or --confirm flag is used). To add a child node, call node.add_child(). 8.2 Prepending an argument The following example shows a decorator that prepends the arguments with the path to a debugger executable. The decorator should be placed in a decorators directory inside a commands directory: # file: my_commands/decorators/debugger.py class Decorator: # noqa def add_arguments(self, parser): # noqa parser.add_argument( '--use-debugger', action='store_true', default=false, help="run the command through the debugger" ) def modify_args(self, dodo_args, root_node, cwd): # noqa (continues on next page) 23

28 if not getattr(dodo_args, 'use_debugger', False): return root_node, cwd (continued from previous page) # Create a new root node with just the path to the debugger # Note that "debugger" is a name tag which is only used internally # to identify nodes in the tree. debugger_node = ArgsTreeNode( "debugger", args=[dodo.get_config('/build/debugger')] ) # Since we want to make the debugger path a prefix, we add the # original arguments as a child node. When the tree is flattened # in a pre-order fashion, this will give the correct result. debugger_node.add_child(root_node) return debugger_node, cwd Note that the decorator returns both the new argument tree and a current working directory. This means that it s possible to change the current working directory for the decorated command as well. 8.3 Appending an argument This is similar to prepending, except that we do not need to create a new node # file: my_commands/decorators/foo.py class Decorator: # noqa def modify_args(self, dodo_args, root_node, cwd): root_node.args.append('--foo') return root_node, cwd # noqa 8.4 Mapping decorators to commands Not all decorators are compatible with all commands. For example, only some commands can be run inside a debugger. Therefore, the configuration contains a list of decorated command for each decorator. In this list, wildcards are allowed, and you can exclude commands by prefixing them with an exclamation mark: ROOT: decorators: # Use a wildcard to decorate all commands, but exclude the foo command debugger: ['*', '!foo'] # The cmake and runserver scripts can be run inside docker docker: ['cmake', 'runserver'] 8.5 Printing arguments The structure of the argument tree determines how arguments are printed when the --echo or --confirm flag is used. We ve seen above that nodes in the tree are created with the ArgsTreeNode constructor. The arguments in this node are indented in correspondence to the node s depth in the tree. The ArgsTreeNode constructor takes an optional argument is_horizontal that determines if arguments are printed horizontally or vertically, e.g. 24 Chapter 8. Decorators

29 docker_node = ArgsTreeNode("docker", args=['docker', 'run']) tty_node = ArgsTreeNode( ["tty", args=['--rm', '--interactive', '--tty'], is_horizontal=true ) docker_node.add_child(tty_node) # add more nodes to the tree... # assume cmake is decorated with the docker decorator dodo cmake --echo produces docker run \ --rm --interactive --tty \ --name=cmake \ dodo_tutorial:1604 \ cmake -DCMAKE_BUILD_TYPE=release /home/maarten/projects/dodo_tutorial/src 8.5. Printing arguments 25

30 26 Chapter 8. Decorators

31 CHAPTER 9 Sharing projects 9.1 Objectives If someone joins your project, then it makes sense to share your working environment with them. At the same time, you want working environments to be independent, so that each project member can arrange it to their preferences. This is achieved by having a shared configuration from which you cherry-pick the parts you need (this is somewhat comparable to how remote repositories work in git). In the explanation below, we ll assume that you have the following project directory layout: ~/projects/foobar ~/projects/foobar/src repository ~/projects/foobar/dodo_commands/res # root of the project # cloned # your Dodo Commands configuration files 9.2 Bootstrapping Let s assume that you wish to share your project with new team members. Follow these steps: copy your Dodo Commands configuration files to the location ~/projects/foobar/src/extra/ dodo_commands/res and commit this to your git repository. These copies are the project s shared Dodo configuration. All team members (including you) will synchronize their local configuration file with this shared file. ask your colleague to create an empty Dodo Commands project using dodo activate FooBar --create. Your colleague then calls the bootstrap command to clone foobar.git and initialize their local Dodo Commands configuration with copies of the shared configuration files: dodo bootstrap src extra/dodo_commands/res --force --git-url foobar.git 27

32 In the above example, the repository is cloned to the src subdirectory of the project directory. The shared configuration files are copied from the extra/dodo_commands/res location (which is relative to src) to ~/projects/ FooBar/dodo_commands/res. Finally, the location of the cloned sources (src) is stored in the configuration under the ${/ROOT/src_dir} key, and the location of the shared configuration files is stored under ${/ROOT/ shared_config_dir}. At this point, your colleague has the same directory structure as you: ~/projects/foobar ~/projects/foobar/src foobar.git repository ~/projects/foobar/src/extra/dodo_commands/res configuration files ~/projects/foobar/dodo_commands/res Commands configuration files # root of the project # cloned # default Dodo Commands # local copies of the default Dodo the ${/ROOT/shared_config_dir} directory is used by the dodo diff command to show the differences between your local configuration files and the shared files. If you are using the recommended diff tool (meld) then you will now be able to copy parts of your local configuration to the shared one. When you push these changes to version control, your colleages will later be able to incorporate your changes (also by copying parts) when they run dodo diff. As mentioned, we recommend to set meld as the diff_tool in ~/.dodo_commands/config: dodo global-config settings.diff_tool meld To synchronize only config.yaml, call dodo diff config.yaml. It s a good practice to use the value ${/ROOT/version} to track whether the copied configuration is up-todate or not (see Checking the config version). 9.3 Checking the config version The dodo check-config --config command compares the ${/ROOT/version} value in your local configuration with the value in the shared configuration. If someone bumped the version in the shared configuration, it will tell you that your local configuration is not up-to-date (in that case, use dodo diff to synchronize). One of the synchronized values is ${/ROOT/required_dodo_commands_version}. The dodo check-version --dodo command reads this value and warns you if your Dodo Commands version is too old. The small script written by dodo autostart on (see Activating the last used project) calls both checks, and this helps you to stay synchronized. 9.4 Starting a project from a cookiecutter template It s convenient to start a brand new project from a cookiecutter template. cookiecutter-url option instead of git-url in the bootstrap call: This can be achieved by using the dodo bootstrap src extra/dodo_commands/res --force --cookiecutter-url com/foo/foobar.git Note that the cookiecutter url can also point to a directory on the local filesystem. 28 Chapter 9. Sharing projects

33 9.5 Symlinking to a local src directory (useful with monolithic repositories) A monolithic repository may contain several projects that each have their own Dodo Commands configuration. In this scenario, each Dodo Commands project should use a symlink to a subdirectory of the monolithic source tree: # Get monolithic repository. cd ~/sources git clone $(dodo activate --create foobar) # Bootstrap the foobar project without cloning the sources, copying the # configuration from ~/sources/monolith/foobar/extra/dodo_commands/res dodo bootstrap --link-dir ~/sources/monolith/foobar extra/dodo_commands/res --force 9.5. Symlinking to a local src directory (useful with monolithic repositories) 29

34 30 Chapter 9. Sharing projects

35 CHAPTER 10 The diagnose command When a colleague shares a Dodo Commands project with you, then you obtain a set of commands for working with this environment. However, it s still necessary to understand how all these commands work. Questions you will have include: which tasks are supported by the environment? which commands are available to perform these tasks? which configuration values are used by these commands? This information needs to be documented. Dodo Commands supports this in a dynamic way through the dodo diagnose command Jinja2 documentation templates Documentation is produced with dodo diagnose using the following workflow: the project author writes documentation in.rst (restructured text) files through the jinja2 templating system, these.rst files can inspect the local system and Dodo Commands configuration. This means that the documentation can directly report on issues with your local environment the dodo diagnose command invokes jinja2 and writes the rendered output into a output directory. finally, it renders the.rst files with sphinx and opens the result in the web browser Diagnose dodo configuration The typical Dodo configuration for the diagnose command is: 31

36 DIAGNOSE: # Documentation.rst is read from here src_dir: ${/ROOT/src_dir}/extra/dodo_commands/diagnose # Rendered documentation is written here output_dir: ${/ROOT/project_dir}/res/diagnose # Import custom jinja2 filters and tests filters: - - ${/DIAGNOSE/src_dir} # added to sys.path - filters # relative module import path Note that you should have a index.rst file in the diagnose source directory, which has a table of contents that references the other.rst files The dodo_expand jinja2 filter The dodo_expand filter is used to print configuration values: to print a configuration value in the documentation output, use {{ '/ROOT/some/key' dodo_expand }}. If the configuration value is a dictionary, then it s automatically placed in a yaml block layout that starts on a new line. You can force or disable this layout behaviour with the layout argument: {{ '/ROOT/some/ key' dodo_expand(layout=true) }} if you want to print both the key and its value, use {{ '/ROOT/some/key' dodo_expand(key=true) }} if you want to print the key and use the value as a browser link, use {{ '/ROOT/some/key' dodo_expand(link=true) }} the quote_key and quote_val arguments control whether the key and value are printed between backtick quotes. 32 Chapter 10. The diagnose command

37 CHAPTER 11 Indices and tables genindex modindex search 33