Improving your Drupal 8 Development Workflow. by Jesus Manuel Olivas / WeKnow

Transcription

1 Improving your Drupal 8 Development Workflow by Jesus Manuel Olivas / WeKnow

2 WHO AM I? Jesus Manuel Olivas jmolivas jmolivas drupal.org/u/jmolivas jmolivas.weknowinc.com

3

4 What we know about? Drupal and surrounding technologies. Symfony, ReactJS, MeteorJS, Docker, Ansible. Creators and maintainers of Drupal Console Project. Distributed team in 12 countries. Partner agencies in the US and Canada. Talent as a service Staff Augmentation Agency. Hire us!

5 Local environment

6 Local environment. Isolate local machine from project dependencies. Easy and simple to start and continue working on existing projects. Facilitate on boarding and adding new team members. Project scope configurations. Shareable with the team using a code repository via git. Mimic production environment and other temporary stages.

7 Local environment. OS Native, LAMP, Dev Desktop XAMPP => MAMP => WAMP => Vagrant => Docker =>

8 Vagrant Docker

9 Local environment. DrupalVM => Docksal => DDEV => Lando => Outrigger => Docker4Drupal =>

10 Download and Install Drupal 8

11 Download Drupal (Composer) # Includes configurations for Docker and DrupalConsole. composer create-project weknowinc/drupal-project /path/to/drupal # Drupal Vanilla composer create-project drupal/drupal /path/to/drupal # Try Drupal (any distro) + Docker composer create-project weknowinc/try-drupal /path/to/drupal

12 Download Drupal (Git Drupal Console) # Clone project using git git clone git@github.com:weknowinc/drupal-project.git # Create project using Drupal Console drupal site:new --repository=weknowinc/drupal-project \ --directory=/path/to/drupal-project/

13 Manage environment variables (.env.dist =>.env) # Copy env file cp.env.dist.env # Customize values vim.env

14 What if this is not a new project? How to automate this? # Create site alias,.env, and docker-compose.yml files. drupal dockerize (Available on ETA DrupalCon Nashville) # This will run drupal generate:site:alias drupal dotenv:init drupal docker:init

15 Environment variables (.env) ENVIRONMENT=develop HOST_NAME=drupal.develop DATABASE_NAME=drupal HOST_PORT=80 DATABASE_USER=drupal DATABASE_PASSWORD=drupal DATABASE_HOST=mariadb DRUPAL_ROOT=/var/www/html SERVER_ROOT=/var/www/html/web DATABASE_PORT=3306

16 Ignore configuration files (.gitignore) # External dependencies vendor # Files web/sites/*/files # Drupal dependencies web/core web/modules/contrib web/themes/contrib web/profiles/contrib web/libraries/contrib # env file.env # DB backups mariadb-init

17 The new directory structure drupal8.dev/ LICENSE README.md composer.json composer.lock scripts vendor web web/ autoload.php core index.php modules profiles sites themes

18 Start and stop containers # Start containers docker-compose up # Start containers daemon docker-compose up -d # Stop containers docker-compose down # Stop containers destroy data docker-compose down -v

19 Docker commands # Check containers docker ps -a # Stop all containers docker stop $(docker ps -a -q) # Remove unused data docker volume prune # Remove all containers docker rm $(docker ps -a -q)

20 docker-compose.yml overview version: "2.3" services: mariadb: php: nginx: mailhog: traefik: volumes: mysqldata: driver: "local"

21 docker-compose.yml mariadb 1/2 mariadb: image: wodby/mariadb: env_file:./.env environment: MYSQL_RANDOM_ROOT_PASSWORD: 'true' MYSQL_DATABASE: ${DATABASE_NAME} MYSQL_USER: ${DATABASE_USER} MYSQL_PASSWORD: ${DATABASE_PASSWORD}

22 docker-compose.yml mariadb 2/2 volumes: - mysqldata:/var/lib/mysql # Uncomment next line and place DB dump.sql file(s) here # -./mariadb-init:/docker-entrypoint-initdb.d healthcheck: test: ["CMD", "mysqladmin","ping", "-h", "localhost"] timeout: 20s retries: 10

23 docker-compose.yml php 1/2 php: image: wodby/drupal-php: env_file:./.env environment: PHP_SENDMAIL_PATH: /usr/sbin/sendmail -t -i -S mailhog:1025 DB_HOST: ${DATABASE_HOST} DB_USER: ${DATABASE_USER} DB_PASSWORD: ${DATABASE_PASSWORD} DB_NAME: ${DATABASE_NAME} DB_DRIVER: mysql

24 docker-compose.yml php 2/2 volumes: # use :cached config for better performance on osx -./:${DRUPAL_ROOT}:cached depends_on: mariadb: condition: service_health

25 docker-compose.yml nginx 1/2 nginx: image: wodby/drupal-nginx: env_file:./.env depends_on: - php environment: NGINX_STATIC_CONTENT_OPEN_FILE_CACHE: "off" NGINX_ERROR_LOG_LEVEL: debug NGINX_BACKEND_HOST: php NGINX_SERVER_ROOT: ${SERVER_ROOT}

26 docker-compose.yml nginx 2/2 volumes: # use :cached config for better performance on osx -./:${DRUPAL_ROOT}:cached labels: - 'traefik.backend=nginx' - 'traefik.port=80' - 'traefik.frontend.rule=host:${host_name}'

27 docker-compose.yml mailhog mailhog: image: mailhog/mailhog env_file:./.env labels: - 'traefik.backend=mailhog' - 'traefik.port=8025' - 'traefik.frontend.rule=host:mailhog.${host_name}'

28 docker-compose.yml traefik traefik: image: traefik env_file:./.env command: -c /dev/null --web --docker --loglevel=info ports: - '${HOST_PORT}:${HOST_PORT}' volumes: - /var/run/docker.sock:/var/run/docker.sock

29 Using Composer to manage project dependencies

30 Composer

31

32 Using composer within the container Drupal 8 really takes a lot of advantages of using composer, you can install/uninstall dependencies and apply patches. Although it s a good practice when using Docker, to run this commands inside your container because if you have a PHP version on your local machine, you could install dependencies that are not suitable for your container instance.

33 Ahoy Ahoy is command line tool that gives each of your projects their own CLI app with with zero code and dependencies.

34 .ahoy.yml ahoyapi: v2 commands: composer: cmd: docker-compose exec --user=82 php composer usage: Run drush commands in the cli service container. stop: cmd: docker-compose stop usage: Stop the docker-compose containers. down: cmd: docker-compose down usage: Stop and remove the docker-compose containers.

35 Execute composer (within the container) # Execute composer within the container docker-compose exec --user=82 php composer install # Execute composer within the container using ahoy ahoy composer install

36 Manage dependencies ahoy composer require drupal/admin_toolbar ahoy composer update drupal/admin_toolbar ahoy composer update drupal/admin_toolbar --with-dependencies ahoy composer remove drupal/admin_toolbar

37 Composer commands ahoy composer why drupal/admin_toolbar ahoy composer show [-o] [drupal/admin_toolbar] ahoy composer check-platform-reqs ahoy composer diagnose

38 How composer know about Drupal dependencies "repositories": [ { "type": "composer", "url": " } ],

39 How composer know where to download dependencies "extra": { "installer-paths": { "docroot/core": [ "type:drupal-core" ], "docroot/modules/contrib/{$name}": [ "type:drupal-module" ], "docroot/theme/contrib/{$name}": [ type:drupal-theme" ] } }

40 Apply patches (composer.json) "extra": { "patches": { "drupal/admin_toolbar": { "Patch description": URL/local path to patch" } } }

41 Composer configuration "minimum-stability": "dev", "prefer-stable": true, "config": { "platform": { "php": 7.0, "sort-packages": true } }

42 Managing external libraries and dependencies

43 Register a drupal-library (composer.json) "repositories": [ { "type": "package", "package": { "name": "ckeditor/codesnippet", "description": "This plugin ", "type": "drupal-library", "homepage": " "version": 4.7.3, "dist": { "type": "zip", "url": " "reference": "master" } } } ],

44 Download a drupal-library (composer) composer require ckeditor/codesnippet # composer.json "extra": { "installer-paths": { "web/libraries/{$name}": [ type:drupal-library" ], } }

45 Drupal Console

46

47 Drupal Console Launcher curl -L -o drupal.phar mv drupal.phar /usr/local/bin/drupal chmod +x /usr/local/bin/drupal

48 Why do I need the Launcher? This is a global executable that enables you to run the command, drupal, from any directory within your site's project. Without it you will be inconvenienced by having to run the command only from your drupal root directory. For example, if you have Drupal root in a /web directory, and a composer.json and your vendor directory in the directory above that, you will be able to run the drupal command from the same directory as the composer.json file. Even better, you can run it from any subdirectory under that as many levels deep as you would like to go.

49 Drupal Console per-site installation composer require drupal/console:~1.0 \ --prefer-dist \ --optimize-autoloader

50 Site alias Drupal Console allows you to run commands from your local server but being able to execute those commands on a local, remote (VPS, PaaS) or virtual (VM, Docker) Drupal installation using site aliases. A site alias file use the YAML format to provide a collection of predefined options. Once an alias is defined you can call them using a short name.

51 Site alias console/sites/drupal.yml (container docker4drupal) develop: root: /var/www/html extra-options: docker-compose exec --user=82 php type: container

52 Site alias console/sites/drupal.yml (ssh drupalvm) drupalvm: root: /var/www/drupalvm/drupal host: user: vagrant extra-options: '-o PasswordAuthentication=no -i ~/.vagrant.d/insecure_private_key' type: ssh

53 Execute Drupal Console using site alias # Using the --target option drupal --target=drupal.develop cr all # Using the identifier cr all

54 Configuration Management Managing configuration changes through the CLI. Install Drupal and import a previously exported configuration. Override configuration per environment Content synchronization.

55 Managing configuration changes through the CLI.

56 Configuration Management In Drupal, configuration is the collection of admin settings that determine how the site functions, as opposed to the content of the site. Configuration will typically include things such as the site name, the content types and fields, taxonomy vocabularies, views and so on.

57 Configuration system The Drupal configuration system handles configuration in a unified manner. By default, Drupal stores configuration data in the database, but it can be exported to YAML files, allowing the configuration to be managed by version control.

58 Configuration Management Configuration is a mapping of keys and values. Stored on the Database by default. Exported as YAML for synchronization between environments. Once a module is installed the configuration provided by the module is owned by the site not the by the module anymore.

59 Exporting Configuration drupal config:export drupal config:export [--directory] [--tar] drupal config:export:single [--directory] [--module] drupal config:export:content:type drupal config:export:view

60 Importing Configuration drupal config:import drupal config:import [--file] [--directory] drupal config:import:single [--directory] [--file] drupal config:import:single [--file] [--file]

61 Useful Configuration commands drupal config:diff drupal config:edit drupal config:override drupal config:delete drupal debug:config

62 Override configuration per environment

63 Configuration override system There are cases when configuration values need to be overridden for specific purposes as custom settings per environment. For this cases the Drupal configuration systems provides the configuration override system. This system allow you to override configuration values.

64 Configuration override system * Maintains these overrides as temporary layers on top of the standard configuration values. * Does not use overridden values for configuration forms. * May store overrides with other configuration files for staging and version control support.

65 Main web/sites/default/settings.php file $env = getenv('environment'); $base_path = $app_root. '/'. $site_path; $servicesfile = $base_path. '/services.'.$env.'.yml'; $settingsfile = $base_path. '/settings.'.$env.'.php'; if (file_exists($servicesfile)) { } $settings['container_yamls'][] = $servicesfile; if (file_exists($settingsfile)) { } include $settingsfile;

66 Using web/sites/default/services.develop.yml parameters: twig.config: { debug: true, auto_reload: true, cache: false } services: cache.backend.null: class: Drupal\Core\Cache\NullBackendFactory

67 Using web/sites/default/settings.develop.php <?php $config['system.performance']['css']['preprocess'] = FALSE; $config['system.performance']['js']['preprocess'] = FALSE; require_once DRUPAL_ROOT. '/modules/contrib/devel/kint/kint/kint.class.php'; Kint::$maxLevels = 5; $settings['cache']['bins']['render'] = 'cache.backend.null'; $settings['cache']['bins']['dynamic_page_cache'] = 'cache.backend.null';

68 Showing overridden configuration # Pass the --show-overridden flag drupal debug:config system.performance --show-overridden

69 Config Override Inspector Config Override Inspector provides indicators to administrators where form fields represent configuration that is overridden. This module is especially useful for developers working on sites where there are configuration overrides in place depending on the environment. It prevents confusion that arise when changing a form setting, but appears to not take effect.

70 Install Drupal and import a previously exported configuration.

71 Configuration Management The Configuration Management (CM) system, is great. But wouldn't it be even more awesome to be able to install a site, export configuration and then re-install site from scratch importing the previously exported configuration?

72 Install site and import previously exported configuration. Not supported by Drupal out-of-the-box. "Site UUID in source storage does not match the target storage."

73 Why would you want to install your site from an existing configuration? Automate the creation of reproducible build/artifacts from scratch at any stage (Development, QA, Production) to test, launch or deploy your site. Simplify on-boarding for new developers to any project without the need to obtain a database-dump. Developers will be able spin-up sites from scratch just by installing the site and importing configuration files.

74 How to achieve this using Drupal Console? [ 1/2 ] Simple as updating your console/config.yml adding: application:... overrides: config: skip-validate-site-uuid: true

75 How to achieve this using Drupal Console? [ 2/2 ] Execute commands to install the site and import your previously exported configuration: drupal site:install --force no-interaction drupal config:import --no-interaction

76 Automate both commands command: name: build:install description: 'Build site by installing and importing configuration' commands: # Install site - command: site:install options: force: true arguments: profile: standard # Import configurations - command: config:import

77 Do I have other alternatives?

78 Using contributed modules to improve configuration management.

79 Configuration Split This module allows to define sets of configuration that will get exported to separate directories when exporting, and get merged together when importing.

80 Configuration Split Export Configuration # Export without development settings drupal config:export # Export including development settings drupal config_split:export --split=develop

81 Configuration Split Import Configuration # Import without development settings drupal config:import # Import including development settings drupal config_split:import --split=develop

82 Configuration Read-only mode This module allows to lock any configuration changes done via the Drupal admin UI. This can be useful in scenarios where for example configuration changes should not be done on the production environment, but only on specific environments.

83 Using web/sites/default/settings.production.php if (PHP_SAPI!== 'cli') { $settings['config_readonly'] = TRUE; } $settings['config_readonly_whitelist_patterns'] = [ system.site, views*', blocks*', ];

84 Deploying alternatives PaaS and VPS.

85 PaaS

86 Deploying site to a stage or production server.

87 VPS

88 Ansistrano (Ansible + Capistrano) ansistrano.deploy and ansistrano.rollback are Ansible Galaxy roles to easily manage the deployment process for scripting applications such as PHP, Python and Ruby. It's an Ansible port for Capistrano.

89 Ansistrano files.ansistrano/ deploy-prod.yml hosts_prod tasks after-symlink.yml

90 Workflow > Setup phase: Creates the folder structure to hold your releases > Code update phase: Puts the new release into your hosts > Symlink phase: After deploying the new release into your hosts, this step changes the current softlink to new the release > Cleanup phase: Removes any old version based in the ansistrano_keep_releases parameter (see "Role Variables")

91

92 hosts_prod production ansible_host=sandcamp-demo.com ansible_user=deploy

93 deploy-prod.yml name: Deploy prod hosts: all vars: ansistrano_keep_releases: 3 ansistrano_deploy_to: "/var/www/html/sandcamp-demo" ansistrano_allow_anonymous_stats: no ansistrano_after_symlink_tasks_file: "{{ playbook_dir }}/tasks/after-symlink.yml ansistrano_shared_paths: - docroot/sites/default/files ansistrano_deploy_via: git ansistrano_git_repo: git@github.com:weknowinc/sandcamp-demo.git ansistrano_git_branch: master roles: - { role: carlosbuenosvinos.ansistrano-deploy }

94 deploy-prod.yml # 1st release -- /var/www/my-app.com -- current -> /var/www/my-app.com/releases/ releases shared # 2nd release -- /var/www/my-app.com -- current -> /var/www/my-app.com/releases/ releases shared # 3rd release -- /var/www/my-app.com -- current -> /var/www/my-app.com/releases/ releases shared

95 .