Deploying Rails with Kubernetes

Transcription

1

2 Deploying Rails with Kubernetes AWS Edition Pablo Acuña This book is for sale at This version was published on This is a Leanpub book. Leanpub empowers authors and publishers with the Lean Publishing process. Lean Publishing is the act of publishing an in-progress ebook using lightweight tools and many iterations to get reader feedback, pivot until you have the right book and build traction once you do Pablo Acuña

3 Contents Acknowledgments Preface Prerequisites Conventions i ii ii ii Development Setting up the Development Environment Ruby/Rails installation Installing Docker Creating a new Rails App Dockerizing the Application Dockerfile webapp.conf docker-compose.yml Adding a resource Adding a production-ready database Adding a Search Engine

4 Acknowledgments I d like to thank all of the people that for any reason have taken part in my learning journey. Especially Becky, for always being there for me and supporting all of my ambitions. i

5 Preface Prerequisites I expect you to have a medium knowledge about Docker and its workflow. It s OK if you have never worked with Docker in a production environment. Since this is a book about deploying Ruby on Rails, you should also have a medium to advanced Rails knowledge. I won t explain basic things about Ruby or Rails. We are going to work with Postgres as our Database and Elasticsearch as our search engine. You don t need previous knowledge about any of these two. The main focus it s going to be on how to create the connexions between our application and these type components. Conventions Navigating between the different layers when you re running containers, specially if you are running Docker inside of a Virtual Machine, can become a little messy. You need to know where you are at every moment. In order to not pollute all of the code with explanations, I ll be using the following conventions: Every line that starts with the $ symbol, means that the command should be run at same same level where Docker is installed. That can be your local machine, or the Virtual Machine where you have your Docker host running. Every line that starts with the # symbol, means that the command should be run inside of a container. Every line that stats with the > symbol, means that the command should be run inside of the Rails console. Sometimes you ll see all the symbols in action so you can have more context. For example: 1 $ docker exec -it myapp bash 2 # rails c 3 > puts 'hello world!' A special case will be to run commands on a remote host. In that case I ll still be using the $ command but it s going to be pretty clear that the commands should be run on the remote host. ii

6 Development In this chapter we will create and dockerize a fresh Rails 5 application. We are going to use production-ready images so we can test the production setup locally before run it in production. That s one of the best advantages of using Docker. We can simulate a similar production environment and that way make sure that all is going to work as a long as we use the same images and versions for the software. The application that we are going to build will be just a regular Rails application with some scaffolds. This application will be connected to a Database and a Search Engine which in this case are going to be Postgres and Elasticsearch. We are not adding to much complexity to the application logic because is not as relevant as the setup for connecting all of the different elements in the architecture and the deployment workflow. This workflow will go from pushing new features to DockerHub to deploying new releases to a staging environment and finally to production. Setting up the Development Environment The only requirements for this part of the book, are Ruby/Rails and Docker. We need Ruby and Rails for creating a new application and Docker for running the application. Ruby/Rails installation If you don t have a recent version of any of these, I recommend you to use some Ruby version manager such as rbenv¹ or rvm². Any of those should work for the Ruby version we need, which in this case is 2.3.0dev. Although any version equal or higher than should work. We also need Rails installed in our system so we can create a fresh application. A couple of days ago the first Rails 5 rc version was released, so I ll be using that one. After setting up your Ruby version, you can use the gem install command and install the most recent version for Rails 5. For example in my case I used: 1 $ gem install rails --version rc1 --no-ri --no-rdoc in order to get the current latest version. ¹ ² 1

7 Development 2 Installing Docker Today we have several options for running Docker. If you re on OS X you can use Docker Machine³, Docker for Mac⁴ which is only available through invitation at the moment but maybe you re reading this when it s already out. You can also run a Linux Virtual Machine and run native Docker. For windows you have the same options and for Linux you can enjoy the best experience with native Docker without any sort of virtualization. Any of these should work, just check that you re using the latest version for Docker and for Docker Compose. In my case I m running Docker version , build 5604cbe and docker-compose version 1.7.1, build 0a9ab35. That s really important since some of the features like networking are only available in the recent versions. Creating a new Rails App I have the latest Rails version (Rails rc1) and Ruby 2.3.0dev version. Let s create a new Rails application: 1 $ rails new myapp We are not using any options by now. We ll be adding more features and dependencies progressively. If you run rails s you should be able to see the application running locally on Dockerizing the Application Luckily for us, dockerize a Rails app it s pretty straightforward if you use the awesome Phusion Passenger Docker image. This image will provide us with a production-ready server for running our application and some other features we will need. In this book I ll show the basics of how to use this image, and if you want to learn more you can go to the official GitHub Repo⁵. They have a very nice documentation for the project. First, we are going to need some extra files. Let s create these files in the root of our project: 1 $ touch Dockerfile docker-compose.yml webapp.conf You should be familiarize with what the Dockerfile and docker-compose.yml files do. The webapp.conf will contain the virtual host for our application. This file will also be added to the Nginx configuration during the build process. Dockerfile Add the following template to the Dockerfile: ³ ⁴ ⁵

8 Development 3 1 FROM phusion/passenger-ruby # Set correct environment variables. 4 ENV HOME /root 5 6 # Use baseimage-docker's init process. 7 CMD ["/sbin/my_init"] 8 9 # additional packages 10 RUN apt-get update # Active nginx 13 RUN rm -f /etc/service/nginx/down # Install bundle of gems 16 WORKDIR /tmp 17 ADD Gemfile /tmp/ 18 ADD Gemfile.lock /tmp/ 19 RUN bundle install # Copy the nginx template for configuration and preserve environment variables 22 RUN rm /etc/nginx/sites-enabled/default # Add the nginx site and config 25 ADD webapp.conf /etc/nginx/sites-enabled/webapp.conf RUN mkdir /home/app/webapp 28 COPY. /home/app/webapp 29 RUN usermod -u 1000 app 30 RUN chown -R app:app /home/app/webapp 31 WORKDIR /home/app/webapp # Clean up APT when done. 34 RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* 35 EXPOSE 80 I ve added comments to every line so you can understand better how this Dockerfile works. For example on these lines: 1 # Add the nginx site and config 2 ADD webapp.conf /etc/nginx/sites-enabled/webapp.conf we are adding the virtual host to the Nginx configuration folder. The rest of the file is pretty self explanatory. We are using a lot of code presented in the official documentation, so nothing too hacky happening here. One thing to notice although is that you want to set the correct permissions for your application. The passenger user has the uid 1000, so all of the chmod and chown commands you see in the Dockerfile are setting the permissions for the application files. We are also running bundle install before copying the entire application into the container in order to have an extra cache

9 Development 4 layer for the gems. This way, the bundle install command will only be run when you actually make changes to the Gemfile. webapp.conf The virtual host configuration can be as simple as: 1 server { 2 listen 80; 3 server_name _; 4 root /home/app/webapp/public; 5 passenger_enabled on; 6 passenger_user app; 7 passenger_ruby /usr/bin/ruby2.2; 8 } We just need to enable passenger for managing our Ruby application from Nginx. docker-compose.yml We just have our application, so let s add a block for building it from our source: 1 webapp: 2 build:. 3 container_name: myapp 4 working_dir: /home/app/webapp 5 ports: 6 - "80:80" 7 environment: 8 - PASSENGER_APP_ENV=development 9 volumes: 10 -.:/home/app/webapp The only environmental variable we need for now is the PASSENGER_APP_ENV variable. This variable is preserved by default and it will indicate the Rails environment in which the container will be running. For now we are in development mode, so we have to declare development for its value. We are also adding a volume in case you want to work locally and you don t want to rebuild your application every time you make changes to your code. Keep in mind that depending on how you re running Docker (natively, with Vagrant, with Docker Machine, etc) you may have to set the correct permissions on the shared folder so you don t lose the permissions for the passenger user. Now, we can build our application with docker-compose. My Docker version is , build 5604cbe and my docker-compose version is 1.7.0, build 0d7bf73. Let s build the app:

10 Development 5 1 $ docker-compose build This can take several minutes the first time since the command will pull the necessary images from DockerHub, update the Linux OS of the container and install the gems. Once it finished, you can run the build again and see that is a lot more faster. Now we can run the application by running: 1 $ docker-compose up Now depending of what you re using for running Docker you cant visit your VM IP address or your Dockerhost address and should be able to see your application running on the port 80. That s great! We our Rails app dockerized. Adding a resource Now that we have our application up and running, let s scaffold a resource so later we can test more than just the Rails welcome page. The application is running inside of the phusion passenger container, and we have two options for interacting with the application. The first option is to run commands by using docker-compose as a wrapper. For example we could run: 1 $ docker-compose run webapp rails c And that would take us to the webapp service defined in our docker-compose which corresponds to the myapp application, and open a rails console inside of it. The second option would be to create a bash session inside of the container and run the commands directly there. For example: 1 $ docker exec -it myapp bash 2 # rails c The first command takes you to the container and from there you can run any rails or rake commands you want. I prefer to use the second approach simply because using docker-compose for running commands will create containers for running everyone of these commands, and those containers will stay there using disk space. You can see this by running some commands using docker-compose run and then listing all of the containers: 1 $ docker ps -a Output:

11 Development 6 1 CONTAINER ID IMAGE COMMAND CREATED STATUS \ 2 PORTS NAMES 3 d2aa78889f2a myapp_webapp "rails c" 11 seconds ago Exited (0) \ 4 3 seconds ago myapp_webapp_run_2 5 bda9ef myapp_webapp "rails c" 2 minutes ago Exited (0) \ 6 About a minute ago myapp_webapp_run_1 7 85f1508c8705 myapp_webapp "/sbin/my_init" 2 hours ago Up 7 minute\ 8 s :80->80/tcp, 443/tcp myapp You see that we have our application running with the name myapp, but we also have two containers that were created just for running the rails commands. Of course the status of those containers is exited since they were created just for that. When you use docker exec instead, you are just going inside of the running container and not creating an extra one. So let s just stick with that approach. Let s go inside of the container and create a very simple scaffold: 1 $ docker exec -it myapp bash 2 # rails g scaffold article title:string body:text 3 # rake db:migrate Now you can visit the path /articles and you should be able to list and add articles. If you want to run the tests created during the scaffold, you can run: 1 # rake Output: 1 Run options: --seed # Running: Finished in s, runs/s, assertions/s runs, 12 assertions, 0 failures, 0 errors, 0 skips We will use these tests later in our Continuous Integration chapter.

12 Development 7 Adding a production-ready database In this section we ll add a real database to our application. Typically you would use MySQL or Postgres or a NoSQL database like MongoDB in production. The setup for any of these would be pretty similar if you re using containers. You just need the correct configuration in order to run the images. In this case we ll go with Postgres. We are going to be using the official Postgres image⁶ from DockerHub. The environment variables that this image uses for running are POSTGRES_PASSWORD, POSTGRES_USER, and POSTGRES_DB. You can find a full list in the documentation, but for us these will be enough for now: First, let s add a Postgres section in our docker-compose.yml file: 1 postgres: 2 image: postgres:9.4 3 container_name: myapp-db 4 environment: 5 - POSTGRES_USER=myapp 6 - POSTGRES_DB=myapp_development 7 - POSTGRES_PASSWORD=secretpassword If you want to persist your data in your development environment, you ll need to mount a data-only container for this database. You can achieve this by adding another section for that container: 1 postgres-data: 2 image: postgres:9.4 3 container_name: myapp-data 4 environment: 5 - POSTGRES_PASSWORD=secretpassword 6 volumes: 7 - /var/lib/postgresql/data 8 command: /bin/true And then mount the correct path on the database container: ⁶

13 Development 8 1 postgres: 2 image: postgres:9.4 3 container_name: myapp-db 4 environment: 5 - POSTGRES_USER=myapp 6 - POSTGRES_DB=myapp_development 7 - POSTGRES_PASSWORD=secretpassword 8 volumes_from: 9 - postgres-data Now we can use the new Docker networking feature in order to connect our database container with our application container. For that we have to use the version 2 of docker-compose, so at the beginning of the docker-compose.yml file add: 1 version: '2' And now let s move the app and the db to a services section with the same network: 1 version: '2' 2 services: 3 webapp: 4 build:. 5 container_name: myapp 6 working_dir: /home/app/webapp 7 ports: 8 - "80:80" 9 environment: 10 - PASSENGER_APP_ENV=development 11 volumes: 12 -.:/home/app/webapp 13 networks: 14 - back-end 15 postgres: 16 image: postgres: container_name: myapp-db 18 environment: 19 - POSTGRES_USER=myapp 20 - POSTGRES_DB=myapp_development 21 - POSTGRES_PASSWORD=secretpassword 22 volumes_from: 23 - postgres-data 24 networks: 25 - back-end 26 postgres-data: 27 image: postgres: container_name: myapp-data 29 environment: 30 - POSTGRES_PASSWORD=secretpassword

14 Development 9 31 volumes: 32 - /var/lib/postgresql/data 33 command: /bin/true 34 networks: 35 back-end: 36 driver: bridge Now we can connect to the database from our web application by using the service alias postgres instead of using the old linking via environmental variables. Now that we re ready with the Docker Compose yaml file, let s add the Postgres gem to our application. Remember that you can modify the code locally since we are mounting a volume from the local project to the project inside of the container. Add the following to your Gemfile: 1 gem 'pg', '~> 0.18' And now from inside the container run bundle install: 1 $ docker exec -it myapp bash 2 # bundle install And then add the following to your config/database.yml file: 1 default: &default 2 adapter: postgresql 3 encoding: unicode 4 pool: development: 7 <<: *default 8 database: myapp_development 9 username: myapp 10 password: secretpassword 11 host: postgres 12 port: test: 15 <<: *default 16 database: myapp_test production: 19 <<: *default 20 database: myapp_production 21 username: myapp 22 password: secretpassword Now let s rebuild our application:

15 Development 10 1 $ docker-compose stop 2 $ docker-compose build And run up: 1 $ docker-compose up This will pull the postgres image, run the container and create the user and database we specified before. Now we just need to run the migrations for our scaffold: 1 $ docker exec -it myapp bash 2 # rake db:migrate And that s it!, now you can visit your application connected to the postgres database and test that that everything works as expected. Adding a Search Engine In order to make things more interesting and to put ourselves in a more realistic scenario, let s add a search engine to our application. Elasticsearch has a really nice integration with Rails via the Elasticsearch-rails gem⁷. The steps we need to follow in order to add this gem to our project are: 1. Add the necessary gems 2. Add the new service to our docker-compose.yml file 3. Add the initializer for connecting with Elasticsearch 4. Add the Elasticsearch modules to our Article model 5. Run docker-compose build and docker-compose up Open the Gemfile an add the following: 1 gem 'elasticsearch-model', git: 'git://github.com/elasticsearch/elasticsearch-rails.git' 2 gem 'elasticsearch-rails', git: 'git://github.com/elasticsearch/elasticsearch-rails.git' And then run bundle install inside of the container: ⁷

16 Development 11 1 $ docker exec -it myapp bash 2 # bundle install We are going to use the official DockerHub image for elasticsearch⁸ in our docker-compose.yml file. Add the following service: Now open the docker-compose.yml file and add the following services: 1 elasticsearch: 2 image: elasticsearch:2.3 3 container_name: myapp-es 4 volumes_from: 5 - elasticsearch-data 6 networks: 7 - back-end 8 elasticsearch-data: 9 image: elasticsearch: container_name: myapp-es-data 11 volumes: 12 - /var/lib/elasticsearch 13 command: /bin/true As you can see, we re also adding a data-only container for persisting the Elasticsearch data locally in case we need to recreate our container. In order to use the gem in our Rails application, we must add an initializer with the configuration. Create the file config/initializers/elasticsearch.rb and add the following code: 1 host = 'elasticsearch' 2 3 Elasticsearch::Model.client = Elasticsearch::Client.new host: " 4 5 # Print Curl-formatted traces in development into a file 6 # 7 if Rails.env.development? 8 tracer = ActiveSupport::Logger.new('log/elasticsearch.log') 9 tracer.level = Logger::DEBUG 10 end Elasticsearch::Model.client.transport.tracer = tracer We can use the host elasticsearch for this service since we declared it with that name in our network in the docker-compose.yml file. The rest of the code is for debugging purposes. Let s also create the task lib/tasks/elasticsearch.rake with the following code: ⁸

17 Development 12 1 require 'elasticsearch/rails/tasks/import' This will gives us a handy command for indexing all the records for the models associated to Elasticsearch documents. Right now we only have the Article model so open app/models/article.rb and add the following: 1 class Article < ApplicationRecord 2 include Elasticsearch::Model 3 include Elasticsearch::Model::Callbacks 4 end Those modules will include the necessary hooks for indexing the records. Rebuild the project: 1 $ docker-compose build 2 $ docker-compose up And add a couple of records to the database: 1 $ docker exec -it myapp bash 2 # rails c 3 > Article.create(title: 'article 1', body: 'body for article 1') 4 > Article.create(title: 'article 2', body: 'body for article 2') 5 > Article.create(title: 'article 3', body: 'body for article 3') These articles should be indexed immediately thanks to the callbacks added by the Elasticsearch modules in the model. Let s perform a search just to be sure: 1 > Article.search("3").results.first 2 => _id="4" _index="\ 3 articles" _score= _source=#<hashie::mash body="body for article 3" created_at=" t\ 4 00:49:37.329Z" id=4 title="article 3" updated_at=" t00:49:37.329z"> _type="article">> You can see in the result that we re in fact searching in the Elasticsearch index and we found a match. If you have previous records you want to index, you can run the task provided by the gem: 1 # bundle exec rake environment elasticsearch:import:model CLASS='Article' 2 [IMPORT] Done And that s it! We have a production-ready search engine for our application.