Amazn Lab: Deplying applicatins n AWS By Nabil Abdennadher 8 th Nvember 2017 OBJECTIVES 1. Hw t cnfigure yur Virtual Private Clud (VPC)? 2. Hw t create and access instances via Amazn Web Services (AWS) management cnsle? 3. Perfrmance analysis f yur instance. 4. Deplying Drupal n AWS 5. Scale applicatins with AWS 1. HOW TO CONFIGURE YOUR VIRTUAL PRIVATE CLOUD (VPC)? If yu are using an AWS educate accunt, use this URL t access yur accunt: https://www.awseducate.cm/sitelgin Amazn Virtual Private Clud (Amazn VPC) lets yu prvisin a lgically islated sectin f the Amazn Web Services (AWS) clud where yu can launch AWS resurces in a virtual netwrk that yu define. Yu have cmplete cntrl ver yur virtual netwrking envirnment, including selectin f yur wn IP address range, creatin f subnets, and cnfiguratin f rute tables and netwrk gateways. T create ur wn VPC: 1. Lg int yur AWS accunt 2. G t the VPC service 3. Click n "Yur VPCs" n the left side bar and then "Create VPC" a. Fill the fields i. Name: The VPC name ex measurevpc ii. CIDR blck is the netwrk f the VPC, we recmmend 10.0.0.0/16 iii. keep tenancy "default" b. Right click n yur VPC and then click n "Edit DNS Hstname". Click n "yes" and save. c. D the same peratin fr "Edit DNS Reslutin. Yu have nw an empty VPC, nw yu need t define a subnet in yur vpc 4. Click n "Subnets" n the left side bar and then "Create Subnet" a. Fill the fields: i. Name: The subnet name e.g "measuresubnet" ii. VPC: Select yur VPC (defined abve) iii. Availability zne: N Preference iv. CIDR blck is a subnet f yur VPC netwrk. We recmmend 10.0.1.0/24 b. Allw autmatic assign Public IP: i. Right click n yur subnet then click n "Mdify Aut-Assign Public IP" -> "Enable..." -> Save. Yu nw have a VPC with a subnet. Yu need nw an internet gateway t have an internet access Intrductin t Clud Page 1 / 9
5. Click n "Internet Gateways" n the left side bar and then "Create Internet Gateway" a. Fill the name field with the name f yur internet gateway. b. Attach it with yur VPC by right clicking n yur internet gateway and then click n "Attach t VPC", finally select yur VPC Yu nw have a VPC with an internet gateway assciated, yu still need a rute table 6. Click n "Rute Tables" n the left side bar and the "Create Rute Table" a. Fill the name field, and select yur VPC b. Edit yur rute table t define the default utbund rule c. Click n yur rute table and select the "Rutes" tab belw. Then, Edit it, add anther rute and put: "0.0.0.0/0" in Destinatin. d. Select yur internet gateway in Target and then save. In ther wrds: everything desn't g t ur subnet ges utside. e. Assciate the rute table with yur subnet i. Click n the "Subnet assciatins" tab and then click n Edit. ii. Finally, assciate yur subnet and click n Save. Yu nw have a well parameterized VPC withut Security grup. 2. HOW TO CREATE AND ACCESS INSTANCES VIA AMAZON WEB SERVICES (AWS) MANAGEMENT CONSOLE? If yu are using an AWS educate accunt, access yur accunt and click n Open Cnsle buttn, then click n Services upper left crner buttn. Yu shuld see a page titled Amazn Web Services that lists mre than 50 services with their crrespnding icns. On the AWS cnsle click n EC2 - Virtual Servers in the Clud. Yu shuld be able t see the EC2 cnsle. Examine the drp-dwn menus in the upper right crner. Search the menu that allws yu t change the cnsle t anther regin. At this stage, we assume that yu created and cnfigured yur VPC, yur security grup and yur public/private key pair. This key pair will allw yu t lg int EC2 instances. Perfrm the fllwing tasks: Launch a Ubuntu Micr Instance (t2.micr). Yu can use: Ubuntu Server 14.04 LTS (HVM), SSD Vlume Type - ami-841f46f. Use the key pair yu have created earlier. SSH t yur instance nce it has been launched. T authenticate use the private key yu have dwnladed befre. Cnsult this URL fr available ptins: http://dcs.aws.amazn.cm/awsec2/latest/userguide/ec2-cnnect-t-instance-linux.html. Since yu are using an ubuntu instance, the lgin user is: ssh i key.pem ubuntu@yurpublicdnsinstance (if a public DNS is available) r ssh i key.pem ubuntu@yurpublicipinstance Try ut varius linux cmmands that yu are familiar with. Questins: Intrductin t Clud Page 2 / 9
Frm the EC2 Management Cnsle nte the public DNS name f the instance. Once yu have successfully lgged int yur EC2 instance, run the hstname cmmands. Try t ping the instance frm yur lcal machine. What d yu see? Explain. Change the cnfiguratin t make it wrk. Ping the instance and check that it wrks. Determine the IP address seen by the perating system in the EC2 instance by running the ifcnfig cmmand. What type f address is it? Cmpare it t the address displayed by the ping cmmand earlier. 3. PERFORMANCE ANALYSIS OF YOUR INSTANCE In this ptinal part yu will run tw simple benchmarks n yur prvisined EC2 instance and yur lcal machine t test cmpute perfrmance and memry thrughput and cmpare the perfrmance f the instance with yur lcal machine. Fr this part it is best t have Linux n yur lcal machine (pssibly in a virtual machine). Attempt t identify the physical CPU f the instance. Dwnlad these tw benchmark applicatins. System stability tester: http://systester.surcefrge.net/ STREAM (http://www.cs.virginia.edu/stream/). Surce cde is available in this URL: http://www.cs.virginia.edu/stream/ftp/cde/stream.c Read the tw web sites t understand the functinalities f these tw benchmarks. Unpack the systester archive (fr Ubuntu instances, dwnlad Linux x86_64 binaries: Generic build file), run the systester-cli benchmark n yur Amazn instance and n yur lcal machine. Cmpile stream.c. Fr cmpilatin yu will need t install the GCC cmpiler with this cmmand: sud apt-get update sud apt-get install gcc. Run the stream benchmark n yur Amazn instance and n yur lcal machine and cmpare the perfrmances f the tw machines. 4. DEPLOYING DRUPAL ON AWS In this sectin, we prpse t install a web-based cntent management applicatin, Drupal. Drupal requires a web server in which t run. It makes use f a relatinal database t stre its data. In this case, the web server is Apache and the database is MySQL. It s recmmended t use Firefx as a brwser. Using the package manager install the Drupal 7 package. The package manager will autmatically install a web server (Apache) and a database (MySQL) needed by Drupal. Befre installing Drupal update the lcal package descriptins t their latest versins by typing: sud apt-get update sud apt-get install drupal7 Yu will be asked a series f questins. Prvide the fllwing answers: Passwrd fr the MySQL "rt" user: Invent a passwrd. Intrductin t Clud Page 3 / 9
Mail cnfiguratin: Chse n cnfiguratin. Hw t cnfigure the database: Chse cnfigure with dbcnfig-cmmn. Which database type t use: Chse mysql. Passwrd fr the administrative accunt fr the MySQL database and user: Invent a passwrd. Passwrd t register with the database server (MySQL applicatin passwrd): Leave it blank s that a randm passwrd will be generated. Change the cnfiguratin f the firewall t allw incming HTTP cnnectins. In the EC2 Management Cnsle navigate t Security Grups. Select the security grup assciated with the instance. Click n the Inbund tab. Create a new custm TCP rule fr prt range 80 and surce 0.0.0.0/0. Click Add Rule. Click Apply Rule Changes. Verify that yu can reach the Apache server frm the brwser n yur lcal machine. Enter the public IP f the instance int yur web brwser. Yu shuld see the Apache2 Ubuntu Default Page. The web server sftware is running but n cntent has been added, yet. Register Drupal t the Apache web server and restart Apache by typing: sud cp /etc/drupal/7/apache2.cnf /etc/apache2/mds-enabled/drupal.cnf sud /etc/init.d/apache2 restart Run the initial Drupal cnfiguratin frm yur web brwser. Enter the fllwing URL int the brwser where yu substitute hstname with the public IP f the instance: http://hstname/drupal7/install.php Yu shuld see a page titled Select an installatin prfile with radi buttns that read Standard and Minimal. Fllw the cnfiguratin steps n this page and give the fllwing answers: Installatin prfile: Select Standard. Language: Select English. Cnfigure the site: Enter the fllwing infrmatin: Site name: Invent a name fr the site, fr example "Clud Cmputing at hepia". Site e-mail address: drupal@yurdmaine. Site maintenance accunt username: admin E-mail address: Enter yur e-mail address. Passwrd: Invent a passwrd fr the admin user and write it dwn. On the last page, click n Visit yur new site. Yu shuld see a functinal Drupal site. The site shuld prminently display the site name yu invented earlier. In the AWS cnsle create an Elastic IP Address and attach it t the instance. Click n Elastic IPs in the left navigatin bar then click Allcate New Address. Select EIP used in: VPC. Select the newly created address and click n Assciate. Select the Drupal instance and click n Yes, Assciate. Ping the Elastic IP Address frm yur lcal machine. Yu may have t wait a few secnds until the address is assciated. Using the brwser navigate t the Drupal start page by using the Elastic IP Address. Intrductin t Clud Page 4 / 9
Fr further details abut Elastic IPs (EIP), read this dcument: http://aws.amazn.cm/articles/1346 Questin: Why is it a gd idea t create an Elastic IP Address fr a web site (ur web applicatin)? Why is it nt sufficient t hand ut, as URL, fr the web site the public DNS name f the instance? 5. SCALING APPLICATIONS WITH AWS In this sectin we will study hw applicatins are scaled with AWS. Yu shuld have a micr instance running Ubuntu Server with Drupal7 installed. Let s call this instance: Drupal instance. Yu will imprve the Drupal site t make it scalable. Yur site will be able t absrb traffic increases by adding virtual machines: The business lgic and presentatin layer will be separated frm the database layer s that the frmer can be replicated in multiple virtual machines. The database mves int Amazn's Relatinal Database Service (RDS), which prvides autmatic backup, data replicatin and failver. Sectins a, b and c describe hw t create Drupal instance t use a relatinal database service (RDS) a. CREATE A DATABASE USING THE RDS The gal is t create a new RDS database t replace the MySQL database currently used by Drupal. Perfrm the fllwing steps: 1. Open the RDS cnsle and switch the cnsle t the regin in which yu created the Drupal instance. 2. Launch a DB instance: Click Launch DB Instance and prvide the fllwing answers (leave any field nt mentined at its default value): Select MySQL Cmmunity Editin Prductin purpse: N (Use the Dev/Test database mde) DB Instance Class: db.t2.micr Multi-AZ Deplyment: N Allcated Strage: 20 GB DB Instance Identifier: yuraccuntname-db Master Username: rt Master Passwrd: Give a passwrd. Select the VPC that defines the virtual netwrk f yur instance (make sure that yur VPC cntains tw subnets in tw different availability znes. Otherwise yur RDS instance wn t be created). Publicly accessible: N (reminder: yur hst VPC des nt supprt DNS reslutin) Availability zne: N preference Database Name: leave empty Return t the instances view and wait fr the DB instance t be created. Intrductin t Clud Page 5 / 9
3. In the RDS cnsle select the newly created DB instance and write dwn the Endpint address. In rder t allw cnnectins t the database, the security grup must be cnfigured t allw inbund MYSQL (default prt: 3306). Take nte that yur Drupal and DB instances are running in the same virtual netwrk. 4. Test whether the database can be reached frm the Drupal instance. Lg int the Drupal master instance. Using the database's endpint address (withut the prt number) and the master passwrd yu wrte dwn, run the cmmand: mysql --hst=endpint_address --user=rt --passwrd=master_passwrd 5. Yu shuld see a welcme message and the MySQL cmmand line prmpt mysql>. Fr further details, please see this URL: http://dcs.aws.amazn.cm/amaznrds/latest/userguide/usingwithrds.iam.html T use RDS database, yu have t migrate the cntent f the lcal MySQL database t the newly created RDS database and change Drupal's database cnfiguratin t use the RDS database. We will use Ubuntu's cnfiguratin system (dpkg-recnfigure) t change Drupal's database cnfiguratin frm using the lcal MySQL database t using the RDS database. Instead f directly editing Drupal's cnfiguratin file (/etc/drupal/7/sites/default/dbcnfig.php) yu will edit the cnfiguratin file f dpkg-recnfigure: 1. Lg int the Drupal instance. 2. Add yur ip t the hst file: sud sh -c sud ech "127.0.0.1 hstname " >> /etc/hsts 3. Stp Apache by typing: sud /etc/init.d/apache2 stp r sud service apache2 stp 4. Run the cmmand sud dpkg-recnfigure drupal7. Prvide the fllwing answers: a. Reinstall database fr drupal7? Yes b. Database type t be used fr drupal7: mysql c. Cnnectin methd fr MySQL database f drupal7: tcp/ip d. Hst name f the MySQL database server fr drupal7: enter the endpint address f the RDS database (withut the prt number) e. Prt number fr the MySQL service: prt number f the Hst MySQL server (3306 by default => leave blank) f. Name f the database's administrative user: rt g. Passwrd f the database's administrative user: the master passwrd f the RDS database h. Username fr drupal7: drupal7 i. Database name fr drupal7: drupal7 The cmmand shuld cmplete withut errr messages. 5. Open the file /etc/drupal/7/sites/default/dbcnfig.php and replace the database passwrd generated by the previus cmmand by: drupal7_pwd. 6. T make the authenticatin in the RDS database less strict, cnnect t the database, create an additinal user 'drupal7'@'%' and give this user access rights t the drupal7 database: a. Launch the mysql cmmand t cnnect t the RDS database b. On the mysql> cmmand prmpt run the fllwing three cmmands where drupal7_pwd is the database passwrd fr user drupal7: i. CREATE USER 'drupal7'@'%' IDENTIFIED BY 'drupal7_pwd'; Intrductin t Clud Page 6 / 9
ii. GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES ON drupal7.* TO 'drupal7'@'%' IDENTIFIED BY 'drupal7_pwd'; iii. FLUSH PRIVILEGES; c. Discnnect frm the RDS database and check that user drupal7 can cnnect t the database: i. mysql --hst=endpint_address --user=drupal7 --passwrd=drupal7_pwd 7. Migrate the database cntent t RDS 1. Lg int the Drupal master instance. 2. Type the fllwing cmmand t migrate the database cntent frm the lcal MySQL database t the RDS database: 1. mysqldump --add-drp-table --user=drupal7 --passwrd=drupal7_pwd drupal7 mysql --hst=endpint_address --user=drupal7 --passwrd=drupal7_pwd drupal7 2. The cmmand shuld cmplete withut errrs. 8. Start Apache by typing: sud /etc/init.d/apache2 start r sud service apache2 start 9. Verify the database cnfiguratin by navigating with yur brwser t the Drupal hme page at http://hstname/drupal7/. If yu receive an errr message "The default settings file des nt exist" r "The settings file is nt writable" run the fllwing cmmands: 1. cd /etc/drupal/7/sites/default 2. sud cp -a settings.php default.settings.php 3. sud chwn www-data:www-data settings.php b. CREATE A DRUPAL VIRTUAL MACHINE IMAGE The aim f this sectin is t describe hw t create a virtual machine image frm the Drupal instance. This image will be used later t create new instances with the same cnfiguratin. 1. In the EC2 cnsle bring up the Instances panel and select the Drupal instance. 2. G t the cntext menu and select Image > Create Image. Prvide the fllwing answers (leave any field nt mentined at its default value): Image Name: Give a name t yur instance. Image Descriptin: Drupal cnnected t RDS database. Click Create Image. The instance will shut dwn temprarily and the image will be created. 3. In the cnsle bring up the AMIs panel. Wait until the status f the AMI ges frm pending t available. c. CREATE A LOAD BALANCER The gal here is t create a lad balancer in AWS that receives HTTP requests frm clients and frward them t the Drupal instances. In the EC2 cnsle g t the Lad Balancers panel and chse Classic Lad Balancer (AWS ffers 3 types f lad balancers). 1. Click n Create Lad Balancer. Prvide the fllwing answers (please leave any field nt mentined at its default value): Intrductin t Clud Page 7 / 9
Define Lad Balancer Lad Balancer Name: yurlastname-drupal. Create LB Inside => the selected VPC The lad Balancer Prtcl shuld already be set t http Add 2 subnets as requested Security Grups: use yur security grup Security settings: ignre Cnfigure Health Check Leave as it is fr the prtcl etc. Health Check Interval: 10 secnds Healthy Threshld: mve dwn t 2. Add EC2 Instances Select the Drupal instance. 2. In the EC2 cnsle select the newly created lad balancer. Write dwn its DNS Name (A Recrd). 3. In the lwer half f the panel, click n the Instances tab. Watch the status f the instance g frm Out f Service t In Service. 4. Lg int the Drupal instance. Examine the Apache access lg /var/lg/apache2/access.lg t see wh is cnnecting t the web server. d. LAUNCH A SECOND INSTANCE FROM THE CUSTOM IMAGE In this task yu will launch a secnd Drupal instance and cnnect it t the lad balancer. 1. Using the custm virtual machine image yu created earlier, launch a secnd instance. Make sure that the instance launches in a different availability zne than the first instance (in a secnd subnet similar t the ne created fr the first instance). 2. Make sure that the instance wrks crrectly by navigating with yur brwser t the Drupal hme page f the new instance at http://hstname/drupal7/. 3. Using the AWS cnsle cnnect the instance t the lad balancer (click the edit instances buttn under the lad balancer and add the new ne). Watch the status f the instance g frm Out f Service t In Service. e. TEST THE DISTRIBUTED APPLICATION (LOAD BALANCER, DRUPAL INSTANCES, RDS INSTANCE) In this sectin yu will try t test the distributed applicatin with a lad generatr and use the mnitring tls f the AWS cnsle. 1. Dwnlad and install n yur lcal machine JMeter frm: http://jmeter.apache.rg/. 2. Fllw the instructins n http://fredpuls.cm/site/sftwaredevelpment/java/test/test_jmeter_quick_start.htm t create a simple test plan. Specify the lad balancer as the target fr the HTTP requests. Run a test. 3. Open tw terminal windws side-by-side and, using SSH, lg int each instance. Bring up a cntinuus display f the Apache access lg by running the cmmand: sud tail -f /var/lg/apache2/access.lg. Intrductin t Clud Page 8 / 9
4. Using the AWS cnsle, enable detailed (1-minute interval) mnitring f the tw instances: Select an instance and click n the Mnitring tab. Click n Enable Detailed Mnitring. 5. Observe which f the instances gets the lad. Increase the lad and re-run the test. Observe respnse times and time-uts. Repeat until yu see inacceptable respnse times and/r timeuts. 6. Immediately after having created a high lad fr the site, re-run the nslkup cmmand t reslve the DNS name f the lad balancer int IP addresses t see if there are any changes. Intrductin t Clud Page 9 / 9