Automate ALL the Phings!

Transcription

1 Let your computer do the hard work Automate ALL the Phings! What if you could automate deploying your sites, restoring your database, and sending out reports? With Phing, you can do all this and more. by Joey Rivera Phing, which stands for Phing is not GNU make, is a command line build tool. You write an XML file with instructions of what you need automated, call the script, and it ll perform all those operations for you. It s the same concept as Apache Ant which is used mainly for Java. Phing is very flexible, extendable and platform independent. It is also written in PHP. Why do we need Phing? Efficiency, time, quality to mention a few Phing lets you streamline processes so you don t have to do them. Anything you or your team find yourselves doing manually daily, weekly, or monthly can be automated with a build tool such as Phing so you can focus on more important tasks. An example of a common process is moving files from one server environment to another. Let s say when you need to deploy your site to production, you usually tar/zip your project folder, then ftp/sftp it up to the server. You wait patiently, and when it's uploaded, you log into the server and un-compress the file. That s a few steps which don t sound like much - but wouldn t you rather make one call in the command line and let all that happen unattended, freeing up your time to be more productive? There are various reasons why we use Phing at my job. First, people aren t perfect and will make mistakes. You can ask a person to do the same task 100 times and chances are they ll make a mistake at least once. On the other hand, you can ask a computer to do the same task 100 times and (assuming you asked it to do the right task) it will perform the task correctly 100 times. The next reason was to reduce the amount of time spent by developers doing manual work. No one here likes to spend their time doing what we all see as being busy work so to improve our quality of work-life we looked around for ways to separate ourselves from those tasks and let Phing take over. This lead us to a Web & PHP Magazine istockphoto.com/mecaleha

2 happier team and because we were able to run these processes more often we also became more efficient. Alright, got it how do we get started? The easiest way to install Phing is through PEAR. The assumption here is that you already have PEAR setup on your machine. While in the command line you simply want to discover the Phing channel and then install the Phing package. The commands are: $ pear channel-discover pear.phing.info $ pear install Phing/Phing Software Source PHP PHPUnit Xdebug SimpleTest beta+ phpdocumentor VersionControl_SVN VersionControl_Git PHP_CodeSniffer Archive_Tar Services_Amazon_S3 HTTP_Request2 PHP Depend PHP Mess Detector PHP Copy/Paste Detector Table 1: A selection of Phing dependencies (source: If you don t have PEAR you can download the packaged file at [1]. After you unpack the files, make sure to add the Phing executable file to your system path so you can call it from anywhere. If you run into any issues, make sure to reference the installation documentation [2] as it has lots of great and useful information. You can also install Phing through composer. The only thing about that is normally I setup Phing as a system wide tool and not on a project by project basis which is how I see composer. If you did want to use composer to install Phing you can find more information at [3]. I tried this route and quickly found some dependencies were not in composer therefor not allowing me to install them so I went back to PEAR for Phing. Out of the box Phing has the capability of doing all kinds of things but depending on the use, you may need some extra dependencies. Table 1 has a few example dependencies you may want to consider after installing Phing (Table 1). The two big ones for deployment use are SVN and/ or Git depending on which you use. We still use SVN so that s the next thing I usually install. You can click on the links to get information on how to install those dependencies most of which are installed through PEAR. Don t feel like you need to install everything upfront, you can just install Phing first and down the road when you find out you need something else, install that piece. If everything is working correctly, you should be able to open up a command line window and type Phing v. After hitting enter you should see something like Phing in the prompt. If you don t, go through the install steps again to see if you missed anything. Structure of a Phing build file Project: At the heart of a Phing application is a build.xml file. This file contains all the instructions you need Phing to perform. The file starts out with a project node. Inside the project you can have one or many targets (think of them as methods you d want to call). Each target can have one or many task the actual individual processes to run. There are also a few other things like types, properties, filters, mappers, and conditions but we ll cover those later. Here is a sample build file taken from [4]. In this example you can see the project node at the top with a name of FooBar and a default property with the value dist. Name is simply the name you want to give this project and default, which is a required property, is the target that will get called by default when running Phing on this build.xml file if you do not specify the target. The documentation is extremely helpful at explaining which properties are required and what they do [5]. Target: Targets are grouping of tasks used to achieve some result so I think of them as a method in a class. You can use as many tasks as you need in a target. Targets can also depend on other targets; in the example above, you ll notice the dist target depends on the build target. This means, if you call the dist target directly and build has not yet ran, Phing will automatically run build first and then call dist. Task: Within targets you usually have one or many tasks the single pieces of executable code. These are the actions that you want to take place such as copy, delete, echo, foreach, ftp, etc. There are many tasks build-in to Phing and you can always create your own. For a list of the built-in tasks check the docs [6]. Property: The other piece you ll be asking yourself about is variables? There are a couple ways to create variables. In Phing variables are called properties. These can be defined globally to all your targets or you can also define some specific to a target. The scope is determined by where you create your property. If you create the property in a target, then it s usable in the target scope. You can declare properties inline, as an external file, or by asking the user for input. Below is an example showing all three: Web & PHP Magazine

3 <property name="name" value="joey"> <property file="property.file"> <input propertyname="last.name">what is your last name?</input> An external property file is a good way to store environment specific variables as well. For example, you can have a dev.properties, stage.properties, prod. properties each with the proper information. To create a properties file, simply create a new file and start adding key/value pairs in each line with an equal sign. You can name the file anything you like with or without an extension. I usually call my files properties. Ex: name=joey last.name=rivera version=1.0.0 Keep in mind there are also various built-in properties in Phing such as: application.startdir: Current work directory. env.*: Environment variables, extracted from $_ SERVER. host.arch: System architecture, i.e. i586. Not available on Windows machines. host.domain: DNS domain name, i.e. php.net. Not available on Windows machines. host.fstype: The type of the filesystem. Possible values are UNIX, WINNT and WIN32. host.name: Operating System hostname as returned by posix_uname(). Not available on Windows machines. host.os: Operating System description as set in PHP_OS variable (see PHP Manual). You can find the entire list in the documentation [7]. The syntax to call a variable is a ${variablename}. For example: Listing 1 <project name="foobar" default="dist"> <!-- Target: prepare --> <target name="prepare"> <echo msg="making directory./build" /> <mkdir dir="./build" /> <!-- Target: build --> <target name="build" depends="prepare"> <echo msg="copying files to build directory..." /> <echo msg="copying./about.php to./build directory..." /> <copy file="./about.php" tofile="./build/about.php" /> <echo msg="copying./browsers.php to./build directory..." /> <copy file="./browsers.php" tofile="./build/browsers.php" /> <echo msg="copying./contact.php to./build directory..." /> <copy file="./contact.php" tofile="./build/contact.php" /> <!-- (DEFAULT) Target: dist --> <target name="dist" depends="build"> <echo msg="creating archive..." /> <tar destfile="./build/build.tar.gz" compression="gzip"> <fileset dir="./build"> <include name="*" /> </fileset> </tar> <echo msg="files copied and compressed in build directory OK!" /> <echo>${name}</echo> <svninfo repositoryurl="${svn.repo.url}" /> <equals arg1="${env}" arg2="development" /> Other Goodies: Phing also has types, filters, mappers, and conditions. You can read the documentation to learn more about them but very high level: Types: more complex data types such as a FileSet to include or exclude types of files based on rules. Filters: These transform data such as Tidy, Strip- PhpComments. Mappers: Like filters but for files and directories like Flatten, RegExp, Glob Conditions: Allow you to create conditional statements such as if/else, comparisons, and can be nested. Examples Hello World : A quick basic hello world example would look like the following. Create a build.xml file and add the following to it: Web & PHP Magazine

4 <project name="helloworld" default="hello"> <target name="hello"> <echo>hello ${name}</echo> Now, in the same folder where you placed the build. xml file, type Phing in the command line and enter. You should see Hello ${name}. But what we really want to do is to pass the name as an argument so it shows up. Let s pass an argument to the script and try again. To do this in command line simply call Phing Dname=Joey. This time you should see Hello Joey. What we did was tell Phing we want to pass a property by using D, the property is name and the value is Joey. If you didn t want to pass the name as an argument, you could also declare the property in the build.xml by adding: <project name="helloworld" default="hello"> <property name="name" value="joey"> <target name="hello"> <echo>hello ${name}</echo> This declares a global property name with value Joey. If you try this example by running Phing without passing any arguments, you should see Hello Joey. Deployments This was our biggest area of improvement which was also our biggest headache before Phing. At first, we used to create a list of all the files that were modified for the iteration and manually copy/paste each file over from one place to another. Quite often we d forget one file and create problems for our customers. Then we went to another strategy where we d export the entire application from SVN to production. This had its own problems since the entire application has thousands of files and it was time consuming and inefficient to copy over all these files just because a handful of files changed. Trying to solve this problem was why we started working with Phing. Thanks to Phing we now follow a continuous delivery approach to make sure when code is committed it ends up in the proper server within a couple minutes in an automated fashion. A continuous integration server might have made this easier but we use a Phing script and setup different cron jobs for the different environments and applications we have. Our application on development gets checked often while our applications in the QA environment only get updated once a day. We have a cron rule that calls our deployment script every 2 minutes passing it the development environment and the name of the application to check. Based on the arguments passed, Phing checks the proper SVN repo, in this case trunk, to see if anything has changed since the last time it was checked. If something changed, it does a diff between the two revisions and deploys only the changes to the proper server location. We deploy to production once a month. To deploy to production we call the deployment script, pass the application name and the production environment argument and the script will do a diff between the tag that is currently in production and the newest tag in the repo and deploys the changes. It usually takes only a few seconds to run and best of all, no files are EVER missed anymore! The script is pretty large to put it all here but below are a few snippets of the build.xml for examples. <svndiff repoa="${url.repo.from}" repob="${url.repo.to}" propertyname="svn.diff" /> This uses a custom task we created to compare two different svn repositories for changes. This is how we know if there are any changes and what they are. It s basically running svninfo xml sumarize repoa repob. Listing 2 is an example of using a try/catch in case you know there are scenarios where things could break. In this case we run svn info on a repository that may or may not be there and create a property check. repo with the value of broke if it s not found. In this example you can see how we used an if/ else to run a different command on linux vs windows. Most of the times we try to make our scripts platform independent but there are times where we need to do these types of checks. <phingcall target="done"></phingcall> The above line is an example of calling another target inside of a target. In Listing 3 you can see how we used an if/else to run a different command on linux vs windows. Most of the times we try to make our scripts platform independent but there are times where we need to do these types of checks. Listing 2 <trycatch property="tryworked"> <try> <svninfo repositoryurl="${url.repo.from}" element="entry" subelement="path" propertyname="check.repo" /> </try> <catch> <property name="check.repo" value="broke" override="true" /> </catch> </trycatch> Web & PHP Magazine

5 DB Restore: Pre-Phing, we rarely used to restore databases to match the data in production. This lead to multiple issues such as being unable to recreate problems in our staging environment since the data didn t match production. Or when finding an issue in our QA server, we didn t know if this was caused by the code changes in the new iteration or a false-positive due to invalid configured data. The reason we were restoring the database was because it just took too long. Having many copies of each database (development, staging, QA, and testing) it would take a person all day to get the latest backup from production and one at a time create all databases. The next issue was after all database were created, make sure to remember to run specific scripts on specific database servers to obfuscate the data and/or create account for our QA testers. After months of trying to do the above more often always seeming to forget a step in the process we decided it was time to let Phing take over. Within a few days of planning, crafting and testing, we came up with our first functional Phing script to take over our Listing 3 <if> <equals arg1="${host.fstype}" arg2="unix" /> <then> <exec command="chmod -R 775 ${path.build}" /> <exec command="${copy.command} ${path.build}/. ${copy.to.path}/" checkreturn="true" /> </then> <else> <exec command='xcopy "${path.build}" "${copy.to.path}" /E /Y' checkreturn="true" /> </else> </if> database restorations [8] (what we call it here). Over the last year it has been tweaked to make it more robust and this is what the script does now: Prompts for ssh username Cleans up environment (delete all tmp folder and create new one) Scp latest production db.dump file for the specific database Loops and sets all sites using the db s to maintenance mode by adding a file to the site Restarts postgres to kick off any active connection to the db Loops and drops/recreates the databases Restores staging from the production dump file All others are templates of staging which in Postgres executing much faster to create from template than to restore from scratch. Runs any database specific scripts Turns all sites back on Reporting: We have a third party vendor we use for handling loan applications for us. Every night we need to grab a report from their site to see who has applied for a loan and what the status of their application is. We had an individual log into the third party s system every morning, click around the site to get to the reports, select the correct date range, generate and download the PDF and finally it out to a few key people. This took enough of someone time where we felt it made sense to simply automate this with Phing. We talked to the third party vendor and asked them to automate this report and sftp it to us every night. We then created a Phing script that goes into our sftp location, grabs the file, and s it out to the few key people at a certain time of the day every day and then moves the file somewhere else. It took us but a couple hours to do and now we can free time from this person s day to focus on something else. The only tricky part about this was the piece. For this, we used the Exec task in Phing to execute a shell command in our Linux environment. The task simply called the mutt command line client passing the proper parameters for the attachment file and recipients and that was it, everything worked like a charm. Conclusion Hope you enjoyed the article. We covered a few benefits of using Phing as well as a few examples on how we use Phing. I do recommend spending some time trying it out, I m sure everyone could find a good use for it. If you have any questions or need help understanding the scripts above feel free to reach me In working with web technologies, Joey Rivera discovered a passion for PHP. He has spent the last few years at Cengage Learning [9] working on multiple projects and improving processes along the way. Not only does he enjoy development but he enjoys sharing as well through his blog [10] or presenting at the local Atlanta PHP Usergroup [11]. When not in front of a computer you can find Joey at the gym, driving his car around the track, or hanging out with his wife and daughter. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] Web & PHP Magazine