Hi. I'm Prateek Baheti. I'm a developer at ThoughtWorks. I'm currently the tech lead on Mingle, which is a project management tool that ThoughtWorks builds. I work in Balor, which is where India's best tech happens. Before Mingle, I used to work on Golang, on another ThoughtWorks product called Gauge. Gauge's test automation solution built by ThoughtWorks products division. Today's talk is going to be heavily influenced by my learnings and experience as a part of Gauge. What I'm going to talk about today is how to do cross browser testing with Gauge and WebDriver. Cross browser testing is a common problem that teams that build web applications face. Hopefully by the end of today's talk, you'll have a basic understanding of what Gauge is like, and you'll learn how to build a cross browser testing suite using Gauge and WebDriver to test your application. Let's get started. We'll start off with a brief introduction of what Gauge is. It's a test automation solution that's being built by ThoughtWorks; it's cross platform, and it's open-source. It supports multiple languages, like Ruby, Java, C#, Python, Javascript, and it has upcoming support for other languages, like Golang as well. These are contributions from the community, as well as the core team that works over at ThoughtWorks. It does have a lot of plugins that come along with it, for report generations and dependency managements. Along with Gauge, we'll be using WebDriver. If you're from the testing industry, you must be familiar with WebDriver; it's a standard browser driving library that a lot of people use for testing web applications. What we'll look at today is a combination of Gauge and WebDriver to build a cross browser testing suite. What I will also be using, is the Java Runner, which is the Java plugin for Gauge. What Gauge does, is it gives you a business layer on top of your test scripts to make it more readable and more understandable. We'll be using the test scripts in Java, and Gauge will be the business layer on top of it. Because we're using Java, we're using IntelliJ. The IntelliJ plugin that comes with Gauge gives it a bunch of useful features that will be required for us to write the test scripts in Gauge. Why I chose Gauge for building this cross-platform suite is some of its powerful features, like parallelization and managing amendments seamlessly. It makes a very simple tool build, as well as maintain the suite as it grows. Without further ado, let's get started with how you go about it. Considering you have Gauge installed, and the IntelliJ plugin installed, what you could do is create a new project. Let's call it "Cross Browser," and I'll be using Java 1.8 with this... In this example that you're going to see today, I will be just running a few scenarios against the Gauge website itself. We'll just be looking at Gauge, "Let's get Started, Contribute, Documentation," etc, just to verify that some elements that are pleasant on that are available. We use WebDriver and JUnit as the session framework. The idea is to be able to run against three different browsers, in this case we will try to run against Firefox, Chrome, as well as PhantomJS. PhantomJS is a headless browser, sometimes used when you don't have a GUI. We'll set it up in such a way that you can run against either of them, or a mix of them, or in parallel as well. Let's see how simplified it can get with the features of Gauge that I was [5:06 - putting on there].
Just to go over a few things in this project, the most important things are the specs directory in this Gauge project, which are test specifications that's basically your business test case, written in a business language. They're written in markdown format in Gauge.Each file's got a specification which can have multiple scenarios, so if you're testing search functionality, they will be in the specification, and there will be multiple cases that you test in it, which would be your different scenarios. These bulleted points here, are steps that actually get executed. Against each bulletpoint, there will be some underlying Java code. If I use the IntelliJ shortcut to navigate, I can actually go to each steps corresponding Java code. The idea is that we write our business flow in this natural language, and be able to have the lang of Java based WebDriver implementation, which will run against different browsers. This is how you initialize a project, and these are the specifications; you can run them, or you can run them with Gauge's first/last command line utility. I have a pre-set up project with a parallel execution against different browsers set up, and I'll walk you through what has been done here; it should be very straightforward to do the same. You can also find this project on GitHub, I will share a link at the end of this talk for that. Jumping right into the specifications, this is the most basic spec I've written. It's very simple, navigate to the Gauge website, got to the Getting Started page, and go to the download section. Go down underneath, and you have "implementations"; you can put them in any class, and annotate them accordingly for an implementation, if you're a writer. New step. You can also just use the helpers to create an implementation, which can actually create the schedule for you, and you can use that...so, how is the WebDriver hooked into the execution code here? If you go to the underlying implementation for "Navigate to [8:13 - http://getgauge.io ]," I also notice that this particular value has been given in codes and has a different color; that's because it's a parameter that this method takes. You can say, in this case, I have a driver class, which has a prior instance, which will get the URL. If you look at this driver class, it's a class with a static instance to the driver instance. This is the interface for WebDriver, and this will change based on the browser that you're running against. If you're running against Firefox, WebDriver has a Firefox instance, or if you're running up on Chrome, depending on how you're set up (in this example I have set up, it would either be Chromedriver, Firefoxdriver, or PhantomJSdriver, which we will change based on a simple amendment [9:16 - value], and be able to run, even, in a mix of browsers at one particular [9:23 - call? code?]. How does WebDriver get initialized and where? For that, you can actually look at how the static instances get set. When you set WebDriver up, this is the settings, and this is the driver class. If you notice, it's getting created here in an initialize driver method, which is a Before scenario, which has an annotation called BeforeScenario. What is BeforeScenario? It's context execution, it's an execution hook for Gauge, where what you can do is run a piece of code or particular step before every scenario starts. This entire block is a scenario; I can write another scenario with a few steps by marking it with a markdown like this, and the next steps will actually be the new scenario. What will happen, is that initialization will run before the scenario, and it also had
a teardown, which is an AfterScenario hook. You can have multiple of these, and they will run before and after all your scenarios. We actually put the browser here... so, making sure that [10:47 - at least our] session, the browser and that gets dominated at the end of each scenario's execution. That's where the initialization goes. Similarly, I have other specifications here. A contributes spec, you will see, if you notice again, it's navigating, and it just verifies that these particular values of sections exist. These tables that are defined in markdown actually get passed into the step implementation as table structures, so they get [11:37 - theorized] right into this table and we can use these tables as a list of rules and, run against this entire [11:47 - daily sys] and still not have to repeat the same step for multiple values. You can posit your structures into the code from this business layer. Also, there is no mention of web browser running, or what exact specific action's happening at this level, so I'll abstract it down so it's more readable to you. Let's go back and see why and how the browser actually changes in the underlying implementation here in WebDriver. For this, I need to talk about a feature in Gauge called Environments. If we go back to this simple project here, you notice this Env directory. There is a default directory inside it with a couple of property files. These are [12:52 - an animate] values that will get set and will be available to you at run time, so those are still environment variables that will get set by Gauge, and you can change behavior at run time based on them. By default, there's just one environment, but coming back to our cross browser project, you'll see that there is four or five environments that I've defined here. One is Chrome, Firefox, PhantomJS. Because we're running against these three browsers, randomizers will be running against a random browser. We'll come back to that. If you look at the property value inside, what I'm doing is I'm setting this browser value to Chrome; I'm also setting the custom reports to retrieve, but it's not very important here. Again. that's the default, and Firefox is setting it to Firefox... What this will do, is at run time I will have an animate value called "browser" available, and I can change the instance of WebDriver that I create, based on that. How do I specify which environment I want to run against? You can run Gauge tests from the IntelliJ addon itself, I can just right-click and click on configuration, fit the environment, so if, let's say, I want to run against Chrome, I say Chrome here, and I start this execution (which will mean that this test will actually run against Chrome). You should see the Chrome browser popping up...here it goes. Similarly, I can change that to Firefox, and it will run against Firefox. How does that get set up? Let's look at that right now. Our class is where we get our WebDriver instance from, and we can see how different drivers get initialized based on the environment. Let me open up the driver /Java, and we initialize driver, which sets the static WebDriver instance here. This driver factor, or getdriver is where the environment is being utilized. If you remember looking at the property files that are getting set, we set an environment variable called "browser" to the name of the browser that needs to be used. What we do is we fetch the environment variable here in Java, and this get WebDriver method. If nothing is set, we default to Firefox. Or else, if it's equal to Chrome, we get the
Chromedriver, if it's equal to PhantomJS, we create a PhantomJS driver; else, it's a Firefox driver. Just so you know, if you're running a Chromedriver, you will need the Chromedriver executable npath. I have Chromedriver executable in User [16:02 - bidden]. You want to make sure you're doing that when you're using Chromedriver to test WebDriver. What this does is essentially sets up the WebDriver instance here, and when a particular scenario runs, for example this "contribute" scenario, it would call the BeforeScenario, which would getdriver each time, and the AfterScenario would quit the driver so the browser session is actually cancelled for each execution, making sure that, each time, the browser is initialized correctly. If I wanted to run this against Firefox, let's say, I would change this to "Firefox," and this test would now run against Firefox as my browser...you can now see Firefox opening here and the test going through. With Gauge, you don't necessarily have to run your test specifications from IntelliJ; it does have first-class command line support. I can go to the project, here, and run Gauge and specify here --env which browser I want to run against, and fit specification, in particular, looking at specification three, against Chrome. I can specify the environment by a simple flag. If I want to change that to FIrefox, all I have to do is erase the environment, and that particular property will be set, which will make sure that the driver initialization takes care of initializing a driver of that particular browser. That's the simple setup. If you notice, it's as simple as you've seen in that one flag when you execute; it makes life a lot easier. One thing i forgot to mention is where the WebDriver dependency sits in the Gauge project. If you notice, we have a LIPS directory, and I have the standalone Selenium 3 JAR inside there. Gauge, by default, picks up those JARS and adds them in class path while running your tests for Java. If you don't want to check on dependencies that way, you could use Gauge's Maven or Cradle plugin to simplify that for you. Just to keep things simple, I'm just now going to all the plugins, I have checked on these dependencies and the LIPS directory in this particular project. Moving on to another very powerful feature of Gauge which I mentioned previously, which was parallel execution. This makes a lot of things simpler for us, especially when you're running a large suite of acceptance tests, that run against web app. They tend to take more time, and parallelization can really help reduce that build time, the time from a checkin to be able to deploy quicker, have faster turnaround times for bugs and features. Gauge actually has a parallel execution feature built in, so it's very, very simple to run specs in parallel. Let's, for example, have four specifications here. I am going to create specifications here...let's run default - that would be against Firefox - and all I need to do is stick this in parallel checkbox, which will ensure that these specifications now run in parallel. Once I hit run, what you should see now is, because we have four spec here. Gauge is going to run it in four parallel streams, which means each stream would initialize Firefox browser. If you notice, there's four Firefox running on my system now, each one running each specification parallely. Because we open and close the browser at a [21:06 - scenario level], it should not share any scenario between each other, and go through them. I can change that, and effectively have this run against
Chrome if I want to. What that should do, is have my specifications run against four different instances of Chrome, in parallel. This is where another new environment that I've introduced called Randomize comes in. What Randomize does in this case, is picks up a browser in random from the browsers that you have selected. If I go to the driver factory class, and if you see Randomize, what it does is return random web browser, and what, essentially it's doing, is it's fetching one of Chrome, PhantomJS, or Firefox. This, together with the parallel execution, what would happen is each of the parallel execution threads would run one of the browsers, or randomly pick a browser. If you had hundreds of tests, and you just wanted to run one small smoke batch, you could run a selection of them parallely against multiple browsers, very randomly chosen - which is a very powerful thing. In this case, it will be really simple to actually do that. All you need is to say "in parallel," and choose the randomize option. We're going to see four specs run in a random one, either Firefox or Chrome or PhantomJS; but you can always configure this for more than these three browsers. If you're looking for Opera or Safari, even a mobile browser like Android, or it can be done using the multiple WebDriver implementations. You notice Firefox as well as Chrome came up in this - PhantomJS did as well, it said less, so you just didn't see it. Also to note, this parallel execution, again, runs from the command line; you can just specify the environment that you will [24:06 - itemize] and the specs, and that's all. It's as simple as that: you have all your test specifications running against randomly picked browsers in parallel. Also, you can configure how many tests you want to run in parallel at once. Let's say you don't want to run four parallel streams, you don't have a machine that powerful. By default, what Gauge does is it picks up the number of calls in your system. This is an eight-code machine, so it will parallelize up to eight streams if I am running it on my machine without a custom specification. If you don't want to run in parallel that many streams, you can use the "minus n" flag, and say just do two parallel streams. What this will do is this will just if not necessary, [25:14 - go in / screen] two parallel streams. Again, you can do that at IntelliJ's site as well. You can mention how many parallel notes you want to run based on your system performance. It makes parallel execution, as well as cross browser execution, very straightforward and simple to toggle. Of course, not without help from WebDriver, which provides a very neat interface onto different browser implementations. With the power of both Gauge and WebDriver, you could achieve this very easily. Not to forget all the good news you get from the business layer itself, that Gauge can give you. This project that I haven't talked about yet is this Resources directory. It has a bunch of files; you notice a CSB file, as well as DXT. Gauge also has this concept of special parameters. Special parameters are different from these regular stream parameters, or these table parameters that you pass into a step. What you can do is you can externalize these parameters, and it turns a couple of pre-defined special parameters, one is a table, and another is a file. If you have a table that has too many columns, let's say hundreds of columns, and a CSB, and you don't really want to add that to your specification and make it really unreadable, what you can do is you can put it in an external file, and just say "Table...section...," just a part of the file.
In this case, it's the CSB file that has all the section titles that it's actually asserting for in that particular scenario. Similarly, you have something called a file parameter, where if your string parameters are too big, let's say spanning multiple lines, you wouldn't want to add them to the specification right here because it would just be very unreadable. You can move it to a file, and Gauge actually reads that file, and when it executes this particular verify step, it gives you the content of that file in the string parameter. What you will essentially be getting is an externalized parameter. This makes your domain look cleaner and more readable. These are just some addons that Gauge provides. Also, we have run so many specifications in this case against different browsers in parallel, Gauge does provide HTML reports through an HGM end report plugin that can generate for each execution. You don't have to worry about any sort of report [28:39 - botching], or anything like that, because it has in-built parallelization capabilities. Just for example, I can open this particular report from executing all scenarios. [28:52 - Vetted] in parallel or not, it doesn't matter; it will still generate a similar cumulative report, even when you do just one particular run or anything else. If you notice, it marks everything in green; you can also see external values here. You can see how many tests passed, failed, what time they were run, what project they were for, in a very nice, clean HTML report. Regardless of what browser you run against, you will have this information available to you. If an HTML report isn't enough, and you're looking for a [29:46 - JUnit side], XML reports, there is also a Gauge plugin. If you look under plugins, you will find an XML report plugin. Adding a plugin to a project is very simple. Under its manifest. JSON file, you can just say which plugin you want to add. Either you can do that, or if you don't want to be editing this file manually, you can say "Gauge minus minus add plugin XML report," and the plugin gets added to your project. Also, it has first-class support to install plugins. It looks like I do already have it installed. If I were to execute it again, I should see an XML report generator. That's particularly useful if you have your build systems or [31:02 - exposing test data to] or any other tools, it's a very standard format, so you can get Junit side XML reports. One strategy to do a cross browser execution in Gauge could be using its table to do an execution model. How that works is given a particular scenario, a specification in a scenario specification, you can run every scenario against a different data set. You do that by defining a table on top of all the scenarios. Let's say if we define a browser, we could say Chrome, Firefox, and PhantomJS. What will then happen is you could have one step defined here...initialize browser. Instead of doing the Before and AfterScenario, you could initialize your browser in this particular step here. How table to an execution works, is it runs all your scenarios against each value. For against Chrome value, this would run once; against Firefox, it would run once; in PhantomJS, it would run once. If you think this table would get repeated in too many specification files, you can also externalize this. This would be another way you could run all your scenarios against different browsers; it's not my favorite, but definitely a possibility. I would still prefer the one where you set up multiple environments; it seems cleaner, and it reduces the amount of text that your business specifications actually have.
I'd like to do a quick recap of what all we talked about, before wrapping this up. What we saw was how you can use Gauge and set up a simple project, initialize a simple project; what Gauge specifications look like, how they're mapped to underlying code; how you can majorly, primarily, use the environments in Gauge and set up different user properties, which will be available through your Java code at run time; and how you can use the scenario hooks and initialize your driver instance; when you actually initialize the particular browser you can select based on the environment that you chose. Another thing we looked at was how to randomize, and run in parallel against multiple browsers. Not to forget [34:51 - getting] an execution, as well as externalizing the parameters. These are some of the features of Gauge, and how you can achieve cross browser parallel execution with them. You could build up this test suite and still have a lot run on time in your suite against multiple browsers. You can choose to run them against a single one, or a combination of two or three, based on your requirements. I hope you found this useful, and that you'll be able to try Gauge, specifically this pattern to cross browser testing. If you are looking for more information, you can find me on Twitter, this is my Twitter handle @prateek_baheti, and my GitHub handle ( github.com/prateekbaheti ). This particular project that we were looking at is available on GitHub open-source, so you can play around with that. You just need to install Gauge on your system before you get started with this Gauge cross browser testing project. I hope you find this content useful. I would like to thank Automation Guild for providing an opportunity to talk about Gauge in a very unique conference setup. I will see you folks in the Q&A section. Thank you.