TERSUBSCRIBE. Making testing more. E s s e n t i a l f o r s o f t w a r e t e s t e r s. It s FREE for testers. April / 5 v2.

Size: px

Start display at page:

Download "TERSUBSCRIBE. Making testing more. E s s e n t i a l f o r s o f t w a r e t e s t e r s. It s FREE for testers. April / 5 v2."

Barry Baker
5 years ago
Views:

TE It s FREE for testers TERSUBSCRIBE E s s e n t i a l f o r s o f t w a r e t e s t e r s April 2014 4 / 5 v2.

1 TE It s FREE for testers TERSUBSCRIBE E s s e n t i a l f o r s o f t w a r e t e s t e r s April / 5 v2.0 number 26 Making testing more Including articles by: Darren Coupland Sogeti UK Leandro Sanchez Build IT Solutions Plinio Aquino Jr Centro Universitário da FEI Sakis Ladopoulos Intrasoft International Harry Sneed Chris Verhoef VU University Amsterdam Llyr Jones Grid-Tools Mike Howse Apica This issue of Professional Tester is sponsored by:

Spot the difference by Leandro Sanchez and Plinio Aquino Jr Making testing more automated Leandro Sanchez and Plinio Aquino Jr propose an automatic verification method A few years ago, the varying

Since the height of the browser wars matters have improved greatly as more browsers have become more compliant with standards, reducing the instances of deliberate incompatibility failure.

2 Spot the difference by Leandro Sanchez and Plinio Aquino Jr Making testing more automated Leandro Sanchez and Plinio Aquino Jr propose an automatic verification method A few years ago, the varying rendering behaviour of different browsers was one of the biggest challenges in testing. Since the height of the browser wars matters have improved greatly as more browsers have become more compliant with standards, reducing the instances of deliberate incompatibility failure. Nevertheless defects (see for example cgi?id=68995 retrieved 20th March hrs UTC) still occur often, and they have become harder to manage because of automatic updating of client software. Mobile apps are just as vulnerable as web to this problem, and device proliferation has multiplied complication for both. Purely visual defects are among the hardest to detect. Even manual test execution is very prone to missing them, particularly if they do not cause deviation from the expected outcomes and are not very obvious. Automated execution (including with tools that feature image recognition to locate interface objects) is much worse, because it seeks only to verify the presence of expected display, usually specific text. There is no viable way to automate testing for the presence of unwanted display or its incorrect visual appearance ( deformation ). This article discusses our research on a proposed method to verify automatically that what is displayed by an application is visually as expected. Possible approaches This should not be confused with (essential) testing done to avoid rather than detect display failure. That includes ensuring the product code is itself compliant with applicable standards, searching for constructs within the code related to known browser differences and comparing how different browsers themselves PT - April professionaltester.com 7

3 <!DOCTYPE HTML PUBLIC -//W3C// DTD HTML 4.01 Transitional//EN > <HEAD><TITLE></TITLE> <style type= text/css > body{margin-top:2px; #main_frame{ height:550px; float:center; text-align:center; background:black; #logo{ width:275px; background-image:url(images/logo. png); background-repeat:no-repeat; #top_center{ width:474px; top:0px; background-color:green; filter:alpha(opacity=50); -moz-opacity:0.5; -khtml-opacity:0.5; opacity:0.5; #tag2{ margin-left:256px; #tag3{ margin-left:512px; background-color:#960; #tag4{ display:inherit; background-color:#990; #top_right{ width:275px; top:0px; background-color:red; margin-left:768px; #main_menu{ height:25px; float:none; background-color:#ff0; #content_frame{ height:400px; #tag1{ background-color:#900; background-color:#930; </style> </HEAD> <BODY><div id= main_frame > <div id= tag1 ></div> <div id= tag2 ></div> <div id= tag3 ></div> <div id= tag4 ></div> <div id= logo ></div> <div id= top_center ></div> <div id= top_right ></div> <div id= main_menu ></div> <div id= content_frame > </BODY></HTML> Figure 1: example HTML/CSS document 8 PT - April professionaltester.com

Figure 2: example document rendered by Chrome Figure 3: deformed example document rendered by IE public

get(args[0]); try { File scrfile = ((TakesScreenshot) driver) getscreenshotas(outputtype.file).

4 Figure 2: example document rendered by Chrome Figure 3: deformed example document rendered by IE public static void main (String [] args) { WebDriver driver = new ChromeDriver(); Dimension dim = new Dimension (1366,768); driver.manage() window() setsize(dim)..; driver.get(args[0]); try { File scrfile = ((TakesScreenshot) driver) getscreenshotas(outputtype.file).; FileUtils.copyFile(scrFile,new File(args[1])); Catch(IOException e1) { e1.printstacktrace(); driver.quit(); Figure 4: capturing an image of browser window content Figure 5: output of phash discrete cosine transformation demo PT - April professionaltester.com 9

5 interpret the code using their DOM inspection tools. These methods don t take account of browser differences that are not already known, in how a browser renders a given DOM, or in the effect of client-side procedural code (eg JavaScript). None of them can detect that what is actually displayed is not correct. We suggest the only way to achieve that is at the pixel level, by taking and analysing images of what is displayed in order to compare them with what is expected to be displayed. This concept could in theory be applied to any application whose display could be affected by any environmental factor, not only web browser. Image acquisition Figure 1 shows an example HTML/ CSS document, simplified but intended to be reasonably typical of commercial, customer-facing applications, which is rendered differently by Chrome 26 (figure 2) and IE 10 with Compatibility View on (figure 3), both running on Microsoft Windows 8. If you are a web tester you are probably already checking the doctype and thinking about the box model, but here we are not concerned with what causes the deformation, but devising a method to detect it automatically. The Java code in figure 4 uses Selenium Webdriver to capture an image of what is in Chrome s main window. The equivalent for other browsers is similar. Image comparison Although it can be useful in regression testing on a particular browser or device, a simple file comparison is no use here for many reasons, most of them obvious: dynamic content, resolution, fonts, colour palettes, graphics hardware, etc. Rather we want to detect the large differences that signify serious deformation, by measuring how visually similar two screenshots are despite all these factors. Algorithms to do this have been developed for web image search functions (see eg images/searchbyimage.html). For this article we experimented with an implementation specially intended to help with testing rendering, PerceptualDiff ( pdiff.sourceforge.net), and one designed for other applications, for example in detecting copyright infringement, phash ( PerceptualDiff uses a computational model of the human visual system to predict whether two images are visibly different, returning a simple pass or fail. It also reports the number of different pixels. Because it works in this way, for it to be effective the compared images need to be of the same pixel size. This is a problem because of the different window layouts between the browsers. We attempted to correct this by calculating the average width and height of the images to be compared, then resampling both to these dimensions. The resampling was done using the ImageMagick API (see org). The total number of pixels in each image is then easily calculated as width height, for comparison with the number of different pixels reported by PerceptualDiff. Of the algorithms implemented by phash, we used discrete cosine transformation which compares low-frequency components of an image and ignores high-frequency ones, similarly to lossy compression algorithms such as JPEG. It returns a hamming distance value indicating how similar are two images (see figure 5): the lower the value, the more similar the images: identical images return 0. A value greater than 10 usually signifies very low similarity. We found that phash works well even on images of different pixel sizes and resolutions, probably because it is designed to detect similar images despite modifications including those intended to prevent that (which its authors call attacks ). Usefulness and improvements The method was tested on various other pages similar to the example, known to render significantly differently in certain browsers. This was detected accurately in all cases, with good correlation between the results returned from the two algorithms. The method was also tested on various third-party websites. Results here were more varied, although the rate of successful detection of gross deformation, especially in IE, was generally good. PerceptualDiff sometimes detected significant deformations which phash did not because of its inability to detect physically small yet visually obvious differences, for example in text and small graphics. Where false positives occurred they were usually caused by attempting to use the proportion of different (reported by PerceptualDiff) to total pixels. More work is needed on how to decide what proportion of different pixels should be considered to indicate significant deformation. This is probably not a constant, but dependent on properties of the page under test. It is possible to imagine ways to calculate it, and/ or the use of an algorithm that can be trained to eliminate false positives by a small amount of human intervention. Inaccuracy might also be caused by the image resampling described above: possibly a more sophisticated resampling algorithm is needed Leandro Sanchez (leandrosanchez@me.com) is a business development executive at Build IT Solutions ( This article is based on his master s degree thesis written with the mentorship of Plinio Aquino Jr (plinio.aquino@fei.edu.br), associate professor and head of human computer interaction and the Intelligent Interfaces Group (HC3iGroup). The authors full affiliations follow for academic credit purposes. Leandro dos Reis Sanchez: MESTRADO PROFISSIONAL ENGENHARIA DE COMPUTAÇÃO, INSTITUTO DE PESQUISAS TECNOLÓGICAS (IPT), USP, SP, BRASIL. Prof. Dr. Plinio Thomaz Aquino Junior: INSTITUTO DE PESQUISAS TECNOLÓGICAS (IPT), USP, SP, BRAZIL. Centro Universitário da FEI, São Bernardo do Campo, SP, BRAZIL 10 PT - April professionaltester.com

INTERNATIONAL UNIVERSITY OF JAPAN Public Management and Policy Analysis Program Graduate School of International Relations

Hun Myoung Park (2/2/2018) Layout & Position: 1 INTERNATIONAL UNIVERSITY OF JAPAN Public Management and Policy Analysis Program Graduate School of International Relations DCC5382 (2 Credits) Introduction