ARC tools for revision and nightly functional tests Jozef Cernak, Marek Kocan, Eva Cernakova (P. J. Safarik University in Kosice, Kosice, Slovak Republic)
Outline Testing strategy in ARC ARC-EMI testing tools Build testing Nightly (every night) Revision (after each commit of developers) Automatic functional testing of nightly builds Future plans Increasing number of automatic functional tests Automatic performance testing Conclusion this field 2
Testing practice in ARC Several roles in project: developers, testers and ARC release manager Adopted tests: Unit tests(developers) Build tests (testers) Revision (Kosice) Nightly (Copenhagen) EMI-RC (ETICS) Functional tests (testers, developers) Manual Automatic (Kosice) Regression testing (testers, developers) Performance testing (testers) Manual Automatic in development phase Large scale testing (in preparation phase) this field 3
ARC-EMI testing tools Relation between ARC and EMI testing tools this field 4
Block diagram of automatic revision and Functional tools Simple database server with PHP and Python scripts this field 5
Technical details Infrastructure (local cluster) 2 SUN 2200M2 servers (2x4GB RAM, 250GB disk, AMD 64 bits) 2 desktops (2x 1GB RAM, 500GB disk 32 bits) 1 MAC mini (2GB RAM, 160GB, 32 bits) Software MySQL, PHP, Python Code is free Distributed data processing SVN this field 6
Revision automatic testing Features Supported are several platforms of OS: CentOS (32,64), Debian(32,64), and MAX OS (64) Detailed identification of errors during building process Autogen Configuration make make dist Build this field 7
continued. User friendly interface to define search criteria Daily Weakly Monthly Custom Revision number, etc (see next examples) Direct connection with SVN Warning messages for developers if code is broken CCCC metric Quick mechanisms to identify breaking event this field 8
Example of www interface for automatic testing tools Home page: http://arc-emi.grid.upjs.sk/ (Revision database is slow) this field 9
Several criteria to search results of revision testing this field 10
Example of the results of daily revisions this field 11
Automatic functional tests Features Automatic download code from SVN based on several criteria: trunk, revision code, nightly builds Automatic build Automatic deployment User friendly interface for: submitting proposals of test cases to search the results of functional tests Tests are grouped into two main groups: functional tests of server, functional tests of client. this field 12
Interface to enter search criteria this field 13
The results of automatic functional tests this field 14
Functional test scenaria client_arcsub_to_arex_gridmap_simplejob_jsdl client_arcsub_to_grid-manager_gridmap_simplejob_jsdl Submission of a simple JSDL job to a GIIS service using fastest queue broker. Failed Submission of a simple JSDL job to an ISIS cloud using fastest queue broker. client_arcsub_fastestqueue_broker_giis_simplejob_jsdl Submission of a simple JSDL job to an GIIS cloud using random broker. client_arcsub_fastestqueue_broker_isis_simplejob_jsdl Submission of a simple JSDL job to an ISIS cloud using random broker. client_arcsub_random_broker_giis_simplejob_jsdl Submission of a simple JSDL job to an ISIS cloud. client_arcsub_random_broker_isis_simplejob_jsdl Submission of a simple JSDL job to a GIIS service. client_arcsub_to_isis_simplejob_jsdl Tries to submit a simple JSDL job to a grid-map secured ARC0 CE. client_arcsub_to_giis_simplejob_jsdl Tries to submit a simple JSDL job to a grid-map secured A-REX service. There was some error before we managed to start testing. The error could have been in code retrieval, installation or somewhere else. this field 15
Automatic performance tests Requirements: several servers and clients Monitoring of several distributed processes: resources: Memory CPU Network usage Performance Scalability Number of concurrent requests Reliability of services: Ratio between successful tasks and total tasks. Examples of tasks: job submission, file transfer and etc. this field 16
continued. Proposed infrastructure: Grid servers and clients, we plan to use existing infrastructure Communication layer XML-RPC client-server architecture Central database server Common Python library on each server and client (for example start stop service) Central control program this field 17
Conclusions Automatic testing covers: Build process (ARC, ETICS), Functional, Performance tests (in progress). Utilization of automatic test tools increases probability to find weak part of the software. On the other hand, manual tests are important. this field 18
References Testing in ARC http://wiki.nordugrid.org/index.php/testing Revision tests http://download.nordugrid.org/builds/ http://arc-emi.grid.upjs.sk/revisiontests.php Functional tests http://arc-emi.grid.upjs.sk/functionaltests.php Code: http://svn.nordugrid.org/trac/workarea/browser/arct estscriptshttp://svn.nordugrid.org/trac/workarea/bro wser/arctestscripts this field 19
Thank you EMI is partially funded by the European Commission under Grant Agreement INFSO-RI-261611 this field 20