UNIVERSIDAD DE CASTILLA-LA MANCHA ESCUELA SUPERIOR DE INFORMÁTICA GRADO EN INGENIERÍA INFORMÁTICA TRABAJO FIN DE GRADO

Transcription

1 UNIVERSIDAD DE CASTILLA-LA MANCHA ESCUELA SUPERIOR DE INFORMÁTICA GRADO EN INGENIERÍA INFORMÁTICA INGENIERÍA DEL SOFTWARE TRABAJO FIN DE GRADO AVEMARESS Tool for the Automatic Verification of Mainframe Replacements using Screen Scraping Sergio Flores Ruiz Junio, 2016

2

3 UNIVERSIDAD DE CASTILLA-LA MANCHA ESCUELA SUPERIOR DE INFORMÁTICA TECNOLOGÍAS Y SISTEMAS DE INFORMACIÓN GRADO EN INGENIERÍA INFORMÁTICA INGENIERÍA DEL SOFTWARE TRABAJO FIN DE GRADO AVEMARESS Tool for the Automatic Verification of Mainframe Replacements using Screen Scraping Autor: Sergio Flores Ruiz Director: Ignacio García Rodríguez de Guzmán Director: Ricardo Pérez del Castillo

4

5 PÁGINA DE CALIFICACIÓN TRIBUNAL: Presidente: Vocal: Secretario: FECHA DE DEFENSA: CALIFICACIÓN: PRESIDENTE VOCAL SECRETARIO Fdo.: Fdo.: Fdo.:

6

7 ABSTRACT A great number of the information systems designed and developed decades ago are still in use. The maintenance and operation costs of those systems are continually increasing which makes their replacement more desirable as years go by. Nevertheless, many of those systems are not replaced because they are critical for the business logic and there are huge risks and costs associated to their replacement. Minimizing the risks and costs of replacement implies the definition of the adequate verification tools and processes. This project provides the design and implementation of an automated verification tool in the replacement of an information system. The tool found defects in the replacement of a real information system within the context of FORTE for a customer of itestra GmbH. Those defects would have affected the business operations of the customer and would have incurred offences against Germany s data protection regulations. I

8

9 RESUMEN Un gran número de los sistemas de información diseñados y desarrollados hace décadas siguen en uso. Los costes de mantenimiento y operación de esos sistemas incrementan con el paso de los años, por lo que reemplazarlos es cada vez más deseable. Sin embargo, muchos no son reemplazados por ser críticos para la lógica de negocio y los enormes riesgos y costes asociados a su reemplazo. Minimizar los riesgos y costes de un reemplazo implica la definición de las herramientas y procesos adecuados de verificación. Este proyecto proporciona el diseño e implementación de una herramienta que permite la verificación automática en el reemplazo de un sistema de información. La herramienta encontró defectos en el reemplazo de un sistema de información real dentro del contexto FORTE para un cliente de la empresa itestra GmbH. Tales defectos habrían afectado a las operaciones de negocio del cliente y habrían implicado sanciones relacionadas con la ley de protección de datos alemana. III

10

11 AGRADECIMIENTOS A mis padres por darme esta gran oportunidad y apoyarme para siempre seguir avanzando y a mi hermano por sus ánimos. A María José y Juan por acogerme en su casa con tanto cariño durante estos meses. A Sofía por tantos consejos desde su experiencia con su TFG. A mis directores de proyecto Ricardo y Nacho por ser los guías durante el desarrollo de este proyecto. También gracias al resto de compañeros y personal de la ESI de Ciudad Real por su ayuda durante mi estancia en la escuela. Y finalmente a mis compañeros en itestra por sus consejos y ayuda a la hora de definir los requisitos y el camino a seguir por este proyecto. V

12

13 Table of Contents 1. Introduction HistAuskunft Justification of the verification process Architectural view of the project Introducción HistAuskunft Justificación del proceso de verificación Vista arquitectónica del proyecto Goals Subgoals Academic goals Precedents and state of the art Legacy information systems IBM 3270 terminals Black box modernization Software replacement Screen scraping GUI testing automation Java AWTRobot Related work Differences with existing GUI testing Methodology and tools Development in itestra GmbH The 7 principles of LSD Kanban boards Zero defects approach Proof of Concept Goal Driven Development principles Getting Things Done method Application of itestra GmbH s method in this project Kanban board Zero defects approach Justification of the PoC GTD based task handling VII

14 4.3. Project planning applying GDD principles Tools and technologies used in this project Hardware tools System Under Test related tools Project management tools Software development tools Quality assurance tools Database oriented tools Documentation tools Results Iteration 0: Initial task definition Iteration 1: Understand the environment of the Systems Under Test T1 - Install the required tools T2 - Connect to the mainframe s VPN T3 - Set up HistViewer s local database T4 - Learn how to use and configure the IBM 3270 terminal T5 - Configure HistViewer T6 - Understand the dialog structure in NatLeben End of iteration 1 with informed review of the Project Plan Iteration 2: Starting the PoC Identification of additional Tasks T7 - Learn more about the technical platform. PoC T8 - Extract one dialog for one contract automatically T9 - Compare one dialog for one contract automatically T10 - Adapt the verification process to verify the specified contracts End of iteration Iteration 3: Navigating through dialogs and new verification modes Identification of additional Tasks T11 - Extract input values to consult dialogs from the output of others T18 - Design the architecture to allow different verification modes T19 - Implement weighted random verification T20 - Adapt the tool to verify the existence of L42 dialogs T21 - Adapt the tool to verify one manually consulted dialog T12 - Identify fields that show equivalent data differently in each system T13 - Include special comparison patterns End of iteration Iteration 4: Output of results VIII

15 Identification of additional tasks T14 - Output the input arguments that led to the compared screens T15 Output whether a dialog is exactly equal in both systems T16 Output which fields do not match in the comparison End of iteration Final iteration: Validation of results T17 Validation of results Challenges Interaction differences between HistViewer and the IBM 3270 emulator Navigation through dialogs in NatLeben Other verification challenges Case study Conclusions and proposals Goal achievement analysis Project conclusions Proposals and future work Future work Future improvements Personal opinion Conclusiones y propuestas Análisis de consecución de objetivos Conclusiones del proyecto Propuestas y trabajo futuro Líneas futuras Mejoras futuras Valoración y opinión personal Acronyms References Appendix A: Probabilities of the input arguments IX

16

17 1 Introduction 1. INTRODUCTION Many organizations began incorporating automatic information systems in the second half of the XX century. A great number of those information systems, especially the ones critical for the business logic, are still in use. The so called Legacy Information Systems (LIS). LIS use technologies that most of the times are considered obsolete resulting in high maintenance and operation costs. Those costs are present in the day to day of the organization and obstruct the application of changes or improvements that could grant competitive advantages. The replacement of LIS with a new system is one of the prevalent solutions for those difficulties. Nevertheless, the replacement of a LIS implies additional risks and costs. Those risks and costs are justified considering that the new system might not include the same functionality of the replaced system [Sommerville 2008b]. Frequently, the initial functionality or the functionality added during the maintenance of the LIS is not fully or reliably documented [Lauder and Kent 2000]. Oftentimes, the documentation of the LIS has been lost (or did not even exist) and the staff that designed the system is no longer available. What is more, the system to replace usually contains information in legacy formats. That information is commonly incomplete or damaged hampering even more the transfer to a new system [Bollig and Xiao 1998]. For these reasons, reverse engineering techniques are frequently applied for the replacement of LIS [Bennett 1995]. Nevertheless, because of the size and complexity of many LIS, some authors compare the application of reverse engineering with gold mining [Lauder and Kent 2000]. A wrong replacement can have terrible consequences, especially in fields where LIS are more abundant (e.g.: banking or insurance). Those consequences justify the application of adequate and sufficient verification tasks to ensure that the replacement is error-free and preserves the LIS business logic. Some examples of the literature suggest that the verification should take even up to the 80% of the effort in replacement projects [Bisbal et al. 1999]. The HistAuskunft project in itestra GmbH is an example of the application of reverse engineering tasks for the replacement of LIS. HistAuskunft is the project in which the author of this document is doing his internship in the FORTE program in collaboration with itestra GmbH. Some of the tasks in that internship are related to the design and development of a tool that automates some of the verification tasks. 1

18 1 Introduction 1.1. HistAuskunft HistAuskunft is a project for an insurance company (itestra GmbH s customer). HistAuskunft began in 2014 and its duration is planned for more years. The main goal of HistAuskunft is to replace a legacy system (z/os [IBM 2011]) and its hierarchical database structure (IMS) [IBM 2016c] with a new system using current technologies (JavaFX [Oracle 2016a] and relational databases IBM DB2 [IBM 2016a, p.2]). The other goal in HistAuskunft is to recover the information stored in the legacy database contained in the mainframe system. HistAuskunft has two parallel approaches. First, itestra GmbH s customer requests the conversion of the hierarchical database to a relational database to allow the retirement of the mainframe. Transferring the data to a relational database will also ease future access and interaction with such data. This process is not as easy as a simple migration. The customer requests the data to be human readable despite these challenges: The hierarchical database s content is encoded in ebcdic 273. Ebcdic 273 is IBM s character encoding for Germany and Austria registered in 1986 [IBM 2016b]. The legacy hierarchical structure is complex due to technological restrictions in the time of its design and development. Design decisions such as primary keys are not easy as those concepts were not considered in the initial design of the hierarchical database. The size and continuous maintenance of the database has increased its complexity during its several decades of life. The second part of HistAuskunft is the extraction by screen scraping of the information shown by the terminal connected to the mainframe. The new system that replaces the mainframe will use the results of this extraction to show the content to the users. The correct replacement of the mainframe is vital for the business operations of itestra GmbH s customer. The mainframe system to replace contains the contracts related with the different dimensions in which itestra GmbH s customer works (insurance, loans, pension schemes, etc.). By the date of writing, it is known that there are eight different environments in the mainframe system, although only details about two of them are known. Those two environments are for life insurance (NatLeben) and loans (Leistung). Each system has different kinds of contracts with a particular dialog and subdialog structure (c.f. Figure 1.1). 2

19 1 Introduction Figure 1.1. Data structure of the environments to verify. Figure 1.2 shows the exact context of this TFG within the HistAuskunft project. This TFG aims for the design and development of a tool that automatically compares the screens of the IBM 3270 (original system) and the HistViewer (new system). The users of the system to replace use the terminal IBM 3270 daily. This terminal provides the connection to the mainframe to access the procedures and persistence of the mainframe. The extraction process uses the screen scraping technique to fetch the content shown by the IBM 3270 and store it in a relational database. The new system shows the information to the users with a recreation process taking as source the data extracted from the screen output of IBM 3270 terminal. If the screen output of those two systems is not equivalent, there might be new requirements or defects in the extraction, persistence and recreation process. Persistence DB2 - HistAuskunft Extraction IBM 3270 terminal Mainframe z/os TFG Automatic verification of screens Recreation HistViewer JavaFX IMS Figure 1.2. Context for this TFG Justification of the verification process The information from the legacy database is very important. That information contains contracts whose information should be conserved without errors. An incorrect extraction of the 3

20 1 Introduction information would imply an offence against Germany s data protection laws [Bundesministerium der Finanzen 2014]. These kinds of offences have serious consequences in Germany (or Spain for that matter). Another reason to justify this verification is that many of the information fields shown to the users are calculated by legacy algorithms stored in the mainframe and written in COBOL. Reverse engineering all those algorithms substantially increases the risks and costs of the project. This limitation is common in Legacy Information Systems [Bisbal et al. 1999]. For these reasons, investing time and resources performing different kinds of verification in several levels is important. Due to the great number of screens (in the order of 16 million only considering the first environment of the eight in the mainframe to replace) the manual verification is unviable in time and error-prone. This TFG presents a solution for that problem with the design and development of a tool that automatizes the verification process. This way, the verification can be performed with higher reliability and in acceptable time. This automatic tool provides a scalable solution that is also adaptable to the available environments to replace in HistAuskunft. The tool satisfies a real need within HistAuskunft. By this date, itestra GmbH did not have any other mean to automatically verify the retirement of the mainframe system. This new tool offers a substantial improvement in the verification times compared to the manual verification by users. The tool compares the screens of both systems (the LIS to replace and the new system) to ensure that they are equivalent. Compared to a manual comparison that is unviable in time and error-prone, this verification tool can verify each screen in less than 3 seconds. In perspective, the tool can verify 100 screens in less than five minutes while a user would need approximately two hours for a manual verification. Aside from a notable time saving, the tool also complements the verification with information of the domain field of the system to replace and is much less error-prone. The automatic verification provides a saving in human time that can be spent in tasks that are more difficult to automatize such as the development of the system that replaces the LIS. The suitability of the verification tool has been proven during its development. The verification already found defects in HistViewer that would have implied defects in the LIS replacement with information loss. The tool has been able to discover defects that would have incurred great costs and complications if they had persisted after the complete retirement of the mainframe system. Those complications fall in the legal context with offences against the data protection laws or time restarting an extraction process that can last more than a whole year. 4

21 1 Introduction Finally, the design and development of this verification tool contributes to the existing knowledge of the domain field of automatic verifications. The problems solved during the development of this tool could be applied to the rest of environments in HistAuskunft or even other LIS retirements Architectural view of the project This project has a Data extractor (c.f. Figure 1.3) that uses two adapted AWTRobot [Oracle 2016c] (3270Scrapterbot and HistViewerScraperbot) to extract the output of the mainframe system and HistViewer. As seen in Figure 1.3, the results are transferred to a Data comparer and processed as input for a Comparison results generator. Those results have to be stored allowing later human interpretation. Figure 1.3. Parts of this project. The verification tool aims to the automation of a process minimizing the interaction of a real user with the system. For that reason, graphical user interfaces are not among the main artefacts to generate in this project. It is true that neither the HistViewer nor IBM 3270 IDS have a Graphical User Interface in the appropriate and that the testing of those systems alone would not be considered GUI testing. Nevertheless, this verification takes place at the same time in both systems, comparing one against the other. This means that both systems are used simultaneously using a window manager and the mouse. For that reason, adapting and using techniques oriented to GUI verification is needed. Both systems will be treated as different windows. The mouse input, keyboard shortcuts and other functionalities offered by the emulator have to be exploited in a rational way because those are all the entry points available for the verification tool. Those entry points generally have more variables and conditions than the interface of a text-based system interface in only one text-based system. 5

22

23 1 Introducción 1. INTRODUCCIÓN Muchas organizaciones comenzaron a incorporar sistemas de información automáticos desde mediados del siglo XX. Algunos de estos sistemas de información, especialmente aquellos críticos para la lógica del negocio, siguen en uso recibiendo el nombre de Sistemas de Información Legados (SIL). Estos SIL disponen de tecnologías que en la mayoría de los casos se consideran obsoletas resultando en altos costes de mantenimiento y operatividad. Dichos costes se manifiestan en el día a día de la organización y dificultan la aplicación de cambios o mejoras que podrían proporcionar ventajas competitivas. Tales dificultades pueden resolverse reemplazando el SIL por un sistema más actual. No obstante, el reemplazo de un SIL incluye riesgos y costes. Estos riesgos y costes se deben a que el nuevo sistema puede no reunir la misma funcionalidad que el sistema reemplazado [Sommerville 2008b]. Frecuentemente, la funcionalidad inicial o aquella añadida al SIL durante su mantenimiento no está documentada de forma completa o fiable en ninguna parte [Lauder and Kent 2000]. En numerosas ocasiones la documentación del SIL se perdió (o no existió desde el principio) y el personal que diseñó el sistema ya no está disponible. Además, el sistema a reemplazar suele contener información en formatos legados. Esta información muchas veces se encuentra incompleta o dañada complicando aún más su traspaso a un nuevo sistema [Bollig and Xiao 1998]. Por estos motivos, se suele recurrir a técnicas de ingeniería inversa para el reemplazo de SIL [Bennett 1995]. Aunque debido al tamaño y complejidad de muchos SIL, algunos autores comparan la aplicación de ingeniería inversa con la minería de oro [Lauder and Kent 2000]. Un reemplazo incorrecto puede tener consecuencias nefastas en casi cualquier sector, sobre todo en aquellos donde los SIL son más frecuentes como banca o seguros. Tales consecuencias justifican la aplicación de tareas de verificación adecuadas y suficientes para asegurar que el reemplazo se realiza libre de errores y preservando la lógica de negocio del SIL. En algunos ejemplos de la literatura, se considera que la verificación debe suponer incluso hasta el 80% del esfuerzo de proyectos de migración [Bisbal et al. 1999]. El proyecto HistAuskunft de la empresa itestra GmbH es un ejemplo de aplicación de ingeniería inversa para el reemplazo de SIL. HistAuskunft es el proyecto en el que el proyectando está realizando sus prácticas dentro del programa FORTE en colaboración con itestra GmbH. La labor del proyectando en el proyecto HistAuskunft consiste en el diseño y desarrollo de una herramienta que permita automatizar las tareas de verificación. 7

24 1 Introducción 1.1. HistAuskunft HistAuskunft es un proyecto para una aseguradora cliente de itestra GmbH. HistAuskunft comenzó en el año 2014 y su duración está planificada para varios años más. El objetivo principal de HistAuskunft es reemplazar un sistema legado (z/os [IBM 2011] con una estructura de datos jerárquica IMS [IBM 2016c]) por un nuevo sistema con tecnologías más actuales (JavaFX [Oracle 2016a] y bases de datos relacionales IBM DB2 [IBM 2016a, p.2].). En HistAuskunft también se pretende recuperar la información almacenada en una base de datos legada contenida en el sistema mainframe. HistAuskunft está evolucionando con dos enfoques diferentes a desarrollar en paralelo. Por un lado, el cliente de itestra GmbH solicita la conversión de la base de datos jerárquica del mainframe a una base de datos relacional para poder retirar el mainframe. Tener los datos en una base de datos relacional también facilitará su futuro acceso. Este proceso no es tan fácil como una simple migración. El cliente solicita que los datos sean humanamente legibles a pesar de los siguientes desafíos: El contenido de la base de datos jerárquica está codificado en ebcdic 273 que es la codificación de caracteres de IBM para Alemania y Austria registrada en 1986 [IBM 2016b]. La estructura jerárquica legada es complicada debido a las restricciones tecnológicas existentes en la época de su desarrollo. Decisiones de diseño como las claves primarias no son triviales pues son conceptos que no fueron contemplados en el diseño inicial de la base de datos jerárquica. El tamaño y continuo mantenimiento de dicha base de datos ha ido aumentando su complejidad a lo largo de sus décadas de vida. La otra parte de HistAuskunft supone la extracción mediante screen scraping de la información mostrada por el terminal conectado al mainframe. El nuevo sistema que reemplaza al mainframe utilizará los resultados de esta extracción para mostrar el contenido a los usuarios. El correcto reemplazo del mainframe es vital para las operaciones de negocio del cliente de itestra GmbH. Dicho mainframe contiene los contratos relacionados con las diferentes dimensiones en las que trabaja la empresa (seguros, préstamos, planes de pensiones, etc.). Aunque se sabe que hay ocho entornos dentro del mainframe, el cliente de itestra GmbH solo ha dado detalles de dos por el momento. Entre estos entornos se encuentran el registro de 8

25 1 Introducción contratos de seguros de vida (NatLeben) y el de préstamos (Leistung). Cada entorno tiene diferentes tipos de contratos con una estructura de diálogos y subdiálogos particular (véase Figura 1.1). Figura 1.1. Estructura de datos de los entornos a verificar. La Figura 1.2 muestra la situación exacta de este TFG dentro de HistAuskunft. En este TFG se pretende diseñar y desarrollar una herramienta que permita comparar la salida del IBM 3270 (sistema original) y el HistViewer (sistema nuevo) de forma automática. Los usuarios del sistema a reemplazar recurren diariamente al terminal IBM 3270 para realizar las tareas del negocio. Este terminal ofrece conexión con el mainframe para acceder a sus procedimientos y persistencia. El proceso de extracción pretende realizar screen scraping para trasladar la información mostrada por el IBM 3270 a una base de datos relacional. El nuevo sistema hace uso de HistViewer para mostrar la información a los usuarios a partir de un proceso de recreación. La no equivalencia entre ambos sistemas supondrá el descubrimiento de nuevos requisitos o defectos en dicho proceso de extracción, persistencia y recreación. Persistencia DB2 - HistAuskunft Extracción Terminal IBM 3270 Mainframe z/os TFG Verificación automática de pantallas Recreación HistViewer JavaFX IMS Figura 1.2. Contexto de este TFG. 9

26 1 Introducción 1.2. Justificación del proceso de verificación Primero de todo hay que tener en cuenta la importancia de los datos a extraer de la base de datos legada. Esta información corresponde a contratos de seguros cuyos datos deben ser conservados sin errores. Una extracción de información incorrecta implicaría un delito contra la ley de protección de datos alemana [Bundesministerium der Finanzen 2014]. Este tipo de delitos tienen muy graves consecuencias tanto en España como en Alemania. Además, muchos campos de información mostrados al usuario por el mainframe son calculados a partir de algoritmos legados en COBOL. Realizar una ingeniería inversa precisa de todos los algoritmos del sistema incrementaría sustancialmente los riesgos y costes del proyecto. Esta limitación es muy común en SIL [Bisbal et al. 1999]. Por este motivo, es importante invertir tiempo y recursos en realizar diferentes tipos de verificación a varios niveles. Debido al gran número de pantallas (en el orden de los 16 millones solo para el primer entorno de los ocho que existen) la verificación manual es inviable en tiempo, laboriosa y por ello propensa a errores. Este TFG presenta una solución a ese problema con el diseño y desarrollo de una herramienta que permita automatizar el proceso de verificación. De esta forma la verificación puede realizarse con mayor fiabilidad y en un tiempo asumible. Esta herramienta automática proporciona una solución escalable y adaptable a los entornos disponibles en el sistema mainframe a reemplazar en HistAuskunft. La herramienta satisface una necesidad real dentro de HistAuskunft pues hasta el momento itestra GmbH no disponía de ningún otro medio para realizar la verificación automática de la retirada del sistema mainframe. Esta nueva herramienta proporciona una mejora muy sustancial en los tiempos de verificación respecto a una verificación manual por usuarios. La herramienta compara las pantallas de ambos sistemas (el SIL a reemplazar y el sistema que lo reemplaza) para asegurar que son equivalentes. Respecto a una comparación manual que es inviable en tiempo y altamente propensa a errores, esta herramienta automática puede verificar cada pantalla en menos de 3 segundos. En perspectiva, la herramienta puede verificar cerca de 100 pantallas en menos de cinco minutos mientras que un usuario necesitaría dos horas aproximadamente en forma manual. Además de un notable ahorro en tiempo, la herramienta proporciona información del campo de dominio del sistema a reemplazar y es mucho menos propensa a errores. La verificación automática supone un ahorro de tiempo humano que puede ser dedicado a otras tareas más difíciles de automatizar como el desarrollo del sistema que reemplaza al SIL. 10

27 1 Introducción La aptitud de la herramienta de verificación se ha comprobado durante su desarrollo. La verificación ya encontró defectos en HistViewer que habrían supuesto un reemplazo no satisfactorio del SIL con pérdida de información. La herramienta ha sido capaz de descubrir defectos que habrían supuesto grandes costes y complicaciones si hubieran persistido tras la retirada completa del sistema mainframe. Estas complicaciones entran tanto en el contexto legal por delitos contra la ley de protección de datos como en tiempo necesitando realizar nuevamente un proceso de extracción que puede durar más de un año. Finalmente, el diseño y desarrollo de esta herramienta de verificación contribuye al conocimiento existente en el campo de dominio de la verificación automática. Los problemas resueltos durante el desarrollo de esta herramienta podrán aplicarse en el resto de entornos contemplados en HistAuskunft e incluso otras retiradas de SIL Vista arquitectónica del proyecto El proyecto consta de un Extractor de Datos (véase Figura 1.3) que comunica sus órdenes a dos AWTRobot [Oracle 2016c] adaptados (3270Scraperbot y HistViewerScraperbot) para extraer las salidas de los sistemas mainframe y HistViewer. Tal y como se ve en la Figura 1.3, los resultados son transferidos a un Comparador de datos que los procesará para servir como entradas de un Generador de resultados de comparación. Esos resultados deberán ser guardados de forma que su posterior interpretación humana sea posible fácilmente. Figura 1.3. Vista arquitectónica del proyecto La herramienta de verificación busca automatizar un proceso minimizando la interacción de un usuario real con el sistema. Por lo tanto, las interfaces gráficas no se encuentran entre los artefactos principales a generar en este proyecto. Es cierto que ni HistViewer ni el emulador de terminal IBM 3270 tienen una interfaz gráfica (GUI) en el sentido estricto y que la verificación de los sistemas por separado no se consideraría verificación de GUI. Sin embargo, esta verificación se produce al mismo tiempo en los dos sistemas, comparando uno con otro. Esto significa que los dos sistemas se utilizarán 11

28 1 Introducción al mismo tiempo a través del gestor de ventanas y haciendo uso del ratón. Por tanto, será necesario el uso y adaptación de herramientas y técnicas orientadas a la verificación de GUI. Ambos sistemas serán tratados como dos ventanas diferentes. La entrada del ratón, atajos de teclado o funcionalidades ofrecidas por los emuladores deberán ser explotadas de un modo racional pues esos son todos los puntos de entrada hacia el sistema bajo test de los que dispondrá la herramienta de verificación. Cabe tener en cuenta que estos puntos de entrada tienen más variables y condiciones que la entrada de una interfaz exclusivamente basada en texto en la verificación de un solo sistema basado en texto. 12

29 2 Goals 2. GOALS The main goal of this project is to design and develop an automatic tool for the verification of whether the information shown by an IBM 3270 terminal connected to a mainframe is equivalent to the information shown in an application replicated using JavaFX Subgoals The main goal of the project is divided into six subgoals. The subgoals exposed in this section give a better understanding of the needed steps to reach the main goal of the project. Those subgoals are identified as SGn. That n suffix shows the priority of the subgoal. The closer n is to 1, the higher the priority of the subgoal. SG1 Understand the structure of contracts, dialogs, sub-dialogs, pages and fields and learn to navigate through it in both HistViewer and the IBM 3270 terminal. SG2 Design and implement the means to automatically fetch the information contained in a dialog for one contract in HistViewer and the IBM 3270 terminal and compare that information automatically as well. SG3 Design and implementation of the logic to consult, extract and compare a dialog for all contracts from a given set automatically. SG4 Design the mechanism for recognizing dialogs, sub-dialogs, pages and fields for each contract to navigate and compare them automatically. SG5 Management of the equivalent data that is shown differently in both systems on purpose. Do not mark equivalent data as a defect. SG6 Output the comparison results so that valuable conclusions can be obtained Academic goals This project has both a business and academic background. The later defines academic goals as the knowledge acquired is one of the most important results of this project in the academic context. AG1 Use and learning of methods from the corporative environment. AG2 Improvement of teamwork capabilities as this project needs to consider the needs of the HistAuskunft team. 13

30 2 Goals AG3 Learning how to collaborate in already running sizeable projects. AG4 Learning and use of technical expressions and vocabulary in the professional environment in Spanish, English and German. AG5 Learn how to do research about technical domain fields and learn how to write academic documents. 14

31 3 Precedents and state of the art 3. PRECEDENTS AND STATE OF THE ART This project is strongly related to the topics of Legacy Information Systems (LIS), GUI testing automation and screen scraping which is the technique that will be used. The chapter ends analyzing previous efforts in the GUI testing automation field and studying the differences between this project and existing GUI testing. The most related sub-topics concerning LIS in the context of HistAuskunft are IBM 3270 terminals, black box modernization and software replacement. IBM 3270 terminals are relevant because this verification tool has to make use of one of them to interact with the mainframe. Black box modernization since both HistAuskunft and the verification tool in this TFG treat the IBM 3270 terminal as a black box, only considering inputs and outputs. And software replacement on the grounds that replacing the IBM 3270 and its mainframe with HistViewer is the ultimate goal of HistAuskunft assisted by this verification tool Legacy information systems As defined by the Merriam Webster dictionary, the word legacy applies to: Something that happened in the past or that comes from someone in the past. Something transmitted by or received from an ancestor or predecessor or from the past. Of, relating to, or being a previous or outdated computer system. Furthermore, Michael L. Brodie and Michael Stonebraker in their work Legacy Information Systems Migration: Gateways, Interfaces, and the Incremental Approach define Legacy Information System (LIS) as any information system that significantly resists modification and evolution [Brodie and Stonebraker 1995]. Although many of these definitions use concepts related to the past or even use the word outdated, LIS are too present nowadays to be considered only an artefact from the past. LIS will always exist considering that today s current systems will be the LIS of tomorrow and organizations usually cannot simply stop using LIS chiefly when those are mission critical. Even if a specific LIS is perceived as outdated and its replacement is considered, the LIS might still be in use for several reasons such as: The replacement might not be as reliable or there are too many risks associated to replacing the LIS with a new system [Sommerville 2008b]. 15

32 3 Precedents and state of the art The system contains legacy data which is difficult to migrate correctly due to being prone to corruption and incompleteness [Bollig and Xiao 1998]. Training the users of the previous LIS in the use of the new system would be too costly in time or money. The users are unwilling to use a different system because they think it might imply more effort in terms of learning and adaptation. The system must be available at all times, so it cannot simply be turned off and then replaced. Good examples lie in banks or computerized reservations systems such as those in airlines, hotels or any system that is critical in the business logic and daily operations [Bollig and Xiao 1998]. This ever growing field of application is very frequent considering that more and more operations and processes are being automated as the years go by. The cost of replacement is high enough to make it unviable or not possible. This cost might come from the replacement process itself or vendor lock-in. In some situations the customer is unable to replace the vendor of the LIS without incurring substantial switching costs. There is little knowledge about the system. The documentation might have been lost or even never existed. Furthermore, the knowledgeable people of aging systems (such as the designers) might be retired. There is no useful documentation about the LIS because it was lost or never existed. The knowledgeable people for that system such as the designers might be retired. The original specification was lost or it is not certain whether it includes all the changes that have been performed to the LIS or not. For this reason, gathering the documentation and requirements of LIS for the purpose of migration is sometimes compared to gold mining [Lauder and Kent 2000]. Designing a new system that is able to perform identically coexists with the risk of not covering all the features of the LIS. Although all those reasons might make the replacement of LIS a difficult task, they should be weighed against the argumentations that make the replacement of LIS convenient: LIS run on dated hardware that is frequently slow, obsolete or has excessive operating 16

33 3 Precedents and state of the art costs [Bennett 1995]. The increasingly higher cost of maintenance has come to a point in these systems where at least one of all four maintenance types defined in the ISO/IEC [ISO 2006] (corrective, preventive, adaptive and perfective) is impossible, unviable or its cost is completely prohibitive. Maintenance costs are also increased when different teams contribute to the development of the system and there are no standard practises among those teams [Sommerville 2008a]. The LIS uses obsolete languages and capable staff would be difficult to find and expensive. The system was optimized for space management or execution speed so that it could be used on old hardware sacrificing readability. This makes the understanding of the system much harder and sometimes deters the application of modern software engineering techniques for engineers more used to modern paradigms [Sommerville 2008a]. Changes that could lead to a competitive advantage or the coverage of new markets are too hard or even impossible to apply due to the increased cost of maintenance. According to Manny Lehman, real word activities are those that involve the mechanization of human or societal activity [Lehman 1980]. Manny Lehman s laws of software evolution state that software meant to perform in such real world activities must evolve and be kept continually changing and adapted so that the systems keep being useful in the real world. In addition, Lehman s second law states that as a given system evolves, its complexity increases. Complexity can only be reduced with active action [Lehman 1980]. This is another reason to consider the retirement or replacement of LIS keeping in mind that their cost of maintenance and their complexity has had many years to be increased and active action to reduce that cost of maintenance is harder and more compromised as time goes IBM 3270 terminals IBM 3270 terminals were first unveiled by IBM s Data Processing Division in 1971, publically demonstrating it as an Information Display System (IDS) [IBM 2003]. This IDS commercialized by IBM was used to communicate users and mainframes. These terminals are no longer manufactured. Despite of that, they are still alive and terminal emulators use their communications protocol (tn3270) to access the content in 17

34 3 Precedents and state of the art mainframes. For instance, itestra GmbH s customer s IT department uses them to access contractual data. There are even terminal emulators for ios devices [C. Buccianti 2011]. This is due to a clear need of communicating with existing IBM Z System platforms and mainframes that are still far from retirement. IBM 3270 terminals were non-programmable and had a display screen and a keyboard. The IBM 3270 only had rudimentary communications capabilities and there was no more GUI than what we can consider plain text. They were not meant to perform calculations or handle business logic. For this reason, they are called Information Display Systems (IDS) and were used to connect and interact with mainframes. The first models had a 12 rows and 80 columns format. The following model used a 24 rows and 80 columns format and managed to be the standard. There were more formats available which were different to the standard [IBM 2013]. Even though by the time this document is being written IBM 3270 terminals are 45 years old, they are still in use. In fact, itestra GmbH s customer is using the one with 32 rows and 80 columns display format and that is the IBM 3270 terminal this project will verify against the new system Black box modernization This technique allows the modernization or replacement of a LIS by examining its inputs and outputs as a means to understand the system. Most of the times, the separated usage of this solution is not practical and white-box techniques have to be employed as well to grant additional knowledge of the system [Comella-Dorda et al. 2000]. Black box modernization has been used in other projects in the past [Bollig and Xiao 1998] helping the engineers understand the business rules and data structures used in the LIS leading to a whole new system that would be much more adaptable. That replacement was feasible basing the process on the requirements of the system itself instead of doing a verbatim copy of the previous system operation by operation. This is the kind of approach used in HistAuskunft considering that the ultimate goal is not just the modernization but the complete retirement of the LIS. The verification tool in this TFG will treat the legacy system and its replacement as black boxes as well only considering the inputs and outputs for said systems. 18

35 3 Precedents and state of the art Software replacement Software replacement is notably useful when LIS cannot keep up with business needs [Bollig and Xiao 1998] or the cost of maintenance is too high [Bennett 1995]. The goal of software replacement is to replace an existing system with a new one. Software replacement is the complete opposite of the wrapping technique where the legacy system is not retired but encapsulated within a new one [Sneed 2000]. The replacement offers a wide range of benefits in terms of flexibility and maintenance feasibility. Nevertheless, replacing a LIS implies several conditions that might increase the risk of the replacement such as the lack of a generic method for the replacement of all systems and the need of understanding the LIS in great detail [Bisbal et al. 1999]. Understanding the LIS is a time demanding task that has to be performed by experts, managers, engineers and users. All of them must be completely involved. The importance of testing in the replacement of a LIS is something to consider in all cases. LIS are usually not only mission-critical but very difficult to completely understand as stated in section 3.1 Legacy information systems. In some migration projects, testing requires up to 80% of the engineers time [Bisbal et al. 1999]. This claiming was based on Bennet s work which highlights the difficulty of maintenance and testing in degrading software structures where understanding is a large task in itself [Bennett 1995]. Most importantly, there are no LIS replacement projects that have been reported to be successful in the literature using a comprehensive or flexible approach. All of them were adhoc solutions and none was generic enough to be applied to the replacement of any other LIS Screen scraping Screen scraping is a technique aimed at extracting the content shown by user interfaces. Screen scraping is especially useful when more convenient options such as useful APIs or direct access to the data source are not available. The use of screen scraping is also frequent when the data source is too complex or its maintenance is not viable, as is the case in many LIS. Screen scraping is one of the main ways to interact with LIS for modernization purposes. The content of the LIS is extracted and then transferred to a new system [Comella-Dorda et al. 2000]. The usual screen scraping application consists of wrapping the previous text based interface with a new graphical interface, that old interface would be actively read and parsed to 19

36 3 Precedents and state of the art the new interface which could be even web browser accessible as added value for its modernization. In case the new interface gets its content from the old one in runtime, the new interface would have to communicate with the old one using a strictly ad-hoc tool that would generate the new screens from the data shown in the old ones. As a drawback, the already existing LIS wasn t replaced, but an additional layer was added to the LIS which means that the system grows in size and complexity. The LIS is still the only source of the business logic and even harder to retire. On the other hand, the screen scraping applied in HistAuskunft and this verification tool is completely different to the standard use of screen scraping. In HistAuskunft, screen scraping is used as a means to fetch the information from the legacy interface to the new interface but not to extend the life of the mainframe. HistAuskunft envisions the complete retirement of the mainframe which acts as source of the data. Thus, screen scraping is being applied to (i) extract all the information displayed by the IBM 3270 terminal, (ii) store it in a relational database and then (iii) completely retire the mainframe. In this verification project screen scraping is used to extract information from both systems (HistViewer and IBM 3270) as well. The verification tool of this project would compare and analyse whether both systems show equivalent information or not. In case the information is equivalent, the mainframe could be retired and HistViewer could take its place GUI testing automation GUI testing is an integral part of the development process because the GUI is the most visible part for the users. Since GUIs often imply concurrence, real time interaction and a broad range of possibilities and permutations, this testing is very time consuming and expensive; most notably when the GUI testing is performed manually. Minimizing or eliminating the human interaction with the System Under Test (SUT) via automation can lead to some of the following advantages: Less tedious and time demanding tests with higher coverage ratios. More easily reproducible tests since they are automated. Minimal amount of human errors, higher accuracy and reliability. Many errors in GUI testing are related to tiredness or brief exposure of the defect because in those situations defects can be unnoticed by humans. Generally, an automatic tool needs less time and is more exhaustive than the human eye. 20

37 3 Precedents and state of the art Testing of characteristics that would be impossible to test manually such as load or performance testing [Rauf and Alanazi 2014]. Increased trust on the testing suite. Automatic tools are usually more deterministic and can be more trustworthy than humans that might not have been looking to the exact location of the defect when it appeared. Minimize the chance of forgetting steps for any test case of the GUI. Chiefly in long tests for which it would be difficult to know for sure if the human tester forgot any step and even harder to discover which one. Testing results are related to the usefulness of the testing algorithm which is more auditable than the actions of a human tester. On the other hand, manual testing benefits from the imagination and creativity of the tester who gets new ideas while the testing is taking place and might take paths more randomly determined. An automated tool cannot really take the initiative to find defects, although, there are recent works such as [Rauf and Alanazi 2014] that consider the application of artificial intelligence (AI) to automated GUI testing. Albeit the authors also admit that nowadays AI techniques are still far from being highly advantageous in the GUI testing domain field Besides, there are common grounds between automated GUI testing and manual GUI testing or even non GUI testing. Automated GUI testing should also consider initial states, goal states and the set of objects and operators that join the initial and goal states combined with the available set of events in each intermediate state [Memon et al. 2001]. An example of an available set of events in a given state is illustrated by the possible actions in the current dialog, that is, in a window that allows the user to send, delete or archive an the set of available events is to send, delete or archive an . However, there are additional challenges in the already complex process of GUI testing [Memon 2002]. For instance, an incorrect GUI state can take the verification to a screen that is not expected. In that case, the test execution would bring confusing reports that could misguide the whole testing and development. Precisely, the biggest challenge in automated GUI testing is having a very long chain of actions as input that give a single output after their execution as a whole. That final output can be unreachable if any intermediate GUI state was incorrect and the whole execution would have been useless [Aho et al. 2015]. In other cases, an unexpected final state can be reached because of an error in the chain of actions. This can remain undetected and hide defects or report false positives when that unexpected final state is mistaken for the 21

38 3 Precedents and state of the art expected one. All things considered, GUI testing is not only important, but the automation of such a process is invaluable in many cases. Of course, it is relevant to consider the advantages and disadvantages exposed previously. Summarizing, GUI testing is harder to deploy at first but can lead to more comprehensive results. Remarkably when there are defects that would have been very difficult to find using more conventional courses of action such as manual testing or testing only the application code. As stated by Mehlitz and Ujma in the NASA Ames Research Center in their work JPF-AWT: Model Checking GUI Applications the application code for GUIs only provides callbacks, which means that only testing the application code is not enough to assure that the behavior of the GUI is appropriate [Mehlitz et al. 2011]. As they claim in their publication, the use of a robot that interacts with the GUI helped them finding a critical null pointer exception that would have been almost impossible to reproduce with traditional testing techniques. This proves further that the benefits brought by GUI testing automation are also applicable to the praxis in real world situations Java AWTRobot The robot class, bundled in the java SDK since version 1.3, facilitates the generation of system input events so that automatic interaction with the system via previously defined system input events can be possible. As stated in the oracle documentations for java [Oracle 2016c], the most notable purposes are test automation, self-running demos, and other applications where control of the mouse and keyboard is needed. Even in this same documentation we can read that the primary purpose of this robot is to facilitate the automation of testing which is the main goal of this project. Input events asked to this robot s methods are at the same abstraction level of the user s input. For instance, mouse movements performed by the robot move the mouse cursor in the very same way the mouse movements performed a user would. The AWTRobot can do exactly the same things a normal user can. For that reason, the AWTRobot is ideal to test GUIs and was chosen for this project. There are some limitations as well; this robot takes the control of the input away from the user, so it cannot be used at the same time the user is doing other tasks since it would be exactly as if two people were using the computer at the same time with one mouse each. 22

39 3 Precedents and state of the art This class has been used as a means to automate the interaction with a computer in a wide range of different approaches such as the implementation of an interface that allows interacting with a computer getting the input from a smartphone over Wi-Fi as designed and implemented in 2014 in the Nanjing University of Information Science and Technology [Ma et al. 2014]. There are also examples that are closer to the field of application of this project that also reinforce the validity of the java AWTRobot as a testing tool to the point of being the key element in an open source library that allows Test-Driven Development of GUIs with the name FEST [Ruiz 2007] Related work As stated in section 3.3 GUI testing automation, GUI testing is a complex process made even more complex by automation. For that reason, most efforts are completely ad-hoc and tailormade for their domain field. Nevertheless, related works have been considered to have a broader view of the state of the art. Building a driver in the SUT s GUI implies the development of a specific interface in the SUT that allows actions and events to be received by the SUT bypassing the GUI [J. Kasik and G. George 1996]. The driver can send commands and events to the SUT from other program that implements the testing process. Still, this method has some limitations which are insolvable in the domain field of this project. Most importantly, the code of the IBM 3270 emulator cannot be modified which renders this solution impossible to be applied. There is also the doubt on whether the actual GUI is being tested or not since the events are not coming to the SUT from the actual GUI. Capture replay GUI testing [Ariss et al. 2010] can be very easy to deploy in small cases and might be convenient in limited and well defined scenarios. As another advantage, it does not need direct access to the code. Nevertheless, this technique cannot do much by itself unless it is coupled with other techniques that extract and process the output. This is especially true with large outputs or many events that demand the extraction and comparison of output constantly. The biggest limitation of this technique is not being able to create new inputs apart from those captured which were those that the tester first performed manually. If the SUT has many different options the tester would have to capture them all first. That is only viable if the SUT has few different input options and events that get repeated constantly. Furthermore, the tester has to know the input options and events with great detail which is also not the case in this project. For those reasons, this technique does not satisfy the need of discovering unknown 23

40 Not bound to strong state definition Possible without access to the code Able to check different SUT s at once Easily scalable Usable in proprietary mainframe IDS 3 Precedents and state of the art characteristics or details or the system. That need is of utmost importance in this project s verification. JavaPathFinder Model Checking, despite being specified by the authors about being more than just a testing tool which performs model checking as well [Mehlitz et al. 2011] the ideas behind this automated verification tool can be relevant to the domain field of this verification. It was first intended to be a software model checker, although it grew in execution modes and other kinds of extensions that increase the complexity and capabilities of this tool. Nevertheless, this technique cannot be applied to our case. Program states cannot be explored until none are left because of the magnitude of the SUT outputs (in the range of 16,000,000 screens). Another limitation is that many yet undefined/undocumented states are waiting to be discovered and cannot be included in a model definition. The other biggest detail that makes JPF Model Checking unviable for this project is the difficulty to adapt it to this verification. JPF needs to interact with the JVM. The interaction is possible for the verification of HistViewer because the code is available, but it is not possible for the IBM 3270 terminal emulator, so the comparison cannot take place. The previous verification options will be compared in Table 3.1 against the verification tool designed and implemented in this project. The relevant characteristics in Table 3.1 were chosen depending on their relevance in the domain field of HistAuskunft s verification. Building a driver in the SUT s GUI Y N Y Y N Capture replay GUI testing N Y Y N Y JPF Model checking Y N 1 N Y N This TFG s tool Y Y Y Y Y Table 3.1. Comparison of different automated GUI testing techniques. 1 Java bytecode is needed. 24

41 3 Precedents and state of the art Differences with existing GUI testing The main purpose of this verification is to assert that HistViewer behaves as the users of the IBM 3270 terminal expect. In traditional GUI testing there are known outputs for a given set of inputs and procedures. This is not the case in this verification which is also meant to discover additional requirements or hidden functionality in the mainframe connected to the IBM 3270 terminal. Automation leads to a much higher coverage and the many advantages pointed out in section 3.3 GUI testing automation. In this project it also brings additional benefits such as the possibility to discover new use cases for the HistViewer and in a bigger scale that manual verification would allow. Most importantly, one of the major differences to regular GUI testing is not having a strong goal state definition. It is not possible to have such strong definition of goal state when the purpose is to discover new states and verify that all the states (initial, intermediate and goal) are equal in the context of two given isolated systems. If such strong definition of goal states was used in this project, the capability of this tool to discover new states or characteristics would be either diminished or even nullified. Far from easing the verification process this means that the tool must get conclusions from the intermediate states as well and log every possible detail that could be considered relevant in the testing process while still not drowning the really important information in irrelevant states or false positives that would render the log unreadable. Finally, tools have not been extensionally used to verify legacy IDS such as the IBM 3270 against other systems. The main reason is that the lack of standards in that age led to system layouts (among other characteristics) that were far to be unified in the age of the target LIS. Since every LIS is so special, general verification solutions would be too complex and ad-hoc solutions too specific to be usable in any slightly different system. 25

42

43 4 Methodology and tools 4. METHODOLOGY AND TOOLS This chapter presents the methodology used in this project. The methodology explains and justifies the methods followed for this project s evolution and the available tools. This chapter also contains the project plan designed using the methods explained Development in itestra GmbH The method used in this project will be, as demanded by itestra GmbH, an adaptation of Lean manufacturing [Ohno 1988], used as standard in their projects. This adaptation is greatly influenced by Lean Software Development (LSD) [Poppendieck and Poppendieck 2003]. In LSD, the Lean principles and the Agile practises are mixed together. This way, LSD considers the evolution of requirements using self-managed and multifunctional teams as well. This adaptation is meant to reduce the creation of waste in favour of the creation of value from the customer s perspective. This value also resides in the actions or processes that create benefit for the customer. As itestra GmbH s method is based on LSD, tasks are managed using Kanban boards (described in section Kanban boards). Another similarity is the zero defects approach that will be covered later in this document. Nevertheless, Lean and itestra GmbH s method are not completely equal as the later also considers the realization of a Proof of Concept (PoC), the Goal Driven Development (GDD) principles and the Getting Things Done (GTD) time management method. All of them will be covered in this chapter The 7 principles of LSD The 7 principles of LSD are based on Toyota s Lean Principles. In LSD Toyota s principles are adapted and heavily influenced by the Agile Software Development. The 7 principles of LSD are [Poppendieck and Poppendieck 2003] [Waters 2010]: 1. Eliminate waste. Activities that do not add value for the customer are regarded as waste. In software development, waste comes principally from the implementation of unrequired features and excessive difficulties in the communication between stakeholders and development teams [Ambler 2010]. Reducing this waste could be done by allowing the teams to be self-managed and reducing scrap and rework later in the lifecycle by reducing waste earlier. 27

44 4 Methodology and tools Build quality in. Ensure quality is considered since the beginning of the project. Look for defects constantly. This is related to the zero defects approach where fixing defects as early as possible reduces costs and increases the overall quality. Create Knowledge. Ensure people get better and improve their skills constantly. Make sure that the created knowledge is accessible and shared. Knowledge can be created, among other ways, via code reviews, proper documentation, wikis or knowledge sharing sessions. Defer commitment. Decide as late as possible. Early decisions usually have risks associated. Something might change and render that early decision a bad one. If a decision can be delayed, delay it so that more information is available by the time the decision has to be made. Despite of this, decisions should not be made too late but just in time. Deliver fast. This principle is heavily influenced by the time to market concept. The time to market refers to the time between the conception of a product and its availability for sale. Delivering fast and being the first one usually gives a competitive advantage. Related to the software development, delivering fast leaves less time for requirements to change. Respect people. Encourage people to give their views. Empower people giving them responsibility to make decisions about their work. Respecting people is strongly related to creating knowledge because knowledge allows informed decisions. Respect also motivates people and leads to commitment with the project. Optimise the whole. Make the appropriate measurements and improvements to the processes constantly. Focus on the integration of the processes as well since the stakeholders are interested in the complete product. Optimise the workflow as a whole to reduce waste and also improve the quality of the product constantly Kanban boards Kanban is a Japanese word that literally means signboard and is a tool for workflow management [Ohno 1988]. Kanban makes identifying forthcoming tasks easier for the team. The Kanban board embodies one of the key principles of Lean manufacturing, produce only what is needed. Kanban boards make task management more visual. The tasks are represented with cards 28

45 4 Methodology and tools that are placed on the board in different vertical lists. A task would be assigned to a list depending on several characteristics. The Kanban board should be kept alive moving the tasks through the lists according to details such as how close those tasks are to the done state. Task flow should be from left to right to represent their progress from the beginning (left) to the end (right). The Kanban itself does not define list s names. It is up to the team to decide on which lists will be used. In software development, the most commonly seen lists are To Do Doing and Done. Cards would be placed on the lists of the Kanban board as seen in Figure 4.1. To Do Doing Done Task 2 Task 1 Figure 4.1. Kanban board example Zero defects approach In a similar manner to the rest of the projects in itestra GmbH, the development will apply a zero defects approach based on the one applied in Toyota [Ohno 1988] and in W. Edwards Deming contributions to Toyota. The highest priority will be to target defects as soon as they get detected; even before new functionality is added. The reasoning behind this statement is that as time goes it will be harder to detect those defects again so that they can be fixed. Also, carrying over defects leads to a worse process with more presence of scrap or rework. In lean manufacturing terms, scrap and rework are two forms of waste which add no value to the customer and thus should be reduced constantly. For clarification, the zero defects approach does not mean there will not be defects at all. It means that action will be taken against every defect found as soon as possible. The full name of this approach could be zero known defects approach. Taiichi Ohno materialized the zero defects approach, among other techniques, with stopping the manufacturing line by pushing a big red button [Ohno 1988]. This red button would stop everything and is meant to be pressed as soon as anyone detects a defect. The manufacturing line would continue after the defect was fixed. Stopping the manufacturing line to fix defects might look like an action that would decrease throughput compared to not stopping at all and getting more hours of continuous manufacturing. In fact, the managers that applied Taiichi Ohno s proposal had a huge productivity drop at first and the managers that did not were having higher productivity rates. But later, the managers that used the red button to stop the manufacturing line and were fixing the defects and inefficiencies as soon as they were 29

46 4 Methodology and tools noticed were improving their processes. After the improvements, those managers were producing faster, more reliably and at a lower cost than the ones that did not stop the manufacturing process to fix defects. As a conclusion, stopping the manufacturing line to fix defects is an investment that ends up reducing costs and increasing value later on Proof of Concept A Proof of Concept (PoC) is according to the Oxford dictionary [Oxford University Press 2015]: Evidence, typically deriving from an experiment or pilot project, which demonstrates that a design concept, business proposal, etc. is feasible. While the term PoC is notably used in economics, it is not strange to see it in engineering or software development. Closer to the domain field of this project, PoCs have been previously used in the definition of verification tools as well [Jaw et al. 2007]. Oftentimes, PoCs are used in software development as a means to check the feasibility of a project or discover the capabilities of the technical platform. To make sense, PoCs should be less costly than the full project they represent. PoCs are similar to prototypes, with the only notable difference being related to coverage. While PoCs focus on whether something can be done or not, prototypes imply further examination and an artefact that is closer to the desired result. A good PoC should cover as many characteristics or features as possible with a depth proportional to the associated risks while keeping costs low. A finished PoC would then showcase if the risks materialize and make the project unviable or it is possible to overcome them leading to a successful project Goal Driven Development principles Instead of requirements, goals will be considered in a way very closely related to the Goal Driven Development (GDD) proposed by [Schnabel and Pizka 2006]. These goals are defined by [Schnabel and Pizka 2006] as an informal description of what a stakeholder wants to change or improve in his business environment. For example, a sample goal from this project is extract content from the IBM 3270 terminal automatically. The GDD principles propose an iterative approach in which all iterations start with the identification of goals. Every iteration ends with an executable version of the software that satisfies the selected goals for that iteration. If the executable version does not satisfy the goals for that iteration it could mean that either there are defects in the implementation or in the definition of goals which should be reviewed as well. 30

47 4 Methodology and tools The goal identification is a three-step process. First, goals are defined without considering technology. Second, the team thinks of how to apply technology to achieve the defined goals. And third, the costs to achieve the goals with different technologies are taken into account to determine the viability and priority of the goals. The defined goals should be SMART: Specific: be able to answer what, why, who and where. Measurable: progress can be measured and accomplishment is verifiable. Attractive: the goal is worth achieving. Realistic: consider the available resources. Time-bound: the goal has defined start and end dates. Finally, while goals are defined in a top-down approach, goals are achieved in a bottom-up approach. This implies a convergence between defining goals (business needs) and achieving them (available technology). This way, the GDD principles avoid situations such as projects that focus on the technology that is going to be used instead of the actual goals Getting Things Done method Time management of this project will be led by the Getting Things Done (GTD) method [Allen 2015]. The main idea is to keep track of the tasks externally and divide them in actionable items according to their category, duration or deadline. The reasoning behind this method is to focus the attention on acting on the items instead of spending time remembering the tasks. Focusing the attention on one actionable item at a time allows the mind to not think about them constantly which is possible having all items in a trusted external place. Those tasks that are being thought constantly while still not making progress are called open loops. Open loops as defined in the GTD method [Allen 2015] are anything pulling at your attention that does not belong where it is, the way it is. In other words, open loops are the thought of something that has to be done, but cannot be done by the time the thought is had. Since no progress is being done, there is a waste of time, energy and attention focus that only adds to anxiety. Open loops have been proven to be a burden on the mind [Baumeister and Tierney 2011] meaning that closing them is highly desirable. Tasks are managed focusing on closing open loops. Those open loops can be closed by being captured outside of the mind to decide about them. Keeping reminders of those open 31

48 4 Methodology and tools loops organized in a system that is going to be reviewed regularly helps resting the mind. The reminders can represent the tasks or actions that should be performed to close the open loops. Reminders are stored in a trusted external place that can be anything from notes in paper to tasks on a Kanban board. The reminders should consist of short sentences that always start with a verb so that it is easier knowing what to do. If the task is estimated to be shorter than 2 minutes or is urgent, it should be done immediately Application of itestra GmbH s method in this project The Lean based method used in itestra GmbH will be applied to this project almost as explained in the previous section. Nevertheless, there will be some slight differences. Some details will be adapted to the specific context of this project and its academic nature. The following paragraphs elaborate on the adaptation of itestra GmbH s method. For generic details on the following aspects of the method applied in this project, please read section 4.1 Development in itestra GmbH in this document Kanban board Workflow will be managed with a Kanban board as expected in a Lean based method. The Kanban board of this project will use the lists defined in Table 4.1. To Add in documentation To Do Doing Done Ask Waiting for answer Answered Table 4.1. Kanban lists in this project. Every task logged in this project s Kanban board would start from To Add in documentation, To Do or Ask. The logged task would then follow the flow seen in Figure 4.2 until Done or Answered. There is a special flow ( Ask, Waiting for answer and Answered ) for logging the questions so that all of them are asked and none is left unanswered. Most importantly, the Kanban board allows logging the question and the answer in a place where they are easily retrievable when needed. Those two different flows (questions and tasks) are in the same board. While having a different board for questions and another for tasks seems feasible, the lists for questions and tasks are going to be kept in the same board because questions are considered very closely related to the tasks of the project. 32

49 4 Methodology and tools Figure 4.2. Task flow through the lists in this project s Kanban board Zero defects approach Despite of the fact that in this project there is no physical red button that can be pressed; the red button will still be present figuratively. Production or in other words, development of new features will be stopped until the detected defects are solved in order to avoid dragging those defects along. Those defects would increase technical debt delaying and reducing the quality of the product as discovered in Toyota before the application of this approach. Reducing technical debt will be considered of utmost importance. Technical debt accumulates interest which would make fixing those defects harder than it was at first. Additional rationale for this approach is that finding a defect locates the attention in the context of the defect which in this project would: Reduce the attention from the task that was being done before finding the defect. This reduces productivity in that one task and makes the probability of introducing new defects higher. Finding the defect and not reacting to it would contribute to the creation of open loops as defined in the Getting Things Done method. The GTD method is explained in the section with the same name in this document. Just logging the detected defect and continuing with something else adds long term overhead. The attention comes from the previous task to the defect so that it gets appropriately logged, then comes back to the previous task and when it is time to fix the defect the attention has to go to the defect once again. What is more, humans cannot change contexts completely in the same way computers do, there is always something about the last context in thought which decreases attention focus. The overhead in the context change might not be so small, chiefly when fixing that 33

50 4 Methodology and tools defect gets delayed by several days or weeks. When it is time to fix the defect, its context is not as recent and could even have been forgotten. The fix then gets burdened by remembering the context, which sometimes implies as much effort as discovering the defect once again Justification of the PoC To demonstrate the feasibility of this project it will start with a PoC. This PoC will cover most of the relevant features of the verification. The PoC will focus on the features with higher associated risks, and especially those that have multiple features depending on them. Additional reasons behind this PoC are: Goals are flexible. This verification tool should be adapted to the new verification needs that might be discovered during this project s development. This tool should progress in the direction that seems most relevant in that time. That relevancy might be discovered while progress is being made in specific directions. Thus, there is a motivation to explore a wide range of possibilities and then refine the capabilities of the tool according to interest, relevance and feasibility. It is not certain that the tool will actually produce relevant results. In other words, the tool might not be viable. This is the initial effort of comparing the interfaces of an IBM 3270 terminal and the system that will replace it. As pointed out in preceding chapters, the implementation of this verification technique is completely ad-hoc and it would be hard to know if it would be viable for specific cases. Analysing the feasibility of different techniques and paths to follow for the verification will minimize risks. Most importantly, minimizing the impact of risks that are related to spending time and resources in verification paths that are later discovered to be unviable. The PoC concept pairs well with the GDD applied in this project. The PoC will define a pragmatic environment to discover the best mapping between the objectives and capabilities of the technical platform. This will help defining more attractive, realistic and specific goals. The PoC will be considered successful if the verification tool is proven to be useful. The usefulness of the verification tool will be determined by the relevance of the issues found in the SUTs, the IBM 3270 terminal and HistViewer. Finding issues would justify the development of 34

51 4 Methodology and tools this tool and make its development viable and desirable GTD based task handling In this TFG, tasks will be captured with a task handling process based on the GTD method. The main purpose is to eliminate open loops as defined in the GTD method so that no task is forgotten and the attention is focused only on one task at a time. This task handling uses some of the tools explained in section 4.4 Tools and technologies used in this project. The process is visually represented in the BPMN diagram of Figure 4.3 and explained in the following list: If the time required to complete the task is less than 2 minutes or it is an urgent task, the task will be started immediately without being registered. These tasks will not be logged unless they cannot be started because the computer is not available. If the task is conceived when the computer cannot be accessed, regardless of its duration, the task will get registered on the online Kanban board with the trello for android application. For relatively small tasks whose context is very important, if that task is related to: o Code, the task will be registered as a TODO in its context, highlighted by Eclipse. o Documentation: the task will be logged as a word comment in its context. o Both: the task will be logged in its context using both word comments and TODOs in the code. All other tasks will be logged in the Kanban board. 35

52 4 Methodology and tools Figure 4.3. Task handling process Project planning applying GDD principles The GDD principles explained in section 4.1 Development in itestra GmbH will be applied to this project with slight variations for this project s planning. The goal hierarchy of this project is defined in the model seen in Figure 4.4. There are three entities: goal, subgoal and task. The possible relationships between the entities are satisfies and depends on. A task can depend on a variable number of other tasks. A task cannot be closed until all its dependencies (if any) are closed. A task satisfies a variable number of subgoals. When all tasks that satisfy a subgoal are closed, the subgoal is satisfied. When all subgoals that satisfy the goal of the project are satisfied, the goal is satisfied and the project complete. For the implementation of that model in this project, the goal and subgoals are the ones defined in the chapter 2 Goals of this document. The tasks that satisfy the subgoals will be defined in the project planning. Additional tasks and depends on relationships can be defined at the beginning of each iteration during the project. 36

53 4 Methodology and tools Figure 4.4. Goal hierarchy in this project. This project, as proposed by the GDD principles will follow an iterative evolution. This iterative evolution is a combination of the GDD principles and the concepts of task, subgoal and goal, proposed in this document. That iterative process, designed for this project can be seen in Figure 4.5. The iterative evolution considers the definition of subgoals and tasks without thinking about the constraints of current technology. Afterwards, the definition of tasks reaches a convergence with their viability as taught by the GDD principles. The convergence is obtained by redefining the tasks as the project evolves depending on their viability and value. Iteration 0 is exposed in this chapter, as it represents the project planning and is essential for the actual development of the project. Table 4.2 shows the initial task list to satisfy the subgoals identified in chapter 2 Goals. Table 4.2 groups the tasks by iteration, the subgoal they satisfy and the estimated effort in person days (PD). The tasks in Table 4.2 are ordered in descending priority, for example T1 is the first task as it is at the top of the table. T17 is the last task because it is at the bottom of the table. Those tasks can be seen with their dependencies in the Gantt chart of Figure 4.6. The subgoals satisfied by those tasks are represented as milestones with the End of the Project as the final milestone. 37

54 4 Methodology and tools Figure 4.5. GDD based iterative process applied to this project. 38

55 Iteration Subgoal Task Effort PD 4 Methodology and tools Description T1 Install the required Tools. 2 T2 Connect to the mainframe s VPN. 1 1 SG1 T3 Set up HistViewer s local database. 1 T4 Learn how to use the IBM 3270 terminal. 2 T5 Configure HistViewer. 1 T6 Understand the dialog structure in NatLeben. 2 T7 Learn more about the technical platform. PoC. 5 2 SG2 T8 Extract one dialog for one contract automatically. 4 T9 Compare one dialog for one contract automatically. 5 SG3 T10 Adapt the verification process to verify the specified contracts. 1 SG4 T11 Extract input values to consult dialogs from the output of others. 5 3 T12 Identify fields that show equivalent data differently in each system. SG5 1 4 SG6 T13 Include special comparison patterns. 3 T14 Output the input arguments that led to the compared screens. 1 T15 Output whether a dialog is exactly equal in both systems. 3 T16 Output which fields do not match in the comparison. 5 Final - T17 Validation of results. 5 Table 4.2. Task overview. 39

56 4 Methodology and tools Figure 4.6. Gantt chart with the initial project plan Tools and technologies used in this project The tools that will be used during the realization of this project are shown in this chapter grouped by their category or main purpose. 40

57 4 Methodology and tools Hardware tools This section presents the hardware tools used in this project. There are two different hardware environments, one is for development and the other one is for testing purposes Development environment The complete development of this project will take its course using the notebook computer provided by itestra GmbH. That notebook computer has the following characteristics: HP Pavilion dm4. 6 GB RAM. Intel Core i3-2330m 2.20GHz. Samsung SSD 840 EVO 250 GB. Internet connection so that communication with the mainframe is possible Testing environment The customer provided another computer with the complete environment that HistViewer s end users will use. That computer gave more knowledge about the end user s environment. Due to the nature of the verification tool, the development could not take place while the mouse was captured by the robot. The verification usually meant hours of captured mouse while the robot did its automatic verification tasks emulating users. Having another computer for testing purposes allowed the execution of the verification tool without hindering its development. The computer provided by the client had following technical characteristics: Fujitsu. I5 3230M 4 GB RAM Connection to two remote servers one via Citrix and the other via Microsoft s remote desktop connection System Under Test related tools These are the tools that are needed for the operation of the SUT. The SUTs (HistViewer and the IBM 3270 terminal emulator) are included in this section. 41

58 4 Methodology and tools IBM Personal Communications Session Manager 6.0 IBM Personal Communications Session Manager 6.0 [IBM 2016d] is used as an IBM 3270 terminal emulator. The emulator allows connection and access to the content of the IMS [IBM 2016c] database via the customer s mainframe. This terminal emulator is used by itestra GmbH s customer to access the mainframe by the time of writing this TFG. This IBM 3270 emulator and the mainframe it connects to will be replaced by HistViewer. This emulator is considered a SUT related tool because the verification tool has to interact with this emulator and HistViewer to compare their outputs Check Point Mobile Check Point Mobile (VPN) [Check Point Software Technologies Ltd. 2016] is used to connect to the customer s network in which the mainframe is located. Check Point Mobile makes the communication between the IBM 3270 terminal emulator and the mainframe possible HistViewer HistViewer is the application in development by itestra GmbH for the replacement of the customer s IBM 3270 terminal emulator and the mainframe. To achieve a successful replacement, HistViewer must have the terminal emulator and the mainframe s functionality and for that reason HistViewer is the application to verify in this TFG. Its frontend is based on JavaFX [Oracle 2016a] and makes use of a relational database. HistViewer s source code will contribute to the comprehension of the data structure used in the whole system in a grey box testing context Windows 8.1 Professional Microsoft Windows 8.1 Professional [Microsoft 2016a] has been chosen as operating system because of the compatibility with the rest of software tools used in the project. Besides, its use was requested by itestra GmbH s customer. It is relevant to test HistViewer s behaviour on Windows 8.1 because that is the operating system that itestra GmbH s customer will use Project management tools The following tools are used for the management of this project. Project management tools focus on managing the dependencies of this project with other modules, the control of the different versions of this project during its life cycle or the management of the tasks themselves. These tools also ease management so that more effort can be spent in tasks that actually increase value for the customer in a lean environment. 42

59 4 Methodology and tools Apache Maven Apache Maven [The Apache Software Foundation 2016] is a software project management tool. The use of Maven is extended from HistAuskunft to this project incorporating other modules as dependencies GIT GIT [Software Freedom Conservancy 2016] is a distributed version control system that supports non-linear development with remote contributions. This project will be in the same repository with the rest of modules in the HistAuskunft project by itestra GmbH since it is part of HistAuskunft s verification tasks Trello Trello [Trello, Inc. 2016] is an online tool focused on project and task management. Trello supports the Kanban board used in this project. Trello will be accessed from the web based client when using the computer. When the computer is not available, Trello will be accessed from its Android application Software development tools These tools are directly related to the development of this project s software. This includes the main programming language environment, remarkable external components that are essential for this project or the main development environment Oracle Java SE JDK 8 Oracle Java SE Development Kit (JDK) 8 [Oracle 2016b] will be used as the main programming language in the development of this project. Java was chosen because HistAuskunft s project manager asked to do so in order to maintain homogeneity and code reusability with the rest of HistAuskunft modules which are written in Java Java AWTRobot Java AWTRobot [Oracle 2016c] will be the base for the robot that verifies the interfaces of HistViewer and the IBM 3270 emulator because of its simple extension by other Java applications while still being useful to interact with GUIs emulating a normal user. Another remarkable characteristic is how customizable it is compared to specialized GUI testing tools or frameworks. Those testing tools or frameworks are usually adapted by default to more standard GUI testing instead of the ad-hoc approach of this project. Adapting those pre-configured GUI 43

60 4 Methodology and tools testing tools or frameworks to verify legacy interfaces would be like swimming against the tide. What is more, the AWTRobot is able to create native input events that are equal to user generated ones. The Java AWTRobot was also considered by HistAuskunft s project manager for the extraction of content from both systems. Further information about this tool can be found in chapter 3 Precedents and state of the art Eclipse Eclipse [Eclipse Foundation 2016] is an Integrated Development Environment (IDE) which will be used for the verifying system s development. Eclipse is a well-known IDE under an open software license (EPL). Eclipse is multiplatform and widely used. The Eclipse IDE was chosen for the development of this project because it is being used in HistAuskunft as standard and in many other projects in itestra GmbH. It was also used because of its proven suitability for software development and the previous experience of the developer of this project. Another key reason for the use of Eclipse is its extensible plugin system that allows the integration of other tools used in this project such as GIT, Apache Maven, or quality assurance tools such as FindBugs or PMD Quality assurance tools Quality assurance tools contribute to the quality of the final product. In a similar way to testing, the lack of issues reported by static code analysis tools does not mean that there are no issues. What is more, reports from quality assurance tools should be subjected to judgement. Quality assurance tools are similar to GPS assistants for driving, they might be right most of the times but they should not prevail over reason. Nevertheless, the reports of quality assurance tools are most of the times suggestions worth considering. The tools considered in this section apply static code analysis, they analyse the code without executing it FindBugs FindBugs [University of Maryland 2016] is used to discover bugs in Java code. It performs static code analysis on Java programs automatically to find potential bugs PMD PMD [PMD 2016] is a source code analyser for Java. PMD reports issues according to default or custom rules defined in rule-sets. Most of the times those issues are not errors that affect the successful execution of the code but flaws that affect the efficiency or maintainability of the code such as dead code, inefficient code, less readable code, overcomplicated expressions or 44

61 4 Methodology and tools classes with high cyclomatic complexity measurements Metrics Metrics [Sauer 2016] is a plugin for eclipse that calculates metrics for projects every time the code is compiled. Among other metrics, this plugin shows the depth of inheritance trees, nested block depth, McCabe s cyclomatic complexity and dependency analyses. These metrics are often useful to detect bad smell (hints that suggest quality issues) in the code ConQAT ConQAT is a static code analysis tool developed by and used in itestra GmbH. ConQAT is a superset of PMD with additional functionality such as improved detection of duplicated code, use of literals or clone units (undesired redundancy). The results of the analysis are contained in csv files and html files that can be opened with a web browser Database oriented tools These tools facilitate the operations with the databases used in this project. This section considers the IBM DB2 engine and Dell Toad for DB2 used to interact with the remote and local IBM DB2 databases IBM DB2 IBM DB2 [IBM 2016a] is a relational database engine. IBM DB2 supports HistViewer s persistence and the storage of the extracted segment binary dumps from itestra GmbH s customer s IMS database. The use of IBM DB2 was asked by itestra GmbH s customer Dell Toad for DB2 freeware 6.1 Dell Toad for DB2 freeware 6.1 [Dell Software Inc. 2016] will be used to communicate with DB2 in tasks such as examining the data structure or fetching dialogs with specific characteristics that should be handled differently by the verifying tool developed in this project Documentation tools These are the tools used for documentation purposes. The tools in this section are used for diagramming, document writing and reference management yed Graph Editor yed [yworks 2016], is a diagramming tool. All graphs and diagrams will be created using this tool unless stated otherwise. 45

62 4 Methodology and tools ObjectAid UML Explorer for Eclipse ObjectAid [ObjectAid LLC 2016] is a lightweight eclipse plugin that supports the creation of class diagrams and sequence diagrams. It can also generate class diagrams by performing reverse engineering from the Java code Visual paradigm for UML 12.2 Visual paradigm [Visual Paradigm 2016] a CASE (Computer Aided Software Engineering) tool that can be used among other things for diagramming, reverse engineering and report and code generation. There are 13 supported UML diagrams such as class or sequence diagrams. Visual Paradigm allows diagramming via reverse engineering and the generation of code from the diagrams Gantt project Gantt project [GanttProject Team 2016] is a project management tool that allows the creation of gantt charts with tasks, milestones and dependencies among other details Microsoft Word Microsoft Word [Microsoft 2016b] is a word processor. Microsoft Word will be used for writing the documentation that needs formatted text such as this document Zotero Zotero [Roy Rosenzweig Center for History and New Media 2016] is a tool used to ease the collection and organization of references. It will be used to manage references in this document with Zotero s integration with Microsoft Word. 46

63 5 Results 5. RESULTS This chapter presents the evolution of this project. The results obtained have used the iterative process exposed in chapter 4 Methodology and tools. Additionally, this chapter presents the challenges related to the design and implementation of the verification tool. This chapter concludes with some empirical evidences after applying the verification tool to the NatLeben system within the HistAuskunft project Iteration 0: Initial task definition Iteration 0 has three phases which are very closely related to the project planning. These are the three phases that lead to the initial task definition: Define and prioritize the subgoals that satisfy the project goal. Define the tasks to satisfy the subgoals. Define the number of iterations and assign the subgoals. The artefacts obtained from these three phases build the project plan. The project plan is exposed in this document in section 4.3 Project planning applying GDD principles. That section also defines the rest of the iterations with their initial subgoal and task association Iteration 1: Understand the environment of the Systems Under Test The purpose of this iteration is to satisfy subgoal 1 with the understanding of the technical context. That understanding will help in the redefinition of the project plan according to the viability of the tasks. The tasks for this iteration are listed in Table 5.1. That table also shows the subgoal that the tasks satisfy, their identifier, description and estimated effort in person days. The tasks are ordered in descending priority. For example, T1 - Install the required tools is to be done first as it is at the top of the table. T6 - Understand the dialog structure in NatLeben is the last task of this iteration. No additional tasks have been identified as there is no more knowledge at the beginning of this iteration than at the end of the project planning in iteration 0. 47

64 Iteration Subgoal Task Effort PD 5 Results Description 1 SG1 T1 Install the required Tools. 2 T2 Connect to the mainframe s VPN. 1 T3 Set up HistViewer s local database. 1 T4 Learn how to use the IBM 3270 terminal. 2 T5 Configure HistViewer. 1 T6 Understand the dialog structure in NatLeben. 2 Table 5.1. Task list for iteration 1 in descending order of priority T1 - Install the required tools The required tools are those listed in this document in section 4.4 Tools and technologies used in this project. This task involves their installation and configuration T2 - Connect to the mainframe s VPN The connection to the mainframe requires connection to itestra GmbH s customer s VPN. That connection is performed by using the tool CheckPoint Mobile which requires a certificate and a password. Itestra GmbH s customer issued the certificate and password to make this verification possible. The connection to the VPN is vital for the verification task in the following dimensions: The IBM 3270 terminal emulator in order to communicate with the mainframe. The content extractor to populate HistViewer s database. HistViewer when it connects to the remote database in the customer s servers. The IBM 3270 emulator and HistViewer are the Systems Under Test (SUT) so it is mandatory to have them up and running for the verification to take place T3 - Set up HistViewer s local database HistViewer is a client which connects to a server with a backend that supplies the data extracted from the mainframe. The verification needs HistViewer s output which comes from the database of that backend. First, the backend needs a DB2 server installed and configured. Second, HistViewer s content extractor has to be launched. That extractor connects to the mainframe, fetches the content from the mainframe s output and stores it in a relational database (DB2). This extraction is performed by a different kind of screen scraping which is 48

65 5 Results being implemented by HistAuskunft s team. The extracted content will be read by HistViewer and compared to IBM 3270 s content by the verification tool developed in this project. The longer the extraction is left running the more screens that will have been extracted. The extraction of all the screens that the customer made available for the development of HistAuskunft is estimated to be completed in a minimum of two months. The extraction is only possible during the operating time of the mainframe which delays the process. For production data, the estimations are still unknown as it depends on the customer s needs by that date but it could be measured in the order of one year. Those times are far longer than the time constraint for this whole project so the extraction ran only until the end of the day this task was being done. That was enough to get approximately all the screens for 4000 contracts which will serve as sample for the verification. Further executions of the content extractor may be considered if a bigger sample is needed T4 - Learn how to use and configure the IBM 3270 terminal The IBM 3270 terminal resides in the emulator provided by the tool IBM Personal Communications. This terminal provides access to the mainframe s content. The connection is established by selecting the Telnet3270 protocol under TCP/IP with a LAN interface and specifying that the type of host is an IBM zseries mainframe. The screen size should be configured to be 32x80 which is the size used by itestra GmbH s customer. The screen size is a very important value to consider in the interaction with the interface performed by the verification tool. After configuring the terminal emulator, the connection to the mainframe is possible. This leads to the window seen in Figure 5.1 where some fields that could compromise itestra GmbH s customer s anonymity have been hidden. That window represents the access control asking for the userid and password among other details. The window displays that the terminal emulator offers more than just the terminal screen which can be seen in the range of actions shown at the top of Figure 5.1. After this task, the IBM 3270 terminal emulator is completely configured to make interaction from the AWTRobot possible. The keyboard is remapped so that the robot can input specific key events such as the PA1 key. The PA1 is a particular key of the IBM 3270 and other terminals. The remapping also allows utilities such as select all and copy shortcuts to get the screen text to the clipboard from the AWTRobot. This allows the extraction of content. The most important result of this task is verifying that the verification tool will be able to extract the 49

66 5 Results data from the terminal emulator as plain text using keyboard shortcuts. Figure 5.1. Mainframe's user authentication with hidden fields T5 - Configure HistViewer HistViewer is one of the modules of the HistAuskunft project available in its git repository. To get HistViewer up and running, the backend has to be built. Apache Maven takes care of the dependencies and the tomcat server for the backend is started according to the DB2 database configuration created in T3 - Set up HistViewer s local database. With the backend started, HistViewer can be launched and the behaviour should be the same learned in T4 - Learn how to use and configure the IBM 3270 terminal. Verifying that the behaviour is identical is one of the responsibilities of the verification tool developed in this project T6 - Understand the dialog structure in NatLeben After the authentication in the mainframe from the IBM 3270 terminal emulator, the navigation through the systems stored in the mainframe is possible. This task is another knowledge-related 50

67 5 Results task that does not involve development but understanding the SUT. The results of this task can be found in the section Navigation through dialogs in NatLeben End of iteration 1 with informed review of the Project Plan Closing task 6 satisfies subgoal 1 and leads to the end of the iteration with the review of the project plan obtained in iteration 0. The knowledge acquired from inspecting the technical aspects of the SUTs and the ways to interact with them allows a more informed revision of the project plan. After this iteration, there is more information about the tasks and what can or cannot be done. Many commands and interaction possibilities were different in HistViewer and the IBM 3270 terminal emulator. Nevertheless, all the actions required to access and extract the information contained in the screens of both systems were available from the keyboard and the mouse. The verification needs a system that can create input events automatically. Those input events must allow entering text in specific positions of the input interface. That interacting system would also have to interact with the operating system s clipboard in order to extract the displayed text in the SUTs. Furthermore, switching between the HistViewer and the IBM 3270 terminal emulator windows has to be possible. The decision of building the verification tool using the Java AWTRobot is still in force as the AWTRobot allows the consecution of all project tasks planned so far and the details explained in this paragraph. Additional reasons for choosing the AWTRobot are explained in section 4.4 Tools and technologies used in this project. According to these results, no unexpected constraints affect the consecution of the planned tasks or notably increase their difficulty. Thus, the project plan does not suffer changes in the planned tasks Iteration 2: Starting the PoC This iteration contains the PoC as proposed in sections Justification of the PoC and 4.3 Project planning applying GDD principles. The closure of this iteration s tasks will achieve Subgoals 2 and Identification of additional Tasks Iteration 2 begins with the identification of additional tasks and dependencies to achieve the 51

68 Iteration Subgoal Task Effort PD 5 Results selected subgoals. In this case, although there is knowledge about the SUT, there is still no actual technical platform. No development-related tasks have been started yet and there is no more knowledge about the development tasks than at the beginning of the previous iteration. For that reason, no additional tasks or relationships between them have been detected. The tasks assigned to iteration 2 can be seen in Table 5.2. The priority of those tasks remains unchanged. Description 2 T7 Learn more about the technical platform. PoC. 5 SG2 T8 Extract one dialog for one contract automatically. 4 T9 Compare one dialog for one contract automatically. 5 SG3 T10 Adapt the verification process to verify the specified contracts. 1 Table 5.2. Task list for iteration 2 in descending order of priority T7 - Learn more about the technical platform. PoC The PoC is the first stage of the development following the method explained in this document. Hence, the PoC will cover the riskiest tasks in order to find constraints and evaluate the feasibility of those tasks. The PoC helps in the definition of the software architecture applying a general view of the project and considering the planned tasks. Further justification for the PoC can be found in section Justification of the PoC. As found out in T6 - Understand the dialog structure in NatLeben, getting the entity id is the first step to access any contract s screen output. For that purpose, the PoC reuses the EntityIdProvider module used in HistAuskunft. Iterating through contracts presents the project with its first challenge. There are differences in the interaction with HistViewer and the IBM 3270 terminal emulator. The main differences in commands are shown in section Interaction differences between HistViewer and the IBM 3270 emulator. The specific commands used for each system or environment are decoupled from the verifier class and its specializations. The screen scraper object (c.f. Figure 5.2) knows which commands to use for each environment (e.g.: NatLeben). The scraperbot object knows how to perform those commands for each system (e.g: HistViewer). The screen scraper object (c.f. Figure 5.2) follows the adapter software pattern [Gamma et al. 1994]. The screen scraper adapts the interface of the scraperbot to the domain of the verification environment. The scraperbots need many conditions such as the window size or commands for each system. Those conditions justify delegating the creation of the scraperbots to a factory (c.f. 52

69 5 Results Figure 5.2). Figure 5.2. Considering different environments and systems using the adapter and factory software design patterns T8 - Extract one dialog for one contract automatically This task is built on the results obtained in the PoC. This task is performed focusing on the L02 dialog since the L02 provides core business information. In particular, the L02 provides the information needed to navigate through other dialogs in the NatLeben environment. The screen scraper presented in previous tasks requests the scraperbot object to perform two actions: (i) place the cursor in the first field and (ii) insert input parameters. The scraperbot knows how to perform such commands depending on the target system and environment. That expected behaviour would be special for each scraperbot object. The screen scraper then gets the screen output encapsulated in an AbstractDialogInput object (cf. Figure 5.3). The screen scraper finally unpacks the input according to the verification environment. Figure 5.3 illustrates how the class NatLebenDialogInput would encapsulate NatLeben s input fields. 53

70 5 Results Figure 5.3. Inserting arguments to consult a NatLeben dialog T9 - Compare one dialog for one contract automatically After closing this task, the verification has to compare the output extracted for the same dialog in both systems. The process of extraction and comparison is shown in the sequence diagram of Figure 5.4. A utility class that serves as prototype in the PoC will handle the output of the comparison results. The output will be redefined in the appropriate tasks. More details about the comparison behaviour of an abstract screen comparer can be seen in Figure 5.5. The comparer first does a quick comparison of the source and target text ignoring some fields that are shown differently on purpose. If the texts are different, a field by field comparison is in order. The result from that field by field comparison is the list of fields that do not match. The field list and the special comparison treatment for those fields are known by the comparer at runtime via object composition. 54

71 5 Results Figure 5.4. Complete extraction and comparison of one dialog for two systems. 55

72 5 Results Figure 5.5. Comparison process. In the case of the comparison, the template method design pattern [Gamma et al. 1994] makes it possible to adapt the comparison process to the environment under verification. As shown in Figure 5.6, the comparison process is common for the different environments. 56

73 5 Results Nevertheless, details such as the fields to ignore or ensuring that the texts are comparable are specific to each environment. The comparison algorithms are contained in the screen comparer classes shown in Figure 5.6. Figure 5.6. Template method design pattern for screen comparers T10 - Adapt the verification process to verify the specified contracts The AbstractVerifier and its specifications can iterate through the screens of different contracts. The iteration is performed using the entity ids from the EntityIdProvider module introduced in T7 - Learn more about the technical platform. PoC End of iteration 2 The most important result of iteration 2 is a successful PoC and as a consequence, a first tool is available. The project is marked as viable for two reasons. First, no serious constraints have been found. Second, the executable version deployed in the testing environment already found issues in the HistViewer. Those issues are some of the ones listed in section 5.7 Case study. After the PoC, reviewing the design architecture and the code is the most immediate step to 57

74 5 Results take. Proper and frequent reviews ensure that the software architecture is adequate for the following development. Finally, the executable version satisfies subgoals 1 to 3. The project plan does not suffer changes after iteration Iteration 3: Navigating through dialogs and new verification modes Iteration 3 starts with the identification of additional tasks. It continues with the closure of all the tasks assigned and ends with an executable version that grants additional value. HistAuskunft s project manager (PM) got involved in the definition of tasks in this iteration proposing some changes Identification of additional Tasks In this iteration, HistAuskunft s PM came to Spain to review the status of this project and redefine some of the tasks. According to the PM, the verification process should be complemented with other verification modes. The navigation mode that was being developed at this time is complemented with the addition of the random, existence and direct verification modes: Navigation: The verification tool navigates using the output of some screens as input for others. This emulates the user that navigates through the environment depending on the output. Random: This mode does not depend on the navigation strategy used in HistAuskunft. Choosing the input values randomly also allows the discovery of additional elements that could have been overlooked. The possible values and probabilities have to be defined and weighted to ensure the obtainment of meaningful results. Existence: This verification tool found some contract screens that should not exist according to NatLeben s specification. This mode is meant to gather more information about them. In the end those contracts should not be available to the end users because of business constraints and this mode prevented their inclusion in the new system. Direct: Some dialogs are consulted manually sometimes. This mode allows the verification to take place on already consulted dialogs giving a faster and less errorprone output. This mode also includes additional details from the domain field of HistAuskunft such as the field s names. 58

75 Iteration Subgoal Task Effort PD 5 Results The design and implementation of the additional modes is explained in T18 - Design the architecture to allow different verification modes. That first task has the design of the additional modes. The other tasks on this iteration are devoted to the implementation of the specific verification modes. Description 3 SG4 SG5 T11 Extract input values to consult dialogs from the output of others 5 T18 Design the architecture to allow different verification modes 2 T19 Implement weighted random verification. 3 T20 Adapt the tool to verify the existence of L42 dialogs 2 T21 Adapt the tool to verify one manually consulted dialog 1 T12 Identify fields that show equivalent data differently in each system. 1 T13 Include special comparison patterns. 3 Table 5.3. Task list for iteration 3 in descending order of priority T11 - Extract input values to consult dialogs from the output of others This is a very important task whose sole closing satisfies subgoal 4. This task is divided in three phases: Detect output values that can be used as input for other dialogs. This detection was done using the IBM 3270 terminal emulator, the documentation for the NatLeben environment and the requirements of the HistViewer. Fetch output values from the screens. The tool collects values such as ERH, IG; ZUO, VSNR (contract number), system messages (e.g.: contract does not exist ) or page delimiters from the screen s output. Navigate through dialog pages using the extracted screen output. The tool uses part of the extracted output as input for the following screens according to the navigation strategy. The results of this task are presented in section Navigation through dialogs in NatLeben T18 - Design the architecture to allow different verification modes This task consists in the design of the architecture for the different verification modes identified at the beginning of this iteration. 59

76 5 Results The verification modes (navigation, random, existence and direct) are designed as specializations of the abstract verifier. The environment (NatLebenVerifier) and the number of systems to verify (IOneSystemVerifier and ITwoSystemsVerifier) are considered for polymorphism and code reuse (c.f. Figure 5.7). The VerificationFullLauncher is in charge of creating the verification environment depending on the input arguments. The environment knows which specialization of AbstractVerifier to use and additional details such as the kind of screen scrapers or the limit of contracts to verify. Figure 5.7. Class diagram for different verification modes T19 - Implement weighted random verification. The random verification does not use the navigation strategy to avoid dependencies with that strategy s design. The main purpose is to discover details that could have been ignored because of the navigation strategy chosen in HistAuskunft. This verification approach has been defined by HistAuskunft s PM and labelled as useful for the whole duration of HistAuskunft. The random verification approach can lead to the discovery of additional details of the mainframe environment that could have slipped out of the elicitation process. The probabilities to use each input argument are computed from the segments database. This SQL DB2 database has the extracted content from the IMS database of the mainframe. The 60

77 5 Results database has the underlying information presented on the screen. The probabilities are calculated regarding the number of occurrences of each possible value for every input field. The complete extraction of probabilities is explained in Appendix A: Probabilities of the input arguments. The probabilities are stored in files to ease their replacement. The available input arguments are generated by reading the files with probabilities when they are needed. That technique is called lazy evaluation or call by need. The arguments and their probabilities are then stored in a hashing data structure to reduce file input and output operations. The software design pattern Flyweight [Gamma et al. 1994] is used to design that call by need. The Flyweight replaces the repetitive creation of similar objects with the reuse of them. As an example, the first documented use of this design pattern had as purpose avoiding the duplicates in the representation of glyphs in a document editor [Calder and Weston 1994]. Those glyphs would be represented as objects that were created when they were needed and only once. Further creations of those glyphs would be replaced with the reuse of the first instance. After the first use of the letter a further repetitions would use the same instance adding only the specific location of that repetition. It was proven beneficial in the large scale as hundreds of repetitions of the same glyph would only need one instance of that glyph and the specific location of the repetitions. In this project, as seen in Figure 5.8 the AbstractInputProvider specializations would serve as Flyweight factories. The Flyweight factories would create and register new Flyweights if they were not already registered. Flyweights (IRandomInputGenerator) are the objects that will be created once and then shared avoiding duplicates. The traditional Flyweight design pattern considers that the Flyweight factory should return the instance that was created (or previously registered), in our case, it is better to return the result of a method executed on them. This encapsulates how the input values are generated and decouples the generation from the verifiers. 61

78 5 Results Figure 5.8. Input providers adapting the Flyweight software design pattern. Figure 5.9 illustrates the random verification. The random verification uses the available input arguments and the probabilities specified before launching the verification to consult and extract the screen text. If the screen text is comparable a high level comparison applying some comparison patterns is in order. If the comparison determines that the screen text does not match, there is another comparison with more detail to identify the specific fields that do not match and their coordinates. In case the text is not comparable, the consult and extraction of screen text takes place as many times as configured to ensure that it was not due to synchronization issues with the SUT. This process continues for as many contracts as specified by the initial number of entities 62

79 5 Results to verify. Figure 5.9. Random verification workflow. 63

80 5 Results T20 - Adapt the tool to verify the existence of L42 dialogs In NatLeben, any entity has one current contract and a variable number of historical contracts (can be zero). It is against NatLeben s specification if there is an historical contract for an entity and there is no current contract. By definition, the last historical contract would be considered the current one and put in that place. Nevertheless, some dialogs that violated this rule were discovered during the development of this tool. Those dialogs were of the L42 kind. The conflicting L42 dialogs are those which have historical contract entries but do not have a current one. This issue was located in the source system (IBM 3270) so there was a conflict between the specification of the environment to replicate and the actual behaviour of the original. The verification tool included a new mode that allowed checking if an entity has historical contracts and does not have current contracts. This mode was used to identify the magnitude of the problem and to analyse its context. According to a design decision made by itestra GmbH s customer, such existing contracts were not supposed to be accessible by the end users. Therefore, the verification tool found contracts that must not be included in HistViewer. This way, the verification tool has reported an important bug that could have been harmful in production T21 - Adapt the tool to verify one manually consulted dialog This verification approach was requested by HistAuskunft s development team, which is the main stakeholder of the verification tool. This verification mode directly extracts the content from both systems any consult. Direct verification is used to verify already consulted contract screens. The verification tool then processes the extracted text to give the comparison results. This verification approach is the automation of a process that was being performed before the development of this verification tool. This verification was previously being performed by manually selecting all the text from the IBM 3270 s output and copying and pasting from the 3270 to the notepad ++ compare plugin and doing the same for HistViewer s output. While that manual process does not seem very time consuming for one screen, it is not scalable and this is a frequent task. Besides, the retirement process of the IBM 3270 terminal and its mainframe is meant to be a several years long project. For that reason, one to one comparisons (IBM 3270 to HistViewer) are going to be very present during the whole length of the project. The direct verification is also capable of naming the fields that do not match, offering 64

81 5 Results additional information from the domain field. This information would be harder to obtain in a manual verification T12 - Identify fields that show equivalent data differently in each system There are fields that show equivalent data in a different format in HistViewer and the IBM 3270 terminal emulator. These differences are on purpose and due to design decisions in the development of HistViewer. Knowing these fields and giving the verification tool the means to handle them is the target of this task. For example, the field called PES (which represents the ID of the user that logged in the system) will have a value in the IBM 3270 terminal emulator but will be left blank in HistViewer for design decisions. The names of the fields, dialogs, coordinates and specific patterns were identified T13 - Include special comparison patterns The verification tool has to apply different comparison patterns depending on the field to verify to improve the verification process. These comparison patters are applied in two ways: Reusing code from the model layer of HistViewer s content extractor developed in HistAuskunft (outside this TFG). Considering the expected position of the fields on the screen (implemented in this task). Those two ways must give the same result in the verification for the source and target outputs to match. Otherwise, the model layers of the reused HistAuskunft module and this verification tool have to be checked. This could help finding issues with HistViewer s model layer End of iteration 3 After this iteration, the executable version satisfies subgoals 1 to 5. There are already many issues that have been found and reported after the execution of the verification tool in the testing environment. Some of the verification results obtained in this iteration can be seen in section 5.7 Case study. On the other hand, there are more tasks to close and subgoal 6 is yet to be satisfied. The development continues with iteration 4 as stated in the project plan. 65

82 Iteration Subgoal Task Effort PD 5 Results 5.4. Iteration 4: Output of results Iteration 4 is focused on the satisfaction of subgoal 6. For that reason, this iteration has the output-related tasks. According to the project plan, this iteration is meant to be the last one that has development tasks Identification of additional tasks This iteration does not add more tasks to the project plan but modifies the estimated effort of the previously identified tasks (T14, T15 and T16). This iteration s tasks can be seen in Table 5.4. The output of the results will be managed by a logging library already used in the HistAsukunft project. That logger replaces the prototype utility class used in the PoC. T14 is estimated to last one day more to learn how to use the library and configure it properly to build csv files and such. T15 is estimated to be one day less and T16 two days less because the architecture of the verification tool allows an easier-than-estimated output management. Description 4 SG6 T14 Output the input arguments that led to the compared screens 2 T15 Output if a dialog is exactly equal in both systems. 1 T16 Output which fields do not match in the comparison. 3 Table 5.4. Task list for iteration 4 in descending order of priority T14 - Output the input arguments that led to the compared screens This task includes learning and adapting the Log4j [Apache Software Foundation 2016] logging library. The configuration of the output lies in an xml configuration file. That configuration follows the standard format used in other HistAuskunft modules. Log4j also eases the generation of output in different streams such as the standard streams for general output and errors or files. Log4j will be used in this project to generate output files in csv and plain text formats with the results of the verification. Spreadsheet programs can open csv files and provide additional means to interact with those values. Spreadsheet programs offer utilities such as filtering and sorting values or plotting graphs to offer more visual results. This task concludes when the verification tool is able to output the input used to extract the compared screens. Knowing the input that led to specific results is very important so that the verification results are traceable and reproducible. 66

83 5 Results T15 Output whether a dialog is exactly equal in both systems The verification tool treats the results differently if the screens match or do not match. The log and the csv files are populated accordingly. This task contains the output of the high level comparison s results. That high level comparison is the one were only the comparison patters were applied and only a match or no match output is given. For performance s sake, the high level comparison is only complemented with a lower level comparison (field by field) only when the screens do not match in the high level comparison T16 Output which fields do not match in the comparison This output contains the results of a low level comparison (or field by field comparison) which is applied when the high level comparison determines that the screens do not match. This output reflects the names of the fields that do not match for the compared screens in both plain text and csv files complemented with the input arguments and a match or no match statement (c.f. Figure 5.10). The coordinates of the fields are also displayed for easier location on the screens. Figure Csv verification sample. As this output can be processed by spreadsheet programs, further analysis is possible. To name a few, it is possible to (i) find trends (fields that do not match often), (ii) apply filters and display only specific screens or omit others, (iii) quantify the impact of deviations for nonmatching screens End of iteration 4 The project plan is reviewed at the end of iteration 4 to ensure that all development tasks have been closed. In iteration 4 all subgoals and the project goal have been satisfied. The next step is the validation of results Final iteration: Validation of results This iteration contains the validation of results to ensure the quality of the product obtained 67

84 5 Results after the previous iterations. The product delivery takes place when the results are validated T17 Validation of results The validation of results starts with the positive analysis of the goal achievement which is further explained in section 6.1 Goal achievement analysis. This validation continues analysing the differences found between HistViewer and the IBM 3270 terminal emulator. The most relevant differences are explained in section 5.7 Case study. The following step is analysing the obtained code. As context, Table 5.5 presents the size of Avemaress (this TFG) compared to the whole HistAuskunft project. Avemaress is also counted towards the total size of HistAuskunft as Avemaress is within HistAuskunft. Furthermore, the number of code duplicates in Avemaress is minimal. The quality assurance tools reported that there were no clone units. LoC Statements Modules HistAuskunft Avemaress % 6.59% 7.61% 11.41% Table 5.5. Size of Avemaress compared to HistAuskunft's. The code compiles with no errors or warnings and the Findbugs tool [University of Maryland 2016] does not find any potential bug in the code. PMD [PMD 2016] analysis has also been considered during the development of the verification tool and in this validation task fixing al blocker or critical violations. Figure 5.11 shows the package view of the project with the dependency and containment relationships between the packages. 68

85 5 Results Figure Package view of the verification tool. The validation ends with the conclusions obtained after the development of this project that are in section 6.2 Project conclusions and the analysis of proposals and future lines of section 6.3 Proposals and future work Challenges This chapter contains some of the challenges found during the development of the verification tool. These challenges are grouped in interaction differences between the source and target systems, the navigation through the NatLeben environment and the other verification challenges Interaction differences between HistViewer and the IBM 3270 emulator The commands used to interact with the IBM 3270 terminal emulator and HistViewer are different (e.g.: ALT-E-C is used to copy text in the English configuration for the IBM 3270 terminal emulator, its equivalent in HistViewer is CTRL-C). The most notable commands and their correspondence in both systems are shown in Table 5.6. Furthermore, the number of input fields and some of their locations is not the same in HistViewer and the IBM 3270 terminal emulator. For instance, all dialogs in the IBM 3270 terminal emulator have 2 extra input fields that do not exist in HistViewer used for authentication. If the verification tool inputs the 69

86 5 Results incorrect text in those fields, the IBM 3270 terminal emulator would not show any content at all. The verification tool has to be aware of which of the two systems it is using for navigation to input the appropriate keyboard shortcuts or write the input arguments in the correct fields. As seen in Table 5.6, language (English or German) affects the behaviour of some commands so the verification tool also has to be aware of the language. There is also a limitation of the AWTRobot that affects the creation of input events. For keys that appear twice in the keyboard such as CTRL (left CTRL and right CTRL), the AWTRobot would use the left one (e.g.: left CTRL) [Oracle 2016c]. Those different positions are relevant for the IBM 3270 terminal emulator. Since the AWTRobot can only input the left key, some keys of the terminal emulator had to be remapped for the navigation to be possible. Command / System HistViewer IBM 3270 (English) IBM 3270 (German) Select All CTRL, A ALT, E, A ALT, B, S Copy CTRL, C ALT, E, C ALT, B, K Select and copy all CTRL, A, C Select All + Copy Select All + Copy Go to first input field PAGEUP, TAB HOME, TAB, TAB HOME, TAB, TAB Go to first cell PAGEUP, HOME HOME HOME Go to the next page in some dialogs F8 PA1 2 PA1 Table 5.6. Main command differences between HistViewer and the IBM Navigation through dialogs in NatLeben After opening a GIMS session, a completely blank screen is shown. Pressing F1 displays help in German about the screen or field selected. That help says that the transaction s codes begin with a capitalized letter and have two figures (e.g.: L00 or S01 ). Those transaction codes lead to different dialogs of different environments. The dialogs for the NatLeben system start with the letter L and have two figures from 00 to 99 depending on the dialog. The list with all possible dialogs for NatLeben and their names is shown in Figure This information comes from asking the mainframe to list the dialogs. 2 Particular key of the IBM 3270 and other terminals. 70

87 5 Results Figure NatLeben dialogs as listed by the mainframe. Figure 5.13 shows the screen for one dialog with transaction code L02. The L02 is a dialog that is available for all contracts and its output is used to navigate through other dialogs. This dialog lists part of the information needed to locate other dialogs of a given contract. Figure 5.13 (1): the current location with the input values that led to this screen. Their meanings are listed in Table 5.7 In this dialog, TC is L02 associated as seen in Figure 5.12 to Allgemeine Vertragsdaten (which means General Contract Data in German). AS L901 which is the value for current contracts. Suchbegriff L represents the contract identifier or entity id (GE-BEZ-NR and GE-BEZ). All these fields contain static data (data that is not editable). Figure 5.13 (2): PES and SW (which are respectively user name and password) are fields used for additional authentication purposes. The system does not display information until correct authentication is performed. The input in the SW field is hidden. These fields are disabled in HistViewer. Figure 5.13 (3): input fields used to navigate through the system (c.f. Table 5.7). TLB- NR and STAND-PER are not used in NatLeben. 71

88 5 Results Figure 5.13 (4): dialog data. The names, locations and purposes of the fields in this area are different for each dialog. Figure 5.13 (5): special field of the L02 dialog named IG. Some available dialogs are listed here. For this contract, the field has the value IG so this contract has the dialogs L01, L02, L04 and L06. Not all available dialogs are listed in this field. Figure 5.13 (6) LETZTE SEITE literally means last page in German. This value is shown in the last page of some dialogs. Figure Sample L02 dialog. Field name Complete name Description TC Transaktion The TC is a code that represents a dialog. Each system and contract has different dialogs available. All different TCs for NatLeben are listed in Figure

89 5 Results AS GE-BEZ-NR GE-BEZ ERH ZUO IG Arbeitsschlüssel Vertrags- oder Versicherungsschein nummer mit Angabe des Quellsystems Erhöhungssatz aus Dynamisierung Zugeordnetes Objekt This field can have two different values: L901: Current contracts. Last contract for that entity id. L902: Historical contracts. Contracts older than the current contract of that entity id. There can be a variable number of them (pages). The entity id (contract number) is a ten-figure number (GE-BEZ-NR) and a letter (GE-BEZ). It identifies a contract. ERH values are two-figure numbers. ERH values are used to access the different pages of dialogs L04, L05, L06, L07, L08, L17 and L42. ERH values can be found in the L13 dialog for that contract. ZUO values are three figure numbers except for L01 dialogs. L01 dialogs use one capitalized letter and a two-figure number, e.g.: B01. ZUO values used to access the different pages of dialogs L01, L05, L06, L07, L08 and L42. Each dialog has its own ZUO values. If the ZUO field is left blank the IBM 3270 automatically fills the ZUO value for AS L901, values can be collected then from the bottom part of the page. This field is contained by the L02 dialog. It lists if dialogs L01, L02, L03, L04, L05, L06, L07, L08 and L42 are accessible for that contract. Table 5.7. Most relevant input fields for navigation. The availability of some dialogs depends on the content of other dialogs. For that reason, the verification tool has to make navigation decisions dynamically at runtime. Notable rules for the navigation through dialogs in the NatLeben environment are shown in Figure

90 5 Results Figure Notable rules for the navigation in the NatLeben environment 74

91 5 Results Other verification challenges This section presents general challenges faced during the development of the verification tool. Those general challenges are: There are different response times for both HistViewer and the IBM 3270 terminal emulator due to not only their different data sources (DB2 and IMS) but also the network status. The access to the mainframe has variable network latency. This is one of the most relevant challenges as the verification tool does not have any direct way to know if the output contains the results of the input. To solve that, the verification tool has to know at every moment which screen and system it is comparing. Any synchronization issue would lead to unexpected screens and tarnish the value of all the verification results from then onwards. The solution designed for this challenge was to process the obtained output to identify whether it was the desired output and repeat the process if it was not. This also forced the verification tool to identify the dialog output in both systems and consider if they are comparable or not before performing the comparison. With faster verification, HistViewer or the IBM 3270 terminal might become unresponsive. This challenge can affect the quality of the results but also allows the verification tool to find potential memory leaks or performance issues. In fact, from facing this challenge, the verification tool found that specific dialogs were taking too long to load and encouraged the redesign of the database queries used in HistViewer. Neither the IBM 3270 terminal emulator nor HistViewer offer any convenient API that could ease the interaction. The verification tool only has access to the user interface like a normal user. As a consequence, the verification tool only has the screen output and the input fields. Those two elements were not intended to be used by a robot so they do not offer explicit output to the robot. Screen scraping is a completely ad-hoc technique. This is against verifying different environments and systems. The project had to find a common ground between the complexity of multiple abstraction layers and the available resources. This is further complicated because most of the other environments to verify are still unknown. The location of the window s elements and the windows themselves are relevant but the verification should not be limited by specific screen resolutions or window size configurations. This was solved moving through the systems using the keyboard and 75

92 5 Results cursor while defining a source and a target system. Each system would take half of the screen each and the robot would identify them depending on the screen resolution of the verification environment. The number and location of the input fields is different for each dialog. The verification tool needs that knowledge to enter the input arguments in the correct fields. This was fixed implementing the results of the analysis of the SUT. The remote connection to the mainframe is only available approximately 54 hours a week. The verification tool takes control of the mouse and keyboard and also needs the windows of the systems to compare on display. Hence, the development cannot continue in the same computer the verification is taking place. Non-availability is indicated by the message "Kann keine virtuelle Sitzung fuer Sitzung "GIMS1" aufbauen." Windows 8.1 introduces limitations on the generation of input events by applications. Input events that are to be handled by the shell need special privileges such as UIAccess. An example is the ALT + TAB input event to switch windows. The verification should be able to run in the testing environment provided by itestra GmbH s customer and that customer s remote servers. This means that the verification is not allowed special privileges. The verification tool gets user level privileges at most. HistViewer and the IBM 3270 terminal emulator show information differently. Those differences are due to design decisions since HistViewer is meant to be a replacement and an improvement of the previous system. The verification system uses different comparison patterns and knows which to apply in each situation according to those special values. These comparison patterns answer to cases such as the PES field that shows as a six-figure number in the IBM 3270 terminal emulator but is left blank in HistViewer on purpose. Another example is how the entity id in the L12 dialog for the mainframe s system is structured completely different from the one used in HistViewer as a design decision. Verification goals can be variable since the verification is being performed on a system that is currently in development. The verification focus can vary depending on the verification needs during the development of the SUT. 76

93 5 Results 5.7. Case study This section provides results of an empirical study that was made during this project. The goal of this study was collecting as much information as possible about the usage of the verification tool and ensure in this way its feasibility and applicability within the HistAuskunft project. This study consisted in the application of the verification tool (following an iterative process) to the NatLeben system with some random contracts. In this empirical validation, 1270 contracts were tested for 1435 screens in total using different verification modes. An 84 % of the screens had at least one issue. As a result of this study, valuable feedback was collected with a twofold purpose: (i) improve the verification tool, but also (ii) produce a bug-free version of the content extractor in the HistAuskunft. Issues detected in the verification tool are explained in the followings paragraphs with the last decision made to solve the issues. Those issues have been found before the HistAuskunft project went to the production phase which proved to be another benefit of this tools development. The issues represent mistakes in the extraction of content that would imply fixing the defect and the repetition of an extraction process estimated to last up to a whole year. The replication details for those issues have not been included in this document as they are only relevant for HistAuskunft s team. The following list contains those found issues and the decisions made to solve them: I1 There are three fields that have not been extracted in some L02 dialogs. This was fixed with high priority and the extraction process got restarted. I2 Database queries in dialogs whose ERH value (used for navigation) is left blank are too slow which results in a bad user experience. The database queries have been notably optimized. I3 The SUCH-NR field in L12 is editable in the IBM 3270 terminal emulator, it is not in HistViewer. It is not needed to replicate this behaviour. I4 The IBM 3270 terminal emulator freezes when asked for a L09, L64 or L19 dialog. The COBOL code for dialog L19 treatment is missing in the mainframe. 77

94 5 Results I5 The IBM 3270 terminal emulator takes you to the L04 dialog when asked for the L74. The customer says it is not relevant for NatLeben. I6 Going to a dialog that does not exist for a given contract from one that exists, the screen keeps the values from the other contract. The values shown for the non-existing contract would be the ones from the previous contract. The information is now cleared and a dialog does not exist error message is shown to the user. I7 L03 dialog has some characters in special colours in the IBM 3270 terminal emulator, it does not in HistViewer. Ask the customer if it is relevant to replicate this behaviour. I8 When L07 is missing for that ZUO, the IBM 3270 terminal emulator says " ** IG 07 RUECKVERSICHERUNGSTEIL HAUPT-VERSICHERUNG **", HistViewer only shows half of it as the other part varies and does not get calculated. There are no problems when the L07 dialog exists for that ZUO value. The customer states that this behaviour does not need to be reproduced in HistViewer. I9 HistViewer does not show some of the general contract details when the ZUO does not exist, the IBM 3270 terminal emulator does. The client states that this behaviour has to be reproduced in HistViewer. I10 In the IBM 3270 terminal emulator most of the fields in dialogs L05 allow input, HistViewer s do not. Ask the customer if it is relevant to replicate this behaviour. I11 For the L06 dialog, when you input an incorrect value of ZUO the mainframe replaces it with blanks. (e.g.: if only ZUO 000 exists for that dialog entering ZUO 013 shows you the same dialog with ZUO 000) this does not happen in HistViewer. Ask the customer if it is relevant to replicate this behaviour. I12 All dialogs show navigation details in the first two lines in the IBM 3270 terminal emulator, they do not in HistViewer. 78

95 5 Results Ask the customer if it is relevant to replicate this behaviour. I13 In the IBM 3270 terminal emulator some dialogs do not exist for AS L901 but do for AS L902. This comes in conflict with the specification as the L901 is the last L902. Also, these contracts are only partially extracted in HistViewer. These dialogs must not be extracted as they should not be available to the users. Any of these dialogs that have been extracted must be deleted. I14 Performing queries in the segments database a lot of values that are not shown by the IBM 3270 terminal emulator were found. Should they be included in HistViewer? Ask the customer if it is relevant to replicate this behaviour. I15 There are not L01 screens with more than one historical page (AS = L902) for each ZUO value. However, in the segments database some of them have up to 15 historical pages. Ask the customer if it is relevant to replicate this behaviour. I16 In the IBM 3270 terminal, visiting dialogs entering blank ZUO values is possible and required to know which ZUO values are available. In HistViewer it is not possible. It is now possible in HistViewer. I17 The g1vp field in the L02 dialog sometimes has * as value instead of a number as expected. That error implies that HistViewer cannot show that dialog at all. Fixed, HistViewer can show those dialogs. I18 Some contracts that were thought to have already been extracted and stored in HistViewer's persistence were not. Trace this issue with the content extractor. I19 In HistViewer there is an unknown field that sometimes shows five 0 in the third line. That field is not present in the IBM 3270 terminal. Trace this issue with HistViewer. I20 The ERH value *0 located in the segments database for segment never appears in the IBM 3270 terminal emulator. 79

96 5 Results That data comes from a corrupt segment of the IMS database and should not be extracted. 80

97 6 Conclusions and proposals 6. CONCLUSIONS AND PROPOSALS This chapter begins presenting a goal achievement analysis. The chapter continues with the conclusions after the closing the project. Proposals for improvements and future work are next. The chapter ends with a personal opinion about this project Goal achievement analysis The goal of this project is, as shown in chapter 2 Goals, to design and develop an automatic tool for the verification of whether the information shown by an IBM 3270 terminal connected to a mainframe is equivalent to the information shown in an application replicated using JavaFX. The project goal was divided in subgoals. The satisfaction of those subgoals would grant the achievement of the project goal. The subgoals were satisfied with the closure of the tasks identified during the project plan and its iterative evolution. The satisfied subgoals can be seen in Table 6.1. The satisfaction of the project goal is also reinforced by the evidence of the results. This project s verification tool already found defects in the replacement of the mainframe system. Those defects are exposed in section 5.7 Case study. SG id SG 1 SG2 SG 3 SG 4 Justification of the achievement The consecution of tasks T1 to T6 satisfied SG1. Those tasks brought the comprehension of the data structure of the environment to verify and how to interact with it. SG1 and tasks T7, T8 and T9 satisfied SG2. This way the base architecture was stablished allowing the automatic extraction of the information contained in a screen in HistViewer and the IBM 3270 terminal. This information could be consulted, extracted and processed. The results of T10 complementing SG2 allowed adapting and replicating the extraction and comparison process of a screen to the screens of all contracts specified. This subgoal was satisfied by T11 and SG3. The design and implementation of the mechanism to identify the available screens for each contract allowed the 81

98 6 Conclusions and proposals satisfaction of this subgoal. The available screens were recognized depending on the information contained in other screens used by real users for the navigation through the system. This subgoal was also satisfied by the design and implementation of the logic to allow the navigation through the screens depending on the information extracted from others. SG 5 SG 6 T12 identified the equivalent data to manage and T13 incorpored the apropiate patterns to the architecture built in tasks T8 and the preceeding ones. This satisfied SG5 creating and incorporating comparison patterns for the comparison of equivalent data shown differently in HistViewer and the IBM 3270 terminal to the verification tool. Finally, satisfying the subgoals before SG6 and completing tasks T14, T15 and T16 achieved the satisfaction of SG6. Those subgoals and tasks allowed the design and development of a module for the persistence of the verification results. Those results are stored with enough information to be reproducible allowing the obtainment of valuable conclusions such as the quantification of the deviation for screens that do not match. Table 6.1. Subgoal achievement analysis Project conclusions As main conclusion, the screen scraping technique has been used in the replacement of a legacy mainframe system successfully with a high level of optimization making the verification of the migration context viable. This tool contributes to the modernization or retirement of LIS process extracting the information contained in terminals connected to mainframes. Modernization or retirement of SIL is a need justified even more by the increasing cost of maintenance of those LIS as years go by. This project adds real value to the HistAuskunft project. As shown in previous chapters, only the extraction of contracts in HistAuskunft is estimated in the order of a year. In case there is a defect in the extraction algorithm, errors could propagate to all the extracted screens. Detecting any defect in the extraction implies its mandatory fix and forces the extraction to be restarted from the beginning. Tolerating no defect at all is justified by legal arguments in the retirement of the mainframe system. The contained information is related to insurance contracts that must be kept during several decades [Bundesministerium der Finanzen 2014]. For that 82

99 6 Conclusions and proposals reason, differences in the information contained in HistViewer and the mainframe system imply legal offences related with data protection laws. Finding defects in the extraction has provided a saving in time and money in the HistAuskunft project. The persistence of defects would imply consequences such as repeating an extraction process that can last in the order of one year or legal because of the offence of data protection laws. As a notable example, this verification tool found that three fields for the L02 dialog were not being extracted properly. This is a very important example as the L02 is one of the few dialogs that exist for all contracts without exception. Any error in this dialog affects all the contracts of itestra GmbH s customer. What is more, the extraction of the L02 dialog was thought to be correct and was close to reach the production phase after several months executing with testing data. As a side note, the defects found in the extraction of screens in HistAuskunft is much less expensive to solve in the development phase with testing data than in the production phase with real data. This is reasoned because if a defect is found in the production phase, the extraction has to be restarted after fixing the defect. Section 5.7 Case study includes additional examples of defects found in the replacement of a legacy system. Nevertheless, some differences in content and behaviour might be still hidden and HistAuskunft is still in development. Furthermore, a complete verification would only be possible with unlimited time and resources. For this reason and quoting Dijkstra, testing shows the presence, not the absence of bugs. Despite of that, this tool contributes to increase the trust on HistViewer. The verification offers another point of view to evaluate the suitability of HistViewer to replace the legacy mainframe system This TFG also proves the acquisition of competences related with software engineering. Requirements proposed in HistAuskunft had to be satisfied considering a reliable and efficient behaviour. This is related to the satisfaction of quality measures, methods and practises of the software engineering stablished by itestra GmbH. That satisfaction allows the project to be maintainable and reused for the rest of environments for which the application of this project is foreseen. Techniques of reverse engineering were applied on LIS and code from other modules in HistAuskunft was reused as seen in chapter 5 Results. Besides, there is a well-defined client. In this case, itestra GmbH and the insurance company that is the customer in the HistAuskunft project. This way, the needs to evaluate and satisfy are defined with real cost and time restrictions. These restrictions are materialized in the need to converge with delivery deadlines with the customer (the insurance customer), the 83

100 6 Conclusions and proposals required functionality and the adaptation to existing systems that were previously developed. These compromises imply solutions using the strategies, standards and technologies available in itestra GmbH. Finally, the evolution of this verification tool has been closely related with practises that encourage flexibility and maintainability. These elements are frequent code reviews to ensure that the development is within industrial quality standards or the use of software design patterns (e.g.: some of the ones exposed in [Gamma et al. 1994]). The application of patterns such as Flyweight, Adapter, Factory or Abstract Method eases the foreseen adaptation of this tool to other verification environments. The use of patterns also eases the corrective and perfective maintenance of this verification tool according to the future lines proposed in section 6.3 Proposals and future work Proposals and future work Different proposals and adaptation measures have been identified during the development of this project. Some are sure to be applied soon, like the need of verifying other mainframe environments apart from NatLeben. Others are presented in this section as future improvements that would add value to the verification tool Future work The project started in this TFG is sure to continue. Since this project is within an industrial context with real needs towards a client, the obtainment of results must continue during the several years HistAuskunft will last. The verification tool developed in this project was used for the verification of the NatLeben environment. Nevertheless, there are at least seven additional environments to verify. The extraction of content for the second environment got started by the end of this TFG. The Leistung environment is different to NatLeben s and implies that the verification tool has to be adapted to that new environment. As a difference to NatLeben, Leistung contains a completely different dialog structure and uses different commands. As an example, the verification tool will have to be able to navigate to other screens depending on the pointer s location. Besides, the use of this tool is also considered for the extraction of content in other environments apart from NatLeben or Leistung. This tool could be adapted to other cases were 84

101 6 Conclusions and proposals the extraction of content is needed and there is no API that allows the extraction. This way, the extraction of content can be used with other purposes aside from the context of verification. These purposes include, but are not limited to, the processing of information for statistics, migrations or even the creation of robots that interact with an interface depending on the output of said interface Future improvements This section presents a couple of future improvements that could add value to the verification process. These future improvements can increase the functionality of the verification tool. Considering that the verification project is to live longer than this TFG to verify more environments. The verification executions should be more configurable and more easily. These executions could be launched from a graphic interface that allows different configurations easily. This module should also offer saving and loading those configurations. The verification emulates a user interacting with the systems and takes full control of the computer. This proposal is to offer the comparison results in real time as a web service. This web service allows consulting the results remotely during the verification. This is interesting considering that the verification could last several days. Showing general information about the progress in real time remotely without stopping the verification would add value to the verification tool Personal opinion This project is notably different to all others I have worked in, not only because of the considerable difference in size but the environment. In this project, I have worked with many technologies I had never seen during my degree. For instance, I could work with the hierarchical data structure IBM IMS and terminals connected to mainframes using real information systems. All that in an industrial environment that complements with a different point of view the learning presented in teaching laboratories. Working in this industrial environment had an impact in the learning and use of different technologies, but also in the learning of processes and methods. Within the FORTE I have experienced how things are done in a real company. Non-technical and communicational skills have also been relevant. This is justified because the communication with the other members of 85

102 6 Conclusions and proposals the HistAuskunft project has been vital for the achievement of goals in this TFG. Even more so, I have been able to take part in processes different to those seen during my academic studies like the archaeology of legacy systems or the international synchronization of tasks in the same project. Finally, I conclude this personal opinion asserting that this experience has been very enriching and I have been able to learn and mature within the software engineering. 86

103 6 Conclusiones y propuestas 6. CONCLUSIONES Y PROPUESTAS Este capítulo comienza presentando un análisis de consecución de objetivos a la finalización del proyecto. Se continuará discutiendo las conclusiones obtenidas del proyecto. Después se estudiarán propuestas de mejora y posibles líneas de trabajo futuro. El capítulo concluye con una opinión personal sobre el proyecto Análisis de consecución de objetivos El objetivo principal de este proyecto es diseñar y desarrollar una herramienta para la verificación automática de la equivalencia de información mostrada en un emulador de terminal IBM 3270 conectado a un mainframe y la mostrada en una aplicación recreada en JavaFX. El objetivo se dividió en objetivos parciales (subobjetivos) que deben satisfacerse para cumplir el objetivo principal del proyecto. Estos subobjetivos fueron satisfechos con la consecución de las tareas identificadas durante el plan de proyecto y la evolución iterativa del mismo. Los subobjetivos cumplidos y la justificación pueden verse en la Tabla 6.1. Otra evidencia que refuerza la consecución del objetivo principal del proyecto es la detección de incidencias en el reemplazo del sistema mainframe. Estas incidencias se encuentran en la sección 5.7 Case study. SG id SG 1 SG2 SG 3 SG 4 Justificación de su consecución La consecución de las tareas T1 hasta T6 satisfizo SG1. En dichas tareas se consiguió comprender la estructura de datos del entorno a verificar y como interactuar con ella. SG1 y las tareas T7, T8 y T9 satisficieron SG2. De esta forma se pudo establecer la base de la arquitectura del sistema permitiendo la obtención automática de la información contenida en una pantalla en HistViewer y el terminal IBM Esta información pudo ser consultada, extraída y procesada. Los resultados de T10 complementando a SG2 permitieron escalar y replicar el proceso de extracción y comparación de una sola pantalla a las pantallas de los contratos que se indicasen. Este subobjetivo se alcanzó a partir de T11 y SG3. De esta forma, se diseñó e 87

104 6 Conclusiones y propuestas implementó el mecanismo para reconocer las pantallas accesibles para cada contrato a partir de las salidas de pantallas que los usuarios utilizarían para la navegación. Este subobjetivo quedó satisfecho diseñando e implementado la lógica para que la herramienta de verificación navegase a través de las pantallas en función de la información contenida en otras. SG 5 SG 6 En T12 se identificaron los datos equivalentes a gestionar y en T13 se incorporaron los patrones de comparación adecuados a la arquitectura construida en las tareas T8 y sus predecesoras. Así se satisfizo SG5 creando e incorporando a la herramienta de verificación los patrones de comparación para datos equivalentes mostrados de forma diferente en HistViewer y el terminal IBM Finalmente, satisfacer los subobjetivos anteriores a SG6 y completar las tareas T14, T15 y T16 consiguieron satisfacer SG6. De este modo se consiguió diseñar y desarrollar un módulo que permite la persistencia de los resultados de la verificación. Estos resultados se guardan con información suficiente como para ser reproducibles. Los resultados son procesables pudiendo obtener conclusiones con valor como una cuantificación del grado de desviación para las pantallas que no coinciden. Tabla 6.1. Análisis de consecución de subobjetivos Conclusiones del proyecto Como conclusión principal, se ha trabajado en la técnica de screen scraping enfocada a la verificación del reemplazo de un sistema mainframe legado con un alto nivel de optimización de forma exitosa, haciendo que la verificación del contexto de migración expuesto sea realmente viable. Esta herramienta contribuye en los procesos de modernización o retirada de SIL extrayendo la información contenida en terminales conectados a mainframes. No se debe perder de vista que modernizar o retirar SIL es una necesidad cada vez mayor justificada por el creciente coste de mantenimiento de dichos SIL con el paso de los años. Este proyecto aporta un valor real al proyecto HistAuskunft. Como se indicó en capítulos anteriores, solo la ejecución de la extracción de contratos en HistAuskunft está estimada en el orden de un año. En caso de haber un defecto en el algoritmo de extracción, los errores se podrían extender a todas las pantallas extraídas. Localizar un defecto tras la extracción implica su obligatoria resolución y ejecutar la extracción desde el principio. Esta tolerancia cero ante los defectos está justificada por los motivos legales tras la retirada del mainframe. La 88

105 6 Conclusiones y propuestas información contenida corresponde a contratos de seguros, los cuales deben ser conservados por ley durante un cierto número de décadas [Bundesministerium der Finanzen 2014]. Por tanto, la disonancia entre la información contenida en HistViewer y el sistema mainframe implicaría problemas legales relacionados con la protección de datos. El descubrimiento de defectos en la extracción por parte de esta herramienta de verificación ha supuesto un ahorro de tiempo y dinero en el proyecto HistAuskunft. Si los defectos hubieran persistido, habrían tenido consecuencias desde tener que repetir un proceso de extracción de un año de duración hasta legales, por incumplimiento de leyes de protección de datos. Como ejemplo notable, esta herramienta de verificación encontró que tres campos de información no estaban siendo extraídos correctamente para el diálogo L02. Este ejemplo es muy importante porque el L02 es uno de los pocos diálogos que existen para todos los contratos sin excepción con lo que un error en este diálogo repercute en todos los contratos de la empresa cliente. Además, se consideraba que la extracción del diálogo L02 era correcta y estaba a punto de pasar a fase de producción tras llevar varios meses ejecutándose con datos de prueba. Cabe destacar que los defectos detectados en la extracción de pantallas de HistAuskunft son mucho menos costosos de solucionar en la fase de desarrollo con datos de prueba que en la de producción con datos reales. Esto se debe a que si en la fase de producción se detecta un defecto, se debe comenzar la extracción desde el principio tras solucionar ese defecto. La sección 5.7 Case study incluye ejemplos adicionales de defectos encontrados en el reemplazo del sistema legado. Sin embargo, no se debe ignorar que algunas diferencias de contenido y comportamiento pueden haber permanecido ocultas. No obstante, HistAuskunft aún está en desarrollo. Además, una verificación completa solo sería posible con una cantidad de tiempo y recursos ilimitada. Por este motivo, citando a Dijkstra, el testing muestra la presencia, no la ausencia de fallos. Aun así esta herramienta contribuye a incrementar la confianza en HistViewer tras la verificación. Ofreciendo así un modo más objetivo para evaluar la precisión con la que HistViewer representa al sistema mainframe que pretende reemplazar. Este TFG también demuestra la adquisición de competencias relacionadas con el itinerario de la Ingeniería del Software. En este proyecto se debieron satisfacer los requisitos planteados en HistAuskunft teniendo además un comportamiento fiable y eficiente. Esto queda unido a su vez con el cumplimiento de las normas de calidad y métodos y prácticas de la Ingeniería del Software establecidos por itestra GmbH. Este cumplimiento consigue que el proyecto pueda ser mantenible y reutilizado en el resto de entornos para los que se prevé la aplicación de este 89

106 6 Conclusiones y propuestas proyecto. Siguiendo en la línea de la Ingeniería del Software, también se aplicaron técnicas de ingeniería inversa sobre SIL y reutilizamiento de código de otros módulos de HistAuskunft tal y como se indica en el capítulo 5 Results. Por otro lado, existe un cliente bien diferenciado, en este caso itestra GmbH y la aseguradora cliente del proyecto HistAuskunft. De esta forma, las necesidades a valorar y satisfacer están definidas con restricciones de coste y tiempo reales. Estas restricciones se materializan en la necesidad de converger con plazos de entrega al cliente (la aseguradora), funcionalidad requerida y adaptación a sistemas existentes desarrollados con anterioridad. Estos compromisos implican soluciones a los problemas planteados haciendo uso de estrategias, estándares y tecnologías disponibles en itestra GmbH. Por último, la evolución de esta herramienta de verificación ha estado estrechamente relacionada con prácticas que propician la flexibilidad y el mantenimiento. Estos elementos toman la forma de revisiones de código frecuentes para asegurar que el desarrollo se encuentra dentro de estándares de calidad industriales o el uso de patrones de diseño de software (p.ej.: algunos de los expuestos en [Gamma et al. 1994]). Mediante la aplicación de patrones como Flyweight, Adapter, Factory o Abstract method se ha buscado facilitar la prevista adaptación de esta aplicación a otros entornos de verificación. El uso de patrones también facilitará el mantenimiento correctivo y evolutivo de la herramienta de verificación de acuerdo a las líneas futuras propuestas en la sección 6.3 Propuestas y trabajo futuro Propuestas y trabajo futuro Durante el desarrollo de este proyecto se han identificado ciertas posibilidades de mejora y adaptación a nuevos entornos. Algunas se conocen con certeza como la necesidad de verificar otros entornos mainframe además de NatLeben. Otras se presentan en esta sección como mejoras futuras que aportarían un valor añadido a la herramienta de verificación Líneas futuras Existe certeza sobre la continuidad del proyecto comenzado en este TFG. Al encontrarse en un contexto industrial con necesidades reales hacia un cliente será necesario continuar obteniendo resultados en los varios años de duración del proyecto HistAuskunft. La herramienta desarrollada en este proyecto fue utilizada para la verificación del contenido del entorno NatLeben. No obstante, existen al menos siete entornos más a verificar dentro del contexto de HistAuskunft. 90

107 6 Conclusiones y propuestas La extracción del segundo entorno (Leistung), empezó a fecha del final de este TFG. El entorno Leistung es diferente a NatLeben y por tanto implica que la herramienta de verificación deberá adaptarse a ese nuevo entorno. A diferencia de NatLeben, Leistung no solo contiene una estructura de diálogos completamente diferente si no que utiliza comandos distintos a NatLeben. Por ejemplo, la herramienta de verificación deberá poder navegar a otras pantallas dependiendo de la posición del cursor. Por otro lado, también se considera el uso de esta herramienta para la extracción de información no solo de entornos mainframe como NatLeben o Leistung. Esta herramienta podría adaptarse a otros casos donde se requiera extraer información y no se disponga de una API conveniente que lo permita. De esta forma, también sería posible utilizar la extracción de contenido para otros propósitos no solo en el marco de la verificación. Estos propósitos incluyen, pero no se limitan a, el procesado de información para elaboración de estadísticas, extracción de información para realizar migraciones o incluso creación de robots que deban responder a una interfaz interpretando las salidas de dicha interfaz Mejoras futuras A continuación se presenta una serie de mejoras futuras que perfeccionarían el proceso de verificación. Estas mejoras aportarían un valor añadido al verificador incrementando su funcionalidad: Al preverse la verificación de más entornos, se considera la necesidad de configurar más fácilmente las ejecuciones de verificación. Esas ejecuciones podrían lanzarse desde una interfaz gráfica que permita seleccionar fácilmente las configuraciones. Este módulo también debería facilitar guardar y cargar tales configuraciones. La verificación simula a un usuario interactuando con los sistemas y por lo tanto secuestra ratón y teclado. Se propone ofrecer los resultados de comparación en tiempo real como servicio web. Ese servicio web permitiría consultar los resultados de forma remota durante la ejecución de la verificación. Este punto es clave teniendo en cuenta que se podría requerir la verificación de decenas de millones de pantallas extraídas. Mostrar información general del progreso en tiempo real remotamente y sin tener que detener la verificación sería esencial. 91

108 6 Conclusiones y propuestas 6.4. Valoración y opinión personal Este proyecto ha sido muy diferente a todos aquellos en los que he participado antes. No solo por la gran diferencia en tamaño, si no por el entorno en el que se sitúa. En este proyecto he trabajado con muchas tecnologías que no había visto durante el grado. Por ejemplo, he podido trabajar con la estructura de datos jerárquica de IBM IMS y con terminales conectados a mainframes operando sistemas de información reales. Todo ello en un entorno industrial real que complementa el aprendizaje presentado en los laboratorios docentes aportando un punto de vista diferente. Trabajar en este entorno industrial no solo ha influido en el aprendizaje y uso de tecnologías, también de procesos y métodos. Dentro del marco FORTE he podido experimentar de primera mano cómo se hacen las cosas en una empresa real, itestra GmbH. En el entorno donde se ha realizado el proyecto también han sido relevantes habilidades no técnicas y comunicativas. Esto se debe a que la comunicación con el resto de miembros del proyecto HistAuskunft ha sido vital para la consecución de los objetivos de este TFG. Además de eso, he podido formar parte de procesos muy diferentes a los vistos durante los estudios académicos como la arqueología de sistemas legados o la sincronización de tareas a nivel internacional para el desarrollo de un mismo proyecto. Por último, concluyo esta memoria afirmando que ha sido una experiencia muy enriquecedora de la que he podido aprender y madurar mucho dentro de la ingeniería del software. 92

109 7 Acronyms 7. ACRONYMS IDS: Information Display System. LIS: Legacy Information System. SIL: Sistema de Información Legado. PoC: Proof of Concept. TFG: Spanish acronym for Trabajo Fin de Grado. This work. SUT: System Under Test. GUI: Graphical User Interface. GDD: Goal Driven Development. ERH: Erhöhungssatz aus Dynamisierung. It is one of the input arguments used for navigating through screens in NatLeben. ZUO: Zugeordnetes Objekt. It is one of the input arguments used for navigating through screens in NatLeben. TC: Transaktion. It is one of the input arguments used for navigating through screens in NatLeben. AS: Arbeitsschlüssel. It is one of the input arguments used for navigating through screens in NatLeben. 93

110

111 8 References 8. REFERENCES P. Aho, M. Suarez, A. Memon, and T. Kanstrén Making GUI Testing Practical: Bridging the Gaps. In th International Conference on Information Technology - New Generations (ITNG) DOI: David Allen Getting Things Done: The Art of Stress-Free Productivity, Penguin. Scott Ambler The Principles of Lean Software Development. (2010). Retrieved May 11, 2016 from oftware_development?lang=en Apache Software Foundation Logger (Apache Log4j API). (2016). Retrieved June 1, 2016 from O. El Ariss, D. Xu, S. Dandey, B. Vender, P. McClean, and B. Slator A Systematic Capture and Replay Strategy for Testing Complex GUI Based Java Applications. In 2010 Seventh International Conference on Information Technology: New Generations (ITNG) DOI: Roy Baumeister and John Tierney Willpower: Rediscovering the Greatest Human Strength, Penguin. K. Bennett Legacy systems: coping with success. IEEE Softw. 12, 1 (January 1995), DOI: Jesús Bisbal, Deirdre Lawless, Bing Wu, and Jane Grimson Legacy information systems: Issues and directions. IEEE Softw. 16, 5 (1999), 103. Scott Bollig and Dan Xiao Throwing off the shackles of a legacy system. Computer 31, 6 (1998), Michael L. Brodie and Michael Stonebraker Legacy Information Systems Migration: Gateways, Interfaces, and the Incremental Approach, San Francisco, CA, USA: Morgan Kaufmann Publishers Inc. Bundesministerium der Finanzen Grundsätze zur ordnungsmäßigen Führung und Aufbewahrung von Büchern, Aufzeichnungen und Unterlagen in elektronischer Form sowie zum Datenzugriff (GoBD). (2014). Retrieved June 6, 2016 from ere_steuerthemen/abgabenordnung/datenzugriff_gdpdu/ GoBD.pdf;jsessionid=619FF42A8591A59204A1721F4BCEDAC8? blob=publication File&v=3 Paul Calder and Upasak Weston Building User Interfaces with Shared Objects. In 17th Australasian Computer Science Conference Flavio C. Buccianti IBM Systems Magazine - The ipad Meets the Mainframe. (2011). Retrieved April 26, 2016 from Check Point Software Technologies Ltd Industry-Leading Cyber Security Solutions for your Network, Data Center, Endpoints & Mobile Devices Check Point Software Technologies. (2016). Retrieved March 17, 2016 from Santiago Comella-Dorda, Kurt Wallnau, Robert C. Seacord, and John Robert A survey of black-box modernization approaches for information systems. In Software Maintenance, Proceedings. International Conference on. IEEE, Dell Software Inc IBM DB2 - Toad World. (2016). Retrieved March 17, 2016 from Eclipse Foundation Eclipse - The Eclipse Foundation open source community website. (2016). Retrieved March 17, 2016 from 95

112 8 References Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides Design Patterns: Elements of Reusable Object-Oriented Software, Pearson Education. GanttProject Team GanttProject: free desktop project management app. (2016). Retrieved May 31, 2016 from IBM IBM Archives: DPD chronology - page 4. (January 2003). Retrieved April 26, 2016 from IBM. 2016a. IBM - DB2 database software. (2016). Retrieved March 22, 2016 from IBM. 2016b. IBM Globalization - Code page identifiers - CP (2016). Retrieved April 28, 2016 from IBM. 2016c. IBM - IMS - Information Management System - database management system - Overview. (2016). Retrieved March 30, 2016 from 01.ibm.com/software/data/ims/ IBM IBM Knowledge Center. (January 2013). Retrieved April 26, 2016 from ork_261.htm IBM. 2016d. IBM - Personal Communications. (March 2016). Retrieved March 17, 2016 from IBM IBM z Systems - Operating System - z/os. (May 2011). Retrieved June 6, 2016 from ISO ISO/IEC 14764: Software Engineering -- Software Life Cycle Processes -- Maintenance. (2006). Retrieved April 25, 2016 from L. Jaw, W.T. Tsai, D. Homan, and K. Keller Verification of Flight Software with Karnough Map-based Checking. In 2007 IEEE Aerospace Conference DOI: David J. Kasik and Harry G. George Toward Automatic Generation of Novice User Test Scripts. (1996). Retrieved April 28, 2016 from Anthony Lauder and Stuart Kent Legacy system anti-patterns and a pattern-oriented migration response. In Systems Engineering for Business Process Change. Springer, Meir M. Lehman Programs, life cycles, and laws of software evolution. Proc. IEEE 68, 9 (1980), Li Ma et al Design and Implementation of Controlling PC Wirelessly by Android Mobile Based on C/S Mode. Int. J. Control Autom. 7, 7 (July 2014), DOI: Peter Mehlitz, Oksana Tkachuk, and Mateusz Ujma Jpf-awt: Model checking gui applications. In Proceedings of the th IEEE/ACM International Conference on Automated Software Engineering. IEEE Computer Society, A.M. Memon, M.E. Pollack, and M.L. Soffa Hierarchical GUI test case generation using automated planning. IEEE Trans. Softw. Eng. 27, 2 (February 2001), DOI: Atif M. Memon GUI testing: Pitfalls and process. Computer, 8 (2002), Microsoft. 2016a. Install the Windows 8.1 Update (KB ) - Windows Help. (2016). Retrieved March 21, 2016 from Microsoft. 2016b. Microsoft Word Document and Word Processing Software. (2016). Retrieved May 25, 2016 from ObjectAid LLC ObjectAid UML Explorer - Home. (2016). Retrieved May 26,

113 8 References from Taiichi Ohno Toyota Production System: Beyond Large-Scale Production, CRC Press. Oracle. 2016a. 1 JavaFX Overview (Release 8). (2016). Retrieved March 29, 2016 from Oracle. 2016b. Java Platform Standard Edition 8 Documentation. (2016). Retrieved March 17, 2016 from Oracle. 2016c. Robot (Java Platform SE 8 ). (2016). Retrieved March 17, 2016 from Oxford University Press proof of concept - definition of proof of concept in English from the Oxford dictionary. (2015). Retrieved May 5, 2016 from PMD PMD. (2016). Retrieved May 25, 2016 from Mary Poppendieck and Tom Poppendieck Lean Software Development: An Agile Toolkit, Addison-Wesley Professional. A. Rauf and M.N. Alanazi Using artificial intelligence to automatically test GUI. In th International Conference on Computer Science Education (ICCSE) DOI: Roy Rosenzweig Center for History and New Media Zotero Home. (2016). Retrieved March 17, 2016 from Alex Ruiz Test-driven GUI development with FEST. (July 2007). Retrieved April 26, 2016 from Frank Sauer Metrics (2016). Retrieved May 26, 2016 from Ingo Schnabel and Markus Pizka Goal-Driven Software Development. In IEEE, DOI: Harry M. Sneed Encapsulation of legacy software: A technique for reusing legacy software components. Ann. Softw. Eng. 9, 1-2 (May 2000), DOI: Software Freedom Conservancy Git. (2016). Retrieved March 17, 2016 from Ian Sommerville. 2008a. Increasing costs of legacy system maintenance. In Software Engineering. Ian Sommerville. 2008b. Risks of replacing legacy systems. In Software Engineering. The Apache Software Foundation Maven Welcome to Apache Maven. (2016). Retrieved March 17, 2016 from Trello, Inc Trello. (2016). Retrieved March 21, 2016 from University of Maryland FindBugs TM - Find Bugs in Java Programs. (2016). Retrieved May 26, 2016 from Visual Paradigm Software Design Tools for Agile Teams, with UML, BPMN and More. (2016). Retrieved June 1, 2016 from Kelly Waters Key Principles of Lean Software Development All About Agile. (2010). Retrieved May 11, 2016 from yworks yed Graph Editor. (2016). Retrieved March 21, 2016 from 97

114

115 9 Appendix A: Probabilities of the input arguments 9. APPENDIX A: PROBABILITIES OF THE INPUT ARGUMENTS This appendix covers the process to obtain the probabilities for input arguments of the random verifier. Those probabilities come from the extracted binary files of the mainframe s IMS database. The binary files where converted and stored in a relational database (SQL DB2). The probabilities were calculated according to appearance ratio of each index (AS, TC, ERH, ZUO). Those indexes are the ones required as input for the navigation through dialogs. Identifying which field of which segment of the IMS database had the values for each input field for each dialog was the task that needed the most effort. The dumps from the IMS were imported to SQL DB2 tables to allow easier queries. Figure 9.1 contains the queries needed to extract the probabilities for the AS and TC fields. Figure 9.2 and Figure 9.3 have the ERH and ZUO respectively. Table 9.1 includes some of the probabilities of the ZUO input values as an example with the probabilities ranging from 0 to 1. ZUO Probability (0 to 1) Table 9.1. Some of the probabilities of the ZUO field as example. 99

116 9 Appendix A: Probabilities of the input arguments Figure 9.1. Extraction of probabilities for the AS and TC (dialog name) fields.. 100

117 9 Appendix A: Probabilities of the input arguments Figure 9.2. Extraction of probabilities for the ERH fields. 101

118 9 Appendix A: Probabilities of the input arguments Figure 9.3. Extraction of the probabilities for the ZUO fields. 102