Test Architect A Key Role defined by Siemens

Test Architect A Key Role defined by Siemens Siemens Munich, Germany January 30 February 3, 2017 http://www.oop-konferenz.de

Agenda Why do we need a Test Architect? What are the responsibilities and tasks of a Test Architect? How can a Test Architect provide value and create impact on the business? Page 2

Systems we build today at Siemens for our customers Power and Gas Wind Power and Renewables Energy Management Building Technologies Mobility Digital Factory Process Industries and Drives Healthcare Page 3

(Core) Learning Programs and other trainings driven by the Siemens Software Initiative (SWI) R&D Manager, 2012 Senior Software Architect (SSWA), 2007 Senior Software Architect Software Architect (SWA), 2009 System Architect (SyA), 2012 Test Architect (TeA), 2016 System Architect Software Architect Test Architect Requirements Engineering for project / product managers, 2010 Functional Trainings Page 4

Key parts of the (core) learning programs Setting standards for software and system development Qualification (courses and on-the-job training) Software Architect Assuring practical experience for critical projects Certification Fostering best practice sharing and experience exchange Architects network Page 5

Test Architect Some external references (1) Page, A., Johnston, K. and Rollison, Bj. 2008. How We Test at Microsoft, Chapter 2. 2008 Alan Page (Microsoft), What is a Test Architect?, 2008 https://blogs.msdn.microsoft.com/alanpa/2008/05/01/what-is-a-test-architect/ John Morrison (Oracle), Test Architect, Oct 8, 2007 https://blogs.oracle.com/johnmorrison/entry/test_architect James Brewer et.al. (Raytheon), The Value of the Test Architect, March 2012 http://www.dtic.mil/ndia/2012test/13942_brewer.pdf ICST InSTA workshops (International Workshop on Software Test Architecture) http://www.aster.or.jp/workshops/insta2015/ UML Testing Profile (UTP) Version 1.2. 2013. Chapter 8, Test Architecture, pp. 19-24 http://utp.omg.org/, http://www.omg.org/spec/utp/ Page 6

Test Architect Some external references (2) ISTQB Glossary (http://www.istqb.org/) Test architect (1) A person who provides guidance and strategic direction for a test organization and for its relationship with other disciplines. (2) A person who defines the way testing is structured for a given system, including topics such as test tools and test data management. That s ok but there is more Page 7

Driving triumvirate for testing & quality engineering Process quality, certification and audits, regulations and norms across lifecycles and versions Quality Manager Process Quality goals Test Manager Budget, people, plans, logistics for the test organization Productivity Strategy Page 8

Driving triumvirate for testing & quality engineering Process quality, certification and audits, regulations and norms across lifecycles and versions Quality Manager Process Quality goals Test Manager Budget, people, plans, logistics for the test organization Productivity Strategy Test Architect Cost efficient, accelerated, innovative, sustainable testing solutions for test methodology and test infrastructure Page 9

As a Test Architect You have one Role but you are wearing two Hats Test Expert for the system under test (SUT) Design the test approach Apply innovative test technologies Drive the quality of the SUT Software / System Architect for the test system Design and realize the test architecture Apply innovative software technologies Drive the quality of the test system This is the architect's job! Page 10

The Test Architect as a technical communication hub Page 11 Customer, Portfolio/Product Manager (Sub) Project manager Supplier Manager Quality Manager / Quality Engineer, Regulation experts Safety, Security, EMC, experts Line Manager System/Software Architect Test Engineer Maintenance -, Service -, Repair -, Refurbishment experts Commissioning, Production, Operation experts Designers, Developers, Engineers Test Manager System Integrator Q-related Roles To interact & motivate efficiently is crucial for Test Architects Close Communication Loose Communication

A Test Architect needs a broad set of competencies Business Understanding Testing and Quality Business model Ecosystems, understanding Partnering, Suppliers Standards, Regulations, Legal issues Global development Lifecycle issues Expert Development processes Domain knowledge Test & Quality management Advanced Product & System requirements Test automation Basic Quality requirements Requirements Engineering Expert drives, coaches and is able to improve in own domain & scope Page 12 Basic Advanced knows about can apply Test strategy Test architecture Design for Testability Test methods & technologies Lean & Agile testing Test process improvement Testing Governance & Guidance Arch. & Design Architecture Quality Project management Social skills Leadership TeA TeM Architecture and Development Integration & Deployment Social Skills & Leadership Test Architect Test Manager

Test Architect Role profile Responsibilities and tasks (extract) Business Understanding Analyze the current testing process and practices and drive improvements Evaluate and select technologies (e.g. by prototyping) Provide detailed planning information for testing-related tasks (esp. w.r.t. testing of requirements and testing infrastructure) Identify and manage technical product risks Setup supplier agreements for outsourced testing infrastructure development Define acceptance criteria for outsourced product development Perform make-or-buy decisions Requirements Engineering Architecture and Design Testing & Quality Maintenance Business Understanding Ensure testability of requirements engineering artifacts Define testability requirements Foster the use of test-driven approaches in requirements engineering Enforce traceability in testing artifacts Prioritize requirements Page 13 black: responsible, grey: involved Requirements Engineering

Test Architect Role profile Responsibilities and tasks (extract) Business Understanding Ensure that testability requirements are incorporated into product design Create design prototypes for testing infrastructure Provide testing infrastructure requirements Define reuse strategy for testing infrastructure components Identify and manage technical risks for the testing environment Setup integration plan for the testing infrastructure Drive integration of testing infrastructure with the SUT Guide/Coach system/software architects in testing related topics Identify and implement reuse for test cases and testing infrastructure components Review architecture artifacts Requirements Engineering Architecture and Design Testing & Quality Maintenance Architecture and Design black: responsible, grey: involved Page 14

Test Architect Role profile Responsibilities and tasks (extract) Business Understanding Requirements Engineering Define and implement a justified, effective test strategy for the project, aligned with the business and quality goals Measure efficiency and effectiveness of the test strategy Architecture and Design Measure, control and continuously optimize coverage and degree of test automation Testing & Quality Maintenance Define and maintain an effective and efficient test architecture in close cooperation with system integration and test managers Govern the design and improvement of test automation framework and test suites Ensure quality of the test code Guide/Coach testing engineers in advanced testing issues Discuss and elaborate test aspects in early development phases Identify and manage testing stakeholders Perform impact analysis for changes in the system/software Monitor and improve the system/software integration process Testing & Quality black: responsible, grey: involved Page 15

Test Architect Role profile Responsibilities and tasks (extract) Business Understanding Perform continuous architecture management for the testing environment Manage the internal quality of the whole testing environment Provide comprehensive documentation of the testing infrastructure's architecture Establish and maintain configuration management for testing artifacts Ensure that testability requirements from post-development phases are taken into account Analyze customer feedback Requirements Engineering Architecture and Design Testing & Quality Maintenance Maintenance black: responsible, grey: involved Page 16

Test Architect Expert Training (TeA) Topics Business Understanding (~15%) Architecture (~15%) Testing & Quality (~40%) Product Lifecycle Models (PLM) Lifecycle models and processes Portfolio management Risk & Uncertainty Estimation and prioritization Systematic architecting Design strategies & tactics Design patterns Architectural views and documentation Architecture quality Architecture reviews Refactoring Value of testing Test strategies, risk-based testing Design for testability Test management Agile testing Test design techniques Test automation Test environment, test architecture Test documentation Internal quality Test Architect revisited Requirements Engineering (~10%) Cross-Cutting Social Skills (~20%) Good requirements Reviewing requirements Configuration and change request management Role of the Test Architect Dealing with quality attributes (non-functional requirements) Agile / Lean development Communication with stakeholders about testing and quality Situational leadership model Leadership and motivation Conflict management Page 17

Test Architect Expert Training (TeA) Schedule Selection of candidates (self perception) Workshop 1 Live as a TeA 4 days Workshop 2 Work as a TeA 3 days Workshop 3 Survive as a TeA 3 days Certification Preparation phase Project phase 1 Project phase 2 Experience phase ~2 months ~2 months ~2 months >6 weeks In parallel setup and maintenance of networking by centrally organized events and exchange platforms Page 18

Systematic architecting Perform user requirements elicitation Create domain model Model dynamics of scenarios Determine scope & boundaries Create first conceptual draft Product / System / Service Structure the baseline architecture Introduce deployment views SWA / SyA Define principles & guidelines Plan and realize increments Page 19

Systematic architecting Links between Test Architect (TeA) and Software / System Architect (SWA / SyA) Test strategy Identify ASR Define qualities Perform user requirements elicitation Create domain model Test System TeA Companion domain model Model dynamics of test system Scope & boundaries of test system Domain model Dynamics Boundaries Review both architectures Model dynamics of scenarios Determine scope & boundaries Create first conceptual draft Product / System / Service Structure the baseline architecture Synchronize Introduce deployment views SWA / SyA Define principles & guidelines Increments, Plan and realize increments Integration Tests Conceptual draft of test system Refine architecture of test system Define test deployment architecture Testing principles governance Incremental evolution Page 20

Scope of Internal Quality As a Test Architect You have one Role but you are wearing two Hats Test Expert for the system under test (SUT) Design the test approach Apply innovative test technologies Drive the quality of the SUT Software / System Architect for the test system Design and realize the test architecture Apply innovative software technologies Drive the quality of the test system This is the architect's job! Internal Quality of the SUT (production code) Page 21 Internal Quality of the test system (test code)

Summary Test Architect A new key role Test Architect on eye level with software / system architects Diverse responsibilities and tasks that require proper skills, competencies, capabilities A new career path for testers A good Test Architect provides value and creates impact on the business Page 22

Principal Key Expert Engineer Siemens AG Research in Digitalization and Automation Software and Systems Innovations CT RDA SSI Otto-Hahn-Ring 6 81739 Muenchen Phone: +49 (89) 636 633509 Fax: +49 (89) 636 45450 Mobile: +49 (172) 4328121 E-mail: peter.zimmerer@siemens.com Internet siemens.com/innovation siemens.com/corporate-technology Page 23