Nicman Group Test Data Management 2.0 Leveraging Copy Data Virtualization Technology in QA for SQuAD November 2016
Agenda Test Data Management 2.0 Leveraging Copy Data Virtualization Technology in QA Copy Data Management, (CDM) History, Issues, Problems CDM: Testing CDM: Continuous Integration / DevOps LIVE! Replication of large data set for Dev and Test Confidential Copyright Nicman Group LLC
Today s Storage Environments Immense Growth Curve Industry has 16 responded by 14 12 10 8 addressed this situation by economics and technology Copies of C Production Data o Better, faster, p cheaper y hardware 6 4 2 0 Storage Explosion Production Data Higher density hard drives Make it affordable to buy more and more storage
Application Layer with Storage Copy Data Storage at App Layer Make a Master Copy Usable Format, Native Language Method of Maintaining via Block Level Deltas BACKUP SNAP SHOT DR/BC DEV/TEST COPY COPY COPY COPY NBU ComV Avamar EMC Hitachi NetApp EMC SRDF Host Based App Based Manual
Real World Example 5
Next Generation Data Management SERVER & COMPUTE VIRTUALIZATION NETWORK VIRTUALIZATION DATA VIRTUALIZATION 2010+ 2012+ 2014+ COMMODITIZATION OF HARDWARE AND VIRTUALIZATION OF SERVICE Enterprise Private & Public Cloud 6
Testing and DevOps Four Problems SPEED QUALITY SECURITY COST 7
The SPEED Problem THE PROCESS THE PHYSICS DBA Creates copies and masks data Prototype Unit Test Build Integration QA Test UAT Tester requests copy. Tickets opened. Compute and storage is provisioned Change Management Request SLA 1 week SLA 1 week Ex: The initial copy + 5 physical copies of 5TB over 1 Gbps will take 5 days SLA 1 2 weeks Developer gets a masked copy Production Data First Copy Competitive Disadvantage, Opportunity Cost Of Revenue From New Apps 8
Low Fidelity Data Leads to QUALITY Problems Unit Testing Works Great!! Integration Testing Works! QA Testing Works! It Does Not Work!!!!!! Prototype Unit Test Build Integration QA Test UAT Production CRITICAL STAGES PROBLEMS USING Low-Fi DATA X Can t Do Scalability Tests Early X Bugs Surface Too Late In DEV Cycles Poor Quality. Low Customer Satisfaction. High Cost Of Support 9
The SECURITY / CONTROL Problem CSO/CISO Concerns Data not masked No control on who can access copies No audit track-ability Non-Compliance, Liability From Sensitive Data Exposure 10
The COST Problem Prototype Unit Test Build Integration QA Test UAT All these physical copies drive up costs Large burden on DBAs 1. Clone DB 2. Mask DB 3. Copy Masked DBs 4. Bring DB Online Production Data First Copy Uncontrolled Infrastructure Cost 11
Solution 12
Dev, QA, Testers Get Self Service & Instant Access Prototype Unit Test Build Integration QA Test UAT / STAGING Production Data Faster TTM Leads To Competitive Advantage & Faster Growth 13
Catch Defects Early Scalability Tests Regression Tests Performance Tests Less Surprises. More Predictability Prototype Unit Test Build Integration QA Test UAT / STAGING 0 TB 0 TB 0 TB 0 TB 0 TB Production Data 50 TB High Quality Release On Time, On Budget 14
High Fidelity Test Data With Lowest Costs Prototype Unit Test Build Integration QA Test UAT / STAGING Test Environments 1. Instant Virtual Copies 2. Self Service 3. Automated Refresh Masked Data CDM Incremental Forever Capture IT Operations 1. Role Based Access Control 2. Data Masking 3. Multi-Hypervisor, Cloud Production Data 15
Control & Security DATA 2 Specify Data Masking Scripts MASKED DATA UNIT TEST BUILD INTERGRATION EXAMPLE: WHO can access SQL DB on WHAT servers WHO can access Oracle DB on WHAT servers QA TEST ORACLE DB UAT \ STAGING 1 Ops: Specify how often to ingest data. 3 Ops: Specify RBAC for testers SUPPORT Reduce Risk Of Non-Compliance & Ensures Data Masking 16
At No Extra Storage!! Prototype Unit Test Build Integration QA Test UAT / STAGING 0 TB 0 TB 0 TB 0 TB 0 TB Production Data 50 TB 17
Example cases for DevOps 18
Central Admin by Ops 2 Data Ingested in an incremental for ever fashion 3 Library of virtual copies for various PIT Prototype Unit Test GOLDEN COPY V1 V2 Vn ORACLE DB V1 V2 Build Integration QA Test EXAMPLE: WHO can access SQL DB on WHAT servers WHO can access Oracle DB on WHAT servers ORACLE DB Vn UAT \ STAGING 1 Ops: Specify how often to ingest data. 4 Ops: Specify RBAC for testers SUPPORT 19
Self Service by Dev Prototype 5 Dev: Self Service: LOGIN. BROWSE. SELECT PIT. PROVISION V1 V2 Vn Unit Test Build Integration 7 Databases Come Up Online. DB Parameters Customizable. No DBA Needed. No Storage Admin Needed. ORACLE DB ORACLE DB Golden Copy V1 V2 Vn QA Test 8 Dev: Protect test environment. 6 Dev: Instant UAT \ STAGING SUPPORT 9 Production Support: Debug \ conduct RCA on high fidelity data. 20
Automated Test Data In Continuous Integration Test Automation 2 1 3 Frequent Builds Code Check In DEV Invoke Actifio API To Provision Use Chef To Provision Platform 4 5 Test Server 1 6 Test Server 2 Deploy Latest Build 7 Framework Execute Automated Test Cases Golden Copy 8 Report Integration Test Results Write Code. Unit Test Using MASKED DATA CDM Production DB Automates High Fidelity Test Data Provisioning 21
1 Test Data Promotion Virtual Copy from golden copy 2 Development wants to modify schema / data structures and they want all testing to happen off this modified schema 8 Continues to protect production and refresh data for Dev, QA, UAT for next iteration Golden Copy 3 Thus dev creates a new data branch (snapshot from v1) and makes the changes V1 4 Multiple virtual copies are created off V2 for testing by multiple teams 6 Multiple virtual copies are created off V3 for testing by multiple teams 5 DEV BUILD INTEGRATION QA TEST UAT \ STAGING PRODUCTION Dev can create further branches if needed V2 V2 V2 V3 V3 V3 V3 7 After UAT, changes are rolled out in production using 3rd party tools 22
Parallel Testing Increases Productivity 1 9:00 AM EST 3 10:05 AM EST V1 V2 TEST SERVER 1 V1 V2 TEST SERVER 2 Golden Copy Vn QA uses this PIT version for testing Golden Copy Vn QA uses the same PIT version on another test server. Continues with other test cases 2 10:00 AM EST TEST CASES Rare issue reproduced. Need DEV to do RCA. But DEV is in 7 am PST. 4 12:00 PM EST V1 V2 TEST SERVER 1 DEV starts RCA, debugs, fixes problem, tests the fix QA also has to execute rest of these test cases Golden Copy Vn TEST SERVER 2 QA executes the rest of the test cases 23
Bookmarks Enable Negative Test Case Execution 1 QA sets up testing environment 3 Testing is performed through completion Golden Copy V1 V2 Vn TEST SERVER VM Tester provisions a PIT image of the database on a test machine. Golden Copy V1 V2 Vn TEST SERVER VM Data can be modified, corrupted and negative test cases can be executed 2 A snapshot/bookmark is set at this PIT 4 QA test environment is rolled back to Golden Copy V1 V2 Vn TEST SERVER VM Before corrupting the data, tester issues a bookmark starting point Golden Copy Nested Bookmarks, Multi Level Grandchildren, Daisy Chained Snapshots V1 V2 Vn TEST SERVER VM After the testing is done, tester rewinds the state to the bookmark 24
5 6 Data Masking Data Refresh happens automatically Ops: Specify data masking scripts and data masking server 2 Configure Actifio to create a live clone 1 Prototype Unit Test Testers can select ONLY masked copies Build Integration Golden Copy Live Clone QA Test ORACLE DB 3 Mounts the data to data masking server. Data is masked here. DATA MASKING SERVER 4 After Masking is done, control returns. UAT \ STAGING SUPPORT 25
ABOUT US Nicman Group: Denver Colorado, Secaucus New Jersey, Philadelphia Pennsylvania 50+ IT consultants and professional services experts specializing in DevOps, software development, business analytics, deployment strategies, performance testing and QA services. Cloud services provider with focused expertise in infrastructure services, private cloud, hosting technologies and data warehousing. Global relationships, partnerships and offshore resources. Confidential Copyright Nicman Group LLC