Actifio Test Data Management

Size: px

Start display at page:

Download "Actifio Test Data Management"

Grant Richard
6 years ago
Views:

1 Actifio Test Data Management Oracle MS SQL

2 Faster Time To Market Start Release Time To Market (TTM) Finish Faster App Releases Faster Application Releases Faster TTM Increases Revenue Market Share Competitive Differentiation COPYRIGHT 2015 ACTIFIO 2

3 Test Environments Needs DEVELOPMENT ENVIRONMENT BUILD ENVIRONMENT INTEGRATION TEST ENVIRONMENT QA ENVIRONMENT UAT \ STAGING SERVER PRODUCTION SERVER TEST ENVIRONMENTS PRODUCTION ENVIRONMENT COPYRIGHT 2015 ACTIFIO 3

5 Low Fidelity Test Data DEVELOPMENT ENVIRONMENT BUILD ENVIRONMENT INTEGRATION TEST ENVIRONMENT QA ENVIRONMENT UAT \ STAGING SERVER PRODUCTION SERVER LOW FIDELITY TEST DATA a.k.a DATA SUBSET COPYRIGHT 2015 ACTIFIO 5

7 The SPEED Problem THE PROCESS THE PHYSICS DBA Creates copies and masks data Prototype Unit Test Build Integration QA Test UAT Tester requests copy. Tickets opened. Compute and storage is provisioned Change Management Request SLA 1 2 weeks SLA 1 week SLA 1 week Developer gets a masked copy Ex: The initial copy + 5 physical copies of 5TB over 1 Gbps will take 5 days Production Data First Copy COPYRIGHT 2015 ACTIFIO 7

8 Low Fidelity Data Leads to QUALITY Problems Unit Testing Works Great!! Integration Testing Works! QA Testing Works! It Does Not Work!!!!!! Prototype Unit Test Build Integration QA Test UAT Production PROBLEMS USING Low-Fi DATA X X Can t Do Scalability Tests Early Bugs Surface Too Late In DEV Cycles RESULTS CRITICAL STAGES Poor Quality Delayed Releases Dev vs. Ops Blame Game COPYRIGHT 2015 ACTIFIO 8

10 The COST Problem Prototype Unit Test Build Integration QA Test UAT All these physical copies drive up costs Large burden on DBAs 1. Clone DB 2. Mask DB 3. Copy Masked DBs 4. Bring DB Online Production Data First Copy COPYRIGHT 2015 ACTIFIO 10

High Fidelity Test Data With Lowest Costs Prototype Unit Test Build Integration QA Test UAT / STAGING Virtual DB Virtual DB Virtual DB Virtual DB Virtual DB Test Environments 1.

12 High Fidelity Test Data With Lowest Costs Prototype Unit Test Build Integration QA Test UAT / STAGING Virtual DB Virtual DB Virtual DB Virtual DB Virtual DB Test Environments 1. Instant Virtual Copies 2. Self Service 3. Automated Refresh Masked Data Incremental Forever Capture IT Operations 1. Role Based Access Control 2. Data Masking 3. Multi-Hypervisor, Cloud Production Data COPYRIGHT 2015 ACTIFIO 12

13 Virtual DB as a Service Financial Services Global Retail and Investment Bank PROD DBs COPIES 14 Copies Per DB Avoided = 1.2 PB DB Data + 2 PB of storage growth avoided 9000 developers. Saved time equivalent to 875 developers = 1.5 M hours per year saved 85 DBAs saved 20,000 DBA hours within 12 months 30 Additional Dev Projects per year within existing budget Time To Value = 7 Months IRR = 294% Every $1.6 M investment in Actifio returned $22 M in value 73% resource benefits 27% hard savings 26 business case stakeholders DB, Storage, Operations, AppDev. 13 Executive Mgnt input Retail, Investment and Data Security 13

14 Use Case: DevOps as a Service Financial Services Global Investment Bank 11 TB Prod, 79 TB Copies, 86% overhead 2,733 monthly refresh hours Reduce $1.6MM Labor costs for manual data refresh Save 16 weeks per year IMPACT: 1+ extra quarter to topline PROD DBs COPIES 45% Return on Investment Cash Positive <6 Months $3.5M 5-Year Business Impact Analysis Done On QA Center Of Excellence Portfolio 14

16 Provision N x Copies Of Multi Tier Apps Rapidly App App App App App App App App App App App App Provision Multiple Virtual Copies Of DB and VMs Instantly ACTIFIO Capture VMs and DB From Production App App App Apps In VMs DB In Physical \ Cluster \ VM Multi Tier Production App COPYRIGHT 2015 ACTIFIO 16

17 Central Admin by Ops 2 Data Ingested in an incremental for ever fashion 3 Library of virtual copies for various PIT Prototype Unit Test MS SQL DB MS SQL DB V1 V2 Vn ORACLE DB Build Integration EXAMPLE: WHO can access SQL DB on WHAT servers WHO can access Oracle DB on WHAT servers ORACLE DB GOLDEN COPY ACTIFIO V1 V2 Vn QA Test UAT \ STAGING 1 Ops: Specify how often to ingest data. 4 Ops: Specify RBAC for testers SUPPORT COPYRIGHT 2015 ACTIFIO 17

18 Self Service by Dev Prototype 5 Dev: Self Service: LOGIN. BROWSE. SELECT PIT. PROVISION MS SQL DB MS SQL DB V1 V2 Vn Unit Test Build Integration 7 Databases Come Up Online. DB Parameters Customizable. No DBA Needed. No Storage Admin Needed. ORACLE DB ORACLE DB Golden Copy ACTIFIO V1 V2 Vn QA Test UAT \ STAGING 8 9 Dev: Protect test environment. Production Support: Debug \ conduct RCA on high fidelity data. 6 Dev: Instant Virtual DB SUPPORT COPYRIGHT 2015 ACTIFIO 18

19 Parallel Testing Increases Productivity 1 9:00 AM EST 3 10:05 AM EST MS SQL DB MS SQL DB V1 V2 TEST SERVER 1 V1 V2 TEST SERVER 2 Golden Copy ACTIFIO Vn QA uses this PIT version for testing Golden Copy ACTIFIO Vn QA uses the same PIT version on another test server. Continues with other test cases 2 10:00 AM EST TEST CASES Rare issue reproduced. Need DEV to do RCA. But DEV is in 7 am PST. QA also has to execute rest of these test cases 4 12:00 PM EST Golden Copy ACTIFIO MS SQL DB V1 V2 Vn TEST SERVER 1 TEST SERVER 2 DEV starts RCA, debugs, fixes problem, tests the fix QA executes the rest of the test cases COPYRIGHT 2015 ACTIFIO 19

20 Test Data Promotion 1 Virtual Copy from golden copy 2 Development wants to modify schema / data structures and they want all testing to happen off this modified schema 8 Actifio continues to protect production and refresh data for Dev, QA, UAT for next iteration Golden Copy 3 Thus dev creates a new data branch (Actifio snapshot from v1) and makes the changes V1 4 Multiple virtual copies are created off V2 for testing by multiple teams 6 Multiple virtual copies are created off V3 for testing by multiple teams ACTIFIO 5 Dev can create further branches if needed DEV Build Integration QA Test V2 V2 V2 UAT \ STAGING V3 V3 V3 V3 PRODUCTION MS SQL DB After UAT, changes are rolled out in production using 3rd party tools COPYRIGHT 2015 ACTIFIO 20 7

Server 1 Virtual DB Test Server 2 Execute Automated Test Cases Golden Copy ACTIFIO Production DB COPYRIGHT 2015 ACTIFIO 21 6

21 2 1 Automated Test Data In Continuous Integration 3 Frequent Builds Code Check In Write Code. Unit Test Using Virtual DB DEV Virtual DB Invoke Actifio API To Provision 5 Virtual DB Use Chef To Provision Platform 4 Test Server 1 Virtual DB Test Server 2 Execute Automated Test Cases Golden Copy ACTIFIO Production DB COPYRIGHT 2015 ACTIFIO 21 6 Virtual DB Deploy Latest Build Actifio Automates High Fidelity Test Data Provisioning 7 Test Automation Framework MASKED DATA 8 Report Integration Test Results

$ONLY masked copies ACTIFIO MS SQL DB Build Integration Golden Copy Masked Copy QA Test ORACLE DB 2 UAT \ STAGING$

22 Data Masking 5 Data Refresh happens automaticall y Ops: Specify data masking scripts and data masking server 1 Actifio has a masked copy that is ready to be consumed by Dev users 3 Prototype Unit Test 4 Testers can select ONLY masked copies ACTIFIO MS SQL DB Build Integration Golden Copy Masked Copy QA Test ORACLE DB 2 UAT \ STAGING Actifio mounts the data to data masking server. Data is masked here DATA MASKING SERVER SUPPORT COPYRIGHT 2015 ACTIFIO 22

24 Copy Data Virtualization Platform ACCESS YOUR DATA PROTECT YOUR DATA MOVE YOUR DATA Self-serve instant data access. Your last backup modernization ever. Across local, remote, cloud infrastructure. Backup, DR, Cloud Mobility COPYRIGHT 2015 ACTIFIO 24

25 Business Impact Summary SPEED Faster Time To Market Increased Topline QUALITY Increased NPS; Repeat Business Lower Support Costs SECURITY Compliance & Control Peace Of Mind COST Lower Infrastructure Costs Eliminate Multiple Point Tools COPYRIGHT 2015 ACTIFIO 25

26 Thank you.

27 Appendix A AWS Related Slides

28 Actifio-Powered DevOps in AWS Incremental For Ever Data Ingestion 4 5 Your dedicated VPC Firewall Actifio Sky in EC2 appliance Near-Instant Virtual DBs presented to Dev and QA On-demand EC2 instances for Dev - QA - UAT MS SQL DB 3 WAN Efficient, deduped replication Dev ACTIFIO Store Local Copy Copy in AWS AQA ORACLE DB QA SOURCE AWS UAT RAC, ASM AIX, Linux RMAN API, Incremental forever data ingestion 2 Data is Deduped AWS EBS Volumes COPYRIGHT 2015 ACTIFIO 28 6

Cross Platform: UNIX to AWS 5 1 Protect data back to Actifio Protect Oracle on UNIX 7 6 Your dedicated VPC Firewall 8 11 Near-Instant Virtual DBs presented to Dev & QA Actifio Sky in EC2 appliance 10

29 Cross Platform: UNIX to AWS 5 1 Protect data back to Actifio Protect Oracle on UNIX 7 6 Your dedicated VPC Firewall 8 11 Near-Instant Virtual DBs presented to Dev & QA Actifio Sky in EC2 appliance 10 On-demand EC2 instances for Dev - QA - UAT WAN Efficient, deduped replication Dev UNIX Prod ACTIFIO Store Local Copy Copy in AWS AQA Linux Server UNIX Staging AWS QA UAT 4 NFS Mount To Linux Server 3 Big-Endian To Little Endian Conversion 2 Present Virtual Volume to another UNIX server AWS EBS Volumes COPYRIGHT 2015 ACTIFIO 29 9

30 Testing Automation consistent starting point 1 QA sets up testing environment 3 Testing is performed through completion MS SQL DB MS SQL DB Golden Copy ACTIFIO V1 V2 Vn TEST SERVER VM QA sets up the environment for testing including a VM for the test server and a PIT image of the database Golden Copy ACTIFIO V1 V2 Vn TEST SERVER VM Testing is done, server and data can be modified without fear. 2 A snapshot/bookmark is set at this PIT Golden Copy ACTIFIO MS SQL DB V1 V2 Vn TEST SERVER VM User protects the QA server and the database image 4 QA test environment is rolled back to starting point Golden Copy ACTIFIO MS SQL DB V1 V2 Vn TEST SERVER VM QA restores environment as it was in steps 1 and 2 COPYRIGHT 2015 ACTIFIO 30

31 Architectural Overview 1. Wide platform support Efficient incremental forever ingestion Stored natively in snap pool; Instant Virtual DB 4. Provisioning Multi-day, month retention in dedup pool 5. Encrypted, B/W efficient replication to remote site 6. Multi year retention in dedup pool 7. Instant Virtual DB Provisioning, Low RTO DR from Snap Pool Actifio Snap Pool Dedup Pool Actifio Dedup Pool Snap Pool TEST STAGING TEST STAGING aws COPYRIGHT 2015 ACTIFIO 31

Nicman Group Test Data Management 2.0 Leveraging Copy Data Virtualization Technology in QA for SQuAD. November 2016

Nicman Group Test Data Management 2.0 Leveraging Copy Data Virtualization Technology in QA for SQuAD November 2016 Agenda Test Data Management 2.0 Leveraging Copy Data Virtualization Technology in QA Copy