Bulletproof Your Application Upgrades with Secure Data Masking and Subsetting

Transcription

1 Bulletproof Your Application Upgrades with Secure Data Masking and Subsetting Jagan R. Athreya, Oracle Senior Director, Database Manageability JT Gaietto, SquareTwo Financial IT Manager, Security and Compliance 1

2 The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle. 2

3 Program Agenda Why Test Upgrades using Production Data Data Masking Data Subsetting Test System Provisioning Customer Case Study 3

4 Why Test Upgrades using Production Data? 4

5 Upgrades are Core to the Success of Your Business Major Release Minor Release Released 11i Planned RUP1 RUP2 RUP3 Nov 04 Jan May 09 Dec 09 Aug 10 Enhanced Product Areas New Products in 12/12.1 PeopleTools Applications 9.1 Feature Pack Feature Pack 9.2 Feature Pack Feature Pack 9.3 Discover Develop Test Migrate 5

6 Upgrade Process Discover Develop Test Migrate Business Analyst Application Developer Software QA tester Application DBA Define business process Develop custom programs and interfaces Unit test and volume test application Upgrade Demonstration Development Test Production Vendor sample PRODUCTION PRODUCTION Production 6

7 Production Data in Development and Test Benefits and Challenges Development Test Benefits Robust application functionality based on real-world production use cases Highly scalable and performant application post-upgrade based on realistic concurrent testing Challenges Sensitive data exposed to unprivileged users High storage costs due to production data in development Sensitive data exposed to unprivileged users 7

8 Provisioning Secure Test Environments Data Masking and Data Subsetting Data Masking Data Subsetting Replace sensitive production data with fictitious yet realistic data in Development and Test Comply with regulations while enabling realistic application testing Bullet proof Application Testing Provision a fractional yet relationally intact representation from production data in Development Reduce storage costs while enabling realistic application development 8

9 Data Masking and Data Subsetting of Enterprise Applications Financials Supply Chain Management Human Capital Management Customer Relationship Management What is the relationship between tables across the various applications? What are the data extraction rules that produces referentially intact subsets? Will the subsetted data fit into the available storage for test systems? What are the various types of sensitive data used across the applications? Where are these sensitive data stored and how are they related to each other? 9

10 Data Masking and Data Subsetting Methodology Production DEPT LAST_NAME SSN SALARY 10 AGUILAR , BENSON , CHANDRA , DONNER ,000 ORACLE IBM DB2 MS SQL Non-Production DEPT LAST_NAME SSN SALARY 10 SMITH , JOHNSON ,000 Find: Identify sensitive data and discover referential relationships Assess: Define the optimal data masking formats and subset criteria Secure: Provision secured test / dev systems using masking+subsetting Test: Verify the integrity of applications and security of data 10

11 Data Masking 11

12 Data Masking Key Concepts Sensitive Data Identification Application Data Modeling Mask format Referential Relationships Data Masking Definition 12

13 Person Name Maiden Name Business Address Business Telephone Number Business Address Custom Name Employee Number User Global Identifier Sensitive Data Identification Party Number or Customer Number Account Name Mail Stop GPS Location Student Exam Hall Ticket Number Club Membership ID Library Card Number Identity Card Number Instant Messaging Address Web site National Identifier Passport Number Driver s License Number Personal Address Personal Telephone Number Personal Address Visa Number or Work Permit Bank Account Number Card Number (Credit or Debit Card Number) Tax Registration Number or National Tax ID Person Identification Number Welfare Pension Insurance Number Unemployment Insurance Number Government Affiliation ID Military Service ID Social Insurance Number Pension ID Number Article Number Civil Identifier Number Hafiza Number Social Security Number Trade Union Membership Number Pension Registration Number National Insurance Number Health Insurance Number Personal Public Service Number Electronic Taxpayer Identification Number Biometrics Data Digital ID Citizenship Number Voter Identification Number Residency Number (Green Card) What is sensitive data? Business driven Privacy law (HIPAA), industry regulations PCIDSS) Confidentiality rules (data protection policies) Where is sensitive data stored? Application Documentation User interface screens Schema-based, e.g. EMP.NATIONAL_ID Data-based, e.g. 16 digit #s beginning with 4 or 5. 13

14 Sensitive Column Types Library Pre-defined search patterns (Schema and Data based) Search entire application for data matching patterns Ranked based on match to search patterns 14

15 Data Masking Format Library 40+ out of the box mask formats for common PII data Built-in mask primitives to expand format library User-defined SQL or PL/SQL based functions for complex formats Define once deploy everywhere Self-updateable for Oracle supplied mask formats 15

16 EMPLOYEE Automatic Referential Integrity Database- and application-enforced CUSTOMER EMPID NAME TITLE CUSTID NAME REP_ID 12 SMITH SALESREP 200 ACME JONES CSR 201 BIG BOX MCNEIL CEO 15 SALVATORE SALES MGR SUPPORT CUSTID CSR_ID Database Referential Integrity Application Referential Integrity Data distribution Read data dictionary to enforce referential relationships Read application meta data (EBS, Fusion Apps) to enforce referential relationships Maintain same data distribution after masking 16

17 Sophisticated Masking Techniques Compound Masking Condition-based Masking Compound masking Multi-column groups, e.g. address (street, city, state, zip) Condition-based masking Mask national identifiers, e.g. social security number, by country-specific format Deterministic (Repeatable) Repeatable consistent masking, e.g. referential integrity across databases, or consistency across refreshes Reversible Key-based format preserving masking, e.g. outsourced data processing Shuffle Randomization within population set. Perturbation Random mask generation around a value, e.g. date of birth Extensible (via PL/SQL) 17

18 E-Business Suite Data Masking Templates Spans entire application suite 950 Columns / 1900 rules HCM - Payroll, Employment Details, Personal Info TCA, ATG, Financials, Projects Best practices for identifying and masking sensitive data Columns, Relationships, and Masking rules for PII and Sensitive attributes Instructions for wiping credentials after cloning (Support Note ) De-identification done across the DB 18

19 What is not masked in the EBS template Financial data Results Forecasts Design Specs Unstructured data Descriptive Flex Fields Developer and Localization flex fields masked where the content is known Notes Attachments Internal primary keys (user_id, person_id) 19

20 Masking Application Users EBS template masks names of Apps (FND) users Existing users not usable after masking For seeded users (e.g. SYSADMIN) the names will be retained but passwords will be reset If there is a need to maintain local application users for testing New interface table provided to populate exempt users : Users exempted from masking Option provided to rename users (JSMITH TEST1) Specify a new password 20

21 Data Masking for PeopleSoft Built-into PeopleSoft 9.2 Configurable via PeopleTools Ships with out of the box masking techniques across the application family 21

22 Database Database Database Application Templates Self-update ` ` ` ` Telco Govt. Retail Mfg. Template Problem: Reduce the manual effort of installing and configuring application templates for data masking Solution: Self-update Application masking templates delivered from EM Oracle to EM software library securely via self-update Mask formats and mask definition transferred and imported into data masking automatically Ready for implementation 22

23 Data Subsetting 23

24 Data Subsetting Key Concepts Sensitive Data Identification Application Data Modeling Subset Parameters Mask format Referential Relationships Data Extraction Clauses Data Masking Definition Data Subset Definition 24

25 FY 2009 FY 2010 FY 2012 FY 2013 Subsetting Production Data America North America Europe Europe Asia 30% 10% 360M 120M Criteria based Fiscal Year Geographical Region Storage goal based Percentage of production size Target storage footprint 25

26 Criteria Based vs. Storage Goal Based Subset Tradeoffs Criteria-based Storage Goalbased Advantage Subsetted data consistent with specified data extraction criteria Simple to specify, e.g. 10% or 120GB. Disadvantage Application meta data knowledge is a prerequisite Extracted data may not fit into target system without trial-and-error. Inconsistent data provisioned in subset Not able to support realistic testing 26

27 Criteria-based subset: Process Time (FY:2011) Dimension (Region:Asia) Space (Size:10%) Select application Automatically select the tables needed to be included in subset Define extraction criteria Automatically traverse down relational hierarchy to identify rows Specify subset parameter Analyzes table statistics to estimate size of the database generated from the specified subset parameter 27

28 Criteria based: Specifying Table Types PO_VENDORS AP_INVOICES FISCAL YEAR REGION MFG_PARTS AP_INVOICE_ LINES BUSINESS UNIT SALES_ FACT PRODUCT Table sizes are a function of their table type CHANNEL Large tables: Transaction, Fact Small tables: Dimension, Lookup, Reference Large Tables: Subset criteria specified including on their descendants Small Tables: No subset criteria specified; tables brought over in full 28

29 Test System Provisioning 29

30 Data Masking and Data Subsetting Integrated with all methods of database provisioning Data Pump Export/Import Data Masking + Data Subsetting RMAN copy Thin Provisioning-based Cloning Backup / restore 30

31 Database (Thin) Cloning Using Copy on Write Production Database NAME SSN SALARY AGUILAR ,000 BENSON ,000 Masking & Sub setting Test Master Database NAME SSN SALARY SMITH ,000 MILLER ,000 Database Cloning in Minutes Leverage storage copy on-write technologies Initial Support for Sun ZFS Storage and Netapp Storage More than Just Flexclone Integrated with Data Management(Masking & Subsetting) Configuration Management (Lineage and Association Tracking) Storage Aware Metering & Chargeback Cloud users can create their own snapshots and time travel to an earlier snapshot Functional Copies User can create multiple copies for functional testing without consuming space 31

32 At-Source Data Masking Maximum Compliance with PCI DSS Production Before Test Production NEW Masked Data Pump File Test Clone-and-Mask Production data is copied to Test and then masked leaving sensitive data vulnerable until it s masked At-Source-Masking Sensitive data is desensitized and deidentified at the source before it leaves the production DB; production DB remains unchanged. 32

33 Integrated Subsetting and Masking High Performance Secure Test System Provisioning Before NEW Production Test Production Masked Data Pump File Test Data Subset Clone and Mask One-step Subset and Mask Production data had to be subsetted first and then sensitive data masked separately. Production data is subsetted and sensitive data masked in one step using At-source Masking 33

34 High Performance Data Masking and Subsetting Performance benchmarks on Exadata X2-2 full rack Mask 600 billion row table in 33 minutes 1% subset of 100TB table in 6.5 hours 1% subset + masking of 110TB table in 5.8 hours 34

35 Oracle Differentiators Centralized comprehensive secure test system provisioning User-extensible library with sophisticated masking techniques Integrated with application management lifecycle One tool for provisioning, monitoring, patching for entire Oracle application infrastructure (application to disk) Integrated with testing tools Optimized for Oracle Applications Application security built by the application development teams High performance database-optimized execution 35

36 Customer Case Study: SquareTwo Financial 36

37 ABOUT US WE ARE AMONG THE TOP DISTRESSED ASSET BUYERS IN THE UNITED STATES. SquareTwo Financial is dedicated to making distressed assets whole. Our sole focus is accelerating financial asset recovery through industry-leading asset modelling, award-winning technology and our pioneering legal Partners Network, which works with consumers to remedy their outstanding debt, while fully protecting the clients brand. Founded in 1994 Based in Denver, Colorado Manage over $9.5 billion in assets Award-winning proprietary technology Pioneering attorney-based Partner Network A+ BBB rating Public company (trade bonds)

38 BUSINESS CHALLENGES eagle is our internally developed financial services collection platform. eagle is responsible for housing 15.9 Million accounts. These accounts have a number of types of NPI data such as Social Security #, Address, Date of Birth, Credit History and so forth. The combination of the growth in accounts housed in eagle and the move to an Agile development methodology with two week sprints created development and testing delays to maintain compliance. In addition, the in-house tool was serial and was not able to multi-thread or utilize multiple cores of a CPU. Once specific fields were masked or redacted the data structure relationship structures were destroyed.

39 BUSINESS CHALLENGES Increased compliance regulation Consumer Financial Protection Bureau [CFPB] PCI-DSS Sarbanes-Oxley HIPAA-Hitech Fast paced business growth 37% Year-over-Year Business Growth Expansion into new Financial Services verticals such as Commercial and HealthCare Business requirement to support the continued momentum and market share growth Agile two-week development cycles In order to aid in the momentum and growth of SquareTwo and our eagle Financial Services Platform Development adopted an Agile approach to development. Legacy Data masking tool issues Growth in database size to 3 TB required day lead time to create a masked copy database Move to 2 week development sprints made it difficult to provide Masked Development and Testing environments Legacy tool was developed in house and was not designed to scale vertically making it unable to use multiple-cpu cores

40 EVALUATION CRITERIA Application aware: Whatever solution must mask or redact specific all NPI data while leaving data structure relationships intact. While also not adding significantly to the DBA workload. High Performance: Creating new Development and Testing environments using the in-house tool currently required day lead time to munge and replicate the eagle account database structure (3 TB). Cost: The initial estimate to move development resources to update our in house application and slate scheduled development work for a later date would cost SquareTwo Financial around $300k

41 WHY ORACLE Cost Ease of Implementation Trusted Partners Scalability Development time vs. Purchasing off the shelf Integration with Exadata platform Time to implement and configure Familiar Partner SquareTwo Financial and Oracle have been strategic partners for over 10 years SquareTwo Financial was the first to install a full Exadata and Exalogic Frame New solution must support multithreading and multi-core architectures Solution must cut down the time to replicate environments (14-15 days for 3 TB of data) CONSIDERATIONS FOR CHOOSING ORACLE

42 BUSINESS BENEFITS Implementation: SquareTwo Financial was able to have Oracle aid in the implementation of the Data Masking product. Since SquareTwo Financial already knew the data definitions and areas of concern, the installation of the Data Masking suite was completed in two weeks. Outcome: Once installed, the DBA team was able to replicate the 3TB eagle account database in 15 hours. This directly lead to improved Development and QA testing support. Leading to an average of 6 active Development and Testing development environments for two separate sprint teams.

43 BUSINESS BENEFITS Two week initial roll out of Data Masking solution Move from Day Database Refresh to 15 hour 96% improvement! Improved support for the QA and Development teams during their two week sprint cycles

44 BEST PRACTICES Understand what your business requirements are based on the information stored in your database and what your compliance requirements are. (e.g. Do you have Credit Card or Tax ID stored in your database?) Know what you want and have to mask based on legal and compliance requirements. Map this to your dataset before starting your masking project. Do NOT under estimate the value of existing partners, while a new partner maybe able to offer a lower price, there is always value with the partner you share more eggs with. Understand the difference between data masking and data subsetting and which of these functions you need. Some vendors include this in one tool, Oracle today does not.

45 Oracle Database Security Solutions Defense-in-Depth for Data Protection and Regulatory Compliance ADMINISTRATIVE Data Discovery and Classification Database Lifecycle Management Vulnerability Scanning DETECTIVE Activity Monitoring Database Firewall Auditing and Reporting PREVENTIVE Masking, Encryption, Redaction, Separation of Duties Multi-Factor Authorization 45

46 Additional Oracle Enterprise Manager Sessions Technical Sessions and Hands-on Labs Time Session Title Location Tuesday 1:15pm-2:15pm Database Lifecycle Management with Enterprise Manager 12c Moscone West :00pm-6:00pm Deep Dive 3D on Exadata Management: From Discovery to Deployment to Diagnostics Moscone West 3018 Wednesday 11:45am-12:45pm Database as a Service using Oracle Enterprise Manager 12c Moscone West :15pm-2:15pm DB Performance Tuning Boot Camp: 10 New Problem-Solving Tips Using ASH & AWR Moscone South 103 3:30pm-4:30pm SQL Tuning Experts Panel: Your Questions on Explain Plan to Exadata Answered Moscone South 102 Time Hands-on Lab Location Tuesday 1:15pm-2:15pm Database Performance Tuning Hands-on Lab Marriott Marquis - Salon 5/6 Wednesday 10:15am-11:15am Database Performance Tuning Hands-on Lab Marriott Marquis - Salon 5/6 11:45am-12:45pm Database Performance Tuning Hands-on Lab Marriott Marquis - Salon 5/6 Thursday 12:45 pm-1:45pm Database Performance Testing, Data Masking, and Subsetting Hands-on Lab Marriott Marquis - Salon 5/6 46