Voltage SecureData Enterprise SQL to XML Integration Guide

Transcription

1 Voltage SecureData Enterprise SQL to XML Integration Guide Jason Paul Kazarian (408)

2 2/18 SQL to XML Integration Guide Copyright 2018, Micro Focus. All rights reserved. This documentation is provided as is without warranty. Customers and prospects assume the entire risk of using the information provided in this guide. This guide makes reference to software source code used for building a user defined function compatible with the PostgreSQL C API. Customers licensed to use Voltage software, either for evaluation or production, may obtain a copy of this software free of charge. Likewise, this software is provided as is without warranty.

3 SQL to XML Integration Guide 3/18 Table of Contents 1.0 About this document Scope Purpose Audience and Resources About Voltage SecureData Enterprise What is Voltage SecureData Enterprise? What is the Simple API? What are Voltage SDE SOAP Web Services? Integration considerations Using SOAP web services from Java Configuration parameters Protection properties Database connection properties Query properties Execution A simple example Format selection Client policy Format selection Selection example Integration solution installation Solution overview Privileged installation tasks Create the deployment directory structure Create an ordinary user Install the latest Java Software Development Kit (SDK) Install a database server Standard installation tasks Load test MySql database tables Install Apache Tomcat Install the Simple API for Java Compile and install the supporting Java libraries Run the integration wizard Integration wizard walk through Start the wizard Property collection Protection properties collection Protection properties verification Writing the protection properties file Database connection and query properties Execute query Normal Operation Logged Output... 17

4 4/18 SQL to XML Integration Guide 7.1 Policy parsing Database querying Applying data protection... 17

5 SQL to XML Integration Guide 5/ About this document This document is a guide to integrating the Voltage SecureData Enterprise (SDE) product family with a Structured Query Language (SQL) to Extensible Markup Language (XML) conversion tool assisting Business Intelligence (BI) platform integration. The tool is published as an Apache Tomcat Java 2 Enterprise Edition (J2EE) application. 1.1 Scope Voltage SDE is a product family offering application-level data protection. As such, Voltage SDE provides virtual servers, command line file processing tools, an Application Programming Interface (API), and Simple Object Access Protocol web services, enabling protection for sensitive data within an application. As infrastructure with a general purpose, client-side software API and file processing tools for deployment and integration into a variety of platforms, Voltage SDE may be integrated by customers into their own applications and data processes. Voltage SDE provides ready-made tools for a variety of platforms. For platforms and systems not directly supported, for example many BI tools, customers may create, own, manage and update integrations using the Voltage Simple API native client library as well as the web services interface. This conversion tool is an example of such an integration. Voltage Security provides unsupported source code, including the Java and Java Server Pages (JSP) source provided with this guide, on an unwarranted, as-is basis to licensed Voltage SDE customers for the express purpose of providing an integration example. This code is neither supported nor warrantied. Licensees assume full risk of use. 1.2 Purpose This document defines the steps necessary to deploy the example Tomcat application using the MariaDb database. Developers must adapt these steps and test the results for a specific environment. 1.3 Audience and Resources This document s primary audience is developers writing Voltage SDE applications. System architects, security officers, and others with an interest in SDE data protection techniques may also find this document useful. The following documents from the Voltage SDE product family may be useful as well: Voltage SecureData Administrator Guide Voltage SecureData Web Service Developer Guide Voltage SecureData Simple API Developer Guide Please contact your Voltage Security account manager or sales engineer to verify you are using the latest version of this integration.

6 6/18 SQL to XML Integration Guide 2.0 About Voltage SecureData Enterprise This chapter discusses basic concepts behind the Voltage SecureData Enterprise (SDE) product family and its various components. We assume familiarity with these concepts as a prerequisite for performing the integration tasks documented in the following chapters. For more details, please consult the references identified in Section 1.4, Developer resources. 2.1 What is Voltage SecureData Enterprise? Voltage SDE is a product family consisting of several components, specifically: Services that may be deployed on one or more servers depending on performance needs. An Application Programming Interface (API) that interacts with these services, specifically the key manager, to perform format-preserving encryption (FPE) within the application (in this example a Tomcat J2EE application). A Simple Object Application Protocol (SOAP) web service to perform both FPE and secure stateless tokenization (SST) entirely on a partitioned server, with no Voltage SDE code running on the client itself. Unsupported, unwarranted example code that application developers may use as an example for implementing a data protection integration project. We discuss these components and general concepts in the following paragraphs. 1.1 What are the Voltage SDE servers? The Voltage SDE product family includes multiple servers: a web management console, an identitybased, stateless (dynamic) key server, an event monitor, and a health monitor. Optionally, a web services server to perform SST is available. These services may all run on a single real or virtual server, and this configuration is known as a singlebox deployment. This type of deployment is especially useful for non-production environments, including development and testing. In addition, the services may deployed on multiple servers to support real-world production workloads. For example, the management console may be deployed on one server, the stateless key server on another, and the web services server on yet another. The actual number of services deployed per physical or virtual instance varies based on performance requirements. Note that two Voltage SDE services are stateless: key management and tokenization. The term stateless in these contexts means except for the master secrets, there is no storage of cryptographic material. Rather this material, be it a key or token, is computed on the fly dynamically using a set of input data, including the identity of the application or person requesting the material. Consequently the Voltage SDE suite stores neither keys nor computed tokens. And some operations such as key rotation are performed by the suite automatically. Since these components and operations are dynamic, assessing, auditing, and deploying data protection is far less complex and costly than the costs associated with solutions that require tracking state information ( state-full solutions). 2.2 What is the Simple API? Voltage SDE provides an API allowing developers to implement identity-based encryption using Voltage FPE on the client (or specifically referencing this integration, the Tomcat server). The API obtains keys from a server, but executes encryption algorithms directly. As of this writing, C, C++, and

7 SQL to XML Integration Guide 7/18 Java on AIX, Voltage Nonstop, Voltage-UX, Linux, OS X, Solaris, Stratus VOS, and Windows are supported. Additionally, a toolkit API is available for custom development purposes. In general, developers use the Simple API by performing the following steps: Per process activation steps: o Create a LibraryContext object storing general parameters for all encryption operations. o Create an FPE object storing parameters for a specific encryption format: credit card, , identification number, and so on. Per data item steps: o Call a protect data function to de-identify sensitive data using FPE prior to storage. o Call an access data function to re-identify this data only when absolutely necessary. Note the Simple API provides FPE services only. To protect the integrity of tokenization and avoid using any Voltage SDE code on the client, SST is available only via web services as described below. Note FPE operations are also available via web services for security organizations wishing to limit the distribution of cryptographic algorithms only to approved platforms. 2.3 What are Voltage SDE SOAP Web Services? Voltage SDE provides SOAP web services for encrypting (via FPE) and tokenizing (via SST) sensitive data items without using any Voltage SDE code on the requesting client. In this case parameters are passed via SOAP to a server. The server processes the data and returns the result to the requester, again via SOAP. Since the operations are entirely network-based, they may be integrated with a web application at the network level. 1 Note that within the Voltage SDE management console, separate identities are provisioned for the Simple API and SOAP web services. To allow a user access to both services, the corresponding identity must be provisioned twice. See the Authentication Overview and the Web Service Overview in the Voltage SDE Web Service Developer Guide for details. 1 Contact your Voltage SecureData sales engineer for a reference implementation of HTTP proxy request and response integration.

8 8/18 SQL to XML Integration Guide 3.0 Integration considerations This chapter discusses considerations for compiling and deploying a Tomcat J2EE application using Voltage SecureData. Installers may find this chapter useful for providing background on the integrated solution. 3.1 Using SOAP web services from Java Since Tomcat uses J2EE for its application architecture, we use Java to call a web services server via SOAP. The Java Development Kit (JDK) includes the wsimport command for converting a Web Services Description Language (WSDL) URL into Java classes, for example: wsimport keep verbose The above command ingests the WSDL URL, generates corresponding Java classes, and compiles those classes. The output is bundled into a Java Archive (JAR) library file for use with Tomcat. We have provided a sample Voltage SecureData WSDL source file called VibeSimpleSoap-wsd.xml with this kit as an integration aid. We highly recommend replacing this file with a similar file generated from a production web services server. 3.2 Configuration parameters The integration wizard generates configuration parameter files based on user input. One may manually modify these parameters using the following criteria: Protection properties POLICY_URL: The URL for the client data protection policy. WSDL_URL: A web services URL location to the web services server. TRUST_STORE_DIRECTORY: The on-disk location of the Simple API certificate store. KEY_IDENTITY: The authentication identity used by the stateless key manager. Different identity strings are given access to different keys. AUTH_TOKEN: The shared secret. This application supports shared secret authentication Database connection properties JDBC_URL: The URL for connecting to a MariaDb database server via JDBC. DATABASE_USER_NAME: The MariaDb login name associated with the application. DATABASE_PASSWORD: The MariaDb user password needed to make a JDBC connection Query properties SQL_STATEMENT: A valid SQL statement that will return a ResultSet via JDBC. PROTECT: A parameter that determines if the output XML file is protected (encrypted, tokenized) or re-identified (decrypted, detokenized). XML_OUTPUT_FILE_NAME: The name of the file for writing the output of the SQL statement.

9 SQL to XML Integration Guide 9/ Execution This Tomcat application includes an integration wizard that offers step-by-step assistance in collecting the previously listed configuration parameters as well as executing the supplied SQL query and returning a result set translated into XML format. In this chapter we explain how we apply Voltage SDE data protection operations to the result set. 4.1 A simple example Consider the following SQL query: SELECT name FROM customers LIMIT 5; This query lists the first five name columns in the customers table. Now consider the following query: SELECT name AS alphanumeric_name FROM customers LIMIT 5; This query does the same while changing the output column label. Now consider the following statement: if (columnlabel.tolower().beginswith(sdeformatname.tolower()) This if statement is true when the column label in a result set starts with the same characters as an SDE data protection format name. Note letter case is ignored. Since we can perform this match, we can select an appropriate data protection format for each column in a result set by matching format names as prefixes to column labels. We can then protect or access each element in the result set. 4.2 Format selection The following sections describe how this Tomcat application selects data protection formats based on column labels. We provide these details to help aid the modification and use of this application example Client policy The Voltage SDE management console publishes a policy containing encryption and tokenization formats at the following URL: The Tomcat application downloads this policy at startup to determine what formats are provided by the Voltage SDE network. The application maintains these formats in two lists: one for tokenization and one for FPE Format selection Using the format name lists, the application selects formats based on the following criteria: Tokenization precedence: The tokenization list is checked for matches before the FPE list. Format order: The application selects the first format name matching a given column name as a prefix based on order of appearance in the client policy. Failure ignored: If the application cannot find a match, it does not apply data protection. The following section provides examples of how this matching algorithm operates in practice.

10 10/18 SQL to XML Integration Guide Selection example Consider a Voltage SDE network with the following format definitions: Tokenization Formats o CC-ST-64O FPE Formats o LowerCaseAlphaNumeric o UpperCaseAlphaNumeric o AlphaNumeric o ORA-DATE o CC o SSN o CC-ST-64O o SSNO Now consider the following SQL query: SELECT name AS alphanumeric_name, cc AS 'cc-st-64o', customers.ssn AS ssno, date FROM customers, loyalty WHERE loyalty.score > 848 AND loyalty.ssn = customers.ssn; The following table describes how the data protection formats are auto-selected: Column Label Format Selected Reasoning name alphanumeric_name AlphaNumeric FPE Only prefix match: others match suffix cc cc-st-64o CC-ST-64O SST Tokenization takes precedence (otherwise CC would be selected) ssn ssno SSN FPE SSN appears first in policy date date null No prefix match: only suffix match Process flow This is the general process flow of the SQL to XML application: Create a LibraryContext using data protection properties, including authentication. Parse the client policy to determine installed protection formats. Execute a database query. Visit every row in the result set and apply data protection based on the matching format. Write the modified result set to an XML file. Appendix A provides an example user session. Appendix B documents the debugging information the application writes to the $CATALINA_HOME/logs/catalina.out file.

11 SQL to XML Integration Guide 11/ Integration solution installation This chapter documents the installation of the Voltage SDE SQL to XML J2EE application. We present details for Linux systems. Those installing this solution on Windows should follow the same process, but tweak the details for that particular environment. 5.1 Solution overview The integrated solution has multiple installation tasks. The first set of tasks must be performed as a privileged or root user. The second set of tasks may be (and perhaps should) be performed by an ordinary user. We present the details for each of these phases in the following sections. 5.2 Privileged installation tasks Perform the steps in this section as the root user Create the deployment directory structure Create the following deployment directory structure: /opt/java /opt/voltage /opt/voltage/config /opt/voltage/simpleapi /opt/tomcat The source code supplied depends on the above structure. However one may modify the source code to match a different deployment directory structure if desired Create an ordinary user Create an ordinary Linux user to perform non-privileged tasks, for example: useradd sqlxml_devel passwd sqlxml_devel Grant this user full access (RWX) to the deployment directory structure chown -R sqlxml_devel:sqlxml_devel /opt/java /opt/voltage /opt/tomcat Install the latest Java Software Development Kit (SDK) We recommend removing all old versions of Java and installing the latest Java SDK. On CentOS systems, the following commands should suffice: yum remove java openjdk yum remove java openjdk yum install java openjdk-devel Modify these commands to suit the environment. One may, for example, use the Oracle JDK instead Install a database server Install the MySql or MariaDB database server and start its corresponding daemon, for example: yum install mysql-server service mysqld start chkconfig mysqld on

12 12/18 SQL to XML Integration Guide Edit the initdb.sql script provided with this kit as appropriate. Use this script to create a test database and database user. Finally, logout as root and login as the ordinary user before proceeding. 5.3 Standard installation tasks Perform the installation steps in this section as an ordinary (non-root) user Load test MySql database tables Read the initdb.sql script to determine the test database user name and password. Load the test tables with the Comma Separated Values (CSV) files in the Sample Data directory. The mysqlimport utility is most useful for this purpose Install Apache Tomcat Install Apache Tomcat using /opt/tomcat as CATALINA_HOME. We tested this solution with Tomcat 9. Other versions should work. Create the /opt/tomcat/bin/setenv.sh file containing the following statement: export JAVA_OPTS= -Djava.library.path=/opt/tomcat/lib Save this file. Finally download the MySQL JDBC connector to the /opt/tomcat/lib directory Install the Simple API for Java Install the Simple API for Java in /opt/voltage/simpleapi following the standard practice. Be sure to follow the directions provided in the installation guide for adding security certificates to the trust store. Finally, copy the entire contents of /opt/voltage/simpleapi/lib to /opt/tomcat/lib. 1.2 Install the Tomcat web application Create the /opt/tomcat/webapps/sqlxml directory for deploying the SQL to XML web application. Copy everything in the Integration Wizard folder to this directory. Then run the following command: /opt/tomcat/bin/startup.sh Test that the application loads by browsing to the appropriate URL, for example: The wizard s welcome page should load Compile and install the supporting Java libraries Copy everything in the Java Source folder of the solution archive to the /opt/java directory. Grant execution rights to the compile.sh script: chmod a+x./compile.sh Run the compile.sh script, which will perform the following tasks: 1. Stop Tomcat. 2. Compile the JAX-WS interface. 3. Compile the Java objects used by the solution. 4. Built a Java Archive (JAR) file containing all of these objects. 5. Copy this custom JAR file to the /opt/tomcat/lib directory. 6. Change to the /opt/tomcat/webapps/sqlxml directory.

13 SQL to XML Integration Guide 13/18 7. Start Tomcat. The integration wizard is now ready to run, which will be the next installation phase. 5.4 Run the integration wizard Start the wizard by browsing to its web application directory on port 8080, for example: The wizard walks you through the configuration process step-by-step. The parameter names are plain English variants of those presented in Section 3.2, Configuration parameters. The following section, Appendix A: Integration wizard walk through, provides an example of how to use the wizard. Modify this example to suit your particular environment.

14 14/18 SQL to XML Integration Guide 6.0 Integration wizard walk through The following screen shots illustrate how to use the installation wizard. Only the success flow is illustrated: error flows will proceed differently than what is show below. 6.1 Start the wizard Browse to the wizard s web application directory, for example to load the welcome page: Proceed by clicking the Next button. 6.2 Property collection The wizard will prompt for the system parameters defined in Section 3.2, Configuration parameters. There is one configuration screen for each of the parameter groups. Once one completes a parameter group, settings are stored for future use. WARNING: This application stores authentication tokens (passwords and shared secrets) as plain text values. This is an example only configuration, not a production suggestion. The following page has a screenshot of the Protection Parameters collection screen. When you are satisfied these properties are correct, click the Submit Protection Properties button.

15 SQL to XML Integration Guide 15/ Protection properties collection 6.4 Protection properties verification The next wizard step allows one to proof the properties entered are correct and also verify the properties file will be written to the correct disk location. If satisfied, click the Next button. Otherwise, click the Edit Protection Properties button to revisit the collection page. A facsimile of this page is shown in the following screen shot: 6.5 Writing the protection properties file The next step is to write the protection properties file. The success case is shown here:

16 16/18 SQL to XML Integration Guide Click the Next button to proceed with script generation. 6.6 Database connection and query properties The wizard runs two additional sets of collection, verification, and writing tasks: the first set is for database connection properties and the second is for query properties. These work in a fashion similar to how the data protection properties operate. After submitting properties for the first time, one can use the wizard to display or edit these property groups. Authentication tokens are not displayed. 6.7 Execute query After completing property collection for the protection, database connection, and query groups, the wizard presents the Execute Query screen: This screen allows one to verify the query is correct as well as edit any of the properties collected so far. Clicking the Next button results in the following Query Results screen: One may now download the generated XML file or run the integration wizard a subsequent time.

17 SQL to XML Integration Guide 17/ Normal Operation Logged Output The following sections document what is logged to the catalina.out file during normal operation. 7.1 Policy parsing The first phase of processing is downloading the client policy to determine what FPE and tokenization formats exist in the Voltage SDE network: Loading properties database. Constructing objects. Parsed XML document with root element clientpolicy Found 11 items for the policy section named FormatMappings. Found 1 items for the policy section named TokenizationFormats. Tokenization Formats Found 1 items for the policy section named TokenizationFormats. CC-ST-64O Finished listing tokenization formats. FPE Formats Found 11 items for the policy section named FormatMappings. LowerCaseAlphaNumeric UpperCaseAlphaNumeric AlphaNumeric US7ASCII-PRINTABLE SFS ORA-DATE Salary CC SSN CC-ST-64O OSSN Finished listing FPE formats. 7.2 Database querying The next phase is to execute the query against the database. Querying the database. Executing sql statement SELECT name AS alphanumeric_name, pan AS 'cc-st-64o', fk AS ossn FROM customers, loyalty WHERE loyalty.score > 848 AND loyalty.ssn = customers.fk Protecting the result set. 7.3 Applying data protection The final phase is to match protection formats with column labels and perform the protection operations. Column 1 tokenization format name is [null] Column 1 data protection format name is [AlphaNumeric] Column 2 tokenization format name is [null]

18 18/18 SQL to XML Integration Guide Column 2 data protection format name is [CC] Column 3 tokenization format name is [null] Column 3 data protection format name is [OSSN] Finished loading protection properties. Values are: Policy URL = sqlxml.voltage.com/policy/clientPolicy.xml WSDL URL = sqlxml.voltage.com/vibesimple/services/VibeSimpleSOAP?wsdl Trust Store = /opt/voltage/simpleapi/truststore Key Identity = sqlxml@voltage.com Restart Tomcat to reload updated values from a new integration wizard session. Loading LibraryContext version Simple API Able to read /opt/voltage/simpleapi/truststore as the trust store directory. singletoncontext initialized. fpelist initialized. Applied Voltage on 30 rows. Writing XML to protect.xml. Columns in data set: 3 Rows in data set: 30 Done! SQL to XML conversion completed successfully.