Analyzing Patent Data in Dell Quickstart Data Warehouse Appliance Using Toad Business Intelligence Suite

Transcription

1 Analyzing Patent Data in Dell Quickstart Data Warehouse Appliance Using Toad Business Intelligence Suite A Dell technical white paper by Mayura Deshmukh and Umesh Sunnapu

2 This document is for informational purposes only and may contain typographical errors and technical inaccuracies. The content is provided as is, without express or implied warranties of any kind Dell Inc. All rights reserved. Dell and its affiliates cannot be responsible for errors or omissions in typography or photography. Dell, the Dell logo, Boomi, and PowerEdge are trademarks of Dell Inc. Intel and Xeon are registered trademarks of Intel Corporation in the U.S. and other countries. Microsoft, Windows, SQL Server, Excel, and Windows Server are either trademarks or registered trademarks of Microsoft Corporation in the United States and/or other countries. Other trademarks and trade names may be used in this document to refer to either the entities claiming the marks and names or their products. Dell disclaims proprietary interest in the marks and names of others. April 2013 Rev 1.0 ii

3 Contents Introduction... 5 Looking at Real World USPTO Patent Data... 6 Toad BI Suite Overview... 6 Connecting to Data Sources... 6 Filtering Data and Creating Views/Snapshots... 8 Toad Intelligence Central Toad Decision Point More Analysis of the USPTO Patent Data for Conclusion iii

4 Figures Figure 1: Data Visualization Using Toad BI Suite... 5 Figure 2: Toad Data Point: Navigation Manager... 7 Figure 3: Toad Data Point: Object Explorer... 7 Figure 4: Toad Data Point: Connection Properties for Toad Intelligence Central... 8 Figure 5: Toad Data Point: Query Builder... 9 Figure 6: Toad Data Point: Query Builder with Where Condition Figure 7: Toad Data Point: Query Builder with Query Figure 8: Toad Data Point: Query Builder with Results Figure 9: Toad Data Point: Publish Result as Snapshot to Intelligence Central Figure 10: Toad Data Point: Access to Snapshot Figure 11: Toad Data Point: Successfully Published Snapshot to Intelligence Central Figure 12: Toad Intelligence Central Admin Console Figure 13: Toad Decision Point: Add Dataset Figure 14: Toad Decision Point: Finding the top 10 Patent Producing Companies for Figure 15: Toad Decision Point: Top 10 Aggregation Figure 16: Toad Decision Point: Top 10 Patent Producing Companies for Figure 17: Toad Decision Point: Chart View Figure 18: Toad Decision Point: Chart View for Top 10 Patent Producing Companies for Figure 19: Toad Decision Point: Chart View for Top 10 Patent Producing Companies for Figure 20: Top Patent Categories for a Given Company for 2012 (Dell) Figure 21: Maximum Time Before a Patent is Awarded for 2012 Per Patent Type Figure 22: List of 10 Patents for 2012 Which Took Maximum Time to be Awarded Figure 23: Average Time for the Patent to be Awarded Figure 24: Top 10 Patent Authors for Figure 25: Types of Patents Most Filled in Figure 26: Patents Awarded for Each Type Every Month per Quarter for Figure 27: Duplicate Records iv

5 Introduction Processes such as business intelligence, data analysis, and data mining help organizations get more insights into their data. Using such tools, business users can get answers to questions critical to their business and business executives can make more informed and better decisions to guide their businesses. For very small businesses, maintaining and analyzing the data using Microsoft Excel would work. However, a growing organization needs a more evolved solution. Dell provides an end-to-end solution that addresses the business intelligence (BI) requirements of a growing organization. Dell Quickstart Data Warehouse Appliance (QSDW) is a bundled data warehousing solution that includes all the hardware (Dell 12G server), software (Windows Server 2008 R2 SP1, SQL 2012, Boomi for ETL, Toad BI Suite for Analytics) and implementation services required for BI. The Business Intelligence on Dell Quickstart Data Warehouse Appliance Using Toad Business Intelligence Suite white paper discusses the hardware architecture, QSDW, and installing and integrating all the components within the Toad BI Suite software. This white paper discusses how to analyze real world patent data that is stored in the QSDW using Toad BI Suite. It describes the process to build reports using Toad BI Suite: Toad Decision point, Toad Data point, and Toad Intelligence Central. Figure 1: Data Visualization Using Toad BI Suite 5

6 Looking at Real World USPTO Patent Data To validate the Dell end-to-end BI solution, we simulated building a data warehouse and business intelligence solution. The dataset used for the project simulation is patent data. This dataset was chosen because it is very complex in terms of various formats, duplicate records, inaccuracies, and so on, and accurately reflects a real time scenario. The patent data is also publically available. This white paper will exemplify how to find the list of top ten organizations that had the most patents granted for the year To know more about the project and read more about the process of extracting, cleansing, inserting patent data, see The Dell Business Intelligence Project Using USPTO Data. Toad BI Suite Overview Toad BI Suite enables you to find and extract meaningful patterns and useful information from the data warehouse, and also provides tools to simplify data management and administration. It consists of three products: Toad Intelligence Central, Toad Data Point, and Toad Decision Point. To understand what role each of the Toad BI Suite components play and how the integration between each component can be performed, see Business Intelligence on Dell Quickstart Data Warehouse Appliance Using Toad Business Intelligence Suite. Toad Data Point Modeling tools are used to build various views or snapshots. Smaller datasets are easier to work with, so only the required data is selected, filters are applied, and the results are published to Intelligence Central as snapshots or views. These snapshots or views are shared with the business users, who work on the datasets to build reports, charts, and find patterns. The following sections describe how to build views or snapshots, publish them to Intelligence Central, and then build reports or charts using these snapshots in Decision Point. Toad Data Point Toad Data Point provides various data modeling tools for data analysts. It also includes visualization tools and cleansing tools. This paper explores few of the basic tools required to create the report for Top 10 patent producing companies for Connecting to Data Sources To create a connection to the data warehouse to access patent data and to create a connection to Intelligence Central to publish the snapshots created in Toad Data Point, see Business Intelligence on Dell Quickstart Data Warehouse Appliance Using Toad Business Intelligence Suite. The connections will be visible in the Navigation Manager as shown in Figure 2. 6

7 Figure 2: Toad Data Point: Navigation Manager Click on the database name to see all the tables, views, indexes in the Object Explorer as shown in Figure 3. Figure 3: Toad Data Point: Object Explorer 7

8 Note: For version 3.3 of Data Point, an error occurs when publishing a snapshot to Intelligence Central if the SQL Server ODBC driver is entered. So this field is left blank as shown in Figure 4. Figure 4: Toad Data Point: Connection Properties for Toad Intelligence Central Filtering Data and Creating Views/Snapshots 1. Right click on each table under the database and select the Query Builder option. This option adds the selected table into a separate window. Note: You can add multiple tables to Query Builder if the data filtering is to be done across multiple tables. 2. To find the Top 10 patent producing organizations for 2012, add Application, Inventor and Mapping tables to Query Builder. Application table contains all data related to the granted patent. Inventor table contains all data related to the inventor and the company the inventor works for. 8

9 Note: For version 3.3 of Data Point, Application is a considered a keyword and an error occurs when publishing data from the Application table to Intelligence Central. A view called Patent_App was built to be used in the query. You also have an error if there is any [ in the query. 3. Select the necessary columns as shown in Figure 5. Smaller datasets are easier to work with. So the dataset is aggregated and only the required data is selected after applying all required filters. Figure 5: Toad Data Point: Query Builder 4. Figure 6 shows the where condition for the Orgname field in the inventor table. Rows which do not have Organization name are removed from the result set. Query is automatically generated in the Query tab as shown in Figure 7. 9

10 Figure 6: Toad Data Point: Query Builder with Where Condition Figure 7: Toad Data Point: Query Builder with Query 10

11 5. Click on (Execute SQL). The results are displayed in the result tab. 6. Right-click on the results page, select Send To Publish option to save the results to Toad Intelligence Central. Figure 8: Toad Data Point: Query Builder with Results 7. For destination, select Intelligence Central connection. 8. Enter the credentials for the data source and select object type as Snapshot. See Figure 9. Note: It is necessary to have a connection established between Toad Data Point and Toad Intelligence Central to publish the results as a snapshot. 11

12 Figure 9: Toad Data Point: Publish Result as Snapshot to Intelligence Central 9. On the sharing link, you can set the security of the snapshot by selecting different options such as share this data with any user/selected users or keep this data private as shown in Figure 10. These snapshots or views are shared with the end users, who will be working on the dataset to build repots, charts, find patterns. Note: Users must already be created in Toad Intelligence Central. 12

13 Figure 10: Toad Data Point: Access to Snapshot 10. Click the Apply on the Sharing Settings screen and then the Publish button to publish Snapshot to Intelligence Central as shown in Figure 11. Figure 11: Toad Data Point: Successfully Published Snapshot to Intelligence Central Toad Intelligence Central Published snapshots can be viewed in Toad Data Point: Object Explorer by selecting the Intelligence Central data source in the Navigation Manager or by logging on the Administrator Console of Toad Intelligence Central. See Figure

14 Figure 12: Toad Intelligence Central Admin Console Toad Decision Point 1. Connect to the Toad Intelligence Central using Toad Decision Point and import the snapshot created previously. For more details on how to create a connection, see Dell end-to-end business intelligence solution using Toad BI suite and Quickstart Data Warehouse Appliance. 2. Import the dataset using the Add Data link as shown in Figure For finding the top 10 companies, select the ApplicationId which gives the number of patents and the organization name. 14

15 Figure 13: Toad Decision Point: Add Dataset 4. Drag and drop the column OrgName and NumberofPatents in the work area of the Dimensional Data Viewer. 5. You can apply filters or merge the dataset with other datasets using the Add Step link. You can also group data using the Dimensional Data view and look at the details of data in Tabular view. 6. To filter and just display the top 10 patent producing companies, right click on the Orgname and select Filter Dialog as shown in Figure 14. The filter Dialog box is displayed. 15

16 Figure 14: Toad Decision Point: Finding the top 10 Patent Producing Companies for Navigate to the Aggregation tab of the Filter Dialog Box. Select Top 10 as shown in Figure 15. Figure 15: Toad Decision Point: Top 10 Aggregation 16

17 Figure 16: Toad Decision Point: Top 10 Patent Producing Companies for 2012 Figure 17: Toad Decision Point: Chart View 17

18 Figure 18: Toad Decision Point: Chart View for Top 10 Patent Producing Companies for 2012 More Analysis of the USPTO Patent Data for 2012 Using the process described in the previous sections, different aspects of the data were examined to derive useful information and patterns. You can create rollups and easily drill down into the data. Toad Decision Point allows you to copy graphs to your presentations, export the reports in excel or publish them to Intelligence Central to share them with other users. Below are some of the sample reports and charts that were built using Toad Decision Point. 18

19 Figure 19: Toad Decision Point: Chart View for Top 10 Patent Producing Companies for 2012 Figure 20: Top Patent Categories for a Given Company for 2012 (Dell) 19

20 Figure 21: Maximum Time Before a Patent is Awarded for 2012 Per Patent Type Figure 22: List of 10 Patents for 2012 Which Took Maximum Time to be Awarded Patents Days taken to grant patent Plasmids for transforming plant cells 9937 Genetically transformed plants 9827 Monitoring system for a ship's radiated noise 9311 Flight control of missiles 9125 Optical sensing arrangements 9007 Method for amplification and detection of RNA sequences 8362 Polynucleotides comprising intercalating entities 7379 Semiclosed Brayton cycle power system with direct combustion heat transfer Cloned DNA sequences hybridizable with genomic RNA of lymphadenopathy associated virus (LAV) Ligation amplification of nucleic acid sequences

21 Figure 23: Average Time for the Patent to be Awarded 21

22 Figure 24: Top 10 Patent Authors for 2012 Figure 25: Types of Patents Most Filled in

23 Figure 26: Patents Awarded for Each Type Every Month per Quarter for 2012 Figure 27: Duplicate Records Duplicates from the data are removed by a SQL procedure but while building the reports we realized there are more scenarios to be considered for cleansing. As shown in Figure 27, for some patents the Inventor has Middle initial added in the Name whereas for some patents it is just the first name and last name. Even though it is the same inventor, two different records are created in the inventor table and are treated as two different inventors. Similarly, we observed duplicate records for company names. For example, Dell Products L.P., Dell Products, LP, Dell Products, Inc, and so on refers to the same company. 23

24 Conclusion Quickstart Data Warehouse along with Toad BI Suite provides a flexible, high performing platform to enable both data provisioners and data analysts to collaborate around data and visualizations in a secure and controlled environment. It allows consumers within the organization to gain insight into their business more quickly, and to make more informed decisions. For more information on Quickstart Data Warehouse Appliance, see For more information on Toad BI Suite, see 24