Dr. Michael Curry. Oregon. The Big Picture: SQL Overview and Getting the Most from SQL Saturday

Transcription

1 Dr. Michael Curry Oregon The Big Picture: SQL Overview and Getting the Most from SQL Saturday

2 Academic Data Management E-Commerce Entrepreneurship Dr. Michael Curry /michaellcurry/ Consultant Information Security IT Audits Data Analytics Scholar Behavioral Cybersecurity Researcher

3 2 Types of DBs: Analytical & Operational Operational data is used to support the day-to-day transactional needs. Analytical data is used for decision support tasks that also require analysis Primarily used by applications (e.g. ERP) Used for analysis to learn insights

4 Todays talk will cover: How SQL technologies are involved in: Managing operational data Transforming into analytical data Doing BI analysis Skill paths for careers in this field Recommendations for SQL Saturday

5 Introduction The Business need for structured data in operational databases Key SQL concepts and terms: Database, table, PK, FK DBMS, SQL, T-SQL, Stored Procedure The Database Administrator (DBA) job & skills

6 The Business Need for Structured Data Database - Organizes data in a way that facilitates efficient access to the information Database Management Systems - such as SQL Server create, modify and manage DBs Enterprise Resource Planning (ERP) System Payroll System Invoicing System Database Management Payroll System Data Integrated Invoicing Payroll, Data Invoicing, & Inventory Data The Database Administrator (DBA) maintains these DBs using SQL Server Inventory System Inventory Data 6

7 SQL Server & Related Technologies SQL Server Database management software (usually on a server) that stores structured data in databases SQL Database A collection of data tables with attributes (e.g. PK, FK, name, price, etc.) Management Studio (Visual Studio) Program to modify a database; make back-ups, write SQL queries, etc. SQL Structured query language for interacting with DB T-SQL is SQL + extensions for more complex operations Stored procedure is T-SQL saved inside a DB

8 Demo 1 Viewing an operational database using management studio Creating an invoice using queries

9 Management Studio Connected to SQL Server Working with a particular database Query results may be used to create useful reports like an invoice Creating an SQL query for invoice details Management Studio is a powerful environment based on Visual Studio

10 Example Operational DB in SQL Server Table Primary Key Foreign Key This design records invoices, customers, employees, shipments, products and vendors plus a few others Inserting a FK creates a relationship between tables

11 Relational Data Data arranged in 2-D Rows and Columns, using primary & foreign keys to create relationships adds relational structure to data This prevents data duplication And supports efficient SQL query results

12 Job Description and Skills * Database Administrator: maintain integrity and performance of key databases Usually a bachelor s degree in an information- or computerrelated subject with data or database focus Good analytical skills ability to perform complex tasks Strong communicator who works well in teams Detail oriented with good problem solving ability Experience with databases is a plus Entry pay: $67K; Average pay: $84K *Glassdoor DBA Salaries Portland, OR

13 DBA Focused SQL Saturday Sessions Time Session 8:30 9:45 The Big Picture: SQL Overview and Getting the Most of SQL Saturday - Dr. Michael Curry 10:15 11:30 Educating yourself will make a SQL career more fulfilling - Chris Wood 11:45 12:45 3D printing at HP and spreadsheets; how HP broke free of data bottlenecks. - Jeff Honsowetz 1:00 2:15 SQL Server Take Over: Establishing a Baseline - Lowry Kozlowski 2:45 4:00 Containerized databases: SQL Server and DevOps - Ned Otter 4:15 5:30 High Availability/Disaster Recovery Glenn Berry

14 Tips for SQL Saturday Introduce yourself and ask questions What type of work do you do with SQL? Could I ask you to review my resume and give feedback? job shadow you one afternoon? get your advice for someone considering this field? talk to you about a possible job? Follow speakers, read their blogs and comment Add 10 people (not students) to your LinkedIn network

15 Analytical Data The Data Analysts role Turning operational data into Business Intelligence Loading a new analytical database

16 Data Analysts Help Turn Data into BI Business Intelligence is patterns, relationships and trends identified in operational (and other) data to help make better decisions Employee Knowledge BI Social Media Sales Data Ops Purchased Data Operational Data Data Account Data

17 Analytical databases support BI analysis An analytical database May contain data from multiple operational sources and external data sources Can have many subject orientations to satisfy multiple analysis goals Time horizon may be longer (years rather than months) Redundancy not a concern (unlike operational) Quantity larger (than operational)

18 Extract, Transform & Load (ETL) Taking data from operational and other sources and putting them into an analytical database Extract refers to getting data from sources, such as operational databases Transform is the conversion from an operational orientation to a summarized analytical orientation. Can be very time consuming and may involve writing small programs (in Python, C#, or T-SQL, for example), to automate the process Once transformed, it is loaded into an analytical database for analysis, often called a data warehouse

19 Example Transformation Using T-SQL Invoice date can be used for different analysis needs, for example sales by quarter T-SQL Transformation CONVERT(VARCHAR, CONVERT(DATETIME, InvoiceDate), 23) DATEPART(DAYofYEAR, InvoiceDate) DATEPART(MONTH, InvoiceDate) DATEPART(DAY, InvoiceDate) DATEPART(QUARTER, InvoiceDate) DATEPART(YEAR, InvoiceDate) New Field FullDate DayOfYr Month Day Quarter Year

20 Demo 2 Transforming data using T-SQL Loading into an analytical database using T-SQL and stored procedure The Star Schema design approach

21 Loading Data into a Data Warehouse T-SQL is used to SELECT and Transform Operational data Into an Analytical DB (aka the data warehouse) For BI Reports Operational DB

22 Star Schema Design Data Warehouse A fact table is at the center and one or more dimension tables are points of a star A fact table contains numeric measures and foreign keys related to the subject of analysis Dimension tables contain descriptive data regarding the measures

23 Job Description and Skills * Data Analyst: collect, analyze, and report on data Usually a bachelor s degree IS, CS or statistics focus Good analytical skills ability to perform complex tasks Strong communicator who works well in teams Detail oriented with good problem solving ability Experience with DBs, Excel, Python & coding a plus Entry pay: $59K; Average pay: $76K *Glassdoor Data Analyst Salaries Portland, OR

24 Data Analyst Focused Sessions Time Session 8:30 9:45 The Big Picture: SQL Overview and Getting the Most of SQL Saturday - Dr. Michael Curry 10:15 11:30 Data Types Do Matter - Angela Henry 11:45 12:45 CozyRoc: Dynamic Data Flows in SSIS Without Programming - Diane Schuster 1:00 2:15 Detecting and Capturing Change for Data Warehousing - Chris Hyde 2:45 4:00 Introduction to SQL Server Reporting Services - Ted Stathakis 4:15 5:30 Data Testing & Integrity BI Edition - Rob Long

25 Business Intelligence Get Power BI Load data from DB for Create a report while investigating the data

26 Power BI Simplifies Analysis & Reporting

27 Demo 3 Power BI orientation Building a Products report Publishing a report online

28 Load Data from SQL Server to Power BI Step 1, Identify the data source Step 2, Identify the Tables Step 3, Import

29 Three Views in Power BI Report Data Relationships

30 Creating a Report The basic principle of analysis with Power BI Put the data of interest onto the report Then adjust the visuals and filters to discover insights BI analysis is an iterative process Don t be afraid to try different things

31 Creating a Report 1. Drag data of interest onto Report

32 Creating a Report 2. Adjust the visual

33 Creating a Report 3. Add dimension

34 Creating a Report 4. Adjust filters

35 Job Description and Skills * BI Analyst: analyze data for better decision making Usually a bachelor s degree IS, CS or statistics focus Good analytical skills ability to perform complex tasks Formulate and test hypotheses Excellent communication skills Experience with DBs, Excel, Python, coding & statistics a plus Entry pay: $70K; Average pay: $85K *Glassdoor BI Analyst Salaries Portland, OR

36 BI Analyst Focused Sessions Time Session 8:30 9:45 The Big Picture: SQL Overview and Getting the Most of SQL Saturday - Dr. Michael Curry 10:15 11:30 Power BI Report Design for the aesthetically challenged - James McGillivray 11:45 12:45 Running SQL Server on Amazon Web Services - Richard Waymire 1:00 2:15 SSIS Deployment Smackdown! Package vs. Project! - Peter Kral 2:45 4:00 Introduction to SQL Server Reporting Services - Ted Stathakis 4:15 5:30 Data Testing & Integrity BI Edition - Rob Long

37 No matter your job, Data Management is a useful skill Sample Job listings: Financial analyst at VanderHouwen SQL data extract and reporting, report modeling Project manager at Black & Decker skills or experience with Word, Excel, SQL and Networks Transportation analyst at Zulily experience with SQL Marketing at Amazon familiarity with HTML, CSS, XML, SQL, or enthusiasm to learn these languages Data Scientist at CenturyLink analyze data to identify previously unknown patterns and insights and leverage them for competitive advantage

38 Thank-you Sponsors

39 Academic Data Management E-Commerce Entrepreneurship Dr. Michael Curry /michaellcurry/ Consultant Information Security IT Audits Data Analytics Scholar Behavioral Cybersecurity Researcher