The NoPlsql and Thick Database Paradigms

Transcription

1 The NoPlsql and Thick Database Paradigms Part 2: Adopting ThickDB Toon Koppelaars Real-World Performance Oracle Server Technologies Bryn Llewellyn Distinguished Product Manager Oracle Server Technologies

2 Who Are We? Part of Oracle ecosystem since 1987 Developer, DBA, Architect, Problem Solver Currently part of Real-World Performance group Author of TheHelsinkiDeclaration.blogspot.com All about Distinguished Product Manager: PL/SQL and EBR Author of "Why use PL/SQL?" 2

3 Database Applications (from Part 1) User Interfaces SQL SQL SQL Business Logic Software Interfaces SQL SQL Table Data Store SQL Conceptually 3 tiers Exposed functionality via services GUI's for human interaction REST, or otherwise, for software interaction Business logic Procedural code composing Q's + TX's Data store, relational database 4

4 Persistence Layer or Processing Engine? SQL SQL SQL SQL SQL SQL DBMS DBMS SQL SQL SQL SQL No PL/SQL Single table, row-by-row SQL NoPlsql PL/SQL All kinds of SQL ThickDB 5

5 Focus of This Part 2 You want to adopt ThickDB approach for development of new application Now, how to do that successfully? Critical success factors: Technical Understand various concepts, architecture etc. How to prevent one big pile of PL/SQL code Organizational Get project roles right 6

6 Roadmap Organizational - Project roles Technical - Architecture of web applications Technical - Code classification and architecture inside database Concluding thoughts 7

7 Organizational: Project Roles "Full-stack" developers with solid Database-design, SQL, PL/SQL, *and* Current UI technology skills, are hard to find High technology change rate You'll inevitably have separate roles in your project Database developers easier to hire without UI skills Hip JavaScript UI developers easier to hire without DB skills 8

8 Organizational: Focus For Each Role Focus for each: Database developers strong (set-based) SQL and PL/SQL skills Focus on maintainability, performance and scalability UI developers strong html/css/javascript skills Focus on extremely friendly UI experience (UX) Separate focus allows both roles to be performed at highest level Database developer skills are more rare, but one database developer can support multiple UI developers 9

9 Organizational: Two Worlds Must Cooperate Separate focus, but must closely cooperate Requires good communication skills Database developers will need to understand UI-developers' world Not vice-versa: they will be "decoupled" via API's Know enough architectural concepts to communicate effectively with them, to jointly design API-layer Database developers, must be in the lead here Why? they own database design and business logic Need to detect API requests that would leave BL in UI 10

10 The New Emerging World Creates Opportunity V+C physically de-coupled now from M 11

11 Roadmap Organizational - Project roles Technical - Architecture of web applications Technical - Code classification and architecture inside database Concluding thoughts 12

12 Architecture of Web Applications In web applications, users: Tick boxes Select values from dropdown LoV's Enter/change editable data items Click on hyperlinks Press buttons Do something that triggers (browser-side) JavaScript These can execute Ajax calls Popular in "Single Page Applications" (SPA's) Partial page (re)rendering These typically initiate request from browser to server Asynchronous Javascript And XML Understanding Technical Architecture 13

13 Browser Server Requests Result in Page Transitions Click on link Press button Initiate Ajax call Browser Browser Browser http-get New page html http-post Form submit New page html Ajax request XML for re-render Web server Web server Web server Understanding Technical Architecture 14

14 Point to Be Made Understand where, when and why UI developer will submit requests For each request that browser can initiate you will need to agree upon some API that handles it Web server A P I Web server acts as proxy: it just translates request to DB-API call Understanding Technical Architecture 15

15 Page Flows These http-get, http-post and Ajax requests result in page transitions Which in turn lead to application "page flows" 16

16 Page Flow Network Customers Edit Customer Login Start Page Orders Edit Order Understanding Technical Architecture 17

17 Incoming Flow Determines What Should Be Displayed Show next/previous ten orders Show 1 st ten orders of this customer E.g. default show 1 st ten "open" orders Edit Customer Start Page Orders Edit Order Refresh list that was shown So here we have four "render" use-cases for Orders-page For each case DB-API is tasked to produce list of orders to be displayed Understanding Technical Architecture 18

18 Transactions Typically Attached to Outgoing Flows Delete-button pressed Update-button pressed Orders Edit Order Two process use-cases for Edit-Order page For each case DB-API is tasked to compose and execute transaction Understanding Technical Architecture 19

19 Render and Process Use-Cases per Page You'll need to discuss per page What are the render use-cases? What are the process use-cases? And per use-case agree upon signature: What are API inputs and outputs Understanding Technical Architecture 20

20 The Connection Pool Application threads use "connection pool" to access database Some number of pre-instantiated Java objects Each owning database session Understanding Technical Architecture 21

21 Connection Pool T1 T2 T3 T4 T5 T6 T7 T8 Servlet execution threads working on browser requests Connection pool C1 C2 C3 Pre-spawned connection pool with three idle connections JDBC JVM S1 S2 S3 Idle database sessions All under same username Understanding Technical Architecture 22

22 Connection Pool Thread 6 requires database access T1 T2 T3 T4 T5 T6 T7 T8 Connection pool C1 C2 C3 JDBC JVM S1 S2 S3 Understanding Technical Architecture 23

23 Connection Pool T1 T2 T3 T4 T5 T6 T7 T8 Thread 6 requests and acquires free connection from pool C1 C2 C3 JDBC Connection pool JVM C2 now unavailable for other threads S1 S2 S3 Understanding Technical Architecture 24

24 Connection Pool T1 T2 T3 T4 T5 T6 T7 T8 Connection pool C1 C2 C3 JDBC JVM Thread 6 submits SQL via C2 S1 S2 S3 DB session S2 wakes up from idle state Understanding Technical Architecture 25

25 Connection Pool T1 T2 T3 T4 T5 T6 T7 T8 Connection pool C1 C2 C3 JDBC JVM S1 S2 S3 S2 executes SQL statement Understanding Technical Architecture 26

26 Connection Pool T1 T2 T3 T4 T5 T6 T7 T8 Connection pool C1 C2 C3 JDBC JVM S2 sends result back to C2 S1 S2 S3 S2 idle again Understanding Technical Architecture 27

27 Connection Pool T1 T2 T3 T4 T5 T6 T7 T8 Connection pool C2 sends result back to T6 C1 C2 C3 JDBC JVM S1 S2 S3 Understanding Technical Architecture 28

28 Connection Pool T6 releases C2 back to pool T1 T2 T3 T4 T5 T6 T7 T8 C1 C2 C3 JDBC Connection pool JVM C2 available again for other threads S1 S2 S3 Understanding Technical Architecture 29

29 Connection Pool These connection/session pairs are sequentially used by requests from different browsers Understanding Technical Architecture 30

30 Points to Be Made Threads should follow strict "request dosql release" pattern Each database request must either: Constitute of queries only, or Represent completed transaction (committed or rolled back) No enduser specific residue must be left in released connection's database session Temp table contents V$context contents Values in global package variables NLS settings Unclosed cursors Unhandled exceptions, during "dosql", should not break release step Leading to connection leakage 31

31 Roadmap Organizational - project roles Technical - Architecture of web applications Technical - Code classification and architecture inside database Concluding thoughts 32

32 First Things First: The Foundation "Proper" = two-fold 1. It must fit business requirements 2. It must follow sound relational practices Proper relational database design 1) is up to you talking to business owners and understanding their problem 2) is about education 33

33 Recommended Practices For a Sound Database Design Use 3 rd normal form Avoid structural redundancies Don't design incidental redundancies E.g. Total value of orderlines at order level Surrogate key 2 nd business key Put different things in different tables Avoid different meaning of column value depending on row Put special things in separate tables (subtypes) Avoid lists of null-able columns due to non-applicability Never allow nulls in columns involved in data integrity constraints Clarification Choose your table- and column-names wisely They are the first steppingstone to understanding your DB design 34

34 Data Integrity Constraints The relational model's meaning: Constraints are second stepping stone to understanding your database design The broader generic concept used in E.F Codd s relational model of data Everything that constrains the data that is allowed I.e. much more that just: CHECK, PRIMARY KEY, UNIQUE KEY, FOREIGN KEY SQL standard includes SQL Assertions for these This is all about predicate logic Boolean expressions that must be true at all times See the book 35

35 SQL Assertions Example Last DB design recommended practice: all columns referenced should be NOT NULL create assertion at_most_one_president as check ((select count(*) as num_president from EMP e where e.job = 'PRESIDENT') <= 1 ) Not supported yet, but Excellent way to document constraints! 36

36

37 Point to Be Made Your database design will have many SQL assertions You will write lot of code to enforce them Enforcement is required during transactional API's All code written to enforce SQL assertions: Data Logic code (DL) Different approaches Table triggers Table API's (TAPI's) Rule packages 38

38 Code Classification Use "du-jour" technology User Interfaces Software Interfaces Procedural code with embedded queries Business TX Logic Business Query Logic Business SQL SQL Logic Data SQL SQL Logic SQL SQL Table Data Store Procedural code with embedded DML and error handling Treat separately from TX-BL Challenge here is: DRY Specify all declarable constraints 39

39 Code Classification API Layer User Interfaces API Layer Software Interfaces Business TX Logic Business Query Logic Business SQL SQL Logic Data SQL SQL Logic SQL SQL Table Data Store 40

40 Code Classification Apply Common Sense DRY User Interfaces API Layer Software Interfaces Business TX Logic Business Query Logic Business SQL SQL Logic Data SQL SQL Logic SQL SQL Shared Views / Functions Table Data Store Business specific - Data views - Data derivations - Computations 41

41 Code Classification Key Message These will then become These will then become straightforward straightforward procedural code User Interfaces API Layer Software Interfaces Business TX Logic Business Query Logic Business SQL SQL Logic Data SQL SQL Logic SQL SQL Shared Views / Functions Table Data Store Make sure that these two are completely separated from other business logic code 42

42 End of Nineties Wisdom Q: How do I prevent one big pile of PL/SQL code? Adversaries argument A: Materialize code classification such that classes, and modules within them, are separately, Visible Maintainable Versionable Etc. 43

43 Schema Architecture: Horizontal Materialization Introduce multiple database schemas Four horizontal layers 1) "Connect schema": object-less schema, used by connection pool to connect to DB 2) "API schema" : holds all API objects jackets on top of code schema objects 3) "Code schema" : holds all business logic and data logic 4) "Storage schema" : holds all tables and declarative constraints 44

44 Schema Architecture T1 T2 T3 T4 T5 T6 Connection pool JVM C1 C2 C3 JDBC Connect schema API schema Code schema Storage schema S1 S2 S3 API objects Business logic Tables 45

45 Remember the Page Network Incoming flows that render Page X Context B Page X Outgoing flows that initiate TX 46

46 Vertically Materialize API's Per Page Page X Create package per page Use page-name/id in its name Render API's Submit API's Add procedure per incoming / outgoing flow U S E C A S E 1 U S E C A S E 2 U S E C A S E 3 U C 1 U C 2 Issue in Page X submit, usecase 2? Impact analysis (dependencies) 47

47 API Object Signature All API's will be packaged procedures Accept and return repeating data via parameters of type: XML JSON (nested) IBI's of varchar2's Index-by-plsinteger record-arrays Accept and return scalar data via parameters of type: Varchar2 48

48 Roadmap Organizational - project roles Technical - Architecture of web applications Technical - Code classification and architecture inside database Concluding thoughts 49

49 Summary We've seen 15 years of MVC in middle-tier With persistence layer role for database Now strong move towards V+C in browser using JavaScript frameworks/libraries M "somewhere else" Both tied together using REST This is opportunity for ThickDB approach to revive 50

50 Summary Separate UI (V+C) and database (M) project roles With DB-developers in the lead who understand technical architecture UI DB Decouple UI via API's that hide all BL+DL Materialize code classes Build UI use-cases vertically Apply common-sense DRY Deploy horizontally using different schemas 51

51 52

52 53