COMP9321 Web Application Engineering

Transcription

1 COMP9321 Web Application Engineering Data Access in Web Applications Dr. Basem Suleiman Service Oriented Computing Group, CSE, UNSW Australia Semester 1, 2016, Week 5 1

2 Acknowledgement/Contributions Service Oriented Computing Group, CSE, UNSW Australia Dr. Helen Paik Prof. Boualem Bentallah Dr. Srikumar Venugopal Dr. Moshe Chai Barukh Dr. Amin Beheshti Dr. Basem Suleiman Many others from service oriented computing group COMP9321, 16s1, Week 3 2

3 Vast Amounts of Data on the Web!! Remote patient monitoring Product sensors Healthcare Social media Manufacturing books, music, videos, etc. Retail Real time location data Digitalization of Artefacts Location-Based Services 3

4 Vast Amounts of Data are Generated!! Air Bus A380: o generate 10 TB every 30 min Twitter: o Generate approximately 12 TB of data per day. Facebook: o Facebook data grows by over 500 TB daily. New York Stock: o Exchange 1TB of data everyday. 4

5 Data Challenge How do we store and access this data over the web? 5

6 Data Design for Web Applications How do we store and access this data over the web? E-Commerce website Data operations are mainly transactions (Reads and Writes) Operations are mostly on-line Response time should be quick but important to maintain security and reliability of transactions. ACID properties are important 6

7 Data Properties Challenge How do we store and access this data over the web? E-Commerce website Data operations are mainly transactions (Reads and Writes) Operations are mostly on-line Response time should be quick but important to maintain security and reliability of transactions. ACID properties are important 7

8 Data Properties Challenge How do we store and access this data over the web? Image serving website Data operations are mainly fetching large files (Reads) Operations are mainly on-line High bandwidth requirement ACID requirements can be relaxed 8

9 Data Properties Challenge How do we store and access this data over the web? Search Web Application Data operations are mainly reading index files for answering queries (Reads) Index compilation is performed off-line due to the large size of source data (the entire Web) Response times must be as fast as possible ACID requirements can be relaxed 9

10 Persistence (Hibernate, pp.5-29) 10

11 Persistence Persistence is: the continuance of an effect after its cause is removed In the context of storing data in a computer system, this means that: the data survives after the process with which it was created has ended In other words, for a data store to be considered persistent: it must write to non-volatile storage (Hibernate, pp.5-29) 11

12 Persistence Persistence is a fundamental concept in application development In an object-oriented applications, persistence allows an object to outlive the process that created it The state of the object may be stored to disk and an object with the same state re-created at some point in the future. Sometimes entire graphs of interconnected objects may be made persistent and later re-created in a new process. (Hibernate, pp.5-29) 12

13 Persistence Not all objects are persistent: o some (transient objects) will have a limited lifetime that is bounded by the life of the process that instantiated it Almost all Java applications contain a mix of persistent and transient Objects This means we need a subsystem that manages our persistent objects (Hibernate, pp.5-29) 13

14 Data Persistence (Hibernate, pp.5-29) 14

15 Data Persistence When we talk about persistence in Java, we normally mean storing data in a relational database using SQL. Relational technology is a common denominator for many disparate systems and technology platforms. Relational technology provides a way of sharing data across different applications or technologies that form part of the same application. The relational data model is often the common enterprise wide presentation of business entities. (Hibernate, pp.5-29) 15

16 Data Persistence When you work with a relational database in a Java application, the Java code issues SQL statements to the database via the JDBC API. The Java Database Connectivity (JDBC) API provides universal data access from the Java programming language. Using the JDBC API, you can access virtually any data source, from relational databases to spreadsheets and flat files. The JDBC API is comprised of two packages: java.sql javax.sql (Hibernate, pp.5-29) 16

17 Data Persistence When you work with a relational database in a Java application, the Java code issues SQL statements to the database via the JDBC API. The Java Database Connectivity (JDBC) API provides universal data access from the Java programming language. Using the JDBC API, you can access virtually any data source, from relational databases to spreadsheets and flat files. The JDBC API is comprised of two packages: java.sql javax.sql (Hibernate, pp.5-29) 17

18 Relational Databases (Hibernate, pp.5-29) 18

19 Relational Databases Data is stored as a collection of tuples that groups attributes e.g. (student-id, name, birthdate, courses). Data is visualized as tables, where the tuples are the rows and the attributes form the columns. Tables can be related to each other through specific columns. Each row in a table has at least one unique attribute. (Hibernate, pp.5-29) 19

20 Structured Query Language (SQL) 20

21 Structured Query Language (SQL) 21

22 Database Concepts 22

23 Database Concepts 23

24 Accessing DB from an Application (JDBC) 24

25 Accessing DB from an Application 25

26 Java DataBase Connectivity (JDBC) 26

27 JDBC Concepts When developers use JDBC, they construct SQL statements that can be executed. A template like query string: SELECT name FROM employee WHERE age =? Can be combined with local data structures so that regular Java objects can be mapped to the bindings in the string. e.g., a java.lang.integer object with the value of 42 can be mapped: SELECT name FROM employee WHERE age = 42 The results of execution, if any, are combined in a set returned to the caller. For example, the query may return: We can browse this result set as necessary. (Barish, p.310) 27

28 JDBC Interfaces 28

29 Typical JDBC Scenario 29

30 PreparedStatement object A more realistic case is that the same kind of SQL statement is processed over and over (rather than a static SQL statement). In PreparedStatement, a place holder (?) will be bound to an incoming value before execution (no recompilation). 30

31 Transaction Management By default, JDBC commits each update when you call executeupdate(). Committing after each update can be suboptimal in terms of performance. It is also not suitable if you want to manage a series of operations as a logical single operation (i.e., transaction). 31

32 Data Access Objects (DAO) 32

33 Data Access Objects (DAO) 33

34 Data Access Objects (DAO) 34

35 Data Access Objects (DAO) 35

36 Data Access Objects (DAO) Example: Cars Database 36

37 Data Access Objects (DAO) Example: Cars Database DTO (Data Transfer Object) 37

38 Data Access Objects (DAO) Example: Cars Database DTO (Data Transfer Object) carries the actual data... 38

39 Data Access Objects (DAO) Example: Cars Database 39

40 Data Access Objects (DAO) Example: Cars Database 40

41 Data Access Objects (DAO) Example: Cars Database 41

42 Data Access Objects (DAO) Example: Cars Database 42

43 Object-Relational Impedance Mismatch Problems 43

44 Object-Relational Impedance Mismatch Problems 44

45 Object-Relational Impedance Mismatch Problems 45

46 Object-Relational Impedance Mismatch Problems 46

47 Impedance (or Paradigm) Mismatch Problem 47

48 Impedance (or Paradigm) Mismatch Problem Granularity (Hibernate, pp.5-29) The problem of granularity 48

49 Impedance (or Paradigm) Mismatch Problem Granularity Observation: Classes in your OO-based model come in a range of different levels of granularity (coarse-grained entity classes like User, finer-grained classes like Address, simple String class like Postcode) Just two levels of granularity in RDB: Tables and Columns with scalar types (i.e., not as flexible as Java type system) Sometimes one ends up forcing the less flexible representation upon the object model (e.g., User class with properties like postcode, state). (Hibernate, pp.5-29) The problem of granularity 49

50 Impedance (or Paradigm) Mismatch Problem Subtypes (Hibernate, pp.5-29) The problem of subtypes 50

51 Impedance (or Paradigm) Mismatch Problem Identity (Hibernate, pp.5-29) The problem of identity 51

52 Impedance (or Paradigm) Mismatch Problem Identity While on the subject of identity Modern object persistence solutions recommend using surrogate key. A surrogate key in a database is a unique identifier for either an entity in the modelled world or an object in the database. The surrogate key is not derived from application data, unlike a natural (or business) key which is derived from application data. (Hibernate, pp.5-29) The problem of identity 52

53 Impedance (or Paradigm) Mismatch Problem Association (Hibernate, pp.5-29) The problem of association 53

54 Impedance (or Paradigm) Mismatch Problem Association (Hibernate, pp.5-29) The problem of association 54

55 Impedance (or Paradigm) Mismatch Problem Object Graph Navigation (Hibernate, pp.5-29) The problem of object graph navigation 55

56 Impedance (or Paradigm) Mismatch Problem Object Graph Navigation Considering the following example: (Hibernate, pp.5-29) The problem of object graph navigation 56

57 Impedance (or Paradigm) Mismatch Problem 1+N selects problem: The N+1 query problem is a common performance issue. It looks like this: Assuming load_cats() has an implementation that boils down to:..and load_hats_for_cat($cat) has an implementation something like this:..you will issue "N+1" queries when the code executes, where N is the number of cats: 57

58 Impedance (or Paradigm) Mismatch Problem The cost of mismatch problems: The DAO pattern helps isolate the mismatch problems by separating the interfaces from implementation, but someone (usually application developers) still has to provide the implementation classes!! (Hibernate, pp.5-29) The cost of mismatch problems 58

59 Object-Relational Mapping (ORM) 59

60 Object-Relational Mapping (ORM) 60

61 Hibernate - ORM Framework Hibernate is an open source ORM solution for JAVA which provides Object-Relational Persistence and Query service for any Java Application Improve development efficiency by relieving the developer from majority of common data persistence related programming tasks Hibernate maps Java classes to database tables and from Java data types to SQL data types Hibernate sits between traditional Java objects and database server to handle all the work in persisting those objects based on the appropriate O/R mechanisms and patterns Supports major RDBMS including MySQL, MS SQL, Oracle, DB2, HSQL 61

62 Hibernate Architecture Java Application (Client Code) Java Persistent Object Hibernate Configuration Criteria Query Session Session Factory Other JNDI JDBC JTA Database 62

63 Hibernate 63

64 Hibernate 64

65 Hibernate 65

66 Continuing with the Cars example... 66

67 Continuing with the Cars example... 67

68 Continuing with the Cars example... 68

69 Continuing with the Cars example... 69

70 Continuing with the Cars example... 70

71 Continuing with the Cars example... 71

72 To use Hibernate, you need: Hibernate packages (hibernate*.jar) A set of mapping (between a table and an object) A Hibernate configuration file (e.g., database connection details) 72

73 NoSQL 73

74 What is NoSQL? Stands for No-SQL or Not Only SQL?? Class of non-relational data storage systems E.g. BigTable, Dynamo, PNUTS/Sherpa,.. Usually do not require a fixed table schema nor do they use the concept of joins Distributed data storage systems All NoSQL offerings relax one or more of the ACID properties (will talk about the CAP theorem) Chapter 19: Distributed Databases 74

75 NoSQL Data Storage Classification: Uninterpreted key/value or the big hash table. Amazon S3 (Dynamo) Flexible schema BigTable, Cassandra, HBase (ordered keys, semistructured data), Sherpa/PNuts (unordered keys, JSON) MongoDB (based on JSON) CouchDB (name/value in text) 75

76 Three properties of a system CAP Theorem Consistency (all copies have same value) Availability (system can run even if parts have failed) Via replication. Partitions (network can break into two or more parts, each with active systems that can t talk to other parts) Brewer s CAP Theorem : You can have at most two of these three properties for any system. Very large systems will partition at some point. 76

77 Why NoSQL? NoSQL Data storage systems makes sense for applications that need to deal with very large semi-structured data : e.g. Social Networking Feeds 77

78 Why NoSQL? share, comment, review, crowdsource, etc. 78

79 Examples NoSQL databases: Employs less constrained consistency models. Simple retrieval and appending operations. Significant performance benefits. Examples: Key value Store Document Store Graph Database 79

80 Docs User Graph Database Social Network Collaborative Filtering Netflix Movie Probabilistic Analysis Text Analysis Wiki Words 80

81 Use a graph structure Graph Stores Labeled, directed, attributed multi-graph Label for each edge Directed edges Multiple attributes per node Multiple edges between nodes Relational DBs can model graphs, but an edge requires a join which is expensive Example Neo4j neo4j.com/ 81

82 Advantages of NoSQL Cheap, easy to implement Data are replicated and can be partitioned Easy to distribute Don't require a schema Can scale up and down Quickly process large amounts of data Relax the data consistency requirement (CAP) Can handle web-scale data, whereas Relational DBs cannot 82

83 Disadvantages of NoSQL New and sometimes buggy Data is generally duplicated, potential for inconsistency No standardized schema No standard format for queries No standard language Difficult to impose complicated structures Depend on the application layer to enforce data integrity No guarantee of support Too many options, which one, or ones to pick 83

84 References (Hibernate) Hibernate In Action, Christian Bauer and Gavin King, Manning Publications (HibernateDOC) docs/reference/en/html/ Some examples are originated from Dr. David Edmond from School of Information Systems, QUT, Brisbane and S. Sudarshan from IIT Bombay. 84

85 85