UNIT 3 XML DATABASES

Transcription

1 UNIT 3 XML DATABASES XML Databases: XML Data Model DTD - XML Schema - XML Querying Web Databases JDBC Information Retrieval Data Warehousing Data Mining XML Databases: XML Data Model The common method of specifying the contents and formatting of web pages is through the use of Hypertext documents. There are various languages for writing these documents; the most common one is HTML (Hyper Text Markup Language). But it is not suited for specifying structured data that is extracted from databases. A new language namely, XML (Extensible Markup Language) has emerged as the standard for structuring and exchanging data over the web. XML can be used to provide information about the structure and meaning of the data in the web pages rather than just specifying how the web pages are formatted for display on the screen. The formatting aspects are specifies separately by using a formatting language such as XSL (Extensible Style sheet Language). Recently, XML has also been proposed as a possible model for data storage and retrieval, although only a few experimental database systems based on XML have been developed so far Structured, Semi structured and Unstructured Data There are 3 characterizations in data. They are Structured Data Semi structured Data and Unstructured Data 1) Structured Data Information stored in databases is known as structured data because it is represented in a strict format. The DBMS then checks to ensure that all data follows the structures and constraints specified in the schema. 2) Semi Structured Data In some applications, data is collected in an ad-hoc manner before it is known how it will be stored and managed. This data may have a certain structure, but not all the information collected will have identical structure. This type of data is known as semi-structured data. 1 PREPARED BY: S.PON SANGEETHA

2 In semi-structured data, the schema information is mixed in with the data values, since each data object can have different attributes that are not known in advance. Hence, this type of data is sometimes referred to as self-describing data. Semi-structured data may be displayed as a directed graph. The labels or tags on the directed edges represent the schema names the names of attributes, object types (or entity types or classes), and relationships. The internal nodes represent individual objects or composite attributes. The leaf nodes represent actual data values of simple (atomic) attributes. 3) Unstructured Data A third category is known as unstructured data, because there is very limited indication of the type of data. A typical example would be a text document that contains information embedded within it. Web pages in HTML that contain some data are considered as unstructured data XML Data Model XML data model is based on tree (hierarchical) structure. The basic object in XML is the XML document. Two main structuring concepts are used to construct an XML document: Elements and Attributes. 2 PREPARED BY: S.PON SANGEETHA

3 Element is as it is used in document description languages such as HTML and SGML. Attributes in XML provide additional information that describes elements. There are some additional concepts in XML, such as entities, identifiers and references. The following is an example of an XML element called <Projects>. Elements are identified in a document by their start tag and end tag. The tag names are enclosed between angled brackets. < > and the end tags are further identified by a slash, </ >. Complex elements are constructed from other elements hierarchically, whereas simple elements contain data values. In XML, tag names are defined to describe the meaning of the data elements in the document. The XML tag names can be defined in another document known as the schema document, to give a semantic meaning to the tag names that can be exchanged among multiple users. 3 PREPARED BY: S.PON SANGEETHA

4 The XML document can be represented either textual model or tree model. In the tree representation, internal nodes represent complex elements whereas leaf nodes represent simple elements. That is why the XML model is called a tree model or a hierarchical model. In the above example, <name> <number> <location> are called simple elements. <Projects> <project> and <worker> are called complex elements. Types of XML documents 1) Data-centric XML documents These documents have many small data items that follow a specific structure, and hence may be extracted from a structured database. They are formatted as XML documents in order to exchange them or display them over the Web. These usually follow a predefined schema that defines the tag names. This can be considered either as semistructured data or as structured data. 2) Document-centric XML documents These are documents with large amounts of text, such as news articles or books. There are few or no structures data elements in these documents. 3) Hybrid XML documents These documents may have parts that contain structured data and other parts that are predominantly textual or unstructured. They may or may not have a predefined schema. XML documents that do not follow a predefined schema of element names and corresponding tree structure are known as schemaless XML documents. XML Documents There are two main category of XML document. 1) Well-Formed XML Documents. 2) Valid XML Documents. 1) Well-Formed XML Documents An XML document is well formed if it follows a few conditions. It must start with an XML declaration to indicate the version of XML being used as well as any other relevant attributes. It must follow the syntactic guidelines of the tree data model. This means that there should be a single root element, and every element must include a matching pair of start and end tags within the start and end tags of the parent element. This ensures that the nested elements specify a well-formed tree structure. 4 PREPARED BY: S.PON SANGEETHA

5 A well-formed XML document is syntactically correct. This allows it to be processed by generic processors that traverse the document and create an internal tree representation. DOM (Document Object Model) A standard model with an associated API functions called DOM allows programs to manipulate the resulting tree representation corresponding to a well-formed XML document. The whole document must be parsed beforehand when using DOM in order to convert the document to that standard DOM internal data structure representation. SAX Another API called SAX (Simple API for XML) allows processing of XML documents on the fly by notifying the processing program whenever a start or end tag is encountered. This makes it easier to process large documents and allows for processing of so-called streaming XML documents, where the processing program can process the tags as they are encountered. This is also known as event-based processing. 2) Valid XML Documents A stronger criterion is for an XML document to be valid. In this case, the document must be well-formed, and in addition the element names used in the start and end tag pairs must follow the structure specified in a separate XML DTD (Document Type Definition) file or XML schema file. 5 PREPARED BY: S.PON SANGEETHA

6 3.2. DTD The following figure shows a simple XML DTD (Document Type Definition) file, which specifies the elements and their nested structures. Any valid documents confirming to this DTD should follow the specified structure. XML DTD Notation A * following the element name means that the element can be repeated zero or more times in the document. This can be called an optional multivalued (repeating) element. A + following the element name means that the element can be repeated one or more times in the document. This can be called a required multivalued (repeating) element. A? following the element name means that the element can be repeated zero or one times. This can be called an optional single-valued (non-repeating) element. An element appearing without any of the preceding three symbols must appear exactly once in the document. This can be called an required single-valued (non-repeating) element. The type of the element is specified via parentheses following the element. If the parentheses include names of other elements, these would be the children of the element in the tree structure. If the parentheses include the keyword #PCDATA or one of the other data types available in XML DTD, the element is a leaf node. PCDATA stands for parsed character data, which is roughly similar to a string data type. Parentheses can be nested when specifying elements. A bar symbol (e1 e2) specifies that either e1 or e2 can appear in the document. Limitations of XML DTD First, the data types in DTD are not very general. Second, DTD has its own special syntax and so it requires specialized processors. It would be advantageous to specify XML schema documents using the syntax rules of 6 PREPARED BY: S.PON SANGEETHA

7 XML itself so that the same processors for XML documents can process XML schema descriptions. Third, all DTD elements are always forced to follow the specified ordering the document so unordered elements are not permitted XML Schema The XML schema language is a standard for specifying the structure of XML documents. It uses the same syntax rules as regular XML documents, so that the same processors can be used on both. To distinguish the two types of documents, we will use the term XML instance document or XML document for a regular XML document and XML schema document for a document that specifies an XML schema. XML schema is based on the tree data model, with elements and attributes as the main structuring concepts. However it borrows additional concepts from database and object models, such as keys, references and identifiers. An XML schema file called company 7 PREPARED BY: S.PON SANGEETHA

8 8 PREPARED BY: S.PON SANGEETHA

9 9 PREPARED BY: S.PON SANGEETHA

10 Some of the Schema concepts 1) Schema Descriptions and XML namespaces It is necessary to identify the specific set of XML schema language elements (tags) by a file stored at a Web site location. The second line in our example specifies the file used in this example, which is: " Each such definition is called an XML namespace. The file name is assigned to the variable xsd using the attribute xmlns (XML namespace), and this variable is used as a prefix to all XML schema tags. 2) Annotations, documentation and language used The xsd:annotation and xsd:documentation are used for providing comments and other descriptions in the XML document. The attribute XML:lang of the xsd:documentation element specifies the language being used. E.g., en 3) Elements and Types We specify the root element of our XML schema. In XML schema, the name attribute of the xsd:element tag specifies the element name, which is called company for the root element in our example. The structure of the company root element is a xsd:complextype 10 PREPARED BY: S.PON SANGEETHA

11 4) First-level elements in the COMPANY database These elements are named employee, department, and project, and each is specified in an xsd:element tag. If a tag has only attributes and no further sub-elements or data within it, it can be ended with the back slash symbol (/>) and termed Empty Element. 5) Specifying element type and minimum and maximum occurrences If we specify a type attribute in an xsd:element, this means that the structure of the element will be described separately, typically using the xsd:complextype element. The minoccurs and maxoccurs tags are used for specifying lower and upper bounds on the number of occurrences of an element. The default is exactly one occurrence. 6) Specifying keys For specifying primary keys, the tag xsd:key is used. For specifying foreign keys, the tag xsd:keyref is used. When specifying a foreign key, the attribute refer of the xsd:keyref tag specifies the referenced primary key whereas the tags xsd:selector and xsd:field specify the referencing element type and foreign key. 7) Specifying the structures of complex elements via complex types Complex elements in our example are Department, Employee, Project, and Dependent, which use the tag xsd: complextype. We specify each of these as a sequence of subelements corresponding to the database attributes of each entity type by using the xsd: sequence and xsd: element tags of XML schema. Each element is given a name and type via the attributes name and type of xsd: element. We can also specify minoccurs and maxoccurs attributes if we need to change the default of exactly one occurrence. For (optional) database attributes where null is allowed, we need to specify minoccurs = 0, whereas for multivalued database attributes we need to specify maxoccurs = unbounded on the corresponding element. 8) Composite (Compound) Attributes Composite attributes from ER Schema are also specified as complex types in the XML schema, as illustrated by the Address, Name, Worker, and WorksOn complex types. These could have been directly embedded within their parent elements. 11 PREPARED BY: S.PON SANGEETHA

12 3.4. XML Querying XPath An XPath expression returns a collection of element nodes that satisfy certain patterns specified in the expression. The names in the XPath expression are node names in the XML document tree that are either tag (element) names or attribute names, possibly with additional qualifier conditions to further restrict the nodes that satisfy the pattern. There are two main separators when specifying a path: single slash (/) and double slash (//) A single slash before a tag specifies that the tag must appear as a direct child of the previous (parent) tag, whereas a double slash specifies that the tag can appear as a descendant of the previous tag at any level. It is customary to include the file name in any XPath query allowing us to specify any local file name or path name that specifies the path. doc( => COMPANY XML doc 1) Returns the COMPANY root node and all its descendant nodes, which means that it returns the whole XML document. 2) Returns all department nodes (elements) and their descendant subtrees. 3) Returns all employeename nodes that are direct children of an employee node, such that the employee node has another child element employeesalary whose value is greater than ) This returns the same result as the previous one except that we specified the full path name in this example. 5) This returns all projectworker nodes and their descendant nodes that are children under a path /company/project and that have child node hours with value greater than 20.0 hours. Some examples of XPath expressions on XML documents that follow the XML schema file COMPANY. 12 PREPARED BY: S.PON SANGEETHA

13 XQuery XQuery uses XPath expressions, but has additional constructs. XQuery permits the specification of more general queries on one or more XML documents. The typical form of a query in XQuery is known as a FLWR expression, which stands for the four main clauses of XQuery and has the following form: o FOR <variable bindings to individual nodes (elements)> o LET <variable bindings to collections of nodes (elements)> o WHERE <qualifier conditions> o RETURN <query result specification> 1. This query retrieves the first and last names of employees who earn more than The variable $x is bound to each employeename element that is a child of an employee element, but only for employee elements that satisfy the qualifier that their employeesalary is greater than This is an alternative way of retrieving the same elements retrieved by the first query. 3. This query illustrates how a join operation can be performed by having more than one variable. Here, the $x variable is bound to each projectworker element that is a child of project number 5, whereas the $y variable is bound to each employee element. The join condition matches SSN values in order to retrieve the employee names. Some examples of XQuery queries on XML 13 PREPARED BY: S.PON SANGEETHA

14 3.5. Web Databases Semantic web architecture and applications are a dramatic departure from earlier database and applications generations. Semantic processing includes these earlier statistical and natural langue techniques, and enhances these with semantic processing tools. First, Semantic Web architecture is the automated conversion and storage of unstructured text sources in a semantic web database. Second, Semantic Web applications automatically extract and process the concepts and context in the database in a range of highly flexible tools. 14 PREPARED BY: S.PON SANGEETHA

15 a. Architecture; not only Application First, the Semantic web is complete database architecture, not only an application program. Semantic web architecture combines a two-step process. First, a Semantic Web database is created from unstructured text documents. And, then Semantic Web applications run on the Semantic Web database; not the original source documents. The Semantic Web architecture is created by first converting text files to XML and then analyzing these with a semantic processor. This process understands the meaning of the words and grammar of the sentence, and also the semantic relationships of the context. These meanings and relationships are then stored in a Semantic web database. The Semantic Web is similar to the schematic logic of an electronic device or the DNA of a living organism. It contains all of the logical content AND context of the original source. And, it links each word and concept back to the original document. Semantic Web applications directly access the logical relationships in the Semantic Web database. Semantic web applications can efficiently and accurately search, retrieve, summarize, analyze and report discrete concepts or entire documents from huge databases. A search for Java links directly to the three Semantic Web logical clusters for Java : (island in Indonesia, a computer programming language, and coffee). The processor can 15 PREPARED BY: S.PON SANGEETHA

16 now query the user for which Java, and then expand the search to all other concepts and documents related to the specific Java. b. Structured and Unstructured Data Second, Semantic Web architecture and applications handle both structured and unstructured data. Structured data is stored in relational databases with static classification systems, and also in discrete documents. These databases and documents can be processed and converted to Semantic Web databases, and then processed with unstrctured data. Much of the data we read, produce and share is now unstructured; s, reports, presentations, media content, web pages. And, these documents are stored in many different formats; text, files, Microsoft word processor, spreadsheet, presentation files, Lotus Notes, Adobe.pdf, and HTML. It is difficult, expensive, slow and inaccurate to attempt to classify and store these in a structured database. All of these sources can be automatically converted to a common Semantic Web database, and integrated into one common information source. c. Dynamic and Automatic; not Static and Manual Third, Semantic Web database architecture is dynamic and automated. Each new document which is analyzed, extracted and stored in the Semantic Web expands the logical relationships in all earlier documents. These expanding logical relationships increase the understanding of content and context in each document, and the entire database. The Semantic Web conversion process is automated. No human action is required for maintaining a taxonomy, meta data tagging or classification. The semantic database is constantly updated and more accurate. Semantic Web architecture is different from relational database systems. Relational databases are manual and static because these are based on a manual process for maintaining a taxonomy, meta data tagging and document classification in static file structures. Documents are manually captured, read, tagged, classified and stored in a relational database only once, and not updated. More important, the increase in new documents and information in a relational database does not make the database more intelligent about the concepts, relationships or documents. d. From Machine Readable to Machine Understandable Fourth, Semantic Web architecture and applications support both human and machine intelligence systems. Humans can use Semantic Web applications on a manual basis, and improve the efficiency of search, summary, analysis and reporting tasks. Machines can also use Semantic Web applications to perform tasks that humans cannot do; because of the cost, speed, accuracy, complexity and scale of the tasks. e. Synthetic vs Artificial Intelligence: Semantic Web technology is NOT Artificial Intelligence. AI was a mythical marketing goal to create thinking machines. The Semantic Web supports a much more limited and realistic goal. This is Synthetic Intelligence. The concepts and relationships stored in the Semantic Web database are synthesized, or brought together and integrated, to automatically create a new summary, analysis, report, , alert; or launch another machine application. The goal of Synthetic Intelligence information systems is bringing together all information sources and user knowledge, and synthesizing these in global networks. 16 PREPARED BY: S.PON SANGEETHA

17 3.6. JDBC JDBC (Java Database Connectivity) API allows Java programs to connect to databases Database access is the same for all database vendors The JVM uses a JDBC driver to translate generalized JDBC calls into vendor specific database calls Anatomy of Data Access Basic steps to use a database in Java 1. Establish a connection a) import java.sql.*; b) Load the vendor specific driver Class.forName ("oracle.jdbc.driver.oracledriver"); c) Make the connection Connection con = DriverManager.getConnection ("jdbc:oracle:thin:@oracle-prod:1521: OPROD", username, passwd); JDBC Establishes connection to database by obtaining a Connection object. JDBC Driver Oracle Server Class.forName ( oracle.jdbc.driver.oracledriver ); con = DriverManager.getConnection ( jdbc:oracle:thin:@bonsai.ite.gmu.edu:1521:ite, accountname", password ); JDBC Driver MySQL Server 17 PREPARED BY: S.PON SANGEETHA

18 Class.forName ( org.gjt.mm.mysql.driver ); con = DriverManager.getConnection ( jdbc: mysql: //localhost/databasename, uid, passwd); 2. Create JDBC Statements Statement stmt = con.createstatement () ; It Creates a Statement object for sending SQL statements to the database. 3. Execute SQL Statements String createlehigh = "Create table Lehigh + "(SSN Integer not null, Name VARCHAR (32), + "Marks Integer)"; stmt.executeupdate (createlehigh); String insertlehigh = "Insert into Lehigh values + "( , abc, 100)"; stmt.executeupdate (insertlehigh); 4. GET ResultSet String querylehigh = "select * from Lehigh"; ResultSet rs = Stmt.executeQuery (querylehigh); //What does this statement do? while (rs.next ()) { int ssn = rs.getint ("SSN"); String name = rs.getstring ("NAME"); int marks = rs.getint ("MARKS"); } 5. Close connections stmt.close (); con.close(); 3.7. Information Retrieval Information retrieval (IR) The goal of IR is finding documents relevant to an information need from a large document set. Info 18 PREPARED BY: S.PON SANGEETHA

19 Information retrieval deals with unstructured data, and the techniques for indexing, searching and retrieving information from large collections of unstructured documents. Unstructured information stored in messages and documents that contain textual and multimedia information. These documents are stored in variety of standard formats, including HTML, XML and several audio and video formatting standards. Information retrieval is the process of retrieving those documents from a collection in response to a query (or a search request) by a user. All forms of unstructured data described above are being added at the rates of millions per day, expanding the searchable space on the web at rapidly increasing rates. A search engine is a practical application of information retrieval to large-scale document collections. IR Systems use a user s information need expressed as a free-form search request (sometimes called a keyword search query) for interpretation by the system. An IR system can be characterized at different levels: by types of users, types of data and the types of the information need, along with the size and scale of the information repository it addresses. 1) Types of Users- The user may be an expert user(for example a librarian), who is searching for specific information that is clear in his/her mind and forms relevant queries for the task, or a layperson user with a generic information need cannot create highly relevant queries for search. 2) Types of data - Search systems can be tailored to specific types of data. The information repository could be hierarchically organized based on a concept or topic hierarchy. These topical domain-specific or vertical IR systems are not as large as or as diverse as the generic World Wide Web., which contains information on all kinds of topics. 3) Types of Information need - In the context of Web search, user s information needs may be defined as navigational, informational or transactional. Navigational search refers to 19 PREPARED BY: S.PON SANGEETHA

20 finding a particular piece of information (particular university web site) that a user needs quickly. The purpose of informational search is to find current information about a topic (Research activities about particular topic). The goal of transactional search is to reach a site where further interaction happens (online reservations, product shopping). 4) Levels of Scale - What information is consumed? On a smaller scale, there are personal information systems such as those on desktops and laptops, called desktop search engines, for retrieving files, folders and different kinds of entities stored on the computer Comparison of IR systems and Databases S.No Databases IR Systems 1 Structured data Unstructured data 2 Schema driven No fixed schema; various data models 3 Relational or object or hierarchical or network model 4 Structured query model Free-form query models 5 Rich metadata operations Rich data operations 6 Query returns data Search request returns list or pointers to documents 7 Results are based on exact matching Results are based on approximate matching and measures of effectiveness Modes of interaction in IR systems A Query is a set of terms used by the searcher to specify an information need. There are 2 main modes of interaction with IR systems a) Retrieval and b) Browsing. a) Retrieval is concerned with the extraction of relevant information from a repository of documents through an IR query. b) Browsing signifies the activity of a user visiting or navigating through similar or related documents based on the user s assessment of relevance. Web search combines both aspects browsing and retrieval. The rank of a web page in a retrieved set is the measure of its relevance to the query that generated the result set Indexing-based IR 20 PREPARED BY: S.PON SANGEETHA