Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Transcription

1 1

2 Oracle Spatial and Graph at OOW 2012 Sessions Date/Time Title Location Tuesday, Oct 2 8:30am-9:30am 10:15am-11:15am 10:15am-11:15am Wednesday, Oct 3 11:45am - 12:45pm 1:15pm-2:15pm JavaOne Session: Leveraging W3C Linked Data for Loosely Coupled Application Integrations Conference Session: Oracle Spatial and Graph: Faster, Bigger, Better 2-D and 3-D Spatial Solutions Conference Session: Integrating Oracle Database with a Social Network Oracle Exadata, Oracle Exalogic, Oracle Exalytics, and Big Data Solutions in the Public Sector (Features panelists from US Census Bureau, New York City) Conference Session: Mash Up All Things Location with Android, ipad, HTML5 Web Maps, and Spatial Parc 55 - Embarcadero Moscone South Marriott Marquis Foothill F Westin San Francisco - City Marriott Marquis Club Room 2

3 Oracle Spatial and Graph at OOW 2012 Test Fest Special offer Get certified on Oracle Spatial onsite! Test Oracle PartnerNetwork Exchange Free for OPN Exchange participants Date/Time Event Location Monday Thursday exam times available (2 hour slots) Test OPN Exchange Certification Testing onsite for Oracle Spatial and many other Oracle technologies Pre-register for an exam slot at Walk-ins available on a first-come, first-served basis Proof of OPN membership and PearsonVue ID required Marriott Marquis, Juniper Room 3

4 Oracle Spatial and Graph at OOW 2012 Demos Date/Time Demo Location Monday-Wednesday Spatial Capabilities in Oracle Spatial and Graph Moscone South, Left S-059 Monday-Wednesday Semantic RDF Graph Technologies in Oracle Spatial and Graph Moscone South, Left S-060 Monday-Wednesday Empowering Business to Drive Process Modeling Moscone South, Right S-234 Monday-Wednesday Oracle Event Processing Moscone South, Right S-230 Monday -Thursday Analyze Big Data Moscone North, Upper Lobby N-013 4

5 Oracle Spatial and Graph: Faster, Bigger, Better 2-D and 3-D Spatial Solutions Jim Steiner, Vice President, Product Management, Oracle Server Technologies Dan Geringer, Senior Software Development Manager, Oracle 5 Copyright 2012, Spatial Oracle and/or its affiliates. and All rights reserved. Graph

6 The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle. 6

7 Program Agenda Introducing Oracle Spatial and Graph Goals for Spatial Features in 12c Vector Performance Acceleration Parallel GeoRaster and Enhanced Raster Operations Real World Feature Model for Network Graph 7

8 Program Agenda Parametric Curve Data Type Extended 3D and Point Cloud data type functionality Web Feature Server Console RDF Semantic Graph Analysis with Spatial Functions Experiences with Exadata Oracle Spatial and Graph 11g 8

9 INTRODUCING 9

10 Why rename this Oracle Database option From Oracle Spatial to Oracle Spatial and Graph Highlights existing graph capabilities in Oracle Spatial W3C RDF graph since Oracle 10gR2 Network Data Model graph since Oracle 10gR1 Addresses increasing market demand for graph database capabilities Social Network Graph database popularity Multimodal and integrated transportation, utility and communications networks 10

11 Oracle Spatial and Graph option Points Web Services (OGC) SPARQL End Point Lines Oracle Spatial and Graph Polygons Geocoding Routing Inferencing Rasters Network Graphs e1 f1 e2 n1 n2 f2 e4 e3 Topologies 3D RDF Semantic Graphs 11

12 Oracle Spatial and Graph option Spatial Features Spatial Data Types and Models Spatial Indexes and Analysis 12

13 Oracle Spatial and Graph option Graph Features Network Data Model graph W3C RDF Semantic graph 13

14 Goals for Spatial features in 12c 14

15 Goals for 12c Dramatically improve performance Core Spatial Functions Vector Performance Acceleration Parallel Raster Operations Reduce application logic and support real world analysis by moving complex spatial logic into the database Real world application features for Network Data Model Graph Server-based Raster Analytics and Image Processing for multi-terabyte cartographic modeling applications Richer 3D and Point Cloud analysis and visualization; Parametric curves RDF Semantic integration with Spatial Functions 15

16 Dramatically Improve performance Core Spatial Functions and Operations ANYINTERACT, INSIDE: 20-30x Oracle Database Locator GEOM DISTANCE: 40X WITHIN DISTANCE: 10X VALIDATE GEOMETRY: 4X 16

17 Vector Performance Acceleration Turbo-charger feature for spatial functions and operators Join: x Spatial and Graph option Performance Improvements Touch: 50x Contains, Overlaps: 50x Complex masks: 50x 17

18 Vector Performance Acceleration Turbo-charger feature for spatial functions and operations GEOM.relate: 5-10x Spatial and Graph option Performance Improvements DML single insert: 3x Coordinate System Transformations: 40-50% General DML operations: 30-50% 18

19 Parallel Raster Operations MANY RASTER FUNCTIONS CAN PARALLELIZE SERIAL OPERATIONS PERFORM UP TO 3X FASTER SCALES TO OVER 100X OF TIMES FASTER ON HIGHLY PARALLEL SYSTEMS 19

20 Virtual Mosaic and Image Processing Virtual Mosaic Support Any large collection of georeferenced GeoRaster objects, rectified or nonrectified, from one or more georaster tables Advanced spatial queries and on-the-fly transformation and mosaicking over it Mosaic of Landsat Images 20

21 Parametric curve support NURBS (non-uniform rational B-spline ) Mathematically precise representation of freeform surfaces and curves which could be exactly reproduced whenever technically needed. Used for highway and rail design, and surface models, like those used for aerospace exterior surfaces, and car bodies, etc. 21

22 Real World Feature Modeling in NDM Graph Feature Representation Network Representation 22

23 Planned Oracle 12c Spatial Features More Details 23

24 To 1 8 Performance Enhancements and Vector Performance Acceleration

25 Improved performance Core Spatial Functions and Operations SPATIAL INDEX STATISTICS New Algorithms to Collect Spatial Index Statistics Optimized Execution Plans 20-90% REDUCTION IN REDO LOGS Improved DML Performance Less Storage Required 25

26 Vector Performance Acceleration Oracle Spatial and Graph Turbo-charger feature OPTIMIZED METADATA QUERIES Kernel level caching Performance gains for DMLs and Spatial function calls Optimization especially noticeable in workflows with many fast running queries 26

27 Vector Performance Acceleration Oracle Spatial and Graph Turbo-charger feature ADVANCED ALGORITHMS Faster more efficient algorithms Improved memory management 27

28 Parallel GeoRaster and Enhanced Raster Operations 28

29 GeoRaster - In Database Raster Algebra Planned 12c GeoRaster Features Raster algebra supported through a set of algebraic expressions and PL/SQL subprograms Raster algebra operations work on individual raster cells, or pixels Apply algebraic functions on raster data to derive new results Generate new raster layer (map) from two or more raster layers Raster algebra operations to implement sophisticated analytical algorithms, such as a Normalized Difference Vegetation Index (NDVI), and TCT (Tasseled Cap Transformation) NDVI helps classify the amount of vegetation in a region, from none to rainforest 29

30 GeoRaster Image Processing Planned 12c GeoRaster Features Rectification of georeferenced raw images Orthorectification of georeferenced raw image with a DEM Image masking, Image stretching, and Image segmentation 30

31 GeoRaster Advanced and Virtual Mosaic Support Planned 12c GeoRaster Feature Mosaic can be persisted, or virtual Virtual (on the fly) mosaic defined by a SQL statement or view Supports gaps, no data, and overlapping regions User defined priority for overlapping regions (Date or SQL ORDER BY) More advanced mosaicking support: georeferenced raw images internal reproject/rectification common point rules (for example, max value, min value, etc...) simple color balancing large-scale image append 31

32 Enhancements to the GeoRaster Core Improve performance through parallel processing of all Raster Algebra and Mosaicking procedures Support relational Raster Definition Tables as an additional raster data storage Users can specify default alpha channel and default pyramid level in the metadata New Bi-quadratic Interpolation Method to interpolate cell value for any point in the grid Biquadratic Interpolation Method is also added as a new Resampling type in image operations and pyramiding Added OTHER resampling type to indicate an unknown or external pyramid resampling type imported from files Operation Progress Reporting is added into many new functions Significant performance improvement of pyramiding with cubic convolution resampling 32

33 Enhancements to the GeoRaster Core Other New Functions and Procedures getrasterrange: retrieve min and max cell values getrasterblocklocator: another new template function generatespatialresolutions: supports resolution generation for pyramids emptyblocks and recreatedmltriggers utility functions GeoRaster Java API Enhancement Support for ground control point (GCP) storage and manipulation, GCP georeferencing, reprojection, grid interpolations, and getcellvalue New Concurrent Batch Loading and Exporting Tool GUI to create GDAL-based batch loading and exporting description files (XML) 33

34 Parametric Curve Data Type 34

35 Parametric curve support NURBS Data type support Support Non-Uniform Rational B-Spline (NURBS) curve geometries NURBS curves represent arbitrary, free-form shapes Control points and knots guide the shape of the curve Enables complex shapes to be represented with little data Support for NURBS curves includes the following Functions to convert WKT/WKB and GML representations into SDO_GEOMETRY Validation functions Spatial indexing Functions and operators approximate NURB curves for computations (implicit densification). 35

36 NURBS Data type support Implementation conforms to SQL/MM standard SQL/MM standard for NURBS curves represents splines, polynomial splines, cubic splines, B-splines and Bezier curves. SDO_GEOMETRY object type supports NURBS curves representation 2D and 3D Curves supported SDO_GTYPE 2002 and 2006 for 2D Curve and MultiCurve SDO_GTYPE 3002 and 3006 for 3D Curve and MultiCurve 36

37 Simplified Java API for 2D and 3D SDOAPI contains an in memory R-Tree for primary filter operations New SDOAPI functions, enhanced to support the following in-memory operations Distance Inside AnyInteract New in memory functions support the following geometries types: Projected 2D and 3D Geodetic 2D and 3D NURB curve types 37

38 Real World Feature Model for Network Data Model Graph 38

39 Feature Modeling/Analysis Model networks with application features (instead of nodes and link) Similar to Topology Data Model but not limited to geometry based features Supports logical networks and physical networks in application representation Data model to manage node and link features with their associated network elements (Nodes/Links) Node features: transformers, sub-stations, etc. Link features: power lines, transit routes, etc. Consistency between network features and network elements automatically maintained Feature level analysis Find the shortest path between two transformers Find the shortest path between two transformers, but use only a certain wire type 39

40 Oracle Spatial and Graph - Network Data Model Best Route Using Traffic Pattern Information Temporal Modeling/Analysis Traffic Patterns Record historical travel patterns for different classes of roads Data collected based on time of day and day of the week NDM can use traffic patterns to compute shortest paths Find shorted path from point A to B with start time of 8 AM Find shortest path from point A to B and reach destination at 5.30PM Support NAVTEQ Traffic Patterns format out of the box 10 PM 8 AM 40

41 Oracle Spatial and Graph - Network Data Model Routes with Preferences (walk/car/bus/subway) Multi-Modal Routing Each mode (car, bus, rail, bike, etc) modeled as a separate network Single logical network represents all modes of transportation Transition nodes where networks meet NDM APIs can specify the modes to consider Out of the box support for transit data published by transit authorities GTFS (General Transit Feed Spec) supported Train & Bus Bus Only 41

42 Large Scale Drive Time/Distance Analysis For millions of customers, find closest store within a specified drive time Same underlying data for geocoder and road network Customers geocode as link id and percentage (instead of longitude/latitude) 5 mile Network Buffer generates all possible paths Each persisted path includes: Covered link IDs, nodes ID, and associated costs Single database query to find closest store and drive time/distance for each customer (join on link_id) Store Location Customer Location 42

43 Enhanced NDM XML API (Callable From PL/SQL) Provides a full fledged web services framework for network analysis Enhanced XML API with Network Constraint and Cost Calculator Support Integrated with Oracle Spatial and Graph Web Service Framework A Simple PL/SQL Wrapper (http put/get) on top of the XML API New Driving Direction API For NDM users with data in Routing Schema Generate step by step driving directions format for final path 43

44 Oracle Spatial and Graph - Routing Engine Enhancements Traveling Salesman Analysis (TSP) Optimal multiple-stop route Open or closed path 44

45 Oracle Spatial and Graph - Routing Engine Enhancements Enhanced Navigation Support Multi-link restrictions Highway has entry point, and left Restrictions where there are no physical barriers Left lane only exit Enter highway with no turn restriction on upcoming left hand exit. 45

46 Router TSP Route Oracle Confidential

47 3D and Point Cloud Enhancements 47

48 US Army Corps of Engineers is making extensive use of the Point Cloud data type in Oracle Database 12c Spatial and Graph option to store, retrieve, subset and analyze LiDAR data used to model 3D landscapes to support in-theatre, mission-critical DoD operations. Across a number of instances for the Corps of Engineers and our DoD partners, we are storing more than 150Tb of data including 100Tb of point cloud data (with more on the way). A single instance currently stores more than 1 Trillion points. Compared to Oracle Spatial and Graph 11g Release 2, the improvements in Oracle Spatial and Graph 12c have dramatically increased scalability for multisession point cloud creation and provide a considerable savings of storage space. Michael Smith US Army Corp, Remote Sensing Center 48

49 Enhancements for Point Cloud data Pyramiding support for PC and TIN data PL/SQL API added to generate pyramids for existing point clouds Points can be repeated in the pyramid levels or mutually exclusive Useful for visualization applications Contour generation from PC data Generate linear geometries that connect points with equal elevation values A polygon can be specified to limit contour generation to a region The contour generation process is grid-based in x and y. Krigging performed to assign heights to grids with no point cloud data. 49

50 3D Geodetic Support DISTANCE, ANYINTERACT and INSIDE operations take height into account LRS functions also support 3D geodetic data Height for geodetic data is usually supplied as meters Or it can be any other UNIT type defined by a coordinate system 50

51 3D Distance Calculation For Geodecic Segment Length of geodetic segment whose endpoints have different heights Endpoint with greatest height (P1) is projected onto the sphere of (P2) and (P3) is generated. The distance between P2 and P3 is spherical The distance of P1 to P2 is calculated as the length of side (P1,P2) of triangle (P1,P2,P3). P2 P1 P3 51

52 RDF Semantic Graph Analysis with Spatial Functions 52

53 Ontology Assisted Query of Gazetteers Ontology-assist Query Standard Spatial Query 53

54 Semantic GIS Spatial and Ontology-based Query Water Features Table Feature_ID Feature_Type Geometry 1 Bourne SDO_GEOMETRY( ) 2 Influent Stream SDO_GEOMETRY( ) 3 Brook SDO_GEOMETRY( ) 4 Canal SDO_GEOMETRY( ) Find all Streams inside a query search window SELECT feature_id FROM TABLE (SEM_MATCH( spatial predicate, rdf predicate); SEM_MATCH combines spatial operators (anyinteract, inside, nearest neighbor, within_distance, etc..) with ontology predicates. 54

55 Towards Qualitative Spatial Reasoning Don t always have quantitative geometry data Textual Descriptions Next to Westin St. Francis Hotel Inside Union Square How can semantics help? Apply description logic (reasoning) to spatial relations Westin St. Francis Hotel is inside Union Square, therefore it is inside San Francisco. inside is an owl:transitiveproperty, inside is owl:inverseof contains, Logical reasoning can be faster than spatial computation of geometries Example: Find all parks inside California Minimizes spatial computation on large number of geometries (parks) 55

56 Web Feature Server

57 Oracle Spatial and Graph - Web Feature Server 1.1 Web-based Administrative console Menu driven GUI simplifies registration of spatial layers with Oracle Spatial and Graph s WFS Browse existing spatial layers Eliminates the need for DBA to run PL/SQL scripts to publish spatial layers Includes tutorial on how to configure and use WFS Provide sample request and response pages for WFS queries Can also be used as a client to other WFS servers In this mode, only browsing options are enabled 57

58 Benefits with Exadata Database Machine 58

59 EHCC and Spatial Point, Line and Polygon geometries can all benefit from EHCC Lines and Polygons, they must be stored inline (less than 4K in size). Options include: COMPRESS FOR QUERY LOW COMPRESS FOR QUERY HIGH COMPRESS FOR ARCHIVE LOW COMPRESS FOR ARCHIVE HIGH 59

60 EHCC and Spatial Two ways to compress: Create Table As Select Direct Path Inserts 1. Create Table As Select CREATE TABLE edges_compressed COMPRESS FOR QUERY LOW NOLOGGING AS SELECT * FROM edges; 60

61 EHCC and Spatial 2. Direct Path Inserts (full code example in presentation appendix) -- PL/SQL Example with append_values hint. DECLARE id_tab ID_TAB_TYPE; edge_tab GEOM_TAB_TYPE; BEGIN -- Population of id_tab and edge_tab shown in presentation appendix FORALL i IN edge_tab.first.. edge_tab.last INSERT /*+ append_values */ INTO edge_ql VALUES (id_tab(i), edge_tab(i)); COMMIT; 61

62 EHCC and Spatial Uniform Geometries Strategy For Much Higher Compression Rates Uniform geometries spatial layers have the same number of coordinates in every row. Some examples: Point data (x NUMBER, y NUMBER) Box polygon (lx NUMBER, ly NUMBER, ul NUMBER, uy NUMBER) Two point line (x1 NUMBER, y1 NUMBER, x2 NUMBER, y2 NUMBER) Four point polygon (x1 NUMBER, y1 NUMBER,, x5 NUMBER, y5 NUMBER) For much higher compression rates, store uniform geometries as a series of NUMBER columns instead of SDO_GEOMETRY 62

63 EHCC and Spatial Uniform Geometries Box Polygon With Function Based Index - Example Create a function based index on uniform geometries to perform spatial queries The following ANYINTERACT queries were run on a 116 million row table Query High compression 18.17x queries still very fast. Anyinteract Query Uncompressed Query Low 3.92x Query High 18.17x Archive High 21.57x 10 acre polygon 1.86 sec 2.02 sec 2.7 sec sec ( rows returned) (1.08x) (1.45x) (6.85x) 63

64 Q & A 64

65 Graphic Section Divider 65

66 66