Oracle Spatial Summit 2015 Effectively Integrate Geospatial Technologies from GIS to Oracle Spatial in Real Estate Sector

Transcription

1 Effectively Integrate Geospatial Technologies from GIS to in Real Estate Sector Francisco Javier Rojas Duran, IT Project Manager Professional DASOFT

2 DASOFT RUV (Housing Registry Authority) - SACP OVERVIEW Customer : RUV (Housing Registry Authority) The Only Housing Registry Authority in Mexico, Supports around 350 Real Estate Companies. Storing all the geospatial data related to dwellings subject of subsidization and data related to Important Points of Interest (Schools, Markets, Recreational Areas, etc.) CHALLENGES / OPPORTUNITIES Need for speed-up the process of Delivery, Reception and Qualification of the Housing Base Maps sent by the Real Estate companies using Autodesk Technology. Need to perform spatial calculation of distance between houses and particular point of interest. Need to display the houses into a Map. SOLUTIONS Database 11g Enterprise Edition Option Autodesk Infrastructure Map Server 2014 AutoCAD Map 3D RESULTS Operation switched from a manual procedure by Geospatial Operators to automatized spatial queries and distances calculations. Achieved 10% performance improvement and lower query s elapsed time vs semi-manual process. From 20 minutes per Map (From 1 up to 1000 houses) with the aid of GIS to 5 seconds per House. (House based performance) using DB. 218 =18 minutes. Less Human Resources needed for operation, some of them were promoted to Geospatial Analyst position. Longer service time (24x7) instead of (8x5). Consolidation of vector data in a central repository. Geospatial data store enabled for Business Intelligence.

3 Speaker Bio Francisco Javier is a Computer Science Engineer with a Master Degree in IT Management based on ITIL/ITSM He has proved experience in Geographic Information Systems for 10 years. He has been working either with Autodesk & Technologies. He has been trained in Technology, PL/SQL, SQL and DBA fundamentals. He has worked on Real Estate projects including the integration between Property Manager and Autodesk software. Currently he is working on spatial analysis performed by from data exchange performed by Autodesk technology, using Autodesk technology to display data into a map. He has been speaker in Autodesk University Extension in Mexico City.

4 Agenda 1) Customer Bio and the Business Requirement, Project Scope and Main Objectives. 2) Best Practices for Importing GIS Data from Autodesk Technology to. 3) Performing Analysis with PL/SQL. 4) Tips & Tricks.

5 Objectives To describe the project and business case. To show how spatial data Integration was done. To show which issues came out regarding spatial analysis. To point out some tips regarding Operators Performance.

6 DASOFT RUV (Housing Registry Authority) OVERVIEW Customer : RUV (Housing Registry Authority) The Only Housing Registry Authority in Mexico. Supports around 350 Real Estate Companies. Storing all the geospatial data related to dwellings subject of subsidization and data related to Important Points of Interest (Schools, Markets, Recreational Areas, etc.) CHALLENGES / OPPORTUNITIES Need for speed-up the process of Delivery, Reception and Qualification of the Housing Base Maps sent by the Real Estate companies using Autodesk Technology. Need to perform spatial calculation of distance between houses and particular point of interest. Need to display the houses into a Map. TECHNOLOGY COMPONENTS Database 11g Enterprise Edition Option ( ) Autodesk Infrastructure Map Server 2014 AutoCAD Map 3D Virtualized Environment.

7 Project Scope To develop a web-based GIS that analyzes the dwellings by performing two main types of spatial queries, Relationship and nearest neighbor, the latter retrieves the nearest distance from each dwelling to pre-defined and user-defined points of interest. By enabling this technology, the customer s aim is to have a shorter response time to Real Estate companies that sent their information. Furthermore, to be accurate in the calculation by using the geometry of the dwelling s perimeter instead of using the centroid as it was being used before the implementation of this project.

8 Project Objectives To redefine the process of Reception, Verification, and Qualification of the housing base maps (just a set of dwellings drawn in a AutoCAD Drawing together with infrastructure features like schools, squares, supermarkets, mini markets, health care centers and so forth) sent by the Real Estate companies using Autodesk Technology. To store the map data after being received and validated into DB.

9 General Procedure for Analysis 1) Creation of an SDF file with the housing base map from an AutoCAD Map Drawing. 2) The developer loads the file through the system. 3) The map is reviewed and validated by Geospatial Operators. 4) The geospatial data from the map is transferred by the operators. 5) The transfer of the elements is performed using FDO Technology (fdo.osgeo.org). 6) The work order is scheduled in order to perform the queries. (3 times a day) 7) When the time comes the dwelling order is being processed with two main groups of queries: Relation between feature geometries. Distances. 8) The result is stored into a DB Table. 9) A data transfer is performed between this application and the main system.

10 Best Practices for Importing data from Autodesk to Feature Complaint Workaround Original Developer s drawing is feature object oriented. Data File format supports OGC geometry validation. Geometry exported in the same dimensionality as defined in the drawing Geometry loaded on the fly could be validated before spatial analysis No No No No Use of AutoCAD Map instead of plain AutoCAD Develop custom tools in order to fix incorrect oriented geometry and duplicated vertices. Use of API to convert dimensionality. VALIDATE_GEOMETRY_WITH_CONTEXT

11 Performing Analysis with PL/SQL. Requirements for Analysis regarding data layers regulated by federal agencies: Layers must have the same dimensionality (2D). They must be imported using AutoCAD Map 3D with FDO. They must be validated with API. VALIDATE_LAYER_WITH_CONTEXT They must have spatial indexes defined in a different table space. They must have unique/non-unique indexes for foreign keys.

12 Dealing with Analysis Queries Create two big groups of queries: Nearest neighbour distance. Needed to be performed for each dwelling against the closest point of interest not exceeding a maximum radius, and retrieving the resulting distance between the house and the point of interest. SDO_RELATE Operator. Needed to be performed either with a set of houses or for each dwelling.

13 Regarding Nearest Neighbour SDO_BATCH_SIZE=10 specifying a max batch size of 10 for improve performance, also allows you to have more than one WHERE condition for the SDO_NN geometry table. DISTANCE=X UNIT=METER specifying a maximum distance as requested in business rules. UNIT by default in geodesic context is meters. ROWNUM to restrict the number of results. SDO_NUM_RES should be used if only proximity matters otherwise use above. SDO_NN_DISTANCE performance improvement with FIRST_ROWS Hint, when queries are optimized for response time returns the first rows in the shortest time possible. (NEW in 11gR2)

14 Example of Nearest Neighbour Query: SELECT SCHOOL_ID,SCHOOL_NAME, D INTO ID,NAME, DIST FROM ( SELECT /*+ FIRST_ROWS */ SCHOOL_ID,SCHOOL_NAME, SDO_NN_DISTANCE(1) D FROM SCHOOLS S --,DWELLINGS DW WHERE S.TYPE = :TYPE More tan one WHERE condition for SDO_NN Geometry -- AND DW.CUV = :PID AND SDO_NN (S.GEOMETRY, :GEOM, 'SDO_BATCH_SIZE=10 DISTANCE=4000',1)='TRUE' ORDER BY D) WHERE ROWNUM < 2;

15 Using SDO_NUM_RES FOR VIV IN ( SELECT NAME FROM BIG_CITIES CITIES,SMB_CONSTRUCCIONES V WHERE V.CUV =PID AND SDO_NN (CITIES.GEOMETRY,V.GEOM, 'SDO_NUM_RES=1')='TRUE' ) LOOP NOM_LOC_URB := VIV.NAME; END LOOP;

16 Performance analysis

17 Why not use SDO_WITHIN_DISTANCE? SELECT MIN(SDO_GEOM.SDO_DISTANCE(S.GEOM,DW.GEOM,0.5,'UNIT=METER')) INTO DIST FROM SCHOOLS S, DWELLINGS DW WHERE DW.CUV = :PID AND S.TYPE = :TYPE AND SDO_WITHIN_DISTANCE (S.GEOM, DW.GEOM,'DISTANCE=4000')='TRUE'; SDO_WITHIN_DISTANCE may return inaccurate results due to a BUG in 11gR In 12c seems that this issue was solved and we have to verify the performance improvement for SDO_WITHIN_DISTANCE vs SDO_NN with SDO_NN_DISTANCE.

18 Performance SDO_WITHIN_DISTANCE SELECT S.DESCRIPCIO NOMBRESALUD, SDO_NN_DISTANCE(1) SALUDDIST FROM SALUD S,SMB_CONSTRUCCIONES V WHERE V.CUV =' ' AND SDO_NN (S.geometry,V.GEOM, 'SDO_NUM_RES=1 DISTANCE=10000 UNIT=METER',1)='TRUE';

19 Performance SDO_WITHIN_DISTANCE SELECT * FROM ( SELECT S.DESCRIPCIO NOMBRESALUD,SDO_GEOM.SDO_DISTANCE(S.GEOMETRY,V.GEOM,0.5,'UNIT=METER') SALUDDIST FROM SALUD S,SMB_CONSTRUCCIONES V WHERE V.CUV = ' ' AND SDO_WITHIN_DISTANCE (S.GEOMETRY,V.GEOM,'DISTANCE=10 UNIT=KILOMETER')='TRUE' ORDER BY 2 ) WHERE ROWNUM <2

20 Regarding SDO_RELATE The order of the geometry and query window matters! It could be common sense to query this way: SDO_RELATE (DWELLING.GEOM,DISTRICTS.GEOMETRY,'MASK=INSIDE+COVEREDBY+ TOUCH+OVERLAPBDYINTERSECT')='TRUE'

21 Regarding SDO_RELATE However since we do filter out all dwellings except one, we had to inverse the order in the spatial operator for performance improvement. SDO_RELATE (DISTRICTS.GEOMETRY,DWELLING.GEOM, 'MASK=COVERS+CONTAINS+TOUCH+OVERLAPBDYINTERSECT')='TRUE' We can use also the HINT /*+ ORDERED*/ but it depends on the explain plan.

22 Bulk vs Single queries We classified SDO_RELATE operations in two big groups: single row and multiple row queries. For the multiple row the resulting spatial relationship applies to all the dwellings, and we did update in a PL/SQL block, however in some queries we did use MERGE statement.

23 Tips Be aware HINTS for Operators may be helpful. Make sure geometries are valid. Use SDO_UTIL.INTERIOR_POINT to avoid always calculating the greatest area of the intersection of overlapping polygons. Analyze the different ways of writing a spatial query and review the explain plan. In 12c better execution plans are generated for spatial.

24 Tips Regarding PL/SQL: Use implicit cursors to avoid handling NO_DATA_FOUND exception. Use MERGE statement wherever is possible, better performance. Use dynamic SQL and create generic sub procedures. Regarding SDO_RELATE: Calculate the number of candidate rows of both tables, and use the fewer one as the query_window driver.

25 Example of Dynamic SQL DECLARE TABLENAME VARCHAR2(30); RADIUS VARCHAR2(100); DIST NUMBER; SQL_STMT VARCHAR2(500); COND VARCHAR2(50); BEGIN SQL_STMT := 'SELECT dist FROM (SELECT /*+ FIRST_ROWS */ SDO_NN_DISTANCE(1) dist FROM ' TABLENAME ' EQM, SMB_CONSTRUCCIONES V WHERE EQM.IDOFERTA = :OFERTA AND SDO_NN (EQM.geom,:PGEOM,:COND,1)=''TRUE'' ' ' ORDER BY SDO_NN_DISTANCE(1)) WHERE rownum < 2'; COND:='SDO_BATCH_SIZE=10 UNIT=METER DISTANCE=' RADIUS; EXECUTE IMMEDIATE SQL_STMT INTO DIST USING PGEOM,OFERTA,COND; EXCEPTION WHEN NO_DATA_FOUND THEN DIST:= NULL; END;

26 Final Achievements Time elapsed processing a house First Run 10 After SDO_RELATE Optimization 5.5 After SDO_NN Hints After using Merge & Interior Point 2 Time in seconds

27 Final Achievements Operation switched from a manual procedure by Geospatial Operators to automatized spatial queries and distances calculations. From 20 minutes per Map (From 1 up to 1000 houses) with the aid of GIS to 5 seconds per House. (House based performance) using DB. 218 =18 minutes. Less Human Resources needed for operation, some of them were promoted to Geospatial Analyst position. Longer service time (24x7) instead of (8x5). Consolidation of vector data in a central repository. Geospatial data store enabled for Business Intelligence.

28

29 References Best Practices with 11g and Fusion Middleware's MapViewer, 2010 OpenWorld 12-September-2010 ( html) Pro for Database 11g By Albert Godfrind y Euro Beinat

30