Paul Ramsey PostGIS Gotchas. The Things You Didn't Know That Lead to Confusion and Dismay. Wednesday, November 18, 15

Transcription

1 Paul Ramsey PostGIS Gotchas The Things You Didn't Know That Lead to Confusion and Dismay

2 Happy GIS day! What do you call the day after GIS day? PostGIS Day is tomorrow

3 Paul Ramsey PostGIS Gotchas The Things You Didn't Know That Lead to Confusion and Dismay

4 Honestly, I have no idea why it works like that...

5 Projections

6 Map projections are easy to misunderstand, and is not obvious how the database handles them

7 I m a web developer, I don t have to care about projections. Latitude, longitude, and leave me alone.

8 POLYGON(( , , , , , , , , , ))

9 every geometry has an srid spatial_ref_sys - srid - srtext

10 srid = 3005 POLYGON(( , , , , , , , , , ))

11 SELECT proj4text srid = 3005 FROM spatial_ref_sys WHERE srid = 3005

12 +proj=aea +lat_1=50 +lat_2=58.5 +lat_0=45 +lon_0=-126 srid = x_0= y_0=0 +ellps=grs80 +towgs84=0,0,0,0,0,0,0 +units=m +no_def

13 Screwing Up Projections

14 I m a web developer, so I loaded the county data in latitude, longitude... shp2pgsql - s 4326 parcels.shp

15 srid = 4326 POLYGON(( , , , , , , , , , ))

16 ALTER TABLE parcels ALTER COLUMN geom srid = 4326 TYPE Geometry(POLYGON, 3005) USING ST_SetSRID(geom, 3005)

17 srid = 3005 POLYGON(( , , , , , , , , , ))

18 ST_SetSRID(geometry, integer) vs ST_Transform(geometry, integer)

19 ALTER TABLE parcels ALTER COLUMN geom srid = 3005 TYPE Geometry(POLYGON, 4326) USING ST_Transform(geom, 4326)

20 srid = 4326 POLYGON(( , , , , , , , , , ))

21 ST_Distance(a,b)

22 Lack of Projections

23 Los Angeles POINT( ) POINT( ) Paris

24 ST_Distance() Los Angeles POINT( ) POINT( ) Paris Actual Distance: 9107km

25 ST_Transform(..., 3857) ST_Distance() Los Angeles POINT( ) POINT( ) Paris Actual Distance: 9107km

26 ST_Transform(..., 3857) ST_Distance() * cos(radians(lat)) Los Angeles POINT( ) POINT( ) Paris Actual Distance: 9107km

27 ST_Distance(a::geography, b::geography) Los Angeles POINT( ) POINT( ) Paris Actual Distance: 9107km

28 Dude, why didn t you just tell me that right away?

29 Geography

30 Pythagoras (Plane) double dx = x2 - x1 double dy = y2 - y1; double d2 = dx * dx + dy * dy; double d = sqrt(d2);

31 Haversine (Sphere) double R = ; /* meters */ double d_lat = lat2-lat1; /* radians */ double d_lon = lon2-lon1; /* radians */ double sin_lat = sin(d_lat/2); double sin_lon = sin(d_lon/2); double a = sin_lat * sin_lat + cos(lat1) * cos(lat2) * sin_lon * sin_lon; double c = 2 * atan2(sqrt(a), sqrt(1-a)); double d = R * c;

32 GEOGRAPHY Functions ST_Area(g1) ST_Distance(g1, g2) ST_DWithin(g1, g2, d) ST_Intersects(g1, g2) ST_Covers(gpoly1, gpt2)

33 ST_Buffer(geography)

34 ST_Buffer(geography) SELECT geography( ST_Transform( ST_Buffer( ST_Transform( geometry($1), _ST_BestSRID($1) ), $2), 4326) )

35 _ST_BestSRID(geog)

36 _ST_BestSRID(geog) _ST_BestSRID(geog1, geog2)

37 ST_StartPoint(geography) SELECT geography( ST_StartPoint( geometry($1) ) )

38 Geodetic Reasoning

39

40

41 ST_Segmentize(geography, )

42 ST_Segmentize(geometry, 0.01)

43 Why does ST_Extent(ST_Transform(geom)) not equal ST_Transform(ST_Extent(geom))?

44 ST_Transform()

45 Dude, stop talking about projections.

46 Unpredictable Performance

47 start-up nerd says...

48 I want to reverse-geocode 1000 points per second... Given a point, tell me what polygon(s) it is in. (a) I have a real-time stream of 1000 points per second (b) I have 1M points I need to process in 1000 seconds

49 (a) I have a real-time stream of 1000 points per second SELECT * FROM polys WHERE ST_Intersects( geom, ST_SetSRID( ST_MakePoint({x},{y}), {srid})); Lots of small queries Find the candidate polygon(s) using r-tree index Test each candidate for point-in-polygon edge by edge: O(N)

50 (b) I have 1M points I need to process in 1000 seconds SELECT pt.id, poly.id FROM poly, pt WHERE ST_Intersects( poly.geom, pt.geom); One big query Nested loop, tends to send the same polygons into the query, over and over PostGIS holds a cache of most recent polygon, with edge index built on it: O(log(N))

51 I want to reverse-geocode 1000 points per second... Given a point, tell me what polygon(s) it is in. (a) I have a real-time stream of 1000 points per second (b) I have 1M points I need to process in 1000 seconds

52 Mysterious Operators

53 Database nerd says, So you mentioned indexes and r-trees, but I haven t seen any operators yet...

54 CREATE OR REPLACE FUNCTION ST_Intersects(geometry, geometry) RETURNS boolean AS 'SELECT $1 && $2 AND _ST_Intersects($1,$2)' LANGUAGE 'sql' IMMUTABLE; Simple Features For SQL

55 &&

56 CREATE OPERATOR CLASS gist_geometry_ops_2d DEFAULT FOR TYPE geometry USING GIST AS STORAGE box2df, OPERATOR 1 <<, OPERATOR 2 &<, OPERATOR 3 &&, OPERATOR 4 &>, OPERATOR 5 >>, OPERATOR 6 ~=, OPERATOR 7 ~, OPERATOR OPERATOR 9 &<, OPERATOR 10 <<, OPERATOR 11 >>, OPERATOR 12 &>, OPERATOR 13 <-> FOR ORDER BY pg_catalog.float_ops, OPERATOR 14 <#> FOR ORDER BY pg_catalog.float_ops, FUNCTION 8 geometry_gist_distance_2d (internal, geometry, int4), FUNCTION 1 geometry_gist_consistent_2d (internal, geometry, int4), FUNCTION 2 geometry_gist_union_2d (bytea, internal), FUNCTION 3 geometry_gist_compress_2d (internal), FUNCTION 4 geometry_gist_decompress_2d (internal), FUNCTION 5 geometry_gist_penalty_2d (internal, internal, internal), FUNCTION 6 geometry_gist_picksplit_2d (internal, internal), FUNCTION 7 geometry_gist_same_2d (geom1 geometry, geom2 geometry, internal);

57 CREATE OPERATOR CLASS gist_geometry_ops_2d DEFAULT FOR TYPE geometry USING GIST AS... CREATE OPERATOR CLASS btree_geometry_ops DEFAULT FOR TYPE geometry USING btree AS OPERATOR 1 <, OPERATOR 2 <=, OPERATOR 3 =, OPERATOR 4 >=, OPERATOR 5 >, FUNCTION 1 geometry_cmp (geom1 geometry, geom2 geom

58 CREATE INDEX myindex ON mytable (geom); CREATE INDEX myindex USING GIST ON mytable (geom);

59 SELECT * FROM mytable ORDER BY geom; SELECT * FROM mytable GROUP BY geom; Sort on xmin of bounding box unless they are equal... Sort on ymin of bounding box unless they are equal... Sort on xmax of bounding box unless they are equal... Sort on ymax of bounding box unless they are equal... Return equality

60 CREATE INDEX myindex USING GIST ON mytable (geom); CREATE INDEX myindex &&& USING GIST ON mytable (geom gist_geometry_ops_nd); CREATE INDEX myindex USING GIST ON mytable (geog);

61 Mysterious Emptiness

62 philosophy nerd says life is emptiness

63 ST_Intersection(A, B) A B

64 ST_Intersection(A, B) POLYGON EMPTY is not NULL

65 ST_Union(C, ST_Intersection(A, B)) A C B

66 ST_Union(C, POLYGON EMPTY) C

67 GEOMETRY EMPTY Intersection of disjoint objects Simplification using tolerance larger than the object Negative buffering using radius larger than the object Differencing larger object from smaller

68 More Unpredictable Performance

69 GIS nerd says

70 Overlay

71 SELECT a.id, b.id, ST_Intersection(a.geom, b.geom) FROM a, b WHERE ST_Intersects(a.geom, b.geom);

72

73 SELECT a.id, b.id, CASE WHEN ST_Contains(a.geom, b.geom) THEN b.geom ELSE ST_Intersection(a.geom, b.geom) END AS geom FROM a, b WHERE ST_Intersects(a.geom, b.geom);

74 Prepared Geometry ST_Contains ST_ContainsProperly ST_Within ST_Covers ST_Intersects

75 1 Problem Many Solutions

76 philosophy nerd says when is a buffer not a buffer?

77 SELECT a.id, b.id FROM a, b WHERE ST_Intersects( a.geom, ST_Buffer(b.geom, 500) ); SELECT a.id, b.id FROM a, b WHERE ST_Distance(a.geom, b.geom) < 500; SELECT a.id, b.id FROM a, b WHERE ST_DWithin(a.geom, b.geom, 500);

78 CREATE OR REPLACE FUNCTION ST_DWithin(geom1 geometry, geom2 geometry, float8) RETURNS boolean AS 'SELECT $1 && ST_Expand($2,$3) AND $2 && ST_Expand($1,$3) AND _ST_DWithin($1, $2, $3)' LANGUAGE 'sql' IMMUTABLE;

79 Raster Disaster

80

81

82

83 Hidden Features

84 SET postgis.backend = geos ; SET postgis.backend = sfcgal ;

85 ST_3DIntersection ST_Tesselate ST_3DArea ST_Extrude ST_ForceLHR ST_Orientation ST_Minkowski ST_StraightSkeleton

86 New to 2.2! ST_ApproximateMedialAxis()

87 CREATE EXTENSION postgis_sfcgal; CREATE EXTENSION postgis_tiger_geocoder; CREATE EXTENSION postgis_topology; CREATE EXTENSION postgis_raster;

88 Even More Hidden Features

89 github.com/pgpointcloud

90 pgrouting.org pgrouting: A Practical Guide

91 Honestly, I have no idea why it works like that...

92 Paul Ramsey PostGIS Gotchas The Things You Didn't Know That Lead to Confusion and Dismay