Integrating Spatial Data with the rest of your E&P Data

Transcription

1 Integrating Spatial Data with the rest of your E&P Data ESRI PUG Houston 11-March-2003 Ian Batty PPDM Association 1

2 PPDM Association Public Petroleum Data Model Association The PPDM Association is a non-profit organization through which members world-wide cooperate to develop standards as a foundation for managing information as an essential asset in the global resource business. Technical Objectives Industry Standard data model Data content Data exchange project Spatial enabling project Background The PPDM is the Public Petroleum Data Model Association- it has a mission statement and a number of technical objectives. The PPDM is a physical implementation of a data model that represents data and data relationships used in E&P. The objective that is particularly pertinent to this presentation is the Spatial Enabling Project. Some of you may be some of you intimately acquainted with PPDM, some only vaguely familiar, and a good group of you are probably wondering what does a data model organization have anything to do with the PUG? or Didn t these guys go out of business years ago?. The answer to the second question, quite obviously, is no. The answer to the first question is a little more involved but will hopefully be answered by the time I stop talking. The key thing to note about the Projects, as opposed to the data model, is that they are completely OPEN to all and ANYONE to participate in. If you want to set the direction that the project takes, step up to the plate and sponsor the project. If you want to observe, come and attend the meetings. If you can t attend, sign up to the list- you will receive the presentation and notes from the meeting. But if you just want to use the results of the project, then that is cool too- but to gain the real benefit you may need to join the Association and be a member of the PUG. Oh, and the final thing to note about the Projects is that as well as producing guidelines, recommendations and methodologies on how to do things, they also produce real live deliverables that are actually implemented in a technology (or 2). The methodologies are as open as they can be- the implementation, by its very nature has to pick a technology. 2

3 Spatial Enabling Background ESRI PUG 2001 & 2002 Companies tried to start spatial initiative March 2002 Funding obtained Technical resources acquired October 2002 First deliverables made available PPDM & the PUG We need a little background to understand how PPDM fits into the GIS world. ESRI has developed a number of data models for different vertical industries that allow people to move from file based data systems into the Geodatabase. The challenge facing the Petroleum Industry was how to move from an existing data model to a Geodatabase. Back in 2001 people began to think that creating this standard Geodatabase model would be a good idea, but being the real world it took another PUG and a lot of background work to actually get cheques signed so that resources could be obtained and start the technical work. By October of last year the first set of deliverables were made available at 3

4 Deliverables available now Spatial enabling methodology Well module Software and scripts Geometry generation for wells Sample dataset SDE Binary Oracle Spatial UML for Geodatabase creation Well header Well test x a,y a,z a, MD a x b,y b,z b, MD b x m,y m,z m, MD m Spatial Enabling Methodology The first problem that people had was simply the problem of how do you get data from a relational database and into a GIS? As wells are quite important to the oil industry we tackled them first and have created a methodology that allows well data in a relational database to be modeled in UML and then the data transformed via some software into a spatial database. The basic approach is to model a well as a 3D line with measures. The measure value is the measured depth. This allows all data in the well module that has a MD value associated with it to be displayed on a GIS. The business tables themselves do not need to be spatial enabled, the geometries do not need to be maintained, but the data can still be displayed. However, just telling people how to do something is not as useful as actually doing it. So you can also download the scripts to create the geometry and can even get sample databases that have been spatially enabled. If you don t want to use a full Geodatabase but need cross application capability too- then that is catered for as well by creating an OGC compliant simple feature database. But the real live implementations of these 2 options are in SDE Binary and Oracle Spatial storage formats. To allow people who are not familiar with a PPDM database we also provided some high level UML diagrams for the well header and well test module. And finally, we provided a detailed UML document that ArcCatalog can read to create a Geodatabase. 4

5 Deliverables for 2003 Data management methodology Software and scripts Data management procedures Geometry generation for all other objects in PPDM UML for Geodatabase creation For all objects in PPDM Sample dataset Simple feature database Geodatabase Day to Day Pragmatic Stuff So then we wanted to use all this good thinking and cool toys and get the wider world to realize that GIS and spatial data were a) pretty neat stuff and b) really should be part of your data management strategy or you are in for some fairly nasty data busts. This meant we had to come up with a way of integrating spatial data into your data management methodology. As with the previous set of deliverables, we didn't just provide the document telling you how to do something, so we created some sample scripts so that people can see how to do it, and provide a couple of sample datasets for people to play with. 5

6 Deliverables for 2003 Methodology to capture object behaviour Assigning a well to its field according to the location of the completion Modeling object changes through time 4D seismic Licensing rounds Modeling objects with stratigraphic depth Never mind 2.5D this is 3D + a bit Head banging cool stuff The other objective for this year is to really try to stretch the capabilities of spatial data and GIS tools and see how we can apply the power of the tools to really make a difference to data management work practices and quality of data. Object Behaviour Database technology allows you to be sure that a well is assigned to a valid field name because it has to exist in the list of field names, but only a spatial database will allow you to check that the well actually exists within the boundaries of that field. Capturing object behaviour and then implementing that behaviour in a spatial database will provide spatial data integrity. Changes through time The oil and gas industry works in a number of dimensions- we have been able to use GIS to address the first 3 dimensions that we typically deal with (Oh, OK, 2.5D) but we are faced with the 4th dimension- how does a GIS handle 4D seismic? How does it deal with governments changing the license blocks over time? The answer may be a bit like versioning, but different, but we shall see where it goes. Stratigraphic depth But oil and gas has even more dimensions to handle: stratigraphic depth: Do we model this as a combination of a linear referencing system on a polygon with depth values and a measure of the stratigraphy and display that in with a TIN? The straightforward answer is we don t know yet- but given the way that we have been able to completely overhaul the way we model wells, who knows, maybe we can do the same for the rest of the data in PPDM 6

7 Data Management in 1995 App 1 App 2 GIS Big Bucket Database Spatial DB Introduction But enough of the stuff that we don t know about- the only way to get spatial data accepted is to show that it is very important and that it needs to be incorporated into your data management practices. To understand why I believe the data isn t incorporated I want to give brief overview of the evolution of data management, purely as I see it, over the last few years. Slide body Back in 1995 we had lots of data, coming in in various different formats and a whole load of people to process it. Their main job was to provide the data to a series of applications so that users could use it. And wouldn t it be nice if we only had to deal with 2 applications. Obviously there were a lot more applications than 2 and so the thinking at the time was let s put it all into 1 big bucket and then all the applications can work of that database There were a couple of different options for the big bucket but that was the way to do it. Guess what? The big bucket didn t work. The experience of a large number of people has shown that if you are still trying to use a big bucket today you are in for a really hard time. Spatial data? Oh yes, spatial data was doing its own thing off to one side. 7

8 Data Management in 1999 App 1 App 2 Project DB 1 Project DB 2 GIS Integration DB Spatial DB Slide Body So we move onto We still have data arriving in lots of different formats- in fact we have more data arriving than ever before. But in a bout a true logical thinking- there has been a round of layoffs and we don t have as many people working on that data. We still have the applications at the top of the picture- but the various vendors, bitten from the failure of the big bucket have gone away and created their own project stores. Some have even created integrated project stores. But even the integration doesn t address all the functionality, so we still have multiple stores. And the end-users are happy.the data management department is working overtime have to load all this data into multiple data sources. And there is a little itch in the back of the minds of the data management people saying we have to make these stores talk to each other The idea that people came up with was to make a little integration layer- but it should be as thin and small as possible- just store the cross references and the system will work. The databases will talk to each other and we won t duplicate any data. Except that when we came to build these systems they got so complex that they became basically unmaintainable. I know, I tried to build one. Meanwhile, spatial data was doing its own thing, off to one side. 8

9 Data Management in 2001 App 1 App 2 App... Project DB 1 Project DB 2 GIS Integration DB Spatial DB Data Management Best Practices Source DB Slide Body So we move on to The world has changed- we are now getting our data from a vendor who does all the formatting that the data management department used to do. So now we only have 1 person working there- but he knows a LOT about the database. The applications are still being used by the end user and they are all working from their own project stores. The problem facing the users now is physically finding the data and knowing when new data arrives or is updated. GIS is a GREAT way of doing that- people can look at a map and in seconds know if there are any changes or if things need to be updated. So we need to move the GIS system into the main data flow. And it can be used to decide which data is moved into the project stores. So that needs a spatial database in the main flow. And fortunately the vendor can supply the data so that we can load it straight into the project store. But you know that the world is never straightforward. The data management department, in their infinite wisdom have created a series of Best Practices and all the data has to go through these best practices. The problem is that the data management department understands attribute data really well- so that can through the best practices, but they don t understand spatial data. So we finish up with the source data going straight into the spatial database. The other issue that we face is that because the source data is coming from a number of different sources, all with best practices different from our own we need to store the attribute data in some sort of integration layer. This layer has to store all the data that most users will ever want to see and everything that can be displayed on a map. The alert people may be thinking this looks a lot like a big bucket The difference is that there are still applications that run of their own source data. Completely outside of the main flow. This was the failing of the big bucket- it tried to cater for every application- the integration layer is much more limited in its ambition- it just wants to cater for most of the people most of the time- but it also wants to allow you to know what data the other applications are using, even if it can t store the details about that data. So we finish up with a main flow and others. But the spatial data and the attribute data in the integration layer don t really talk to each other. That itch that the data management department had? It is still there. Not as big, but itching none the less and the fear of data busts is just as large as ever 9

10 Data Management in 2003 App 1 App 2 App... Project DB 1 Project DB 2 GIS Spatially Enabled Integration DB Data Management Best Practices Source DB Slide Body Which brings us up to date. The good news is that unlike the real world, the data management world has not changed that much since Source data still arrives, applications exist, they use their own data stores, people still like to use a GIS to understand the data and best practices still cause as much trouble as ever, and I should add, still solve as many problems as ever. The only difference is that there is a methodology to integrate your spatial data with your attribute data and create a single, consistent, clean, contained integration layer. 10

11 Integration Database Data storage Normal" attribute data Anything that can be displayed on a map Storing spatial data In a spatial storage format Integrating multiple data stores Structured data Integrating data types Unstructured data Values to catalogue EDMS Integration Database The question then becomes "what is the integration database" Normal Attribute Data It is very difficult to define what is normal - but whatever you think is normal, you can be sure that someone else thinks a bit differently. So you will need a pretty comprehensive data model. The other thing to note is that the integration layer is going to duplicate data. But, if you do it right- the data in the database has been passed through all your best practices and data quality processes. You can call it information. Go on- you know you want to- and your manager will love it when you do. Just don t call it knowledge. That gets all sorts of attention. And most of it the wrong sort. The level of data duplication can be decided company by company but to use GIS and Geodatabase there needs to be a minimum of data in it Spatial Data Not only do you need to store the spatial data- you need to store it in a format that the GIS can understand- a spatial storage format. You can choose any one- but when you come to implementing a system you will probably need to choose 1. Or maybe 2. Integrating data stores This was the original objective of this layer- so you had better make sure you can handle the fact that in Argentina a well can have up to 7 names - and 2 of them were provided by the government. Or in the UK the official government name for a well not only has spaces in it, but also slashes. Most of our applications can t handle that- so we had better find a way of naming wells so that the computer doesn t barf. Oh yeah, and when it comes to the spatial data, don t forget that Columbia and Ecuador have different definitions of where a basin is and neither of them agree with the one that the geologists in your company want to use. The integration database should be able to store all the information associated with these problems. However, actually solving these problems will involve a large amount of process work - there is no way that technology alone can solve all these problems 11

12 Integration Implementation Define best practices Naming conventions Implement integration database Populate cross reference tables Populate attribute tables Relational model provides data constraints Populate spatial storage format data Implement data management processes Technology Think of this as the box- there probably isn t one out there that exactly meets your needs. Your choice will depend on the skills of the personnel in your organization- do they need a GUI? Are they comfortable with handling data and databases? How much data are you dealing with? How often does it change? What is the cost to your organization of the problem at the moment? How quickly do you need the solution? How many datatypes are you dealing with? How many databases storing the same information are you dealing with? What data do you users want to see? Spatial integration You need to have a system so that when a seismic line goes from being a planned line, to one that has been surveyed in, your users get to see the new, correct information as soon as possible. And they will want to see it on the map. When a deviated well is resurveyed you need to make sure that the x,y locations of the formation tops that your geologists have picked get plotted in the correct place. Data Management Objectives Most organisations have a visionary who drives data management with objectives like the data needs to be clean, correct, complete, contained etc. Those objectives will be captured in the Best Practices, make sure that they are not lost in the integration implementation. 12

13 Information Observations Best practices are essential Specialized applications will always exist outside the main data flow Use a GIS to display Integration database Integration occurs before display Best Practices Most organizations recognize that best practices are essential. However, actually getting round to designing them, and more importantly, implementing and maintaining them is expensive and it hurts your head. However, in order to maintain a system you MUST have them. End of sermon. Specialized application The big bucket failed by trying to cater to all applications. Accept that there will always will be applications outside the flow. But if you accept that, at least you can manage it. Use a GIS GIS's are really cool. Use them- a lot. And good things will happen to you. End of sales pitch for GIS technology 13

14 Contact Information PPDM Web Vendor Faire Booth 12 14