Visualization and clusters: collaboration and integration issues. Philip NERI Integrated Solutions Director

Transcription

1 Visualization and clusters: collaboration and integration issues Philip NERI Integrated Solutions Director

2 Overview Introduction, Paradigm & Clusters The Geoscience task map Seismic Data Processing / specifics of imaging sector Interaction, before, during and after CORBA technologies for integrated operation Visualization needs for terabyte seismic volumes Conclusions

3 Introduction Seismic data for hydrocarbon exploration / production "Identify and spatially locate percentile variations in acoustic properties 5 miles below surface, over 100s of square miles, using ship- born measurement systems" 30,000 x 15 kbytes recorded ever 15 seconds Months of 24 x 7 processing on super- large computers Purpose: define drilling locations for M$50 wells

4 Processing seismic data Data unit: terabytes Very large volumes From raw data to final result involves tens of processes Data volume does reduce as processing progresses Clusters: thousands of nodes Multi-tiered, 128-node building blocks Challenge: moving data to the nodes (and back) Data storage devices Interconnect bandwidth

5 Paradigm and clusters Paradigm: geoscience and engineering software Diversified range of software includes seismic processing Traditional trace processing, Pre Stack Depth Migration (PSDM) Leading visualization tools, both voxel analysis and 3D modeling Full software suite (15+ million lines of code) delivered on Linux Paradigm: global geophysical / reservoir services Processing centers on 5 continents Clusters replaced most multi-cpu architectures in 2001

6 Simplified Geoscience Task Map First IA32 clustered applications Geophysics Data Management & Interoperability Petrophysics Seismic Data Processing Reservoir Imaging Visualization & Interpretation Reservoir Characterization Well Log & Petrophysics Analysis Geology Mapping & Model Building Well Planning & Drilling Design Reservoir Engineering Drilling Engineering Facility Engineering Production Engineering Petroleum Engineering Oilfield Economists

7 Sequential / Trace by Trace processing Small data collections to work on 1 MB to 500 MB each Data requests can be anticipated No input during processing No changes in parameters, variables, data flows After trial run(s), process can run for hours, days Relatively little interaction between streams Interconnect bandwidth is seldom a bottleneck Average interconnect bandwidth used, little interconnect traffic Scalable to large amount of nodes (in tiered configuration) Move from 32 bit to 64 bit will improve some metrics

8 Simplified Geoscience Task Map Advanced clustered applications Geophysics Data Management & Interoperability Petrophysics Seismic Data Processing Reservoir Imaging Visualization & Interpretation Reservoir Characterization Well Log & Petrophysics Analysis Geology Mapping & Model Building Well Planning & Drilling Design Reservoir Engineering Drilling Engineering Facility Engineering Production Engineering Petroleum Engineering Oilfield Economists

9 Ray tracing in discrete volumes

10 Large aperture processes Variable data collections to work on 100 MB to the whole dataset, for each node / trace set Data requests cannot be anticipated (ray tracing, etc..) Iterative refinement of model, properties Observations during processing can lead to suspending processing, editing input data and resuming work Can run for days High interaction between streams Interconnect bandwidth is always a bottleneck Limit to the amount of nodes

11 Backplane becomes a major cost As the cluster size grows Larger data sets Larger data flows Larger node interconnection Interconnect costs dominate Overtake node costs Still a bottleneck Limits the size of largest cluster $ Nodes

12 Solutions Shared memory All nodes share a common access to memory Increased bandwidth in interconnect subsystem Brick formats for data Important caching in file server and nodes 64 bit clusters with stronger nodes (xx cpus w/ xxx GB RAM) Obvious questions about alternative architectures Cluster versus multi-cpu systems

13 Extending into interactive modes Interactive QC and parameter editing Geophysics Data Management & Interoperability Petrophysics Seismic Data Processing Reservoir Imaging Visualization & Interpretation Reservoir Characterization Well Log & Petrophysics Analysis Geology Mapping & Model Building Well Planning & Drilling Design Reservoir Engineering Drilling Engineering Facility Engineering Production Engineering Petroleum Engineering Oilfield Economists

14 Blocked Seismic : An optimized data format Allow consistent access times regardless of direction Multi- threading for improved performance Avoid excessive file sizes Concurrent access in different areas without conflict Relieve cache usage

15 Data Integration Inter-Process Communication Visualization Canvas Application Application Application Application Application Visualization Application Solid Earth Modeler Application Application Application Modeling Application Relational Database Tight Data Management through rdbms Distributed Data Management Loose Data Integration with CORBA

16 Platform mix, system mix " build pieces and get them talking!" Distributed Data Management Standards-based XML, CORBA, Java Local and Remote Data Access Multi- Vendor Interoperability Platform- Independent Data Management Hardware Optimized Data Access

17 Paradigm εpos Data Management & Interoperability Integration 1) Data stores, repositories and databases Distributed Data Management: A 5-Layered Model

18 Paradigm εpos Data Management & Interoperability Integration 1) Data stores, repositories and databases 2) Data Servers execute CORBA instructions for each repository Distributed Data Management: A 5-Layered Model

19 Paradigm εpos Data Management & Interoperability Integration 1) Data stores, repositories and databases 2) Data Servers execute CORBA instructions for each repository 3) Services & catalogs for project management and applications Distributed Data Management: A 5-Layered Model

20 Paradigm εpos Data Management & Interoperability Integration 1) Data stores, repositories and databases 2) Data Servers execute CORBA instructions for each repository 3) Services & catalogs for project management and applications 4) Data Servers transmit data requests from applications Distributed Data Management: A 5-Layered Model

21 Paradigm εpos Data Management & Interoperability Integration Distributed Data Management: A 5-Layered Model 1) Data stores, repositories and databases 2) Data Servers execute CORBA instructions for each repository 3) Services & catalogs for project management and applications 4) Data Servers transmit data requests from applications 5) Applications issue data requests to Data Servers

22 Distributed data and clusters The management of projects On separate systems (security, accessibility, availability) Catalogs reference data locations / structures Data flows are point-to-point Segregation Visualization can take place outside of the processing cluster Brick format allows access to ongoing jobs data concurrently

23 Visualization cluster? Data roaming tools Purpose "Navigate" terabyte datasets to view / QC / interpret data Not memory-resident, use of brick format and disk caching Relatively simple displays, tessellated surfaces are the most complex graphical objects Viewing totally unpredictable in most cases: 360º horizontal axis 360º vertical axis Zoom from overall view to high magnification Selection / removal of any object at any time No obvious benefit from clustered graphics Handled effectively on a single graphic card, I/O bottlenecks

24 Reservoir Navigator

25 Visualization cluster Voxel rendering tools Purpose Analysis of data clouds, properties, spatial bodies, anomalies Data memory resident Same un-predictable view-point / magnification / selection process Clustering voxel visualization? Very high bandwidth backplane required Shared memory, or replication of a large % of the raw data volume on each node The assembly layer would require high level functions relative to view points, etc.. Can a cluster compete with a bespoke system?

26 VoxelGeo

27 Conclusions Clusters are adopted for scalability and price/performance in appropriate application domains Clusters are preponderant for seismic data processing Some specific processes do not fit (current) clusters Brick formatted data crucial to cluster architecture CORBA distributed data to access cluster jobs "live" Visualization (roaming or voxel) does not automatically fit the constraints of cluster architecture

28