GPC 2007 Round Table on Pervasive Grids

Transcription

1 GPC 2007 Round Table on Pervasive Grids Manish Parashar The Applied Software Systems Laboratory ECE, Rutgers University (Ack: NSF, DoE, NIH)

2 The Grid Concept Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations The Grid concept is pervasive! But is this what we mean by a pervasive grid? Explosive growth in computation, communication, information and integration technologies Pervasive on-demand anytime-anywhere access computing, communication, information and data resources is ubiquitous Ack: I. Foster

3 Pervasive Grid (Cyber) Ecosystems Pervasive Grid Systems Seamless, secure, on-demand access to and aggregation of, geographically distributed computing, communication and information resources Computers, networks, data archives, instruments, observatories, experiments, sensors/actuators, ambient information, etc. Context, content, capability, capacity awareness, Mobility Knowledge-based, information/data-driven, context/content-aware computationally intensive, pervasive applications Symbiotically and opportunistically combine services/computations, real-time information, experiments, observations, and to manage, control, predict, adapt, optimize, Crisis management, monitor and predict natural phenomenon, monitor and manage engineered systems, optimize business processes Close to loop! acquire, assibilate, analyze/predict, control,. seamless access seamless aggregation seamless (opportunistic) interactions/couplings

4 Pervasive Computational Ecosystems and Dynamic Information Driven Applications Experts query, configure resources Computers, Storage, Instruments,... Resources discovered, negotiated, co-allocated on-the-fly. components deployed Experts interact and collaborate using ubiquitous and pervasive portals Scientist Scientist Laptop Computer PDA Resources Experts monitor/interact with/interrogate/steer models ( what if scenarios, ). Application notifies experts of interesting events. Component s write into the archive Applications & Services Model A Model B Experts mine archive, match real-time data with history Automated mining & matching Data Archives Data Archive & Sensors Sensors, Non- Traditional Data Sources Real-time data assimilation/injection (sensors, instruments, experiments, data archives), Components dynamically composed. WebServices discovered & invoked.

5 Pervasive Grid Application Characteristics Close to loop! acquire, assibilate, analyze/predict, control,. Opportunistic applications do not depend on the availability of the information, but can opportunistically use information if it is available Safety, Convenience, Collaborative multiple application entities cooperate with each other, each providing partial information, to make collective decisions in an autonomous manner Control provide autonomic control capabilities using actuation devices

6 Many Application Areas. Hazard prevention, mitigation and response Earthquakes, hurricanes, tornados, wild fires, floods, landslides, tsunamis, terrorist attacks Critical infrastructure systems Condition monitoring and prediction of future capability Transportation of humans and goods Safe, speedy, and cost effective transportation networks and vehicles (air, ground, space), V2V systems Energy and environment Safe and efficient power grids, safe and efficient operation of regional collections of buildings Health Reliable and cost effective health care systems with improved outcomes Enterprise-wide decision making Coordination of dynamic distributed decisions for supply chains under uncertainty Next generation communication systems Reliable wireless networks for homes and businesses Report of the Workshop on Dynamic Data Driven Applications Systems, F. Darema et al., March 2006, Source: M. Rotea, NSF

7 Information-driven Management of Subsurface Geosystems: The Instrumented Oil Field (with UT-CSM, UT-IG, OSU, UMD, ANL) Model Driven Detect and track changes in data during production. Invert data for reservoir properties. Detect and track reservoir changes. Assimilate data & reservoir properties into the evolving reservoir model. Use simulation and optimization to guide future production. Data Driven

8 Management of the Ruby Gulch Waste Repository (with UT-CSM, INL, OU) Ruby Gulch Waste Repository/Gilt Edge Mine, South Dakota ~ 20 million cubic yard of waste rock AMD (acid mine drainage) impacting drinking water supplies Monitoring System Multi electrode resistivity system (523) One data point every 2.4 seconds from any 4 electrodes Temperature & Moisture sensors in four wells Towards Dynamic Data-Driven Management of the Ruby Gulch Waste Repository, M. Parashar, et al, DDDAS Workshop, ICCS 2006, Reading, UK, LNCS, Springer Verlag, Vol. 3993, pp , May Flowmeter at bottom of dump Weather-station Manually sampled chemical/air ports in wells Approx 40K measurements/day

9 Diverse Geosystems Similar Solutions Landfills Oilfields Underground Pollution Models Control Simulation Data Undersea Reservoirs

10 Adaptive Fusion of Stochastic Information for Imaging Fractured Vadose Zones (with U of AZ, OSU, U of IW) Near-Real Time Monitoring, Characterization and Prediction of Flow Through Fractured Rocks System responses Inverse Modeling Parameters, Boundary & Initial Conditions Prediction Forward Modeling Network design bad Comparison With observations good Application

11 Data-Driven Forest Fire Simulation (DOE, U of AZ) Predict the behavior and spread of wildfires (intensity, propagation speed and direction, modes of spread) based on both dynamic and static environmental and vegetation conditions factors include fuel characteristics and configurations, chemical reactions, balances between different modes of hear transfer, topography, and fire/atmosphere interactions. Self-Optimizing Large Scale Wild Fire Simulations, J. Yang*, H. Chen*, S. Hariri and M. Parashar, Proceedings of the 5th International Conference on Computational Science (ICCS 2005), Atlanta, GA, USA, Springer-Verlag, May 2005.

12 Pervasive Grid Computing Uncertainty Challenges System Uncertainty Very large scales Ad hoc structures/behaviors p2p, hierarchical, etc, architectures Dynamic entities join, leave, move, change behavior Heterogeneous capability, connectivity, reliability, guarantees, QoS Lack of guarantees components, communication Lack of common/complete knowledge (LOCK) number, type, location, availability, connectivity, protocols, semantics, etc. Information Uncertainty Availability, resolution, quality of information Devices capability, operation, calibration Trust in data, data models Semantics Application Uncertainty Dynamic behaviors space-time adaptivity Dynamic and complex couplings multi-physics, multi-model, multiresolution,. Dynamic and complex (ad hoc, opportunistic) interactions Software/systems engineering issues Emergent rather than by design

13 Pervasive Grid Computing Research Issues, Opportunities Programming systems/models for data integration and runtime selfmanagement components and compositions capable of adapting behavior and interactions policy driven deductive engine correctness, consistency, performance, quality-of-service constraints Content-based asynchronous and decentralized discovery and access services semantics, metadata definition, indexing, querying, notification Data management mechanisms for data acquisition and transport with real time, space and data quality constraints high data volumes/rates, heterogeneous data qualities, sources in-network aggregation, integration, assimilation, caching Runtime execution services that guarantee correct, reliable execution with predictable and controllable response time data assimilation, injection, adaptation Security, trust, access control, data provenance, audit trails, accounting

14 Project AutoMate: Semantic + Autonomics Accord Programming Framework Meteor/Squid Content-based Middleware Ontology, Taxonomy Autonomic Grid Applications Programming System Autonomic Components, Dynamic Composition, Opportunistic Interactions, Collaborative Monitoring/ Control Decentralized Coordination Engine Agent Framework, Decentralized Reactive Tuple Space Semantic Middleware Services Content-based Discovery, Associative Messaging Content Overlay Content-based Routing Engine, Self-Organizing Overlay Sesame/DAIS Protection Service Rudder Coordination Middleware Conceptual models and implementation architectures programming systems based on popular programming models object, component and service based prototypes content-based coordination and messaging middleware amorphous and emergent overlays

15 Project Rutgers Programming System Semantically meaningful abstraction for in-network processing and external querying E.g., GridMap, abstractions for scientific/engineering applications Adaptive behaviors and interactions Deductive engine for policy driven self-management Asynchronous, content-based messaging and coordination support Data Processing Services In-network algorithms for modeling, interpretation of phenomenon, and decision making. Acquisition of data/information with dynamic qualities and properties from streams of data from the physical environment Address issues of data quality assurance, statistical synthesis and hypotheses testing, and in-network data assimilation. Applications control of sensors and data acquisition. System Management Services Application-driven dynamic management of pervasive Grid systems Overlay management, autonomic management/adaptations for computation/communication/power tradeoffs, dynamic load-balancing, etc. Prototype deployments on RUGrid, ORBIT, PlantLab, etc.

16 Summary Pervasive Grid Ecosystems & Information-Driven Investigation Knowledge-based, data and information driven, context-aware, computationally intensive Building a community Use cases/application scenarios Benchmarks Testbeds experimental infrastructure Orbit, VINI Community position paper