Application Projects in VIOLA

Transcription

1 Application Projects in VIOLA The integration of applications with high communication demands is a crucial element of the vertically integrated approach of VIOLA. These demands include guaranteed high bandwidth as well as quality-of-service requirements between dynamically changing communication partners. In the first phase, four application projects covering visualisation and Grid- Computing in several scientific disciplines have been selected for funding. An additional infrastructure project is responsible for setting up an UNICORE-based Compute-Grid for the applications and to develop and integrate additional middleware components, which are needed by the applications but are not yet available. This presentation gives an overview of goals of these projects and the results achieved so far.

2 Overview about application projects in VIOLA April 27 th, 2005 Thomas Eickermann John von Neumann Institute for Computing Research Centre Jülich , Th.Eickermann, NIC, Research Centre Jülich Page 1

3 Content Introduction Application projects VIOLA-Support MetaTrace TechSim AMG-OPT KoDaVis Status and Outlook Offline Demo Page 2

4 Application Projects: Partners & Contributors NIC, Research Centre Jülich U.Detert, T.Düssel, M.-A.Hermanns, W.Frings, L.Kirtchakova, M.Rambadt, K.Stollwerk, H.Zilken, Fraunhofer Institute for Media Communication S.Conrad, Fraunhofer Institute for Algorithms and Scientific Computing H.Grund, A.Krechel, A.Leventi-Peetz, H.Schwichtenberg,S.Springstubbe, B.Steckel, O.Waeldrich, G.Winter, K.Wolf, W.Ziegler, Research Centre caesar R.Backofen, N.Botkin, K.B.Duong, A.Goepfert, O.Pykhteev, V.Turova, S.Vey, A.Voigt, A.Zollorsch, Fachhochschule Bonn-Rhein-Sieg R.Berrendorf, J.Seidel, Technical University of Aachen (RWTH) I.Assenmacher, T.Kuhlen, [N.Berr, B.ierbaum, R.Finocchiaro, K.Klaus, M.Poeppe, ] Page 3

5 Project structure: Vertical Integration Networks Middleware Applications Environmental Sciences: Simulation / Visualization Material Science & Engineering: Grid-Middleware and -Services (Secure access, Scheduling, Communication, I/O, Tools) Control Plane (Bandwidth and QoS reservation, user-network-interface) Network components (SDH/Ethernet Switches/Router) optical fiber infrastructure Technical Simulations Dissemination / Co-operation Page 4

6 VIOLA Applications - Rationale Networker's perspective From laboratory experiment to deployment in NREN: Evaluation of network technology in close-to-production environment Involve users from leading scientific institutions in the local region Applications with a broad spectrum of requirements Users perspective: Early access to next-generation technology may be competitive advantage: Deployment in X-WiN in Evaluation of Grid-System by means of applications Page 5

7 Application Areas & Ressources RWTH Aachen Forschungszentrum Jülich FH Bonn-Rhein-Sieg Sankt Augustin caesar Bonn Universität Bonn Fraunhofer Gesell. Sankt Augustin Complex distributed scientific simulations Bundling power of PC-cluster at partner sites: CPUs, memory, I/O capabilities Load sharing: use an appropriate free resource Visualization Access to huge data repositories in the network Control and steer running computer simulations Collaborate in distributed teams 4 Intel/AMD Linux-Clusters: FZ Jülich Caesar Fraunhofer SCAI FH Bonn-Rhein-Sieg High-End Visualisation / VR: RWTH Aachen FZ Jülich Fraunhofer IMK Page 6

8 VIOLA-Support: Provisioning of Infrastructure Networks Middleware Applications Environmental Sciences: Simulation / Visualization Material Science & Engineering: Grid-Middleware and -Services (Secure access, Scheduling, Communication, I/O, Tools) Control Plane (Bandwidth and QoS reservation, user-network-interface) Network components (SDH/Ethernet Switches/Router) optical fiber infrastructure Technical Simulations Dissemination / Co-operation Page 7

9 VIOLA-Support: Application Support and Grid-Integration Partners: FZJ/ZAM, FhG/SCAI, FH BRS, caesar Task: provide a production-quality environment for the users Secure and seamless access to the parallel computers of the project partners Allow coordinated reservations of CPUs, I/O-Servers and network bandwidth Leverage network bandwidth for applications via standard APIs (MPI) Transparent access to data stored anywhere in the testbed Page 8

10 VIOLA-Support: Application Support and Grid-Integration Actions: Deploy a UNICORE-Grid at the partner sites develop a meta-scheduler based on Global Grid Forum standards, capable of managing compute and network resources port, enhance MetaMPICH (RWTH) and integrate it into UNICORE Adapt and integrate the coupling-tool MpCCI into the VIOLAenvironment (UNICORE) [in sub-project TechSim] extend MetaMPICH by a high-performance parallel-i/o-component (MPI-IO) for distributed remote file-servers and memory Prototypes of all components are available As required by additional applications: attach additional computers to the testbed JUMP (FZJ s 9 TeraFLOPS IBM Supercomputer) SUN SMPs (SCAI s Cluster) Page 9

11 VIOLA-Applications: Using the Infrastructure Networks Middleware Applications Environmental Sciences: Simulation / Visualization Material Science & Engineering: Grid-Middleware and -Services (Secure access, Scheduling, Communication, I/O, Tools) Control Plane (Bandwidth and QoS reservation, user-network-interface) Network components (SDH/Ethernet Switches/Router) optical fiber infrastructure Technical Simulations Dissemination / Co-operation Page 10

12 MetaTrace: Simulation of pollutant transport in groundwater with distributed SMP-Clusters Partners: FZJ/ZAM, FH BRS Environmental research in Jülich: distribution of chemicals in the soil experiments in a test field simulations on parallel computers Optimal distribution of work to several PC-clusters: calculation of water-flow distribution and chemical reactions of pollutants exchange of intermediate results: 1 GigaByte in 1 second Goal: simulate more accurate models in less time Page 11

13 AMG-OPT: Grid-enabled technology to accelerate simulations Partners: FhG/SCAI, FZJ/ZAM Development of algorithms at the Fraunhofer-Institut SCAI: optimal numerical solvers for special large linear systems of equations e.g. equations with unknowns in simulations of mechanical structures or liquids A stress-test for the VIOLA-network: bandwidth and latency are just sufficient to solve such equations efficiently distributed over a network Goal: SAMGP-Software as a building block for simulation-programs in the Grid Page 12

14 TechSim: Distributed Simulation of complex technological Systems Partners: caesar, FhG/SCAI Different kinds of physics often interact in technological systems: makes modeling and simulation difficult, is well suited for distribution to several computers Two forward-looking applications from caesar in VIOLA simulation of crystal growth of 300mm silicon-wafers for microchip-production : process-optimization (uses coupling software MpCCI from SCAI) biosensors (liquid film on piezo-crystal) detect smallest amounts of organic molecules: 3D-simulations should lead to improvements Page 13

15 KoDaVis: Making Atmospheric Processes visible Partners: FZJ/ZAM, RWTH Aachen, FhG/IMK, FH BRS Joint project from FZ Jülich (ICG) and MPI Hamburg simulations of physical and chemical processes in the atmosphere help to understand the effect of human activities on the climate: data sets ~ 1000 GigaByte visualization of data needed to gain insight into processes VIOLA: high bandwidth allows for interactive access to the data over the network short signal delays enable collaboration in distributed teams: common view of the data video-conference support Demo at the end of the presentation! Page 14

16 Application Requirements Application Bandwidth Latency Hardware Middleware MetaTrace 8 Gbit/s (< 0.1 msec) 2-5 Clusters UNICORE MetaMPI VISIT TechSim 2 8 Gbit/s 2-3 Clusters UNICORE MetaMPI MpCCI AMG-OPT 8 Gbit/s < 1 ms 2 Clusters UNICORE MetaMPI KoDaVis 8 Gbit/s 25 msec QoS (Video) 1 Cluster VR / Visualisation ViSTA AVS/Express VISIT Page 15

17 Status & Current work Initial version of middleware is deployed Integration of new middleware components in progress Application projects have finished preparatory work in time - until network testbed had been set up First runs over the network have been made in line with project plan Applications are beginning to stress network & middleware Stability and functionality of all components are rapidly improving Next steps: Functional and performance improvements of middleware and application Integration of new hardware (Cray XD1, Sun Cluster) Inclusion of additional applications & partners (Erlangen, Garching, ) Page 16

18 KoDaVis Use Case Scenario Visualisation Client 1 Linux-Workstation AVS/Express Visualisation Client 2 Powerwall ViSTA File Server A (parallel, caching) Data Sets: a,b,c, Interaction Server File Server X (parallel, caching) Data Sets: x,y,z, Data Broker Page 17

19 KoDaVis Demo Setup Visualisation Client 1 Linux-Workstation AVS/Express Visualisation Client 2 Powerwall ViSTA 1 Gbit/s Ethernet File Server A (parallel, caching) Data Sets: a,b,c, Interaction Server File Server X (parallel, caching) Data Sets: x,y,z, Data Broker Page 18