OpenSees on Teragrid

Transcription

1 OpenSees on Teragrid Frank McKenna UC Berkeley OpenSees Parallel Workshop Berkeley, CA What isteragrid? An NSF sponsored computational science facility supported through a partnership of 13 institutions. Provides over 20 high performance computational systems, specialized visualization services and massive storage archives to national community. Resources growing exponentially. At end of 2006 the fastest HPC had 2000 processors and Teraflops. At end ,000 processors and 400 teraflops. By 2010 Petascale flops. 1

2 The Teragrid Strategy Create a unified national HPC infrastructure that is both heterogeneous and extensible Single user support resources. Single authentication point Common software functionality Common job management infrastructure Globally-accessible data storage Access Secure Shell - A Protocol that allows data to be transferred securely over the network using public key encryption Common programs ssh for login scp or sftp for file transfer Teragrid portal. NEES portal under development at NEESit. 2

3 OpenSees on Teragrid We are in the process of building OpenSees, OpenSeesSP and OpenSeesMP on all Teragrid machines. For now, the executables will be located in my home/bin directory. The location of this and the machines that the applications are installed on can be seen on the parallel web page. Feel free to help out! File System Global File System Common Environment Variables 1. TG_CLUSTER_HOME limited size but regular backup 2. TG_CLUSTER_GPFS cluster wide global file system, faster than home, no backup, loose after 7 days. 3. TG_NODE_SCRATCH node local scratch, no backup, loose access after run. You can access these in script and use to save files, e.g. set scratch $::env(tg_node_scratch) For optimimum performance 1. Get your files ready in TG_CLUSTER_HOME 2. Before you run, copy them to TG_CLUSTER_GPFS/dir 3. When running get recorders to write to TG_NODE_SCRATCH 4. Remember to copy from SCRATCH in your script and from GPFS when the job is completed 3

4 Job Submission 2 Modes of Submission Interactive - limited to a few processors and a small amount of time, for debugging & testing Batch - for large jobs & lengthy runs. Requires creation of batch submission script. The type of script differs from machine to machine, but all essentially the same: Example Submission Script #!/usr/bin/ksh. #@ wall_clock_limit = 05:00:00 #@class = normal #@node_usage = not_shared #@node = 32 #@tasks_per_node = 8 #@job_type = parallel #@network.mpi = sn_single,not_shared,us,high #@notification = always #@notify_user = fmckenna@berkeley.edu #@job_name= job.&(jobid) #@output = LL_out.$(jobid) #@error = LL_err.$(jobid) #@initialdir = /users03/ucb/ux455280/results #@queue poe /gpfs/fmk/bin/openseessp /gpfs/fmk/bridge/bridgenonlin.tcl 4

5 TeraGrid is heterogeneous Multiple architectures: Alpha (PSC), IA-32 (TACC, NCSA, Purdue,etc.), IA- 64 (SDSC, NCSA,etc.), Bluegene (SDSC), etc. Multiple resource managers: PBS (NCSA, SDSC, etc.), LSF (TACC, NCSA), SGE (TACC), etc. Multiple resource usage policies: Per-user limits, submission semantics, etc. Science Gateways A new initiative for the TeraGrid Science communities increasingly building their own cyberinfrastructure: Science Gateways. Provides an easy-to-use solution(s) for community computational needs. Both software and hardware Science Gateways will increase overall science throughput by making computers easier to use for knowledge discovery 3 Types: 1. Portals - web based interface 2. Service Gateways - applications on users access Teragrid 3. Bridges to Community Grids. 5

6 Science Gateways An expanding user base 6000 Users A new generation 6000 of users that access TeraGrid via Science Gateways, scaling well beyond the traditional user with a 5000 shell login account. Projected 4000 user community size by each science gateway project Impact on society from gateways enabling decision support is much larger! 2000 OSG OSG Flood Flood HEP HEP SNS SNS NESC/CCEGA NESC/CCEGA OLSG OLSG NCN NCN NVO NVO LEAD LEAD