CSinParallel Workshop. OnRamp: An Interactive Learning Portal for Parallel Computing Environments

Transcription

1 CSinParallel Workshop : An Interactive Learning for Parallel Computing Environments Samantha Foley ssfoley@cs.uwlax.edu Josh Hursey jjhursey@cs.uwlax.edu Project supported by SIGCSE Special Projects Grant (May 2015), Blue Waters Student Internship program (2015), and the UW-L Computer Science Department 1

2 to Parallel Computing Project There exists a significant barrier to entry for learning how to become productive in a Parallel Computing Environment () due to often unfamiliar and complex system software, programming interfaces, and tools. UNIX Shell (Scripting) Remote System Access & File Transfer Compilation Environment (Makefile, mpicc, nvcc) Runtime Environment (mpirun, srun, aprun) Batch Submission Systems (PBS, SLURM, LSF) Parallel Programming Paradigms (MPI, OpenMP) Parallel Architectures Parallel Programming Patterns, Tradeoffs, Scaling CS1/CS2 (Java & Eclipse IDE) Traditional Instruction Path 2

3 to Parallel Computing Project The Project, provides a web-based portal that coaches users through interactive tutorials that teach them about the software ecosystem and parallel computing while allowing them to launch & explore parallel applications from day one. UNIX Shell (Scripting) Remote System Access & File Transfer Compilation Environment (Makefile, mpicc, nvcc) Runtime Environment (mpirun, srun, aprun) Level 3 Batch Submission Systems (PBS, SLURM, LSF) Level 2 Parallel Programming Paradigms (MPI, OpenMP) Level 1.5 Parallel Architectures Level 1 Parallel Programming Patterns, Tradeoffs, Scaling CS1/CS2 (Java & Eclipse IDE) 3

4 to Parallel Computing Project High-Level Goals Encourage students to explore parallel and distributed computing concepts without the overhead of system software peculiarities. Help students transition to using the native, eventually. Bring together existing educational hardware & curriculum modules efforts into a flexible, portable architecture. Basic Architecture VM Cluster XSEDE Cluster 4

5 to Parallel Computing Project Later release will allow users to edit code VM Cluster XSEDE Web (client facing side of the ) Users do not need accounts on each (use a shared UNIX account). Users can be grouped into one or more Workspaces. Workspaces are assigned & Module combinations. Users view Module instructions, launch jobs, view results, transfer files. Automatically generated & validated custom forms for each Module. Administrative panel to manage s, one-click deploy Modules, manage Users, manage Workspaces, monitor usage, Cluster 5

6 to Parallel Computing Project Developing turn- key scripts to setup virtual clusters. VM Cluster XSEDE Broker between the users of the and the s. Enforce policy, authenticate users, and manage security. Cache files between Users and s. Custom drivers for each type of software environment (SLURM, PBS, LSF). Manage modules, and user files associated with jobs. Launch and monitor jobs on the system. Cluster 6

7 to Parallel Computing Project Curriculum modules A Z B Z A B VM Cluster Z XSEDE Cluster Curriculum Modules Complete freedom to structure the module as you like. Write a few python scripts to hook into the architecture. Configuration files allow you to specify custom, module-specific parameters and validation requirements for users running the code. Custom documentation/instructions Can be tailored to the environment R 7

8 to Parallel Computing Project Curriculum Module Life cycle A A Admin deploys Module on using Module can be deployed from git, svn, tar.gz, file system unpacks Module and runs onramp_deploy.py Script can compile the code, generate documentation, sends back Module generated docs and configuration files. Once deployed: Admin can test the module by trying to run it. Admin can assign it to the users in one or more Workspaces. 8

9 to Parallel Computing Project Curriculum Module Life cycle A Users configure & run Modules View Module and documentation and instructions Custom parameters defined in onramp_uioptions.spec On job submission: (user defined parameters are available) runs onramp_preprocess.py sets up batch script and submits to queue Batch script will run onramp_run.py On job completion Batch script will run onramp_postprocess.py 9

10 to Parallel Computing Project Curriculum Module Life cycle A After job completion, users can: View result output (stdout/stderr) Compare results to other runs Access files from the run and module Resubmit the job after adjusting some parameters (Future) Visualize, graph results (Future) Share results and/or collaborate 10

11 to Parallel Computing Project Roadmap May 2015 SIGCSE Grant started (project development started) Sept (alpha release) s: SLURM, (PBS) : Workspaces/Users, job launch/monitoring, basic admin. capabilities Modules: MPI Hello World & Ring, MPI/OpenMP Area Under the Curve (Shodor) Jan (beta release) s: Rocks Cluster, Virtual Machine (scripted) : Module customized submissions, full admin. Capabilities Modules: HPL, Monte Carlo (CSinParallel), Parameter search exemplar March 2016 (v1.0 release) s: Cloud provisioned s (scripted) : Demonstrate Coaching up to Level 2 11

12 to Parallel Computing Project The Project, provides a web-based portal that coaches users through interactive tutorials that teach them about the software ecosystem and parallel computing while allowing them to launch & explore parallel applications from day one. UNIX Shell (Scripting) Remote System Access & File Transfer Compilation Environment (Makefile, mpicc, nvcc) Runtime Environment (mpirun, srun, aprun) Level 3 Batch Submission Systems (PBS, SLURM, LSF) Level 2 Parallel Programming Paradigms (MPI, OpenMP) Level 1.5 Parallel Architectures Level 1 Parallel Programming Patterns, Tradeoffs, Scaling CS1/CS2 (Java & Eclipse IDE) 12