Using LSF with Condor Checkpointing

Transcription

1 Overview Using LSF with Condor Checkpointing This chapter discusses how obtain, install, and configure the files needed to use Condor checkpointing with LSF. Contents Introduction on page 3 Obtaining Files on page 4 Setup and Use on page 5 Platform Computing

2 Copyright Copyright Platform Computing Corporation All rights reserved. Although the information in this document has been carefully reviewed, Platform Computing Corporation does not warrant it to be free of errors or omissions. Platform Computing Corporation reserves the right to make corrections, updates, revisions or changes to the information in this document. UNLESS PROVIDED OTHERWISE IN WRITING BY PLATFORM COMPUTING CORPORATION, THE PROGRAM DESCRIBED IN THIS DOCUMENT IS PROVIDED AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. IN NO EVENT WILL PLATFORM BE LIABLE TO ANYONE FOR SPECIAL, COLLATERAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING ANY LOST PROFITS OR LOST SAVINGS, ARISING OUT OF THE USE OF OR INABILITY TO USE THIS PROGRAM. We d Like to Hear from You You can help us make this manual better by telling us what you think of the content, organization, and usefulness of the information. If you find an error, or just want to make a suggestion for improving this manual, please address your comments to doc@platform.com. Your comments should pertain only to the LSF documentation. For product support, contact support@platform.com. Trademarks LSF Base, LSF Batch, LSF JobScheduler, LSF MultiCluster, LSF Analyzer, LSF Make, LSF Parallel, Platform Computing, and the Platform Computing and LSF logos are trademarks or registered trademarks of Platform Computing. UNIX is a registered trademark of The Open Group. Other products or services mentioned in this document are identified by the trademarks or service marks of their respective companies or organizations. 2 Using LSF with Condor Checkpointing

3 Introduction Introduction LSF is integrated with the checkpointing capabilities of Condor. This integration allows jobs submitted to LSF to use the native Condor checkpointing facilities to perform user-level job checkpointing. Condor checkpointing is supported on many platforms including Linux. The Condor checkpoint support files are installed in place of the LSF checkpointing files. The support files are available as a separate distribution from Platform Computing. After Condor checkpoint support is installed, jobs are built, re-linked, checkpointed, and restarted the same way as user-level checkpointable job are in LSF. For more information about using LSF to checkpoint your jobs, see the LSF Administrator s Guide. Condor is a High Throughput Computing (HTC) environment from the University of Wisconsin. For more information about Condor visit the Condor Project Home page at Limitation The current version of Condor checkpoint support only supports the linking of C language programs. Using LSF with Condor Checkpointing 3

4 Obtaining Files Obtaining Files LSF Overview The files for Condor checkpoint support are available from Platform Computing and are distributed as a compressed tar archive named: lsfx.x_condor_chkpnt.tar.z Where x.x is the current LSF version. The file is available on your LSF CD or the Platform Computing web ( and FTP sites (ftp.platform.com). Access to the download area of the Platform Web site and the Platform FTP site is controlled by login name and password. If you are unable to access the distribution files, send to info@platform.com. Distribution File Contents The Condor checkpoint support distribution contains the following: Item bsub ckpt_ld echkpnt erestart README setup uninstall Description a wrapper script used to detect checkpoint options and modify arguments for submissions to Condor shell script to link checkpoint jobs with Condor libraries Condor specific echkpnt Condor specific erestart LSF documentation for Condor checkpoint support installation script uninstall script Condor The Condor version 6.1.x distribution needs to be downloaded and extracted according to Condor instructions. See the Condor Project Home page at for more information about obtaining the Condor distribution. You do not need to configure Condor, it only needs to be extracted from its distribution and made accessible to LSF. 4 Using LSF with Condor Checkpointing

5 Setup and Use Setup and Use Requirements LSF 4.0 is installed and operational. Condor has been extracted and is accessible to LSF. What the Installation Does The installation script, setup, does the following: Renames: bsub to bsub_real echkpnt to echkpnt.lsf erestart to erestart.lsf ckpt_ld to ckpt_ld.lsf Installs a bsub wrapper in LSF_BINDIR Installs new versions of: echkpnt erestart ckpt_ld Setup Steps To install Condor checkpoint support: 1 Log on to your host and become the superuser (root). % su 2 Make the distribution directory current, the directory where lsfx.x_condor_chkpnt.tar.z is located. 3 Uncompress the distribution file with the zcat and tar commands: # zcat lsfx.x_condor_chkpnt.tar.z tar xvf - 4 Invoke the setup installation script and follow the prompts: #./setup 5 After Condor checkpoint support has been installed, verify that the Condor configuration file can be found by one of the following: CONDOR_CONFIG environment variable /etc/condor/condor_config ~condor/condor_config 6 Add the Condor binary path to your PATH environment variable. For example, if Condor is installed in /usr/local/condor: % setenv PATH /usr/local/condor/bin:$path Using LSF with Condor Checkpointing 5

6 Setup and Use Use Review the limitations of Condor checkpointing before building checkpointable jobs. Limitations are discussed on the Condor Project Web site at For more information about using LSF to checkpoint your jobs, see the LSF Administrator s Guide. 6 Using LSF with Condor Checkpointing