Anaconda Python Distribution

Transcription

1 Anaconda Python Distribution Introduction Usage Useful Commands Installation Extra Notes Dry Run Create an environment Installed Environment Useful commands Sample PBS file Activate an environment De-Activate an environment Deleting an environment n060 anaconda installation Installing packages without root access Reference Introduction Anaconda is a completely free Python distribution (including for commercial use and redistribution). It includes over 195 of the most popular Python packages for science, math, engineering, data analysis. Usage

2 The usage is: module load anaconda/4.00py2.7 OR: module load anaconda/4.00py3.5 You need to run this after loading the modules: source /sw/anaconda/py27-ver400/etc/profile.d/conda.sh #if using module load anaconda/4.00py2.7 OR source /sw/anaconda/py3-ver400/etc/profile.d/conda.sh #if using module load anaconda/4.00py3.5 Manual Usage: You will need to activate the environment. For example, to activate biopython source activate biopython discarding /sw/anaconda/2.0.1/bin from PATH prepending /sw/anaconda/2.0.1/envs/biopython/bin to PATH To deactivate this environment, use: source deactivate Module Usage: ============= For example,you can use this to load biopython using python 2.7 version (anaconda) : module load anaconda/py27biopython1.66 For python 3.5 (anaconda env), use this: module load anaconda/py35biopython1.66 Anaconda environment need not be loaded separately. However, it can be loaded as follows if needed for other reasons: module load anaconda/4.00py2.7 module load anaconda/4.00py3.5 Useful Commands

3 To get a list of environment in anaconda: conda info -e To create an environment: conda create -n biopython biopython De-Activate an environment source deactivate Deleting an environment conda remove -n biopython conda create -h usage: conda create [-h] [--yes] [--dry-run] [-f] [--file FILE] [--unknown] [--no-deps] [-m] [--use-index-cache] [--use-local] [--no-pin] [-c CHANNEL] [--override-channels] [-n NAME -p PATH] [-q] [--alt-hint] [--clone ENV] [--no-default-packages] [package_spec [package_spec...]] Create a new conda environment from a list of specified packages. To use the created environment, use 'source activate envname' look in that directory first. This command requires either the -n NAME or -p PREFIX option. positional arguments: package_spec optional arguments: -h, --help --yes --dry-run -f, --force installed), --file FILE --unknown (which --no-deps -m, --mkdir --use-index-cache --use-local --no-pin package versions to install into conda environment show this help message and exit do not ask for confirmation only display what would have been done force install (even when package already implies --no-deps read package versions from FILE use index metadata from the local package cache are from unknown channels) do not install dependencies create prefix directory if necessary use cache of channel index files use locally built packages don't use pinned packages

4 -c CHANNEL, --channel CHANNEL additional channel to search for packages. These are URLs searched in the order they are given (including file:// for local directories). Then, the defaults or channels from.condarc are searched (unless --override-channels is given). You can use 'defaults' to get the default packages for conda, and 'system' to get the system packages, which also takes.condarc into account. You can also use any name and the.condarc channel_alias value will be prepended. The default channel_alias is --override-channels Do not search default or.condarc channels. Requires --channel. -n NAME, --name NAME name of environment (in /sw/anaconda/2.0.1/envs) -p PATH, --prefix PATH full path to environment prefix (default: /sw/anaconda/2.0.1) -q, --quiet do not display progress bar --alt-hint Use an alternate algorithm to generate an unsatisfiable hint --clone ENV path to (or name of) existing local environment --no-default-packages ignore create_default_packages in condarc file examples:

5 conda create -n myenv sqlite Installation bash Anaconda Linux-x86_64.sh Anaconda will now be installed into this location: /root/anaconda - Press ENTER to confirm the location - Press CTRL-C to abort the installation - Or specify an different location below [/root/anaconda] >>> /sw/anaconda/2.0.1 PREFIX=/sw/anaconda/2.0.1 installing: python installing: conda py27_0... installing: conda-build py27_0... installing: _license-1.1-py27_0... installing: argcomplete py27_0... installing: astropy np18py27_0... installing: atom py27_0... installing: beautiful-soup py27_0... installing: binstar py27_0... installing: bitarray py27_0... installing: blaze np18py27_1... installing: blz np18py27_0... installing: bokeh np18py27_1... installing: boto py27_0... installing: cairo installing: casuarius-1.1-py27_0... installing: cdecimal-2.3-py27_0... installing: chaco np18py27_0... installing: colorama py27_0... installing: configobj py27_0... installing: cubes py27_4... installing: curl installing: cython py27_0... installing: datashape np18py27_1... installing: dateutil-2.1-py27_2... installing: docutils-0.11-py27_0... installing: dynd-python np18py27_0... installing: enable np18py27_2... installing: enaml py27_1... installing: flask py27_1... installing: freetype installing: future py27_0...

6 installing: gevent py27_0... installing: gevent-websocket py27_0... installing: greenlet py27_0... installing: grin py27_1... installing: h5py np18py27_0... installing: hdf installing: ipython py27_2... installing: ipython-notebook py27_0... installing: ipython-qtconsole py27_0... installing: itsdangerous-0.24-py27_0... installing: jdcal-1.0-py27_0... installing: jinja py27_0... installing: jpeg-8d-0... installing: kiwisolver py27_0... installing: launcher py27_0... installing: lcms installing: libdynd installing: libpng installing: libsodium installing: libtiff installing: libxml installing: libxslt installing: llvm installing: llvmpy py27_0... installing: lxml py27_0... installing: markupsafe-0.18-py27_0... installing: matplotlib np18py27_1... installing: mock py27_0... installing: mpi4py-1.3-py27_0... installing: mpich p installing: multipledispatch py27_0... installing: networkx py27_0... installing: nltk np18py27_0... installing: nose py27_0... installing: numba np18py27_0... installing: numexpr np18py27_0... installing: numpy py27_0... installing: openpyxl py27_0... installing: openssl-1.0.1h-0... installing: pandas np18py27_0... installing: patsy np18py27_0... installing: pep py27_0... installing: pil py27_1... installing: pip py27_0... installing: pixman installing: ply-3.4-py27_0... installing: psutil py27_0... installing: py py27_0... installing: py2cairo py27_1... installing: pycosat py27_0... installing: pycparser-2.10-py27_0... installing: pycrypto py27_0... installing: pycurl py27_2...

7 installing: pyface py27_0... installing: pyflakes py27_0... installing: pygments-1.6-py27_0... installing: pyparsing py27_0... installing: pyqt py27_0... installing: pytables np18py27_0... installing: pytest py27_0... installing: pytz py27_0... installing: pyyaml-3.11-py27_0... installing: pyzmq py27_0... installing: qt installing: readline installing: redis installing: redis-py py27_0... installing: requests py27_0... installing: rope py27_1... installing: runipy py27_0... installing: scikit-image np18py27_0... installing: scikit-learn np18py27_1... installing: scipy np18py27_0... installing: setuptools-3.6-py27_0... installing: sip py27_0... installing: six py27_0... installing: sphinx py27_0... installing: spyder-2.3.0rc1-py27_0... installing: spyder-app-2.3.0rc1-py27_0... installing: sqlalchemy py27_0... installing: sqlite installing: ssl_match_hostname py27_0... installing: statsmodels np18py27_0... installing: sympy py27_0... installing: system installing: theano np18py27_0... installing: tk installing: tornado py27_0... installing: traits py27_0... installing: traitsui py27_0... installing: ujson-1.33-py27_0... installing: util-linux installing: werkzeug py27_0... installing: xlrd py27_0... installing: xlsxwriter py27_0... installing: xlwt py27_0... installing: yaml installing: zeromq installing: zlib installing: anaconda np18py27_0... installing: _cache-0.0-x0... Python :: Continuum Analytics, Inc. creating default environment... installation finished. Do you wish the installer to prepend the Anaconda install location to PATH in your /root/.bashrc? [yes no]

8 [no] >>> No

9 Thank you for installing Anaconda! The binaries are in /sw/anaconda/2.0.1/bin. You can use the module file to load the environment (recommended): module load anaconda/2.0.1 OR prepend (not advised) to your ~/.bashrc file: PATH=/sw/anaconda/2.0.1/bin:$PATH Extra Notes Dry Run To see what packages will be downloaded and/or used in an environment before creating it, you can use the --dry-run option. conda create --dry-run -n foo2 python e.g.: conda create --dry-run -n biopython biopython Fetching package metadata:.. Solving package specifications:. Package plan for installation in environment /sw/anaconda/2.0.1/envs/biopython1.64: The following packages will be downloaded: package build biopython-1.64 np18py27_0 2.1 MB python MB Total: 13.5 MB The following packages will be linked: package build biopython-1.64 np18py27_0 hard-link numpy py27_0 hard-link openssl-1.0.1h 0 hard-link python hard-link readline hard-link sqlite hard-link system hard-link tk hard-link zlib hard-link Dry run: exiting

10 Create an environment conda create -n biopython biopython Fetching package metadata:.. Solving package specifications:. Package plan for installation in environment /sw/anaconda/2.0.1/envs/biopython: The following packages will be downloaded: package build biopython-1.64 np18py27_0 2.1 MB python MB Total: 13.5 MB The following packages will be linked: package build biopython-1.64 np18py27_0 hard-link numpy py27_0 hard-link openssl-1.0.1h 0 hard-link python hard-link readline hard-link sqlite hard-link system hard-link tk hard-link zlib hard-link Proceed ([y]/n)? y Fetching packages... biopython % ########################## Time: 0:00: kb/s python % ########################## Time: 0:00: kb/s Extracting packages... [ COMPLETE ] ####################### 100% Linking packages... [ COMPLETE ] ####################### 100% # # To activate this environment, use: # $ source activate biopython # # To deactivate this environment, use: # $ source deactivate # Installed Environment

11 Useful commands Useful commands: conda info --envs conda list conda activate Sample PBS file #!/bin/bash #PBS -N condatest #PBS -m abe #PBS -M #PBS -l select=1:ncpus=1:mem=4gb,walltime=10:00:00 module load anaconda/4.00py2.7 source /sw/anaconda/py27-ver400/etc/profile.d/conda.sh ##use /sw/anaconda/py3-ver400/etc/profile.d/conda.sh if using module load anaconda/4.00py3.5 echo "Starting job: " cd $PBS_O_WORKDIR python <pythonfile> >>>An example of the python file>>>>>>>>> cat pythonfile.py import tensorflow as tf >>>>>>>>>>>>>> Activate an environment

12 source /sw/anaconda/py27-ver400/etc/profile.d/conda.sh OR source /sw/anaconda/py3-ver400/etc/profile.d/conda.sh if using module load module load anaconda/4.00py3.5 source activate biopython e.g: source activate biopython discarding /sw/anaconda/2.0.1/bin from PATH prepending /sw/anaconda/2.0.1/envs/biopython/bin to PATH De-Activate an environment source deactivate Deleting an environment conda remove -n biopython n060 anaconda installation module load anaconda/5.3.1py3 conda info --envs # conda environments: # base accelerate biopython deeplearning keras tensorflow-gpu * /sw/centos7/anaconda/5.3.1/py3 /sw/centos7/anaconda/5.3.1/py3/envs/accelerate /sw/centos7/anaconda/5.3.1/py3/envs/biopython /sw/centos7/anaconda/5.3.1/py3/envs/deeplearning /sw/centos7/anaconda/5.3.1/py3/envs/keras /sw/centos7/anaconda/5.3.1/py3/envs/tensorflow-gpu e.g module load anaconda/5.3.1py3 source activate keras pip install soundfile Installing packages without root access

13 Instructions from: -sudo-or-root-privileges-using-anaconda-59571ec0ecad First, navigate to your home directory. module load anaconda/5.3.1py3 wget sh Miniconda3-latest-Linux-x86_64.sh Now, we have a local installation of the Conda package manager at ~/miniconda3 Let s add conda to our bash profile so that we can call it from the command line vim.bash_profile i Add this line to the file: export PATH=$HOME/miniconda3/bin:$PATH esc :wq Now, let s reload our.bash_profile source.bash_profile And check that our installation was completed successfully which conda Which should return ~/miniconda3/bin/conda Now that Conda is installed, we can install Spacy or other Python packages conda install spacy python -m spacy download en Now, let s make sure that Spacy is installed correctly python import spacy nlp = spacy.load('en') doc = nlp(u'hello World!') print([w.lemma_ for w in doc]) Now that you have Conda installed, you can use it to install many packages locally. Ref: Reference