Tutorial-1b: The Linux CLI

Similar documents
Tutorial-1b: The Linux CLI

Computer Systems and Architecture

Introduction to remote command line Linux. Research Computing Team University of Birmingham

Introduction to Unix The Windows User perspective. Wes Frisby Kyle Horne Todd Johansen

Perl and R Scripting for Biologists

Computer Systems and Architecture

Introduction: What is Unix?

Introduction to UNIX. Logging in. Basic System Architecture 10/7/10. most systems have graphical login on Linux machines

Introduction to Linux Workshop 1

Working with Basic Linux. Daniel Balagué

This lab exercise is to be submitted at the end of the lab session! passwd [That is the command to change your current password to a new one]

Introduction to UNIX command-line

Contents. Note: pay attention to where you are. Note: Plaintext version. Note: pay attention to where you are... 1 Note: Plaintext version...

Linux Command Line Interface. December 27, 2017

Practical Session 0 Introduction to Linux

Unix/Linux Basics. Cpt S 223, Fall 2007 Copyright: Washington State University

Introduction to UNIX command-line II

Read the relevant material in Sobell! If you want to follow along with the examples that follow, and you do, open a Linux terminal.

Carnegie Mellon. Linux Boot Camp. Jack, Matthew, Nishad, Stanley 6 Sep 2016

UNIX. The Very 10 Short Howto for beginners. Soon-Hyung Yook. March 27, Soon-Hyung Yook UNIX March 27, / 29

Linux at the Command Line Don Johnson of BU IS&T

Linux Operating System Environment Computadors Grau en Ciència i Enginyeria de Dades Q2

Chapter-3. Introduction to Unix: Fundamental Commands

Introduction to UNIX/Linux

Linux Bootcamp Fall 2015

Unix as a Platform Exercises + Solutions. Course Code: OS 01 UNXPLAT

Introduction to the UNIX command line

Introduction p. 1 Who Should Read This Book? p. 1 What You Need to Know Before Reading This Book p. 2 How This Book Is Organized p.

A Brief Introduction to the Linux Shell for Data Science

Files

Table of contents. Our goal. Notes. Notes. Notes. Summer June 29, Our goal is to see how we can use Unix as a tool for developing programs

CS Fundamentals of Programming II Fall Very Basic UNIX

Introduction To. Barry Grant

First of all, these notes will cover only a small subset of the available commands and utilities, and will cover most of those in a shallow fashion.

Files and Directories

CS 215 Fundamentals of Programming II Spring 2019 Very Basic UNIX

Introduction to Unix: Fundamental Commands

When talking about how to launch commands and other things that is to be typed into the terminal, the following syntax is used:

commandname flags arguments

Unix/Linux Primer. Taras V. Pogorelov and Mike Hallock School of Chemical Sciences, University of Illinois

Introduction to Linux Part 1. Anita Orendt and Wim Cardoen Center for High Performance Computing 24 May 2017

Mills HPC Tutorial Series. Linux Basics I

CISC 220 fall 2011, set 1: Linux basics

CSC209H Lecture 1. Dan Zingaro. January 7, 2015

Introduction to Linux Organizing Files

Linux Command Line Primer. By: Scott Marshall

Operating Systems. Copyleft 2005, Binnur Kurt

Operating Systems 3. Operating Systems. Content. What is an Operating System? What is an Operating System? Resource Abstraction and Sharing

Unix basics exercise MBV-INFX410

CENG 334 Computer Networks. Laboratory I Linux Tutorial

Session 1: Accessing MUGrid and Command Line Basics

IMPORTANT: Logging Off LOGGING IN

Unix Basics. Benjamin S. Skrainka University College London. July 17, 2010

Unix Introduction to UNIX

Hitchhiker s Guide to VLSI Design with Cadence & Synopsys

INSE Lab 1 Introduction to UNIX Fall 2017

Introduction to the Linux Command Line

Introduction to Linux

Lab 2: Linux/Unix shell

Introduction to Linux

Linux Essentials Objectives Topics:

UNIX COMMANDS AND SHELLS. UNIX Programming 2015 Fall by Euiseong Seo

5/20/2007. Touring Essential Programs

Unix as a Platform Exercises. Course Code: OS-01-UNXPLAT

CS 460 Linux Tutorial

Useful Unix Commands Cheat Sheet

Unix/Linux Operating System. Introduction to Computational Statistics STAT 598G, Fall 2011

CS CS Tutorial 2 2 Winter 2018

Connecting to ICS Server, Shell, Vim CS238P Operating Systems fall 18

Basic Linux (Bash) Commands

Research. We make it happen. Unix Basics. User Support Group help-line: personal:

Recap From Last Time:

Introduction to Linux. Fundamentals of Computer Science

acmteam/unix.pdf How to manage your account (user ID, password, shell); How to compile C, C++, and Java programs;

BGGN 213 Working with UNIX Barry Grant

The kernel is the low-level software that manages hardware, multitasks programs, etc.

COMS 6100 Class Notes 3

Introduction to the shell Part II

Exercise 1: Basic Tools

Recap From Last Time: Setup Checklist BGGN 213. Todays Menu. Introduction to UNIX.

EECS Software Tools. Lab 2 Tutorial: Introduction to UNIX/Linux. Tilemachos Pechlivanoglou

Unix tutorial. Thanks to Michael Wood-Vasey (UPitt) and Beth Willman (Haverford) for providing Unix tutorials on which this is based.

UNIX and Linux Essentials Student Guide

Using UNIX. -rwxr--r-- 1 root sys Sep 5 14:15 good_program

Intro to Linux. this will open up a new terminal window for you is super convenient on the computers in the lab

Introduction. File System. Note. Achtung!

Command-line interpreters

BIOINFORMATICS POST-DIPLOMA PROGRAM SUBJECT OUTLINE Subject Title: OPERATING SYSTEMS AND PROJECT MANAGEMENT Subject Code: BIF713 Subject Description:

Getting started with Hugs on Linux

5/8/2012. Creating and Changing Directories Chapter 7

Introduction to the Linux Command Line January Presentation Topics

Using LINUX a BCMB/CHEM 8190 Tutorial Updated (1/17/12)

Shell Programming Overview

Lab Working with Linux Command Line

CSE 303 Lecture 2. Introduction to bash shell. read Linux Pocket Guide pp , 58-59, 60, 65-70, 71-72, 77-80

Crash Course in Unix. For more info check out the Unix man pages -orhttp:// -or- Unix in a Nutshell (an O Reilly book).

Linux crash lecture by Andrey Lukyanenko

Introduction to Linux. Woo-Yeong Jeong Computer Systems Laboratory Sungkyunkwan University

CSE Linux VM. For Microsoft Windows. Based on opensuse Leap 42.2

Chapter 1 - Introduction. September 8, 2016

Transcription:

Balazs Konya (Lund University) Programming for Scientists Tutorial1b 1 / 22 HTTP://WWW.HEP.LU.SE/COURSES/MNXB01 Introduction to Programming and Computing for Scientists Tutorial-1b: The Linux CLI

Balazs Konya (Lund University) Programming for Scientists Tutorial1b 2 / 22 Why Command Line Interface (CLI)? May look old-fashioned compared to GUI & touchscreens BUT: Powerfull way of interacting with the computer CLI is the best option for complex actions: Repeat and automate Operate with many objects To restart a chain of actions at various phases Options and actions are invoked in a consistent form Offers the simplest user environment Consumes little system resources (cpu, memory) Offers much more control over the system Working with CLI is faster than most of the GUIs Best suited for remote sessions with limited bandwidth More stable interface: not changing as much as the GUIs

Balazs Konya (Lund University) Programming for Scientists Tutorial1b 3 / 22 Launching a terminal Linux distributions come with numerous terminal programs Usually available under system or accessories lxterminal, xterm, uxterm Exercise: start up a terminal by: Navigating the menu Typing the name of the program (lxterminal)

Balazs Konya (Lund University) Programming for Scientists Tutorial1b 4 / 22 The prompt and the shell The Linux prompt: user@machine:directory$ all the typing is done after the prompt The linux SHELL: Linux uses a program called SHELL to accept and interpret commands entered in text mode Wide range of shells exists: bash, tcsh, csh When you log into Linux or start a terminal you are dropped into the default shell The linux shell is a very powerfull command language interpreter Built-in commands (e.g. pwd, cd, echo, exit, logout) Variables, functions, arrays Logical expressions Controll structures Expansion, substitution, pattern matching (regular expressions) Command history Special characters, e.g.: ~. $ & *? Exercise: Take a look at the actual prompt of your session, what is your directory? Find out what is your shell by using a built-in command: echo $0 Try out the logout command

Executing, suspending, killing commands You execute a command by typing its name at the linux prompt. BUT: When you type a command that is not recognized as an internal shell command the shell searches for a program on the system with that name under locations specified by the PATH environment variable Alternatively, you can specify the command name including its full path: /directory1/directory2/program_name (see details later) Program files for commands have to be set as executable. This usually occurs during installation or can be done manually by the user (see details later) To stop (suspend) a program use ctrl + z To continue the program either use fg or bg commands To kill a program use ctrl + c Exercise: Try to run the toy program xeye. What is the problem? Suspend it (ctrl + z) to get back the prompt Find out the location of the program (which xeyes), check if the PATH contains that directory (echo $PATH) Resume the program with fg and bg. What is the difference? Kill the program with ctrl + c Balazs Konya (Lund University) Programming for Scientists Tutorial1b 5 / 22

Balazs Konya (Lund University) Programming for Scientists Tutorial1b 6 / 22 The power of the linux prompt Tab expansion: To help minimize errors and increase typing speed the shell offers automatic command/file name completion feature. Type a section of a word and press Tab key Command history: Use the up and down arrow keys to cycle through the commands Use the ctrl + r to search the command history Text modifications at the prompt Delete texts after the cursor: ctrl + k, transpose characters: ctrl + t, transpose words: esc + t Exercise: Try out tab expansion, command history and some of the text manipulation (e.g. transpose two words)

Getting help & and information Linux provides a text-based help system, the man pages: man command_name Navigate with page up/down Search with /text, press n to repeat search Exit with Q Many of the commands come with built-in short help: - - help cli option Use the - - help command line option after the command Another built-in help is the info command that prints the info pages info command_name In case you feel lost, you can try some of the following commands: whoami, who, pwd Exercise: Browse the bash documentation (man bash), navigate, search the exhaustive manual Find out what the who command does Balazs Konya (Lund University) Programming for Scientists Tutorial1b 7 / 22

Text processing, filtering Linux is very strong at text file and stream processing. System adminstration is done via configuration files that are mostly text files Command outputs are text streams Read the content of a file: cat, head, tail, more, less Manipulate or measure text file (content): wc, sort, nl, uniq, od More advanced tools (including a programming language): grep, sed, awk Chaining tools together: the linux pipe will feed the output of the first command as the input for the second command: command1 command2 Exercise: Inspect the content of a text file (e.g. the /etc/services config file) with cat,head,tail, more or less (bonus material: read the manpage of less Play with some of the text manipulation commands, try to chain them together, e.g. run sort /etc-services more Use the grep command to search text patterns in a file: grep ftp /etc/services Balazs Konya (Lund University) Programming for Scientists Tutorial1b 8 / 22

Files & Directories Linux uses a file system to organize and store files so they can be easily accessed These files can be made up of text, data, or program source code, or can represent hardware devices. Actually, all major entities are represented as files Each of the files in a file system has its own unique filename: ASCII symbols, 255 characters Case sensitive: Backup12 and backup12 are two different files! A file name may contain extension(s): detector.data.tgz Avoid: -! # & @ $? * / (e.g. openfile may be seen as a command option) Files are stored, organized in folders (or directories). Directories can also contain directories (called subdirectories), which can in turn store files This structure is the hierarchical tree structure The base or parent directory of the file system is called root (/) The rules for naming directories are the same as filenames Balazs Konya (Lund University) Programming for Scientists Tutorial1b 9 / 22

Balazs Konya (Lund University) Programming for Scientists Tutorial1b 10 / 22 Files & directories (advanced topic) Linux supports a number of file types: Regular or ordinary files Directory entries Device or special files (character or block device) Sockets or named pipes Symbolic links: Hard links: a hard-linked file is accessible from multiple directories. Changes made to a hard-linked file are synched with all instances. Each hard link must be deleted in order to make the file inaccessible. Share the same inode number. Symbolic links: Very similar to shortcuts in Windows. Allows users to refer to files in other locations. You can rename symbolic links. These links are simply references to a filename and won't work when the original file is deleted. Have different inode numbers. Exercise: Create, modify, delete and compare hard and soft links ln origfile hardlink_to_orig ln s origfile softlink_to_orig ls li Modify the content of the original file ls -li rm origfile ls -li

Navigating the directory tree The topmost directory is the root, represented by / When a user or program is working in the file system, their location is known as the active or current working directory, represented by./ Use the pwd command to determine the full path to the current directory Another special directory is the home, it is represented by the tilda symbol ~ Each directory, except the root, has a parent directory, represented by../ A path in the tree is the route leading to a file or directory: Absolut path (always from the root) /root/home/john/data.txt Relative path (e.g. from home or from the active directory) ~/data.txt or./experiment/data2.txt Moving in the tree is done using the change directory cd command: cd absolute_path, cd.., cd ~, cd relative_path Exercise: Navigate the directory tree using the cd command Change to root, home, determine the active directory, move to the parent directory Balazs Konya (Lund University) Programming for Scientists Tutorial1b 11 / 22

Navigating the directory tree Balazs Konya (Lund University) Programming for Scientists Tutorial1b 12 / 22 Exercise: Navigate the directory tree using the cd command Change to root, home, determine the active directory, move to the parent directory

Searching in the directory tree Shell wildcards are useful in searching for files and directories Asterisk (*): matches zero or more instances of any character ls a*.exe will return a.exe, aa.exe, a1.exe, aaaa.exe Question mark (?): matches a single instance of any character ls a?.exe will return a1.exe, ab.exe Square brackets []: matches a set of characters specified via explicit list or inclusive ranges ls a[a-c].exe will return aa.exe, ab.exe, ac.exe Exclamation mark in square brackets [! ]: match any character that is NOT listed in the bracket ls a[!e].exe will NOT return ae.exe but every other combination The powerful find command can search files by name, owner, access/modification time, etc.. find /home name *.cpp find /tmp user courseuser Exercise: Try to list files using wildcard-based ls searches Search for your executable files (hint: use the executable option of find) Balazs Konya (Lund University) Programming for Scientists Tutorial1b 13 / 22

Balazs Konya (Lund University) Programming for Scientists Tutorial1b 14 / 22 Listing, creating, copying,moving files and directories Listing the content of a directory: ls directory_name Without argument the ls command lists the current directory Useful switches: -a (all files), -F (file types), -l (long outpout) Creating new files: The touch command creates an empty file: touch newfile Using > redirection, i.e. saving the output of a command to a new file: ls -la > dirlist.txt Using an application (e.g. editor) Creating a directory: mkdir mkdir directory_name Copy files and directories: cp cp original new_copy Move files and directories: mv mv old_location new_location

Balazs Konya (Lund University) Programming for Scientists Tutorial1b 15 / 22 Listing, creating, copying,moving, deleting files and directories Deleting files and directories: rm rm filename(s) rm d dir_name or rm r dir_name There is no undelete command in Linux! rm rf is very powerfull! r is for recursive deletion Exercise: The ls command has many useful other switches, e.g. find out how to list files time ordered, including reverse ordering Create new files with > redirection of various commands such as echo, ls, cat Create multi-level directory structure, cp or mv files into it Then, use rm to remove subdirs

Permissions Every Linux file and directory has three ownership levels three set of permissions associated with them. WHO: owner, group, other users WHAT: read, write, execute Means slightly different actions for files and directories. e.g. execute for directories grant permission to enter into the directory The permissions can be set using symbolic or octal notation: read r=4; write w=2; execute x=1, no permission -=0 Changing permissions: chmod, there is also a chown command to change owner/group of a file or directory. Exercise: Check the permissions of a newly created file and directory (use the ls l or ls ld command) Remove permissions, e.g. try chmod u-x new_dir, and see if you can list the content or change to the new_dir Balazs Konya (Lund University) Programming for Scientists Tutorial1b 16 / 22

Some additional handy commands tar: creates/extracts archive files (file boundles) tar cvf myarchive.tar /home/user/ tar xvf myarchive.tar ps and top: check/monitor the running processes; kill to terminate a process kill processid export and unset to define and clear environment variables export TODAY=Wednesday unset TODAY wget: download files wget some_url Exercise: Download a tarball e.g. from download.nordugrid.org with wget and extract its content wih tar and gzip Balazs Konya (Lund University) Programming for Scientists Tutorial1b 17 / 22

Balazs Konya (Lund University) Programming for Scientists Tutorial1b 18 / 22 Accessing remote computers Secure Shell (SSH) is a secure way of accessing remote computers, executing commands remotely or moving data between computers. All network traffic is encrypted The de-facto protocol for remote login & computer access SSH clients are available on non-linux platforms too (putty and winscp on windows) SSH servers are listening to incomming connections on the standard TCP 22 port Login is done with username/passwd or using keypairs (advanced topic) Exercise: use the Linux ssh and scp commands: ssh remote_user@machine X scp localfile user@machine:remote_dir

Editors You need some tool to type in your code: the editor Full screen terminal editors: vi, joe, emacs Graphical editors: leafpad, gedit, nedit, xemacs, geany Syntax highlighting, autoindentation Some of these are much more than editors and can be called IDE (Integrated Development Environment) A full-scale IDE: Eclipse Exercise: Choose your preferred editor Try out some of the above ones: create new files, download some c++ code from the web and open it in the editors You may need to install the editor with sudo apt-get install editor_name Or, if you have already one, tell us why that one is the best Balazs Konya (Lund University) Programming for Scientists Tutorial1b 19 / 22

Balazs Konya (Lund University) Programming for Scientists Tutorial1b 20 / 22 and now some Dangerous stuff Recursive remove: rm rf Recursively force-remove all the files it can Without prompting you The fork bomb: :(){ : : & };: An innocently looking short code that creates bash function that reproduces itself. A Denial Of Service (DOS) attack. A very serious bug: The bash shellshock, Existed since 1989, discovered only 14 Sept. 2014. Allows the attacker to execute arbitrary commands on vulnerable versions of bash that can lead to unauthorized access to a computer system Testing vulnerability: env x='() { :;}; echo vulnerable' bash -c "echo this is a test Don t run stuff you don t understand! Just because someone recommended it on a webpage/forum...

Balazs Konya (Lund University) Programming for Scientists Tutorial1b 21 / 22 Further reading Online interactive Linux fundamentals tutorial (4 modules). Very much recommended: http://linuxsurvival.com Introduction to Linux CLI: Not a tutorial, rather online textbook. For those who would like to read more than the just these slides http://linuxtutorial.todolistme.net One-page Linux reference card: http://cheat-sheets.s3.amazonaws.com/for-mobile/linux-commands-cheat-sheetnew.pdf 3-pages Linux reference from O Reilly: http://www.linuxdevcenter.com/excerpt/linuxpg_quickref/linux.pdf

Balazs Konya (Lund University) Programming for Scientists Tutorial1b 22 / 22 Take away message: Linux Command Line is a very powerful toolbox It is much more than file management, there are tools (commands) that look more like full-scale programming environments After mastering it, the CLI gives you full control over the system, directory structure, file content, processes and much more (almost) everything can be done in the CLI be careful, you might destroy your system! Linux is the native environment for scientific computing, many scientific tools are deeply rooted in the Linux culture The more you use it, the more addicted you ll become