UNIX, GNU/Linux and simple tools for data manipulation
|
|
- Karin Craig
- 6 years ago
- Views:
Transcription
1 UNIX, GNU/Linux and simple tools for data manipulation Dr Jean-Baka DOMELEVO ENTFELLNER BecA-ILRI Hub Basic Bioinformatics Training Addis Ababa Wednesday December 13 th 2017 Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 1 / 37
2 1 UNIX & GNU/Linux: brief history and introduction 2 Using the Bash shell Your first commands Filesystems and permissions Bash special characters and features Quoting in Bash 3 So many tools You CANNOT live without your man Data manipulation commandline tools Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 2 / 37
3 Outline 1 UNIX & GNU/Linux: brief history and introduction 2 Using the Bash shell Your first commands Filesystems and permissions Bash special characters and features Quoting in Bash 3 So many tools You CANNOT live without your man Data manipulation commandline tools Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 3 / 37
4 UNIX & GNU/Linux: introduction GNU/Linux is an operating system (OS). GNU/Linux fully belongs to a broad family of OSes, the UNIX family. Operating system: definition unique interface between the computer (hardware) and the different programs (software) users run on it allows different programs and different users to use concurrently the same machine implements a filesystem, a console environment, a graphical environment, drivers for keyboard and mouse, etc examples of operating systems: Windows (Microsoft), Mac OS X (Apple), Android (Google), GNU/Linux, FreeBSD, etc Linux is only the kernel of GNU/Linux systems, responsible for granting access to the resources on the host and for time-sharing between processes. Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 4 / 37
5 UNIX & GNU/Linux systems: timeline GNU/Linux: a fairly recent member of an old and huge family (see : UNICS 1971: UNIX Time-Sharing System V1 1982: SunOS : UNIX System V 1991: GNU project (GNU/Hurd) ; Linux : Linux : Darwin 0.1 ; Mac OS X Server : Android 1.0 (derived from Linux ) 2013: Linux 3.9 Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 5 / 37
6 Linux distributions: different flavours of the same OS The GNU/Linux operatring system comes in different distributions. Three distributions have ever been true beacons and gave many offsprings: 1 Debian (1993) Ubuntu, 2004 and Linux Mint, Slackware (1993), from SLS (1992) SuSE, RedHat (late 1994) CentOS and Fedora, both 2003 For a full account, see Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 6 / 37
7 What makes UNIX systems superior to the Windows family UNIX gives you more control over your computer (no hidden actions, no undesired pieces of software). Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 7 / 37
8 What makes UNIX systems superior to the Windows family UNIX gives you more control over your computer (no hidden actions, no undesired pieces of software). UNIX environments are free from viruses. Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 7 / 37
9 What makes UNIX systems superior to the Windows family UNIX gives you more control over your computer (no hidden actions, no undesired pieces of software). UNIX environments are free from viruses. UNIX enables you to harness the full computational power of your machine. Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 7 / 37
10 What makes UNIX systems superior to the Windows family UNIX gives you more control over your computer (no hidden actions, no undesired pieces of software). UNIX environments are free from viruses. UNIX enables you to harness the full computational power of your machine. UNIX systems have been designed from their origin to be massively multi-user and multi-process systems. Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 7 / 37
11 What makes UNIX systems superior to the Windows family UNIX gives you more control over your computer (no hidden actions, no undesired pieces of software). UNIX environments are free from viruses. UNIX enables you to harness the full computational power of your machine. UNIX systems have been designed from their origin to be massively multi-user and multi-process systems. UNIX systems are much more secure than any Windows. Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 7 / 37
12 What makes UNIX systems superior to the Windows family UNIX gives you more control over your computer (no hidden actions, no undesired pieces of software). UNIX environments are free from viruses. UNIX enables you to harness the full computational power of your machine. UNIX systems have been designed from their origin to be massively multi-user and multi-process systems. UNIX systems are much more secure than any Windows. Take-home message The true power of UNIX (and so of GNU/Linux) lies in its commandline interface. Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 7 / 37
13 Outline 1 UNIX & GNU/Linux: brief history and introduction 2 Using the Bash shell Your first commands Filesystems and permissions Bash special characters and features Quoting in Bash 3 So many tools You CANNOT live without your man Data manipulation commandline tools Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 8 / 37
14 Bash: a shell environment Bash is the most popular shell environment on GNU/Linux systems. It stands for "Bourne Again Shell". Shell environments are designed to: interact with the host filesystem (browse and create directories, see the content of files, etc), Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 9 / 37
15 Bash: a shell environment Bash is the most popular shell environment on GNU/Linux systems. It stands for "Bourne Again Shell". Shell environments are designed to: interact with the host filesystem (browse and create directories, see the content of files, etc), interact with the installed software (install, run, etc), Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 9 / 37
16 Bash: a shell environment Bash is the most popular shell environment on GNU/Linux systems. It stands for "Bourne Again Shell". Shell environments are designed to: interact with the host filesystem (browse and create directories, see the content of files, etc), interact with the installed software (install, run, etc), login to distant hosts (telnet, ssh), Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 9 / 37
17 Bash: a shell environment Bash is the most popular shell environment on GNU/Linux systems. It stands for "Bourne Again Shell". Shell environments are designed to: interact with the host filesystem (browse and create directories, see the content of files, etc), interact with the installed software (install, run, etc), login to distant hosts (telnet, ssh), perform all of the above through automated processes scripts. Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 9 / 37
18 Bash: a shell environment Bash is the most popular shell environment on GNU/Linux systems. It stands for "Bourne Again Shell". Shell environments are designed to: interact with the host filesystem (browse and create directories, see the content of files, etc), interact with the installed software (install, run, etc), login to distant hosts (telnet, ssh), perform all of the above through automated processes scripts. Shells are at the same time commandline environments (run one command at a time) and scripting environments (write and run scripts). Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 9 / 37
19 Bash: a shell environment Bash is the most popular shell environment on GNU/Linux systems. It stands for "Bourne Again Shell". Shell environments are designed to: interact with the host filesystem (browse and create directories, see the content of files, etc), interact with the installed software (install, run, etc), login to distant hosts (telnet, ssh), perform all of the above through automated processes scripts. Shells are at the same time commandline environments (run one command at a time) and scripting environments (write and run scripts). On most GNU/Linux distributions, Bash is accessible through the "Terminal" icon. Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 9 / 37
20 Outline 1 UNIX & GNU/Linux: brief history and introduction 2 Using the Bash shell Your first commands Filesystems and permissions Bash special characters and features Quoting in Bash 3 So many tools You CANNOT live without your man Data manipulation commandline tools Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 10 / 37
21 Standard structure of a UNIX command Synopsis of a command <command> <options> <objects> For example: ls (only the command) Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 11 / 37
22 Standard structure of a UNIX command Synopsis of a command <command> <options> <objects> For example: ls (only the command) ls -l (command plus an option) Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 11 / 37
23 Standard structure of a UNIX command Synopsis of a command <command> <options> <objects> For example: ls (only the command) ls -l (command plus an option) ls -l -h h3a* (command, two options and one object) Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 11 / 37
24 Standard structure of a UNIX command Synopsis of a command <command> <options> <objects> For example: ls (only the command) ls -l (command plus an option) ls -l -h h3a* (command, two options and one object) ls -lh h3a* (single-letter options can be concatenated) Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 11 / 37
25 Standard structure of a UNIX command Synopsis of a command <command> <options> <objects> For example: ls (only the command) ls -l (command plus an option) ls -l -h h3a* (command, two options and one object) ls -lh h3a* (single-letter options can be concatenated) cp one two (command and two objects) Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 11 / 37
26 Standard structure of a UNIX command Synopsis of a command <command> <options> <objects> For example: ls (only the command) ls -l (command plus an option) ls -l -h h3a* (command, two options and one object) ls -lh h3a* (single-letter options can be concatenated) cp one two (command and two objects) man head (command and one object) Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 11 / 37
27 Standard structure of a UNIX command Synopsis of a command <command> <options> <objects> For example: ls (only the command) ls -l (command plus an option) ls -l -h h3a* (command, two options and one object) ls -lh h3a* (single-letter options can be concatenated) cp one two (command and two objects) man head (command and one object) head -n 2 one (an option with a value) Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 11 / 37
28 Standard structure of a UNIX command Synopsis of a command <command> <options> <objects> For example: ls (only the command) ls -l (command plus an option) ls -l -h h3a* (command, two options and one object) ls -lh h3a* (single-letter options can be concatenated) cp one two (command and two objects) man head (command and one object) head -n 2 one (an option with a value) head --lines=2 one (same command, POSIX-style long option) Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 11 / 37
29 Outline 1 UNIX & GNU/Linux: brief history and introduction 2 Using the Bash shell Your first commands Filesystems and permissions Bash special characters and features Quoting in Bash 3 So many tools You CANNOT live without your man Data manipulation commandline tools Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 12 / 37
30 UNIX filesystems Filesystems are hierarchies. The filesystem of a UNIX machine is standardized. Under the root (/) are: /bin essential command binairies /boot static files of the boot loader /dev device files (special files to access your devices) /etc host-specific system configuration files /home user home directories (e.g. /home/peter, /home/sarah, etc) /lib essential shared librairies and kernel modules /media mount point for removable media (e.g. CD-ROMs & flash disks) /mnt old-style mount point for any media /tmp system-wide temporary folder, writable by anyone Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 13 / 37
31 File permissions Three (four) types of rights: right to read from a file (r) right to write to it (w) right to execute a binary file or a script (x) right to traverse a directory (x) Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 14 / 37
32 File permissions Three (four) types of rights: right to read from a file (r) right to write to it (w) right to execute a binary file or a script (x) right to traverse a directory (x) Three types of people: the owner of a file (u) the other members of the user s group (g) the rest of the world, the others (o) Typical line of output from ls -l -rw-r--r-- 1 jbde jbde juil. 6 12:48 awk.dvi Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 14 / 37
33 File permissions explained Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 15 / 37
34 Outline 1 UNIX & GNU/Linux: brief history and introduction 2 Using the Bash shell Your first commands Filesystems and permissions Bash special characters and features Quoting in Bash 3 So many tools You CANNOT live without your man Data manipulation commandline tools Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 16 / 37
35 Why it is often necessary to quote strings or escape chars Some characters have a special meaning for the tools you use, e.g. the commandline interpreter Bash: spaces or tabs are logical separators between elements on the commandline: cd /tmp a dollar sign introduces Bash variables: echo $PATH a star means all the files (wildcard): cat * the greater than sign is interpreted as a redirection: cat * > listing.txt the vertical bar pipes the output of some command into the input of another: grep h3a long_course.htm wc -l... Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 17 / 37
36 Why it is often necessary to quote strings or escape chars Some characters have a special meaning for the tools you use, e.g. the commandline interpreter Bash: spaces or tabs are logical separators between elements on the commandline: cd /tmp a dollar sign introduces Bash variables: echo $PATH a star means all the files (wildcard): cat * the greater than sign is interpreted as a redirection: cat * > listing.txt the vertical bar pipes the output of some command into the input of another: grep h3a long_course.htm wc -l... escaping or quoting prevents these characters from being interpreted by the shell. Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 17 / 37
37 Escaping a single character In Unix, prepending a backslash (\) escapes the character following the backslash. Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 18 / 37
38 Escaping a single character In Unix, prepending a backslash (\) escapes the character following the backslash. > echo $PATH /home/jbde/bin:/usr/local/bin:/usr/bin:/bin > echo \$PATH $PATH Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 18 / 37
39 Escaping a single character In Unix, prepending a backslash (\) escapes the character following the backslash. > echo $PATH /home/jbde/bin:/usr/local/bin:/usr/bin:/bin > echo \$PATH $PATH And if a filename contains spaces, e.g. named with spaces.txt: > cat named with spaces.txt cat: named: No such file or directory cat: with: No such file or directory cat: spaces.txt: No such file or directory > cat named\ with\ spaces.txt <produces the content of the file> Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 18 / 37
40 Outline 1 UNIX & GNU/Linux: brief history and introduction 2 Using the Bash shell Your first commands Filesystems and permissions Bash special characters and features Quoting in Bash 3 So many tools You CANNOT live without your man Data manipulation commandline tools Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 19 / 37
41 Strong quoting with single quotes You can also quote a string to prevent included spaces to be interpreted: > cat 'named with spaces.txt' <produces the content of the file> Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 20 / 37
42 Strong quoting with single quotes You can also quote a string to prevent included spaces to be interpreted: > cat 'named with spaces.txt' <produces the content of the file> Generally speaking, simple quote do not allow any kind of interpretation/substitution/expansion. > echo 'Your PATH variable contains $PATH' Your PATH variable contains $PATH Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 20 / 37
43 Weak quoting with double quotes While preventing included spaces to be interpreted, double quotes allow expansion of Bash variables: > cat "named with spaces.txt" <produces the content of the file> > echo "Your PATH variable contains $PATH" Your PATH variable contains /home/jbde/bin:/usr/local/bin:/usr/b Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 21 / 37
44 Using Bash every day Bash has nice features you should use to work efficiently: the history of previous commands (browse vith,, Ctrl+R) autocompletion with the <TAB> key everywhere you can (commands, filenames, etc) wildcards and regexps use quoting appropriately pipe commands into each other ( ) redirect output (> erases previous file, >> appends) Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 22 / 37
45 Outline 1 UNIX & GNU/Linux: brief history and introduction 2 Using the Bash shell Your first commands Filesystems and permissions Bash special characters and features Quoting in Bash 3 So many tools You CANNOT live without your man Data manipulation commandline tools Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 23 / 37
46 Outline 1 UNIX & GNU/Linux: brief history and introduction 2 Using the Bash shell Your first commands Filesystems and permissions Bash special characters and features Quoting in Bash 3 So many tools You CANNOT live without your man Data manipulation commandline tools Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 24 / 37
47 Asking for help on a command: man This is the absolute basic command, to learn first! man ls To browse within the manpage: <Space>: next page b: previous page G: goto the bottom g: goto the beginning /: search an expression (indicate pattern or string and press <Enter>) q: quit and return to commandline Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 25 / 37
48 Sectioning of a manpage Manpages are all written using the same format/sectioning: 1 NAME: the name of the command 2 SYNOPSIS: the syntax of the command (sometimes several lines to describe several ways of using the command) square brackets ([...]) indicate optional components pipes ( ) within a construct separates alternatives ellipsis (...) usually indicate that the previous object is repeatable 3 DESCRIPTION and OPTIONS: meaning and behaviour of the different options and objects to give on the commandline 4 EXAMPLES: the most useful section, provides real-world examples along with some explanation of what they do 5 EXIT STATUS: useful in scripts, to monitor automatically whether the command execution produced and error 6 SEE ALSO: also useful when you don t know exactly the name of a command but know a similar/sister one (e.g. uniq and join are cross-referenced) Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 26 / 37
49 Outline 1 UNIX & GNU/Linux: brief history and introduction 2 Using the Bash shell Your first commands Filesystems and permissions Bash special characters and features Quoting in Bash 3 So many tools You CANNOT live without your man Data manipulation commandline tools Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 27 / 37
50 Reading files: cat and less cat produces the full content of file(s) to the standard output can concatenate several files: cat FILE1 FILE2 > FILE3 is non-interactive: prints all and quits Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 28 / 37
51 Reading files: cat and less cat produces the full content of file(s) to the standard output can concatenate several files: cat FILE1 FILE2 > FILE3 is non-interactive: prints all and quits less is a pager produces the full content of file(s) to the standard output, one page at a time several files are processed one after the other: less FILE1 FILE2 and then :n (next) and :p (previous) to browse is fully interactive: <space> for next page, b for the previous, / to search, q to quit, etc useful option: -S not to have your lines automatically wrapped (preserves column alignment on long lines) Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 28 / 37
52 Count the numbers of chars, words or lines: wc wc stands for "word count" wc -l FILE number of lines wc -c FILE number of bytes ( chars) wc -w FILE number of words wc -L FILE length of longest line in file Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 29 / 37
53 Select columns from a file: cut Simplified syntax cut -f <fields> -d <delimiter> FILE be sure you quote the delimiter, e.g. ``;'' <fields> can be a comma-separated list (ranges indicated with hyphens) Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 30 / 37
54 Select columns from a file: cut Simplified syntax cut -f <fields> -d <delimiter> FILE be sure you quote the delimiter, e.g. ``;'' <fields> can be a comma-separated list (ranges indicated with hyphens) Example: select fields 2 and 5 from a semicolon-separated file cut -f 2,5 -d ';' cut_example.csv Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 30 / 37
55 Select columns from a file: cut Simplified syntax cut -f <fields> -d <delimiter> FILE be sure you quote the delimiter, e.g. ``;'' <fields> can be a comma-separated list (ranges indicated with hyphens) Example: select fields 2 and 5 from a semicolon-separated file cut -f 2,5 -d ';' cut_example.csv Example: specify output separator cut -f 1-3 -d ';' --output-separator=$'\t' cut_example.csv Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 30 / 37
56 Select columns from a file: cut Simplified syntax cut -f <fields> -d <delimiter> FILE be sure you quote the delimiter, e.g. ``;'' <fields> can be a comma-separated list (ranges indicated with hyphens) Example: select fields 2 and 5 from a semicolon-separated file cut -f 2,5 -d ';' cut_example.csv Example: specify output separator cut -f 1-3 -d ';' --output-separator=$'\t' cut_example.csv Example: extract only the first three characters of each line cut -c 1-3 cut_example.csv Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 30 / 37
57 Sort a file according to some rules: sort sort sorts text files according to the content of some fields, called keys. Example: sorting lines alphabetically sort cut_example.csv Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 31 / 37
58 Sort a file according to some rules: sort sort sorts text files according to the content of some fields, called keys. Example: sorting lines alphabetically sort cut_example.csv But it s usually not a good idea not to control the way sort sorts. Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 31 / 37
59 Sort a file according to some rules: sort sort sorts text files according to the content of some fields, called keys. Example: sorting lines alphabetically sort cut_example.csv But it s usually not a good idea not to control the way sort sorts. Example: sort according to 2 nd and then 3 rd field (semicol-separated fields) sort -t ';' -k 2,3 cut_example.csv Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 31 / 37
60 Sort a file according to some rules: sort sort sorts text files according to the content of some fields, called keys. Example: sorting lines alphabetically sort cut_example.csv But it s usually not a good idea not to control the way sort sorts. Example: sort according to 2 nd and then 3 rd field (semicol-separated fields) sort -t ';' -k 2,3 cut_example.csv Example: sort numerically (-n) according to 9 th field only sort -t ';' -n -k 9,9 cut_example.csv # to check results: sort -t ';' -n -k 9,9 cut_example.csv cut -f 9 -d ';' Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 31 / 37
61 sort, continued -g option to sort numerical fields containing scientific notation: sort -k 2,2 -n with_sci_notation # unexpected result sort -k 2,2 -g with_sci_notation # GOOD! Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 32 / 37
62 sort, continued -g option to sort numerical fields containing scientific notation: sort -k 2,2 -n with_sci_notation # unexpected result sort -k 2,2 -g with_sci_notation # GOOD! WARNING!! sort relies heavily on your locale setting! Try: LC_ALL=fr_FR.utf8 sort -k 2,2 -g with_sci_notation Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 32 / 37
63 sort, continued -g option to sort numerical fields containing scientific notation: sort -k 2,2 -n with_sci_notation # unexpected result sort -k 2,2 -g with_sci_notation # GOOD! WARNING!! sort relies heavily on your locale setting! Try: LC_ALL=fr_FR.utf8 sort -k 2,2 -g with_sci_notation One-letter sorting options can be used as flags, and several fields specified: Ascending order on the 5 th field, descending on the 6 th and then alphabetically on the 1 st field sort -k 5,5g -k 6,6nr -k 1,1 hmmsearch_raw_output less -S Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 32 / 37
64 sort, some caveats WARNING!! by default, sort separates fields on blank to non-blank transitions. careful with empty fields! One should specify the delimiter. A precise delimiter to prevent sort from merging delimiters sort -k 11,11 -t $'\t' CDS_top_100.txt cut -f 11 less Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 33 / 37
65 join lines of two files sharing a common field join allows you to perform the relational join operation on two files. Example: I want to select the lines of FILE2 whose 11 th field corresponds to an entry in FILE1. join t $'\t' dg_top_100.txt CDS_top_100.txt WARNING!! join operates on files already sorted on the join field! Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 34 / 37
66 Produce only the n last lines of a file: tail Convenient to cut parts you are not interested in, for instance because: the final lines of a log file contain the error that matters to you the header (first few lines) of the file is of no interest for the next tool in the pipeline the file is sorted and the last lines contain the samples of interest: you set a cutoff Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 35 / 37
67 Produce only the n last lines of a file: tail Convenient to cut parts you are not interested in, for instance because: the final lines of a log file contain the error that matters to you the header (first few lines) of the file is of no interest for the next tool in the pipeline the file is sorted and the last lines contain the samples of interest: you set a cutoff Produce the last 30 lines of a file tail -n 30 input_file or simply: tail -30 input_file Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 35 / 37
68 Produce only the n last lines of a file: tail Convenient to cut parts you are not interested in, for instance because: the final lines of a log file contain the error that matters to you the header (first few lines) of the file is of no interest for the next tool in the pipeline the file is sorted and the last lines contain the samples of interest: you set a cutoff Produce the last 30 lines of a file tail -n 30 input_file or simply: tail -30 input_file Produce all the lines from the 30th tail -n +30 input_file Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 35 / 37
69 Symmetrical to tail: head Produce the first 30 lines of a file head -n 30 input_file or simply: head -30 input_file Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 36 / 37
70 Symmetrical to tail: head Produce the first 30 lines of a file head -n 30 input_file or simply: head -30 input_file Produce all but the last 30 lines head -n -30 input_file Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 36 / 37
71 Translate chars with tr tr helps you change any occurrence of a character into another: Translating Windows end-of-lines into UNIX ones cat Win_formatted_file tr '\r' '\n' > UNIX_formatted_file Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 37 / 37
72 Translate chars with tr tr helps you change any occurrence of a character into another: Translating Windows end-of-lines into UNIX ones cat Win_formatted_file tr '\r' '\n' > UNIX_formatted_file Warning! tr only processes its standard input! Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 37 / 37
73 Translate chars with tr tr helps you change any occurrence of a character into another: Translating Windows end-of-lines into UNIX ones cat Win_formatted_file tr '\r' '\n' > UNIX_formatted_file Warning! tr only processes its standard input! But tr also comes handy to change separators in a CSV file: Translating semicols into tabulations cat example_mj.txt tr ';' '\t' Dr Jean-Baka DOMELEVO ENTFELLNER UNIX, GNU/Linux and simple tools for data manipulation 37 / 37
A Brief Introduction to the Linux Shell for Data Science
A Brief Introduction to the Linux Shell for Data Science Aris Anagnostopoulos 1 Introduction Here we will see a brief introduction of the Linux command line or shell as it is called. Linux is a Unix-like
More informationTable of contents. Our goal. Notes. Notes. Notes. Summer June 29, Our goal is to see how we can use Unix as a tool for developing programs
Summer 2010 Department of Computer Science and Engineering York University Toronto June 29, 2010 1 / 36 Table of contents 1 2 3 4 2 / 36 Our goal Our goal is to see how we can use Unix as a tool for developing
More informationBioinformatics? Reads, assembly, annotation, comparative genomics and a bit of phylogeny.
Bioinformatics? Reads, assembly, annotation, comparative genomics and a bit of phylogeny stefano.gaiarsa@unimi.it Linux and the command line PART 1 Survival kit for the bash environment Purpose of the
More informationPrinciples of Bioinformatics. BIO540/STA569/CSI660 Fall 2010
Principles of Bioinformatics BIO540/STA569/CSI660 Fall 2010 Lecture Five Practical Computing Skills Emphasis This time it s concrete, not abstract. Fall 2010 BIO540/STA569/CSI660 3 Administrivia Monday
More informationIntroduction To Linux. Rob Thomas - ACRC
Introduction To Linux Rob Thomas - ACRC What Is Linux A free Operating System based on UNIX (TM) An operating system originating at Bell Labs. circa 1969 in the USA More of this later... Why Linux? Free
More informationIntroduction to UNIX command-line
Introduction to UNIX command-line Boyce Thompson Institute March 17, 2015 Lukas Mueller & Noe Fernandez Class Content Terminal file system navigation Wildcards, shortcuts and special characters File permissions
More informationUnix System Architecture, File System, and Shell Commands
Unix System Architecture, File System, and Shell Commands Prof. (Dr.) K.R. Chowdhary, Director COE Email: kr.chowdhary@iitj.ac.in webpage: http://www.krchowdhary.com JIET College of Engineering August
More informationIntroduction: What is Unix?
Introduction Introduction: What is Unix? An operating system Developed at AT&T Bell Labs in the 1960 s Command Line Interpreter GUIs (Window systems) are now available Introduction: Unix vs. Linux Unix
More informationChapter-3. Introduction to Unix: Fundamental Commands
Chapter-3 Introduction to Unix: Fundamental Commands What You Will Learn The fundamental commands of the Unix operating system. Everything told for Unix here is applicable to the Linux operating system
More informationShell Programming Overview
Overview Shell programming is a way of taking several command line instructions that you would use in a Unix command prompt and incorporating them into one program. There are many versions of Unix. Some
More informationChapter Two. Lesson A. Objectives. Exploring the UNIX File System and File Security. Understanding Files and Directories
Chapter Two Exploring the UNIX File System and File Security Lesson A Understanding Files and Directories 2 Objectives Discuss and explain the UNIX file system Define a UNIX file system partition Use the
More informationIntroduction to Linux
Introduction to Linux University of Bristol - Advance Computing Research Centre 1 / 47 Operating Systems Program running all the time Interfaces between other programs and hardware Provides abstractions
More informationShells and Shell Programming
Shells and Shell Programming 1 Shells A shell is a command line interpreter that is the interface between the user and the OS. The shell: analyzes each command determines what actions are to be performed
More informationChapter 1 - Introduction. September 8, 2016
Chapter 1 - Introduction September 8, 2016 Introduction Overview of Linux/Unix Shells Commands: built-in, aliases, program invocations, alternation and iteration Finding more information: man, info Help
More informationShells and Shell Programming
Shells and Shell Programming Shells A shell is a command line interpreter that is the interface between the user and the OS. The shell: analyzes each command determines what actions are to be performed
More informationLinux Operating System Environment Computadors Grau en Ciència i Enginyeria de Dades Q2
Linux Operating System Environment Computadors Grau en Ciència i Enginyeria de Dades 2017-2018 Q2 Facultat d Informàtica de Barcelona This first lab session is focused on getting experience in working
More informationComputer Systems and Architecture
Computer Systems and Architecture Stephen Pauwels Computer Systems Academic Year 2018-2019 Overview of the Semester UNIX Introductie Regular Expressions Scripting Data Representation Integers, Fixed point,
More informationPerl and R Scripting for Biologists
Perl and R Scripting for Biologists Lukas Mueller PLBR 4092 Course overview Linux basics (today) Linux advanced (Aure, next week) Why Linux? Free open source operating system based on UNIX specifications
More informationScripting Languages Course 1. Diana Trandabăț
Scripting Languages Course 1 Diana Trandabăț Master in Computational Linguistics - 1 st year 2017-2018 Today s lecture Introduction to scripting languages What is a script? What is a scripting language
More informationOverview LEARN. History of Linux Linux Architecture Linux File System Linux Access Linux Commands File Permission Editors Conclusion and Questions
Lanka Education and Research Network Linux Architecture, Linux File System, Linux Basic Commands 28 th November 2016 Dilum Samarasinhe () Overview History of Linux Linux Architecture Linux File System
More informationUnix Introduction to UNIX
Unix Introduction to UNIX Get Started Introduction The UNIX operating system Set of programs that act as a link between the computer and the user. Developed in 1969 by a group of AT&T employees Various
More informationCSC209H Lecture 1. Dan Zingaro. January 7, 2015
CSC209H Lecture 1 Dan Zingaro January 7, 2015 Welcome! Welcome to CSC209 Comments or questions during class? Let me know! Topics: shell and Unix, pipes and filters, C programming, processes, system calls,
More informationOperating Systems. Copyleft 2005, Binnur Kurt
3 Operating Systems Copyleft 2005, Binnur Kurt Content The concept of an operating system. The internal architecture of an operating system. The architecture of the Linux operating system in more detail.
More informationOperating Systems 3. Operating Systems. Content. What is an Operating System? What is an Operating System? Resource Abstraction and Sharing
Content 3 Operating Systems The concept of an operating system. The internal architecture of an operating system. The architecture of the Linux operating system in more detail. How to log into (and out
More informationIntroduction to Unix: Fundamental Commands
Introduction to Unix: Fundamental Commands Ricky Patterson UVA Library Based on slides from Turgut Yilmaz Istanbul Teknik University 1 What We Will Learn The fundamental commands of the Unix operating
More informationIntroduction to UNIX command-line II
Introduction to UNIX command-line II Boyce Thompson Institute 2017 Prashant Hosmani Class Content Terminal file system navigation Wildcards, shortcuts and special characters File permissions Compression
More informationITST Searching, Extracting & Archiving Data
ITST 1136 - Searching, Extracting & Archiving Data Name: Step 1 Sign into a Pi UN = pi PW = raspberry Step 2 - Grep - One of the most useful and versatile commands in a Linux terminal environment is the
More informationStd: XI CHAPTER-3 LINUX
Commands: General format: Command Option Argument Command: ls - Lists the contents of a file. Option: Begins with minus sign (-) ls a Lists including the hidden files. Argument refers to the name of a
More informationIntroduction of Linux
Introduction of Linux 阳 oslab2018_class1@163.com 寅 oslab2018_class2@163.com PART I Brief Introduction Basic Conceptions & Environment Install & Configure a Virtual Machine Basic Commands PART II Shell
More informationExamples: Directory pathname: File pathname: /home/username/ics124/assignments/ /home/username/ops224/assignments/assn1.txt
ULI101 Week 03 Week Overview Absolute and relative pathnames File name expansion Shell basics Command execution in detail Recalling and editing previous commands Quoting Pathnames A pathname is a list
More informationIntroduction. Let s start with the first set of slides
Tux Wars Class - 1 Table of Contents 1) Introduction to Linux and its history 2) Booting process of a linux system 3) Linux Kernel 4) What is a shell 5) Bash Shell 6) Anatomy of command 7) Let s make our
More informationComputer Systems and Architecture
Computer Systems and Architecture Introduction to UNIX Stephen Pauwels University of Antwerp October 2, 2015 Outline What is Unix? Getting started Streams Exercises UNIX Operating system Servers, desktops,
More informationModule 8 Pipes, Redirection and REGEX
Module 8 Pipes, Redirection and REGEX Exam Objective 3.2 Searching and Extracting Data from Files Objective Summary Piping and redirection Partial POSIX Command Line and Redirection Command Line Pipes
More informationShells. A shell is a command line interpreter that is the interface between the user and the OS. The shell:
Shells A shell is a command line interpreter that is the interface between the user and the OS. The shell: analyzes each command determines what actions are to be performed performs the actions Example:
More informationINTRODUCTION TO LINUX
INTRODUCTION TO LINUX REALLY SHORT HISTORY Before GNU/Linux there were DOS, MAC and UNIX. All systems were proprietary. The GNU project started in the early 80s by Richard Stallman Goal to make a free
More informationIntroduction to Linux. Woo-Yeong Jeong Computer Systems Laboratory Sungkyunkwan University
Introduction to Linux Woo-Yeong Jeong (wooyeong@csl.skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu What is Linux? A Unix-like operating system of a computer What is an
More informationReading and manipulating files
Reading and manipulating files Goals By the end of this lesson you will be able to Read files without using text editors Access specific parts of files Count the number of words and lines in a file Sort
More informationGetting to grips with Unix and the Linux family
Getting to grips with Unix and the Linux family David Chiappini, Giulio Pasqualetti, Tommaso Redaelli Torino, International Conference of Physics Students August 10, 2017 According to the booklet At this
More informationIntroduction to Linux. Fundamentals of Computer Science
Introduction to Linux Fundamentals of Computer Science Outline Operating Systems Linux History Linux Architecture Logging in to Linux Command Format Linux Filesystem Directory and File Commands Wildcard
More informationLinux for Beginners. Windows users should download putty or bitvise:
Linux for Beginners Windows users should download putty or bitvise: https://putty.org/ Brief History UNIX (1969) written in PDP-7 assembly, not portable, and designed for programmers as a reaction by Bell
More informationPart III. Shell Config. Tobias Neckel: Scripting with Bash and Python Compact Max-Planck, February 16-26,
Part III Shell Config Compact Course @ Max-Planck, February 16-26, 2015 33 Special Directories. current directory.. parent directory ~ own home directory ~user home directory of user ~- previous directory
More informationMineração de Dados Aplicada
Simple but Powerful Text-Processing Commands August, 29 th 2018 DCC ICEx UFMG Unix philosophy Unix philosophy Doug McIlroy (inventor of Unix pipes). In A Quarter-Century of Unix (1994): Write programs
More informationIntroduction to Linux
Introduction to Linux Prof. Jin-Soo Kim( jinsookim@skku.edu) TA Sanghoon Han(sanghoon.han@csl.skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu Announcement (1) Please come
More informationOperating System Interaction via bash
Operating System Interaction via bash bash, or the Bourne-Again Shell, is a popular operating system shell that is used by many platforms bash uses the command line interaction style generally accepted
More informationEssential Unix and Linux! Perl for Bioinformatics, ! F. Pineda
Essential Unix and Linux! Perl for Bioinformatics, 140.636! F. Pineda Generic computer architecture Memory Storage Fig. 1.2 From Designing Embedded Hardware, 2 nd Ed. by John Catsoulis OS concepts Shell
More information22-Sep CSCI 2132 Software Development Lecture 8: Shells, Processes, and Job Control. Faculty of Computer Science, Dalhousie University
Lecture 8 p.1 Faculty of Computer Science, Dalhousie University CSCI 2132 Software Development Lecture 8: Shells, Processes, and Job Control 22-Sep-2017 Location: Goldberg CS 127 Time: 14:35 15:25 Instructor:
More informationLinux & Shell Programming 2014
Unit -1: Introduction to UNIX/LINUX Operating System Practical Practice Questions: Find errors (if any) otherwise write output or interpretation of following commands. (Consider default shell is bash shell.)
More informationEECS2301. Lab 1 Winter 2016
EECS2301 Lab 1 Winter 2016 Lab Objectives In this lab, you will be introduced to the Linux operating system. The basic commands will be presented in this lab. By the end of you alb, you will be asked to
More informationLinux & Shell Programming 2014
Practical No : 1 Enrollment No: Group : A Practical Problem Write a date command to display date in following format: (Consider current date as 4 th January 2014) 1. dd/mm/yy hh:mm:ss 2. Today's date is:
More informationVirtual Machine. Linux flavor : Debian. Everything (except slides) preinstalled for you. https://www.virtualbox.org/
Virtual Machine Anyone have problems installing it? VM: Virtual Box - allows you to run a different operating system within the current operating system of your machine. https://www.virtualbox.org/ Linux
More informationUNIX COMMANDS AND SHELLS. UNIX Programming 2015 Fall by Euiseong Seo
UNIX COMMANDS AND SHELLS UNIX Programming 2015 Fall by Euiseong Seo What is a Shell? A system program that allows a user to execute Shell functions (internal commands) Other programs (external commands)
More informationWeek Overview. Simple filter commands: head, tail, cut, sort, tr, wc grep utility stdin, stdout, stderr Redirection and piping /dev/null file
ULI101 Week 05 Week Overview Simple filter commands: head, tail, cut, sort, tr, wc grep utility stdin, stdout, stderr Redirection and piping /dev/null file head and tail commands These commands display
More informationLinux Command Line Primer. By: Scott Marshall
Linux Command Line Primer By: Scott Marshall Draft: 10/21/2007 Table of Contents Topic Page(s) Preface 1 General Filesystem Background Information 2 General Filesystem Commands 2 Working with Files and
More informationIntroduction to Linux
Introduction to Linux The command-line interface A command-line interface (CLI) is a type of interface, that is, a way to interact with a computer. Window systems, punched cards or a bunch of dials, buttons
More informationFilesystem Hierarchy and Permissions
and Linux Prepared by Steven Gordon on 19 April 2017 Common/Reports/linux-file-permissions.tex, r1417 1/15 Multiuser and Server Operating System Linux systems are commonly used as a multi-user system E.g.
More informationCENG 334 Computer Networks. Laboratory I Linux Tutorial
CENG 334 Computer Networks Laboratory I Linux Tutorial Contents 1. Logging In and Starting Session 2. Using Commands 1. Basic Commands 2. Working With Files and Directories 3. Permission Bits 3. Introduction
More information*nix Crash Course. Presented by: Virginia Tech Linux / Unix Users Group VTLUUG
*nix Crash Course Presented by: Virginia Tech Linux / Unix Users Group VTLUUG Ubuntu LiveCD No information on your hard-drive will be modified. Gives you a working Linux system without having to install
More informationContents. Note: pay attention to where you are. Note: Plaintext version. Note: pay attention to where you are... 1 Note: Plaintext version...
Contents Note: pay attention to where you are........................................... 1 Note: Plaintext version................................................... 1 Hello World of the Bash shell 2 Accessing
More informationBasic Unix Command. It is used to see the manual of the various command. It helps in selecting the correct options
Basic Unix Command The Unix command has the following common pattern command_name options argument(s) Here we are trying to give some of the basic unix command in Unix Information Related man It is used
More informationChap2: Operating-System Structures
Chap2: Operating-System Structures Objectives: services OS provides to users, processes, and other systems structuring an operating system how operating systems are designed and customized and how they
More informationIntroduction to Linux Basics
Introduction to Linux Basics Part-I Georgia Advanced Computing Resource Center University of Georgia Zhuofei Hou, HPC Trainer zhuofei@uga.edu Outline What is GACRC? What is Linux? Linux Command, Shell
More informationUnix/Linux Primer. Taras V. Pogorelov and Mike Hallock School of Chemical Sciences, University of Illinois
Unix/Linux Primer Taras V. Pogorelov and Mike Hallock School of Chemical Sciences, University of Illinois August 25, 2017 This primer is designed to introduce basic UNIX/Linux concepts and commands. No
More information5/20/2007. Touring Essential Programs
Touring Essential Programs Employing fundamental utilities. Managing input and output. Using special characters in the command-line. Managing user environment. Surveying elements of a functioning system.
More informationCOMS 6100 Class Notes 3
COMS 6100 Class Notes 3 Daniel Solus September 1, 2016 1 General Remarks The class was split into two main sections. We finished our introduction to Linux commands by reviewing Linux commands I and II
More informationLinux Essentials Objectives Topics:
Linux Essentials Linux Essentials is a professional development certificate program that covers basic knowledge for those working and studying Open Source and various distributions of Linux. Exam Objectives
More information- c list The list specifies character positions.
CUT(1) BSD General Commands Manual CUT(1)... 1 PASTE(1) BSD General Commands Manual PASTE(1)... 3 UNIQ(1) BSD General Commands Manual UNIQ(1)... 5 HEAD(1) BSD General Commands Manual HEAD(1)... 7 TAIL(1)
More informationGNU/Linux 101. Casey McLaughlin. Research Computing Center Spring Workshop Series 2018
GNU/Linux 101 Casey McLaughlin Research Computing Center Spring Workshop Series 2018 rccworkshop IC;3df4mu bash-2.1~# man workshop Linux101 RCC Workshop L101 OBJECTIVES - Operating system concepts - Linux
More informationFilesystem Hierarchy and Permissions
2 and Prepared by Steven Gordon on 19 April 2017 Common/Reports/linux-file-permissions.tex, r1417 1 Multiuser and Server Operating System systems are commonly used as a multi-user system E.g. multiple
More informationLinux Essentials. Programming and Data Structures Lab M Tech CS First Year, First Semester
Linux Essentials Programming and Data Structures Lab M Tech CS First Year, First Semester Adapted from PDS Lab 2014 and 2015 Login, Logout, Password $ ssh mtc16xx@192.168.---.--- $ ssh X mtc16xx@192.168.---.---
More informationCS370 Operating Systems
CS370 Operating Systems Colorado State University Yashwant K Malaiya Fall 2016 Lecture 5 Slides based on Text by Silberschatz, Galvin, Gagne Various sources 1 1 User Operating System Interface - CLI CLI
More informationLecture 5. Essential skills for bioinformatics: Unix/Linux
Lecture 5 Essential skills for bioinformatics: Unix/Linux UNIX DATA TOOLS Text processing with awk We have illustrated two ways awk can come in handy: Filtering data using rules that can combine regular
More informationUNIX files searching, and other interrogation techniques
UNIX files searching, and other interrogation techniques Ways to examine the contents of files. How to find files when you don't know how their exact location. Ways of searching files for text patterns.
More informationDATA 301 Introduction to Data Analytics Command Line. Dr. Ramon Lawrence University of British Columbia Okanagan
DATA 301 Introduction to Data Analytics Command Line Dr. Ramon Lawrence University of British Columbia Okanagan ramon.lawrence@ubc.ca Why learn the Command Line? The command line is the text interface
More informationWhy learn the Command Line? The command line is the text interface to the computer. DATA 301 Introduction to Data Analytics Command Line
DATA 301 Introduction to Data Analytics Command Line Why learn the Command Line? The command line is the text interface to the computer. DATA 301: Data Analytics (2) Understanding the command line allows
More informationIntroduction to Linux
Introduction to Linux Prof. Jin-Soo Kim( jinsookim@skku.edu) TA - Kisik Jeong (kisik@csl.skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu What is Linux? A Unix-like operating
More informationOverview of the UNIX File System
Overview of the UNIX File System Navigating and Viewing Directories Adapted from Practical Unix and Programming Hunter College Copyright 2006 Stewart Weiss The UNIX file system The most distinguishing
More informationUNIX. The Very 10 Short Howto for beginners. Soon-Hyung Yook. March 27, Soon-Hyung Yook UNIX March 27, / 29
UNIX The Very 10 Short Howto for beginners Soon-Hyung Yook March 27, 2015 Soon-Hyung Yook UNIX March 27, 2015 1 / 29 Table of Contents 1 History of Unix 2 What is UNIX? 3 What is Linux? 4 How does Unix
More informationBasic Linux Command Line Interface Guide
This basic Linux Command-Line Interface (CLI) Guide provides a general explanation of commonly used Bash shell commands for the Barracuda NG Firewall. You can access the command-line interface by connecting
More informationIntroduction. File System. Note. Achtung!
3 Unix Shell 1: Introduction Lab Objective: Explore the basics of the Unix Shell. Understand how to navigate and manipulate file directories. Introduce the Vim text editor for easy writing and editing
More informationUnix Filesystem. January 26 th, 2004 Class Meeting 2
Unix Filesystem January 26 th, 2004 Class Meeting 2 * Notes adapted by Christian Allgood from previous work by other members of the CS faculty at Virginia Tech Unix Filesystem! The filesystem is your interface
More informationBasic Survival UNIX.
Basic Survival UNIX Many Unix based operating systems make available a Graphical User Interface for the sake of providing an easy way for less experienced users to work with the system. Some examples are
More informationIntroduction to Linux (Part I) BUPT/QMUL 2018/03/14
Introduction to Linux (Part I) BUPT/QMUL 2018/03/14 Contents 1. Background on Linux 2. Starting / Finishing 3. Typing Linux Commands 4. Commands to Use Right Away 5. Linux help continued 2 Contents 6.
More informationThe input can also be taken from a file and similarly the output can be redirected to another file.
Filter A filter is defined as a special program, which takes input from standard input device and sends output to standard output device. The input can also be taken from a file and similarly the output
More informationWorking With Unix. Scott A. Handley* September 15, *Adapted from UNIX introduction material created by Dr. Julian Catchen
Working With Unix Scott A. Handley* September 15, 2014 *Adapted from UNIX introduction material created by Dr. Julian Catchen What is UNIX? An operating system (OS) Designed to be multiuser and multitasking
More informationFile Commands. Objectives
File Commands Chapter 2 SYS-ED/Computer Education Techniques, Inc. 2: 1 Objectives You will learn: Purpose and function of file commands. Interrelated usage of commands. SYS-ED/Computer Education Techniques,
More informationTNM093 Practical Data Visualization and Virtual Reality Laboratory Platform
November 8, 2016 1 Introduction The laboratory exercises in this course are to be conducted in an environment that might not be familiar to many of you. It is based on open source software. We use an open
More informationIntroduction to Linux
Introduction to Linux Prof. Jin-Soo Kim( jinsookim@skku.edu) TA - Dong-Yun Lee (dylee@csl.skku.edu) Computer Systems Laboratory Sungkyunkwan University http://csl.skku.edu What is Linux? A Unix-like operating
More informationThe Unix Family. Linux 101. GNU/Linux distributions. Even more. MULTICS MIT/GE/Bell Labs. UNIX AT&T (Bell Labs) Mach, Carnegie Mellon
The Unix Family MULTICS MIT/GE/Bell Labs Linux 101 Johan Montelius KTH 2017 Mach, Carnegie Mellon GNU/Hurd OS X, Apple UNIX AT&T (Bell Labs) BSD Berkeley SysV AT&T Minix, Tanenbaum FreeBSD NetBSD GNU/Linux,
More informationThe Online Unix Manual
ACS-294-001 Unix (Winter Term, 2018-2019) Page 14 The Online Unix Manual Unix comes with a large, built-in manual that is accessible at any time from your terminal. The Online Manual is a collection of
More informationIntroduction to the Shell
[Software Development] Introduction to the Shell Davide Balzarotti Eurecom Sophia Antipolis, France What a Linux Desktop Installation looks like What you need Few Words about the Graphic Interface Unlike
More informationChapter 4. Unix Tutorial. Unix Shell
Chapter 4 Unix Tutorial Users and applications interact with hardware through an operating system (OS). Unix is a very basic operating system in that it has just the essentials. Many operating systems,
More informationIntroduction to Unix and Linux. Workshop 1: Directories and Files
Introduction to Unix and Linux Workshop 1: Directories and Files Genomics Core Lab TEXAS A&M UNIVERSITY CORPUS CHRISTI Anvesh Paidipala, Evan Krell, Kelly Pennoyer, Chris Bird Genomics Core Lab Informatics
More informationIntroduction to Linux. Roman Cheplyaka
Introduction to Linux Roman Cheplyaka Generic commands, files, directories What am I running? ngsuser@ubuntu:~$ cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu
More informationWelcome to getting started with Ubuntu Server. This System Administrator Manual. guide to be simple to follow, with step by step instructions
Welcome to getting started with Ubuntu 12.04 Server. This System Administrator Manual guide to be simple to follow, with step by step instructions with screenshots INDEX 1.Installation of Ubuntu 12.04
More informationLecture 3. Essential skills for bioinformatics: Unix/Linux
Lecture 3 Essential skills for bioinformatics: Unix/Linux RETRIEVING DATA Overview Whether downloading large sequencing datasets or accessing a web application hundreds of times to download specific files,
More informationBasics. I think that the later is better.
Basics Before we take up shell scripting, let s review some of the basic features and syntax of the shell, specifically the major shells in the sh lineage. Command Editing If you like vi, put your shell
More informationGetting your department account
02/11/2013 11:35 AM Getting your department account The instructions are at Creating a CS account 02/11/2013 11:36 AM Getting help Vijay Adusumalli will be in the CS majors lab in the basement of the Love
More informationIntroduction to Linux (and the terminal)
Introduction to Linux (and the terminal) 27/11/2018 Pierpaolo Maisano Delser mail: maisanop@tcd.ie ; pm604@cam.ac.uk Outline: What is Linux and the terminal? Why do we use the terminal? Pros and cons Basic
More informationBasic Linux Command Line Interface Guide
This basic Linux Command-Line Interface (CLI) Guide provides a general explanation of commonly used Bash shell commands for the Barracuda NG Firewall. You can access the command-line interface by connecting
More informationFiles and Directories
CSCI 2132: Software Development Files and Directories Norbert Zeh Faculty of Computer Science Dalhousie University Winter 2019 Files and Directories Much of the operation of Unix and programs running on
More information