CSE-205: Unix and Shell Programming Unit I The UNIX Operating system

Transcription

1 Unit I The UNIX Operating system The Unix Operating System, The UNIX architecture and Command Usage, The File System An Operating system: An operating system (OS) is a collection of software that manages computer hardware resources and provides common services for computer programs. The operating system is a vital component of the system software in a computer system. Application programs usually require an operating system to function. An operating system is a program that acts as an interface between the software and the computer hardware. It is an integration set of specialized programs that are used to manage overall resources and operations of the computer. It is specialized software that controls and monitors the execution of all other programs that reside in the computer, including application programs and other system software. Time-sharing operating systems schedule tasks for efficient use of the system and may also include accounting for cost allocation of processor time, mass storage, printing, and other resources. For hardware functions such as input and output and memory allocation, the operating system acts as an intermediary between programs and the computer hardware, although the application code is usually executed directly by the hardware and will frequently make a system call to an OS function or be interrupted by it. Operating systems can be found on almost any device that contains a computer from cellular phones and video game consoles to supercomputers and web servers. 1.2 Common features: Process management Interrupts Memory management File system Device drivers Networking (TCP/IP, UDP) Security (Process/Memory protection) I/O - By Prof. P.M.Katkar. Page 1

2 1.3 Operations of an Operating System: Making a computer system convenient to use in an efficient manner To hide the details of the hardware resources from the users To provide users a convenient interface to use the computer system. To act as an intermediary between the hardware and its users and making it easier for the users to access and use other resources. Manage the resources of a computer system. Keep track of who is using which resource, granting resource requests, according for resource using and mediating conflicting requests from different programs and users. The efficient and fair sharing of resources among users and programs 1.4 Characteristics of Operating System: Memory Management -- It keeps tracks of primary memory i.e. what part of it are in use by whom, what part are not in use etc. Allocates the memory when the process or program requests it. Processor Management -- Allocate the processor (CPU) to a process. Deallocate processor when processor is no longer required. Device Management -- Keep tracks of all devices. This is also called I/O controller. Decides which process gets the device when and for how much time. File Management -- Allocates the resources. De-allocates the resource. Decides who gets the resources. Security -- By means of passwords & similar other techniques, preventing unauthorized access to programs & data. Job accounting -- Keeping track of time & resources used by various jobs and/or users. Control over system performance -- Recording delays between request for a service & from the system. Interaction with the operators -- The interaction may take place via the console of the computer in the form of instructions. Operating System acknowledges the same, do the corresponding action and inform the operation by a display screen. Error-detecting aids -- Production of dumps, traces, error messages and other debugging and error-detecting methods. - By Prof. P.M.Katkar. Page 2

3 Coordination between other software and users -- Coordination and assignment of compilers, interpreters, assemblers and other software to the various users of the computer systems. Manufacturers of operating system 1.5 The UNIX, Linux, and variant history DOS WINDOWS ENTERPRISE LINUX (Macintosh) Mac OS, i-os(iphone OS). UNIX development was started in 1969 at Bell Laboratories in New Jersey. Bell Laboratories was ( ) involved on the development of a multi-user, time-sharing operating system called Multics (Multiplexed Information and Computing System). Multics was a failure and in in early 1969, Bell Labs withdrew from the Multics project. Bell Labs researchers who had worked on Multics (Ken Thompson, Dennis Ritchie, Douglas McIlroy, Joseph Ossanna, and others) still wanted to develop an operating system for their own and Bell Labs programming, job control, and resource usage needs. When Multics was withdrawn Ken Thompson and Dennis Ritchie needed to rewrite an operating system in order to play space travel on another smaller machine (a DEC PDP-7 [Programmed Data Processor 4K memory for user programs). The result was a system called UNICS (UNiplexed Information and Computing Service) which was an 'emasculated Multics'. The first version of Unix was written in the low-level PDP-7 assembler language. Later, a language called TMG was developed for the PDP-7 by R. M. McClure. Using TMG to develop a FORTRAN compiler, Ken Thompson instead ended up developing a compiler for a new high-level language he called B, based on the earlier BCPL language developed by Martin Richard. When the PDP-11 computer arrived at Bell Labs, Dennis Ritchie built on B to create a new - By Prof. P.M.Katkar. Page 3

4 language called C. Unix components were later rewritten in C, and finally with the kernel itself in Unix V6, released in 1975 became very popular. Unix V6 was free and was distributed with its source code. In 1983, AT&T released Unix System V which was a commercial version 1.6 What is UNIX? The UNIX operating system is a set of programs that act as a link between the computer and the user. The computer program that allocates the system resources and coordinates all the details of the computer's internal is called the operating system or kernel. Users communicate with the kernel through a program known as the shell. The shell is a command line interpreter; it translates commands entered by the user and converts them into a language that is understood by the kernel. The first "Unix" system was developed by Ken Thompson in the Bell AT&T laboratories at Murray Hill in New Jersey in the United States from Ken Thompson's aim was to develop a simple interactive operating system, called "Multics" (Multiplexed Information and Computing System) in order to run a game which he had created (space travel, a simulation of the solar system). In April 1969 the AT&T laboratories decided to use the GECOS (General Electric Comprehensive Operating System ) instead of Multics. However, Ken Thompson and Dennis Ritchie who joined the team needed to make the space travel game work on a smaller machine (a DEC PDP-7, Programmed Data Processor which only had 4K of memory to make user programs run), this is why they recreated the system in order to create a limited version of Multicscalled UNICS (UNiplexed Information and Computing Service), quickly shortened to Unix. There are various Unix variants available in the market. Solaris Unix, AIX, HP Unix and BSD are few examples. Linux is also a flavor of Unix which is freely available. Several people can use a UNIX computer at the same time; hence UNIX is called a multiuser system. A user can also run multiple programs at the same time; hence UNIX is called multitasking. 1.7 Architecture of UNIX O.S.: - By Prof. P.M.Katkar. Page 4

5 A system architecture or systems architecture is the conceptual model that defines the structure, behavior, and more views of a system. An architecture description is a formal description and representation of a system, organized in a way that supports reasoning about the structures of the system, which comprise system components, the externally visible properties of those components, the relationships (e.g. the behavior) between them, and provides a plan from which products can be procured, and systems developed, that will work together to implement the overall system. Unix is a layered operating system. The innermost layer is the hardware that provides the services for the OS. The operating system, referred to in Unix as the kernel, interacts directly with the hardware and provides the services to the user programs. These user programs don't need to know anything about the hardware. They just need to know how to interact with the kernel and it's up to the kernel to provide the desired service User: It is the person which gives instructions as input to computer to operate the computer resources and data. It can create it s own data or can able to modify system data. Users are generally demands for utility or applications from computer by an instruction to complete a cretin task. - By Prof. P.M.Katkar. Page 5

6 Here are three types of accounts on a UNIX system: 1. Root user, 2. System user, 3. User or limited user. 1. Root account: This is also called superuser and would have complete and unfettered control of the system. A superuser can run any commands without any restriction. This user should be assumed as a system administrator. 2. System accounts: System accounts are those needed for the operation of system-specific components for example mail accounts and the sshd accounts. These accounts are usually needed for some specific function on your system, and any modifications to them could adversely affect the system. 3. User accounts: User accounts provide interactive access to the system for users and groups of users. General users are typically assigned to these accounts and usually have limited access to critical system files and directories. UNIX supports a concept of Group Account which logically groups a number of accounts; every account would be a part of any group account. UNIX groups plays important role in handling file permissions and process management. Utility: A program that performs a very specific task, usually related to managing system resources. Operating systems contain a number of utilities for managing disk drives, printers, and other devices. Utilities differ from applications mostly in terms of size, complexity and function. For example, word processors, spreadsheet programs, and database applications are considered applications because they are large programs that perform a variety of functions not directly related to managing computer resources. The final layer of the Unix OS is the Utilities and Applications layer. This layer includes the commands, word processors, graphic programs and database management programs. Traditionally, these programs were accessed by typing the commands to start the program on the command line. They can still be accessed in this way, but they can now also be accessed through the GUI. UNIX utilities are commands that, generally, perform a single task. It may be as simple as printing the date and time, or a complex as finding files that match many criteria throughout a directory hierarchy. Commands can be combined through pipelines that connect the standard output of one utility to the standard input of another. - By Prof. P.M.Katkar. Page 6

7 Some utilities are built into the shell; test (and its synonym [) and echo have been in every Bourne-type shell since (at the very at least) Shell: The shell is the program that sits between the user and the kernel. It is the interpreter that translates the commands that are typed into the terminal session. Users can type commands directly into the terminal, or they can create a text file containing a series of commands that can be sent to the shell. The series of commands are called a shell script. There are multiple shells that are used by the Unix OS. They include the Bourne shell (sh), the C shell (csh), the Korn shell (ksh) and the Bourne Again shell (bash). Each shell has own set of shell commands. Operating system commands are the same across all the shells. The initial shell that the user logs into is defined by the system administrator. The user can change her default shell by using the "chsh" command. Users may want to change their shells in order to use particular features that are available in one shell or not another, or they may simply prefer a particular shell environment. Kernel: The kernel is a program that constitutes the central core of a computer operating system. It has complete control over everything that occurs in the system The kernel is the first part of the operating system to load into memory during booting (i.e., system startup), and it remains there for the entire duration of the computer session because its services are required continuously. Thus it is important for it to be as small as possible while still providing all the essential services needed by the other parts of the operating system and by the various application programs. Because of its critical nature, the kernel code is usually loaded into a protected area of memory, which prevents it from being overwritten by other, less frequently used parts of the operating system or by application programs. The kernel performs its tasks, such as executing processes and handling interrupts, in kernel space, whereas everything a user normally does, such as writing text in a text editor or running programs in a GUI (graphical user interface), is done in user space. This separation is made in order to prevent user data and kernel data from interfering with each other and thereby diminishing performance or causing the system to become unstable (and possibly crashing). When a computer crashes, it actually means the kernel has crashed. If only a single program has crashed but the rest of the system remains in operation, then the kernel itself has not crashed. A crash - By Prof. P.M.Katkar. Page 7

8 is the situation in which a program, either a user application or a part of the operating system, stops performing its expected function(s) and responding to other parts of the system. The program might appear to the user to freeze. If such program is a critical to the operation of the kernel, the entire computer could stall or shut down. The kernel provides basic services for all other parts of the operating system, typically including memory management, process management, file management and I/O (input/output) management (i.e., accessing the peripheral devices). These services are requested by other parts of the operating system or by application programs through a specified set of program interfaces referred to as system calls. Process management, possibly the most obvious aspect of a kernel to the user, is the part of the kernel that ensures that each process obtains its turn to run on the processor and that the individual processes do not interfere with each other by writing to their areas of memory. A process, also referred to as a task, can be defined as an executing (i.e., running) instance of a program. Categories of Kernels Kernels can be classified into four broad categories: monolithic kernels, microkernel, hybrid kernels and exokernels. Each has its own advocates and detractors. Monolithic kernels: Monolithic kernels, which have traditionally been used by Unix-like operating systems, contain all the operating system core functions and the device drivers (small programs that allow the operating system to interact with hardware devices, such as disk drives, video cards and printers). Modern monolithic kernels, such as those of Linux and FreeBSD, both of which fall into the category of Unix-like operating systems, feature the ability to load modules at runtime, thereby allowing easy extension of the kernel's capabilities as required, while helping to minimize the amount of code running in kernel space. Microkernel: A microkernel usually provides only minimal services, such as defining memory address spaces, interprocess communication (IPC) and process management. All other functions, such as hardware management, are implemented as processes running independently of the kernel. Examples of microkernel operating systems are AIX, BeOS, Hurd, Mach, Mac OS X, MINIX and QNX. Hybrid kernels: - By Prof. P.M.Katkar. Page 8

9 Hybrid kernels are similar to microkernels, except that they include additional code in kernel space so that such code can run more swiftly than it would were it in user space. These kernels represent a compromise that was implemented by some developers before it was demonstrated that pure microkernels can provide high performance. Hybrid kernels should not be confused with monolithic kernels that can load modules after booting (such as Linux). Most modern operating systems use hybrid kernels, including Microsoft Windows NT, 2000 and XP. DragonFly BSD, a recent fork (i.e., variant) of FreeBSD, is the first non-mach based BSD operating system to employ a hybrid kernel architecture. Exokernels: Exokernels are a still experimental approach to operating system design. They differ from the other types of kernels in that their functionality is limited to the protection and multiplexing of the raw hardware, and they provide no hardware abstractions on top of which applications can be constructed. This separation of hardware protection from hardware management enables application developers to determine how to make the most efficient use of the available hardware for each specific program. Exokernels in themselves they are extremely small. However, they are accompanied by library operating systems, which provide application developers with the conventional functionalities of a complete operating system. A major advantage of exokernel-based systems is that they can incorporate multiple library operating systems, each exporting a different API (application programming interface), such as one for Linux and one for Microsoft Windows, thus making it possible to simultaneously run both Linux and Windows applications. 1.8 Features of UNIX Several features of UNIX have made it popular. Some of them are: Portable UNIX can be installed on many hardware platforms. Its widespread use can be traced to the decision to develop it using the C language. Multiuser The UNIX design allows multiple users to concurrently share hardware and software Multitasking UNIX allows a user to run more than one program at a time. In fact more than one program can be running in the background while a user is working foreground. Networking - By Prof. P.M.Katkar. Page 9

10 While UNIX was developed to be an interactive, multiuser, multitasking system, networking is also incorporated into the heart of the operating system. Access to another system uses a standard communications protocol known as Transmission Control Protocol/Internet Protocol (TCP/IP). Organized File System UNIX has a very organized file and directory system that allows users to organize and maintain files. Device Independence UNIX treats input/output devices like ordinary files. The source or destination for file input and output is easily controlled through a UNIX design feature called redirection. Utilities UNIX provides a rich library of utilities that can be use to increase user productivity. 1.9 Advantages of UNIX Unix is more flexible and can be installed on many different types of machines, including main-frame computers, supercomputers and micro-computers. Unix is more stable and does not go down as often as Windows does, therefore requires less administration and maintenance. Unix has greater built-in security and permissions features than Windows. Unix possesses much greater processing power than Windows. Unix is the leader in serving the Web. About 90% of the Internet relies on Unix operating systems running on Apache, the world s most widely used Web server, which is free. Software upgrades from Microsoft often require the user to purchase new or more hardware or prerequisite software. That is not the case with Unix. The mostly free or inexpensive open-source operating systems, such as Linux and BSD, with their flexibility and control, prove to be very attractive to (aspiring) computer wizards. Many of the smartest programmers are developing state-of-the-art software free of charge for the fast growing "open-source movement. - By Prof. P.M.Katkar. Page 10

11 Unix also inspires novel approaches to software design, such as solving problems by interconnecting simpler tools instead of creating large monolithic application programs Introduction to UNIX files Files are collection of data items stored on disk. Or, it's device which can store the information, data, music (mp3 files), picture, movie, sound, book etc. In fact whatever you store in computer it must be inform of file. Files are always associated with devices like hard disk, floppy disk etc. File is the last object in your file system tree. See Linux/UNIX - rules for naming file and directory names. In UNIX systems any element is represented in the form of a file. All files are architecture around a single tree structure where the base, called the root, is written "/". UNIX systems define different file types: Ordinary Files: An ordinary file is a file on the system that contains data, text, or program instructions. In this tutorial, you look at working with ordinary files.ordinary files entered on the hard disk. This is a file in the generally understood sense of the word; Directories are files (nodes) of the tree structure able to contain files or other directories. A directory contains at least a parent directory (written..), relating to the directory at the higher level, and a current directory (written.), i.e. itself; Directory is group of files. Directory is divided into two types: Root directory - Strictly speaking, there is only one root directory in your system, which is denoted by / (forward slash). It is root of your entire file system and cannot be renamed or deleted. Sub directory - Directory under root (/) directory is subdirectory which can be created, renamed by the user. Directories are used to organize your data files, programs more efficiently. Device files located in the /dev/ directory relate to the system devices. This concept may be disconcerting for a new user for the first time The concept of mount points Files in a UNIX system are organized into a single tree structure. It is however possible to have several partitions using a mechanism called mounting which makes it possible to connect a partition to a directory in the main tree structure. So, the fact of mounting a - By Prof. P.M.Katkar. Page 11

12 partition in the directory /mnt/partitionmakes all the files in the partition accessible from this directory, called the "mount point" The file system of UNIX: All of the files in the UNIX file system are organized into a multileveled hierarchy called a directory tree. A family tree is an example of a hierarchical structure that represents how the UNIX file system is organized. The UNIX file system might also be envisioned as an inverted tree or the root system of plant. At the very top of the file system is single directory called "root" which is represented by a / (slash). All other files are "descendents" of root. The number of levels is largely arbitrary, although most UNIX systems share some organizational similarities. The "standard" UNIX file system is discussed later. To ensure compatibility and portability, UNIX systems comply with the FHS (File Hierarchy Standard) standard. The basic hierarchy of a Unix system is as follows: - By Prof. P.M.Katkar. Page 12

13 The tree of the file system starts at the trunk or slash, indicated by a forward slash (/). This directory, containing all underlying directories and files, is also called the root directory or "the root" of the file system. Directories that are only one level below the root directory are often preceded by a slash, to indicate their position and prevent confusion with other directories that could have the same name. When starting with a new system, it is always a good idea to take a look in the root directory. - By Prof. P.M.Katkar. Page 13

14 - By Prof. P.M.Katkar. Page 14

15 The extended file system, or ext, was implemented in April 1992 as the first file system created specifically for the Linux kernel. It has metadata structure inspired by the traditional Unix File System (UFS) and was designed by Rémy Card to overcome certain limitations of the MINIX file system. It was the first implementation that used the virtual file system (VFS), for which support was added in the Linux kernel in version 0.96c, and it could handle file systems up to 2 gigabytes (GB) in size. It is the first in the series of the extended file systems, superseded by both ext2 and xiafs, between which there was a competition, which ext2 won because of its long-term viability. ext2 remedied issues with ext, such as the immutability of inodes andfragmentation. There are other members in the extended file system family: ext2, the second extended file system. ext3, the third extended file system. ext4, the fourth extended file system. Inode As per definition, an inode is a data structure on a unix filesystem which stores information about filesystem objects (files, directories and so on). The type of information stored in an inode is: file type, permissions, owner, group, size, file access time, change and modification, number of links, acl s. The inode does not contain the name of the object, only the object s properties. An inode can be viewed with ls -i or stat command. You can also delete a file that contains special characters with the find -inum - exec rm -fr {} \;,where is the inode of the respective filesystem object. In the EXT2 file system, the inode is the basic building block; every file and directory in the file system is described by one and only one inode. The EXT2 inodes for each Block Group are kept in the inode table together with a bitmap that allows the system to keep track of allocated and unallocated inodes. - By Prof. P.M.Katkar. Page 15

16 the EXT2 Inode Figure : EXT2 Inode Figure shows the format of an EXT2 inode, amongst other information, it contains the following fields: Mode This holds two pieces of information; what this inode describes and the permissions that users have to it. For EXT2, an inode can describe one of file, directory, symbolic link, block device, character device or FIFO. Owner Information The user and group identifiers of the owners of this file or directory. This allows the file system to correctly allow the right sort of accesses, Size The size of the file in bytes, Timestamps The time that the inode was created and the last time that it was modified, Datablocks Pointers to the blocks that contain the data that this inode is describing. The first twelve are pointers to the physical - By Prof. P.M.Katkar. Page 16

17 blocks containing the data described by this inode and the last three pointers contain more and more levels of indirection. For example, the double indirect blocks pointer points at a block of pointers to blocks of pointers to data blocks. This means that files less than or equal to twelve data blocks in length are more quickly accessed than larger files. You should note that EXT2 inodes can describe special device files. These are not real files but handles that programs can use to access devices. All of the device files in /dev are there to allow programs to access Linux's devices. For example the mount program takes as an argument the device file that it wishes to mount. File Names UNIX permits file names to use most characters, but avoid spaces, tabs and characters that have a special meaning to the shell, such as: & ; ( )? \ ' " ` [ ] { } < > $ -! / Case Sensitivity: uppercase and lowercase are not the same! These are three different files: NOVEMBER November november Length: can be up to 256 characters Extensions: may be used to identify types of files libc.a - archive, library file program.c - C language source file alpha2.f - Fortran source file xwd2ps.o - Object/executable code mygames.z - Compressed file Hidden Files: have names that begin with a dot (.) For example:.cshrc.login.mailrc.mwmrc Uniqueness: as children in a family, no two files with the same parent directory can have the same name. Files located in separate directories can have identical names. - By Prof. P.M.Katkar. Page 17

18 Reserved Filenames: / - the root directory (slash). - current directory (period).. - parent directory (double period) ~ - your home directory (tilde) Pathnames Specify where a file is located in the hierarchically organized file system Must know how to use pathnames to navigate the UNIX file system Absolute Pathname: tells how to reach a file begining from the root; always begins with / (slash). For example: /usr/local/doc/training/sample.f Relative Pathname: tells how to reach a file from the directory you are currently in ( current or working directory); never begins with / (slash). For example: training/sample.f../bin ~/projects/report.001 For example, if your current directory is /usr/home/johnson and you wanted to change to the directory /usr/home/quattro, you could use either of these commands: cd../user1 cd /usr/home/user1 What is command? - relative pathname - absolute pathname A command is an instruction given by a user telling a computer to do something, such a run a single program or a group of linked programs. Commands are generally issued by typing them in at the command line (i.e., the all-text display mode) and then pressing the ENTER key, which passes them to the shell. Commands on Unix-like operating systems are either builtins or external commands. The former are part of the shell. The latter consist of both executables, which are programs that have been written in a programming language (e.g., C, C++, Java or Python) and then compiled into a binary, and shell scripts. - By Prof. P.M.Katkar. Page 18

19 The general syntax for commands is command [options] [arguments] An option, also sometimes referred to as a switch or a flag, is a single-letter code, or sometimes a single word or set of words, that modifies the behavior of a command in some predetermined way. When multiple single-letter options are used, all the letters are placed adjacent to each other (i.e., not separated by spaces) and can be in any order. The set of options must usually be preceded by a single hyphen, again with no intervening space. An argument, also called a command line argument, is a file name or other data that is provided to a command in order for the command to use it as an input. A command consists of a command name usually followed by one or more strings(i.e., sequences of characters) that comprise options and arguments. Each of these strings is separated by white space (which consists of one or more spaces or tabs). A command-line interface (CLI) is a means of interacting with a computer program where the user (or client) issues commands to the program in the form of successive lines of text (command lines). The CLI was the primary means of interaction with most popular operating systems in the 1970s and 1980s, including MS- DOS, CP/M,Unix, and Apple DOS. The interface is usually implemented with a command line shell, which is a program that accepts commands as text input and converts commands to appropriate operating system functions. - By Prof. P.M.Katkar. Page 19