Part 1. Introduction to File Organization

Transcription

1 Part 1 Introduction to File Organization Data processing from a computer science perspective: Storage of data Organization of data Access to data This will be built on your knowledge of Data Structures 1. File structure A combination of: Representation for data in files Operations for accessing the data Allow applications to read, write, and modify data Support: Searching about data in files using search criteria. Reading the data in some particular order. 2. Data Structures vs. File Structures Both involve: Representation of Data + Operations for accessing data Difference: Data Structures deal with data in main memory File Structures deal with data in secondary storage device (File). 3. Where do File Structures fit in CS?

2 4. Goal of file structure Minimize number of trips to the disk in order to get desired information. ( get what we need in one or few disk access ) Grouping related information so that we are likely to get everything we need with only one trip to the disk (e.g. name, address, phone number, account balance). 5. Good File Structure Design Fast access to great capacity Reduce the number of disk accesses By collecting data into buffers, blocks or buckets Manage growth by splitting these collections 6. Basic File system functions The allocation of all space on the secondary devices The control of the devices themselves. The processing of all the I/O terminations to ensure that the correct processing is performed. It must then initiate the next I/O request targeted for that device. Notify the user by returning a status field, which indicates how the I/O request terminated. 7. What is a FILE? A file is: A collection of data is placed under permanent or non-volatile storage Examples: anything that you can store in a disk, hard drive, tape, optical media, and any other medium which doesn t lose the information when the power is turned off. 8. Physical Files & Logical Files Physical file: Physically exists on secondary storage; known by the operating system; appears in its file directory. Logical file: What your program actually uses, a pipe though which information can be extracted, or sent.

3 Operating system: get instruction from program or command line; link logical file with physical file or device Why is the distinction useful? Why not allow our programs to deal directly with physical files? 9. Computer Architecture Computer Architecture 10. Memory Hierarchy Systems with 32bit can address only 2 32 bytes in main memory. The number of data objects may exceed this number! Data must be maintained across program executions. This requires storage devices that save information when the computer is restarted. We call such storage nonvolatile. Primary storage is usually volatile, whereas secondary and tertiary storage are nonvolatile. Typical times for getting info

4 Main memory: ~120 nanoseconds Magnetic Disks: ~30 milliseconds An analogy keeping same time proportion as above Looking at the index of a book: 20 seconds versus Going to the library: 58 days 11. Comparison between Main Memory Fast (since electronic) Small (since expensive) Volatile (information is lost when power failure occurs) Secondary Storage Slow (since electronic and mechanical) Large (since cheap) Stable, persistent (information is preserved longer) 12. History of File Structure Design In the beginning it was the tape Sequential access Access cost proportional to size of file Analogy to sequential access to array data structure Disks became more common Direct access: Analogy to access to position in array Indexes were invented: list of keys and points stored in small file allows direct access to a large primary file Great if index fits into main memory. As file grows we have the same problem we had with a large primary file Tree structures emerged for main memory (1960`s) Binary search trees (BST`s) Balanced, self-adjusting BST`s: e.g. AVL trees (1963) A tree structure suitable for files was invented: B trees (1979) and B+ trees Good for accessing millions of records with 3 or 4 disk accesses. What about getting info with a single request? Hashing Tables (Theory developed over 60 s and 70 s but still a research topic) Good when files do not change too much in time.

5 Expandable, dynamic hashing (late 70 s and 80 s) one or two disk accesses even if file grows dramatically Basic File Operation Most programs will perform the following kinds of functions: File Operations CREATE new files OPEN, or start using, a file that already created. EXTEND: Increase the size of the file. Close: Stop using the file after the program has finished using it. DELETE a file that already exists. Block Data Transfer operation: READ: transfer data from a device. WRITE: transfer data to a device. 1. Create File a. File Operation Allow all users to create new files whenever it is necessary Tasks: Device name /user name: tells where to place the file. Filename: identifies the file among all the other files. File size: tell how much room to allocate when creating the file File organization : access methods, can know how to insert and later retrieve the blocks within the file Block size amount of data that will be read or written in every operation. Validate the User s Input Data Check to see if the file already exists Check if there was an enough space exists. And Allocate space on the device for the file: Insert a pointer to the first block of the file Allocate and initialize I/O buffers and control blocks Build a file control block, to save critical information about the. Return success or fail status back to the user.

6 Errors Another file with the same name already exists. The device has no available. The device is corrupted, or has become, nonoperational. Wrong Input Parameter. 2. Open File OPEN request is allow users to locate and access files that were created at an earlier time. Tasks: Errors: Device name, User name, or account identification Filename Allocate space for Internal Control Blocks (ICB) Get the correct volumes mounted on the devices Locate the file on the devices Allocate internal memory space for I/O buffers Protect against unauthorized users Maintain a list of all the open file in the system File not found on the specified Device. File already in use by some other user application. Hardware error has occurred. How to open file in c File *outfile; Outfile=fopen( account.txt, W ); The First argument indicates the physical name of the file. The second argument indicates the mode of operation Mode of Operation: R : Open an existing file for reading W: Open an existing file for writing A: Create a new file or append to an existing file. R+: Open an existing file for reading and writing A+ create a new file or append an existing one for reading and writing.

7 3. Extend File Allow the user(s) to explicitly increase the size of a file. Tasks: Errors: 4. Close File Device and directory name where the file is located Name of the file Amount by which the file should be increased in size. Validate the parameters in this request Check to see if there is enough room on this device to increase. If space exists, Add the space to the file in question and mark in such a way that no other user can use. Return a success or failure status code to the user. Device has no space available in which to extend the file File was not previously opened. Allow the user to decrease, stop using the file and allow other users to access the file. Tasks Errors 5. Delete File The name of the file to be closed. Write out any I/O buffers that have been modified in main memory. Release any internal memory space dedicated to the file. Return status to the user, indicating success or failure. The file to be closed was not currently open Any of the errors that can occur on a WRITE operation. A hardware problem occurs.

8 Erase a specific file from the system when the user no longer needs it. Tasks Errors: Device and user account name File name Validate the user s input parameters Return all the space on the media taken up by the file. Remove the file name from the directory where it resides. Return status to the user. File could not be found (an invalid file or user account name). Other users were currently active on the file A hardware problem was detected. b. Block Operation A block of data is the unit of data that is transferred to, or from, a device. The size of the block is definable by the user. The larger the block, the more data that will be transferred in that I/O operation and the greater the performance of the program. 1. Write Request Provide users with the capability of transferring data out of the computer s main memory and onto one of the devices connected to the system. Writing in C: char c ; // a character char a[100] ; // an array with 100 characters FILE * outfile ; outfile = fopen( myfile.txt, w ) ; fwrite(&c,1,1,outfile) ; // writes one character fwrite(a,1,10,outfile) ; // writes 10 characters Writing in C++: char c ; // a character char a[100] ; // an array with 100 characters

9 fstream outfile ; outfile.open( myfile.txt, ios::out) ; outfile << c; // writes one character outfile.write(&c,1) ; outfile.write(a,10); // writes 10 bytes Tasks Errors: Number of the block to be written Address of the I/O buffer in many memory that contains the block to be written Size or amount of the block to be written Name of the status field into which the file system will insert the termination status of the I/O request. Validate the input parameters Convert the logical block number to a physical device address Check if space exists in the file for this block If space does not exist in the file, extend the file. This is called an implicit extend to the File could not be found (an invalid file or user account name). Other users were currently active on the file A hardware problem was detected. 2. Read Request The function of the READ routine is to transfer a specific block of data from the medium into main memory. The unit of data transfer is the block. Example fread(&c,1,1,infile) ; // reads one character fread(a,1,10,infile) ; // reads 10 characters 1st argument: destination address 2nd argument: element size in bytes 3rd argument: number of elements 4th argument: logical file name

10 Tasks Errors: Number of the block to be read into memory Address of the I/O block buffer in main memory Size or amount of data to be read into main memory The address of a field into which the file system can store the status of the I/O request Validate the input parameters Validate input parameters Convert the logical block number to a physical block Read the block into an I/O buffer in main memory Return status to the user program File could not be found (an invalid file or user account name). Read error detected by the device Invalid device block number requested Hardware failure detected c. Calculating file size int main(int argc, char* argv[]) { FILE *hfile=fopen(argv[1],"r"); fseek(hfile, 0L, SEEK_END); int filelength = ftell(hfile); printf( \nfile size is %d,filelength) ; fclose(hfile) ; Return 0; }