2009 S2 COMP File Operations

Transcription

1 2009 S2 COMP File Operations Oliver Diessel Last updated: 16:00 22 Sep 2009

2 9. File Operations Topics to be covered: Streams Text file operations Binary file operations Interleaving reading & writing Thanks: Alistair Moffat COMP1921_09s2 Week 9 9. File Operations 2

3 File Operations All input and output (I/O) so far by: Keyboard and screen Unix-level file redirection C also offers text files and binary files as I/O mechanisms Differ in type of data stored But are handled in a similar manner to each other COMP1921_09s2 Week 9 9. File Operations 3

4 Streams C always automatically opens 3 files, or streams, when a program starts running stdin is the standard input file, usually associated with the keyboard, but can take input from a file using Unix input redirection; functions scanf and getchar take their input from stdin stdout standard output file; associated with the terminal screen; can write to a file using Unix output redirection; printf and putchar send their output to stdout stderr standard error output file; usually associated with the terminal screen; can be redirected to a file using Unix redirection These streams are automatically closed when the program exits COMP1921_09s2 Week 9 9. File Operations 4

5 Implementing error messages To write to stderr, use a variant of printf with a filename argument: fprintf(stderr, error: null pointer\n ); See example of use: simple.c The function printf is actually implemented as a call to fprintf with stdout specified as the first argument In C all error messages and non-standard outputs are written to stderr, where the user may see them even if stdout is being directed to a file. COMP1921_09s2 Week 9 9. File Operations 5

6 File redirection % man bash (or your favourite Unix shell) around line 1180 describes the file redirection options %./simple sends stdout and sderr to the terminal screen %./simple >simple.out sends stdout to simple.out and stderr to the screen %./simple &>simple.out sends both to simple.out %./simple 2>simple.err sends stderr to simple.err and stdout to the screen %./simple >simple.out 2>simple.err sends stdout to simple.out and stderr to simple.err COMP1921_09s2 Week 9 9. File Operations 6

7 Flushing a stream Output is normally buffered by low-level (screen and disk) device drivers before being output. The buffers are normally flushed when they are full or when the program terminates. Due to this buffering, output to different streams can appear to be out of sequence The programmer can force a write using the fflush command fflush(file *stream); will force the buffer for stream to be flushed when it is executed Example: observe the difference in behaviour of %./simple &>simple.out with and without the fflush(stdout); statement following the printf statement COMP1921_09s2 Week 9 9. File Operations 7

8 Text files The stdio library contains a large number of other functions for manipulating streams (see Table 11.1, page 194). In a text file, information is stored as a sequence of ASCII printable characters The stream I/O functions that relate to text files include: FILE *fopen(char *, char *) Returns a pointer to the file (stream) named by the first argument that has been opened for the operation given by the second argument: r for reading, w for writing and a for appending int fclose(file*) Flushes pending output to file and closes it. In this course include an fclose for every fopen int getc(file*) Reads a single character, as for getchar; returns EOF if end of file detected int putc(int, FILE*) Outputs a single character, as for putchar; returns EOF if op fails int fscanf(file*, const char*, ) Input from specified stream, as for scanf; returns number of values read int fprintf(file*, const char*, ) Output to specified stream, as for printf; returns the number of bytes written COMP1921_09s2 Week 9 9. File Operations 8

9 Example writing a text file #include <stdio.h> // program textout.c int main(int argc, char **argv) { FILE *fp; if (argc < 2) { fprintf(stderr, "Not enough arguments\n"); else { fprintf(stderr, "Opening %s", argv[1]); if ((fp = fopen(argv[1], "w")) == NULL) { fprintf(stderr, "...failed\n"); else { fprintf(stderr, "\n"); fprintf(fp, "One output line\n"); fclose(fp); return 0; COMP1921_09s2 Week 9 9. File Operations 9

10 Example reading a text file #include <stdio.h> // program textin.c int main(int argc, char **argv) { FILE *fp; char c; if (argc < 2) { fprintf(stderr, "Not enough arguments\n"); else { fprintf(stderr, "Opening %s", argv[1]); if ((fp = fopen(argv[1], "r")) == NULL) { fprintf(stderr, "...failed\n"); else { fprintf(stderr, "\n"); while ((c = getc(fp))!= EOF) { putc(c, stdout); fclose(fp); return 0; COMP1921_09s2 Week 9 9. File Operations 10

11 Notes on fopen The term stream is use to refer to a sequence of bytes moving to or from a file A FILE* variable is created to refer to a particular stream and to keep track of its state The name of the file to open can be in the current directory or specify a full pathname such as /usr/share/dict/words If a file is to be read and it doesn t exist when opened, an error occurs A file opened for writing is created if it doesn t exist, or is truncated to a file size of 0 bytes if it already does exist When a file is opened for appending, output operations commence at the end of an existing file, or at the start if the file did not previously exist COMP1921_09s2 Week 9 9. File Operations 11

12 Binary files The string to number and number to string conversions performed by scanf and printf involve non-trivial computations (consume significant time and energy). They also require relatively expensive storage compared to internal representations. For example, an int variable requires 4 bytes of storage, but when printed as a text string might need 11 bytes. C also supports direct input and output of byte sequences exactly as they would be stored in memory while the program is executing: int fwrite(const void*, size_t, size_t, FILE*) Writes binary data to the file starting at the memory address indicated by the first argument, writing objects of size given by the second argument, and the number of given by the third argument; returns the number of objects written int fread(const void*, size_t, size_t, FILE*) As above with read* replacing writ* The resulting binary files cannot be processed in a text editor, or printed. They are also architecture-dependent, so data portability is a real issue. But they offer the ability of efficiently reading and writing whole chunks of data, including structures and arrays. COMP1921_09s2 Week 9 9. File Operations 12

13 Example See fread.c program by Moffat Note that 7 double values, or a total of 7*8 = 56 bytes are written to the file These are stored as a sequence of bytes with no suggestion of how they should be interpreted Could be read as 56 chars Or as 14 variables of type int Or as any mixture of char, int & double variables If the data is to be interpreted sensibly, it must be read using exactly the same type or sequence of types used to create it COMP1921_09s2 Week 9 9. File Operations 13

14 od dump files in octal & other formats % man od or % od --help Useful for viewing the contents of binary files in arbitrary formats % od t fd temp.dat to view the output of fread.c as a sequence of doubles Try some of the other options! Note: the leftmost column of the output reports in octal the offset from the start of the file of the leftmost byte on that line of output Note: the user can specify the number of bytes od outputs per line COMP1921_09s2 Week 9 9. File Operations 14

15 Interleaving writing and reading The function fseek allows the current location in an open file to be moved: int fseek(file*, long, int) Moves the current location within the file by the byte amount indicated by the second argument to a position relative to the starting point specified by the third argument (symbolic values defined in stdio): SEEK_SET (the start of the file), SEEK_CUR (the current position) and SEEK_END (the current EOF); returns 0 if successful and -1 otherwise. A file that is opened with the addition of a "+" modifier to the file opening access mode string (i.e., r+, w+, or a+ ) can be both read and written in interleaved operations as long as an fseek is done each time prior to switching between access modes (reading and writing, or vice-versa) This allows the file to be used like an on-disk array However, performance will be dramatically worse than for an in-memory array, especially if the access pattern is non-sequential. See program fseek.c COMP1921_09s2 Week 9 9. File Operations 15

16 Example #include <stdio.h> // program textseek.c #include <string.h> int main(int argc, char **argv) { FILE *fp; char w1[10] = {0; char w2[10] = {0; char *s1 = "One output line\n"; if (argc < 2) { fprintf(stderr, "Not enough arguments\n"); else { fprintf(stderr, "Opening %s", argv[1]); if ((fp = fopen(argv[1], "w+")) == NULL) { fprintf(stderr, "...failed\n"); else { fprintf(stderr, "\n"); fprintf(fp, "%s", s1); fseek(fp, 0L, SEEK_SET); fread(w1, 1, 3, fp); fseek(fp, 0L, SEEK_SET); fwrite(".1.", 1, 3, fp); fseek(fp, 1L, SEEK_CUR); fscanf(fp, "%s", w2); fseek(fp, -1L, SEEK_END); fprintf(fp, " %s %s\n", w1, w2); fclose(fp); return 0; COMP1921_09s2 Week 9 9. File Operations 16

17 Case study Write a program that reads a set of text files, with the lines in each file presumed to be in sorted order; and writes a single sorted file to the standard output. Lines may be assumed to be at most 1024 characters long. As many as ten input files must be allowed for, specified as command-line arguments. Figure 11.5 on page 201 gives a solution (file mergefiles.c). Points to note: Use of an array of FILE* pointers Use of the function fgets to read a complete line of input into a character array (At most) one line of each file is held in memory at any time COMP1921_09s2 Week 9 9. File Operations 17

18 Conclusion We ve looked at: Use of stdin, stdout, stderr streams & redirection Writing and reading ASCII textfiles Writing and reading binary files; advantages thereof Updating arbitrary bytes in ASCII and binary files Next steps: Files & File Operations (Moffat, Ch. 11) Wk 10: Tuesday Aaron Carroll on Bit Level Operations which are highly relevant to Assignment 2 Wk 10: Thursday Simonne Mautner on gdb & Joseph Gentle on refactoring, an approach to efficient functional structuring of your programs using Assignment 1 as an example COMP1921_09s2 Week 9 9. File Operations 18