Memory Management and

Transcription

1 Introduction to Computer and Program Design Lesson 5 Memory Management and File I/O James C.C. Cheng Department of Computer Science National Chiao Tung University

2 Dynamic memory allocation l malloc Syntax: #include <stdlib.h>!!!!void* malloc( size_t n );! size_t è unsigned integer Ø scanf("iu", &n); printf("%iu", n); // for Windows! Ø scanf("zu", &n); printf("%zu", n); // for UNIX! malloc() returns a pointer to a chunk of memory of n bytes, or NULL if there is an error. The memory pointed to will be on the heap, not the stack, so make sure to free it when you are done with it. The returned pointer must be typecast 2

3 Dynamic memory allocation l free Syntax: #include <stdlib.h>!!!!void free( void* ptr );! free() deallocates the space pointed to by ptr, freeing it up for future use. ptr must be NULL or used in a previous call to malloc(), otherwise a runtime error will occur on free(). 3

4 Dynamic memory allocation l Example: // 10 characters:! char *pc = (char *)malloc( sizeof(char) * 10);! // 10 integers:! int *pi = (int *)malloc( sizeof(int) * 10);! // 10 doubles:! double *pd = (double *)malloc( sizeof(double) * 10);!! if( pc!= NULL && pi!= NULL && pd!= NULL){! for(int i=0; i<10; ++i)! pd[i] = pi[i] = pc[i] = i + 65;! for(int i=0; i<10; ++i)! printf("%c, %d, %f\n", pc[i], pi[i], pd[i]);! }!! // Deallocation! free(pc);! free(pi);! free(pd);! 4

5 Dynamic memory allocation l free a NULL pointer int *p; free(p); p = NULL;! free(p);!// Non-NULL!!// Runtime error!!!// OK! 5

6 Dynamic memory allocation l calloc Syntax:!#include <stdlib.h>!!!!void* calloc (size_t n, size_t size );! calloc() returns a pointer to a chunk of memory of n*size bytes with elements initialized t zero, or NULL if there is an error. The returned pointer must be typecast 6

7 Dynamic memory allocation int n = 1024 * 1024 * 128 * sizeof(char); // 128M chars! double rtime;! clock_t clk0, clk1;!! // The time of malloc and initialization is proportional to n! clk0 = clock();! char *pc = (char *)malloc(n);! for(int i=0; i<n; ++i) pc[i] = 0;! clk1 = clock();! rtime = (double)(clk1 - clk0) / (double)clocks_per_sec;! printf("iterative clearing time: %f\n", rtime);!! free(pc);!! // The time of calloc is constant if hardware supports calloc! clk0 = clock();! pc = (char *)calloc(n, sizeof(char));! clk1 = clock();! rtime = (double)(clk1 - clk0) / (double)clocks_per_sec;! printf("calloc time: %f\n", rtime);!! free(pc);! 7

8 Dynamic memory allocation l realloc Syntax:!#include <stdlib.h>!!!!void * realloc( void * ptr, size_t n);! ptr: Pointer to a memory block previously allocated with malloc, calloc or realloc, or a null pointer (to allocate a new block). n: New size for the memory block, in bytes. Ø If n is zero, the return value depends on the particular library implementation, but the returned pointer shall not be used. A pointer to the reallocated memory block, which may be either the same as ptr or a new location. The returned pointer must be typecast If the function failed to allocate the requested block of memory, a null pointer is returned, and the memory block pointed to by argument ptr is not deallocated (it is still valid, and with its contents unchanged). 8

9 Dynamic memory allocation int i = 0, n = 0, x = 0, *p = NULL;! while(scanf("%d", &x)!= EOF){! ++n;! p = (int *)realloc(p, n * sizeof(int));! p[n-1] = x;! for(i=0; i<n; ++i)! printf("%d, ", p[i]);! printf("\n");!! }! free(p);! 9

10 Dynamic memory allocation l Memory leaks and dangling Normal allocation Pointer Memory Memory leaks Memory dangling Pointer Memory Pointer Memory 10

11 Dynamic memory allocation l Memory Leaks There is no any pointer to point an allocated memory space Take look the following example: Ø Executing the system monitor to watch the memory usage Ø Using two threads to execute the following program Ø If the memory usage approach to the limitation, just stop the program char! key = 0, end = 0;! do{ // Create 64M byte for input buffer! int n = 1024 * 1024 * 64;! char *pc =(char *)malloc(n);! memset(pc, 0, n); // Clear all data! scanf("%s", pc);! key = pc[0]; end = pc[1];! }while(key!= 'q' && key!='q' end!= 0);! If the OS does not provide Garbage Collection, the allocated memory will never be released. Notice that the timing for releasing by garbage collection is when the program has been terminated. 11

12 Dynamic memory allocation l Memory Dangling A pointer points an unallocated memory space char *pc;! *pc = 100; // Dangling! pc = (char *)malloc(12);! int *pi = (int *)pc;! free(pi);! *pc = 50; // Dangling!! pc pi Memory free 12

13 Dynamic memory allocation l Memory Manipulation Functions memset, memory setting: #include <memory.h> or #include <string.h>! void* memset( void *dest, int c, size_t count );! Ø dest: the destination pointer Ø c: set the value, c & 0xFF, to each byte of dest Ø count: the number of byte to set Ø returns the value of dest int n = 3, *p = (int *)malloc(sizeof(int)*n);! memset(p, 0, sizeof(int)*n);! for(int i=0; i<n; ++i) printf("%d, 0x%0X\n", p[i], p[i]);! memset(p, 255, sizeof(int)*n);! for(int i=0; i<n; ++i) printf("%d, 0x%0X\n", p[i], p[i]);! memset(p, 65537, sizeof(int)*n);! for(int i=0; i<n; ++i) printf("%d, 0x%0X\n", p[i], p[i]);! memset((int *)memset(p, 0, 12) + 1, 255, 4) ;! for(int i=0; i<n; ++i) printf("%d, 0x%0X\n", p[i], p[i]);! free(p); 13

14 Dynamic memory allocation l Memory Manipulation Functions memcpy, memory copy: #include <memory.h> or #include <string.h>! void* memcpy(void *dest, const void *src, size_t count );! dest: the destination pointer src: the source pointer count: the number of byte to copy returns the value of dest If the source and destination overlap, this function does not ensure that the original source bytes in the overlapping region are copied before being overwritten. 14

15 Dynamic memory allocation l Memory Manipulation Functions memcpy, memory copy: char s1[] = "Hello! My friend!"; // Why do we use char[]?! const char *s2 = "Hi! Guys."; // Why do we use const char*?! memcpy(s1+7, s2+4, 6);! printf("%s\n", s1);!!// Hello! Guys.!! memcpy(s1+5, s1+7, 4);! printf("%s\n", s1); // Maybe HelloGuysys., maybe not 15

16 Dynamic memory allocation l Memory Manipulation Functions memmove, memory move: #include <string.h>! void *memmove(void *dest, const void *src, size_t count);! dest: the destination pointer src: the source pointer count: the number of byte to move returns the value of dest It is similar to memcpy, but memmove ensures the copy of overlapping region. char s1[] = " ";! memcpy(s1 + 2, s1, 5);! printf("%s\n", s1);!! // ! 16

17 Dynamic memory allocation l Memory Manipulation Functions memcmp, memory compare: #include <memory.h> or #include <string.h>! int memcmp(const void *p1, const void *p2, size_t n );! Ø p1 and p2: the pointers of memory Ø n: the number of byte to compare Ø Return Value: relationship of first n bytes of buf1 and buf2 Ø return (p1[i] p2[i]), if (p1[i] p2[i]!= 0 or i == n-1), i = 0, 1,, n-1 < 0: p1 less than p2 0: p1 identical to p2 > 0: p1 greater than p2 char first[] = " ";! char second[] = " ";! printf("%d\n", memcmp( first, second, 19 ) ); // 0! printf("%d\n", memcmp( first, second, 20 ) ); // <0! printf("%d\n", memcmp( first + 2, second, 18 ) ); // >0! 17

18 Dynamic memory allocation l Case study: recording the weights and heights int main(void)! {! const size_t n = 5;! float A[n][2] = {0};! int i=0;! for(i=0; i<n; ++i){ // Data input.! printf("w H: ");! scanf("%f %f", &A[i][0], &A[i][1]);! }! qsort(a, n, sizeof(float) * 2, cmpw );!// By weight! for(i=0; i<n; ++i) // Data input.! printf("(%f, %f) ", A[i][0], A[i][1]);! printf("\n");! qsort(a, n, sizeof(float) * 2, cmph ); // By height! for(i=0; i<n; ++i) // Data input.! printf("(%f, %f) ", A[i][0], A[i][1]);! printf("\n");! }!! 18

19 Dynamic memory allocation l Case study: recording the weights and heights int cmpw( const void* p1, const void*p2){! float *pf1 = (float*)p1;! float *pf2 = (float*)p2;! return pf1[0] - pf2[0];! }!! int cmph( const void* p1, const void*p2){! float *pf1 = (float*)p1;! float *pf2 = (float*)p2;! return pf1[1] - pf2[1];! }!! 19

20 Dynamic memory allocation l Case study: recording the weights and heights (dynamic memory alloc) size_t n = 0;! printf("n="); scanf("%iu", &n); getchar(); // n=?! // Memory allocation, watch the array pointer declaration! float (*A)[2] = (float (*)[2])calloc(n, sizeof(float)*2);! int i=0;! for(i=0; i<n; ++i){! printf("w H: ");! scanf("%f %f", &A[i][0], &A[i][1]);! }! qsort(a, n, sizeof(float) * 2, cmpw ); // By weight! for(i=0; i<n; ++i)! printf("(%f, %f) ", A[i][0], A[i][1]); // By height! printf("\n");! qsort(a, n, sizeof(float) * 2, cmph );! for(i=0; i<n; ++i)! printf("(%f, %f) ", A[i][0], A[i][1]);! printf("\n");! free(a);!! 20

21 Struct l struct To design a custom data type To group multiple relevant data Heterogeneous array struct Student{! char name[8];! float w, h;! };! Student x = {0};! printf("name:%s, w:%f, h:%f\n", x.name, x.w, x.h);!! Student y = {"James", 75.2f, 175.6f};! printf("name:%s, w:%f, h:%f\n", y.name, y.w, y.h);! Student z = y;! printf("name:%s, w:%f, h:%f\n", y.name, y.w, y.h);! scanf("%s %f %f", z.name, &z.w, &z.h);! getchar();! printf("name:%s, w:%f, h:%f\n", z.name, z.w, z.h); 21

22 Struct l member selection operator x.m x is the object name m is the member name p->m p is a pointer points an object m is the member name member = field Student x;!!! x.w = 65.0f; x.h = 178.5f;!!! Student * px = &x;! px->w = 75.4f, px->h = 190.0f; 22

23 Struct l Passing a struct object to a function by its address DO NOT by value! void InputStudent(Student* px){! scanf("%s %f %f", px->name, &px->w, &px->h);! }!! void PrintStudent (const struct Student* px){! printf("name:%s, w:%f, h:%f\n", px->name, px->w, px->h); }! 23

24 Dynamic memory allocation l Case study: recording the weights and heights (struct) int cmpw( const void* p1, const void*p2){ /*??? */ }! int cmph( const void* p1, const void*p2) {/*??? */}! int main(void){! size_t n = 0, i = 0;! printf("n="); scanf("%iu", &n); getchar(); // n=?! // Memory allocation, watch the array pointer declaration! Student *A = //???! // ---??? ---!! // ! printf("sorted by weight:\n");! // qsort(???);! for(i=0; i<n; ++i)! // PrintStudent??;! printf("sorted by hight:\n");! // qsort(???);! for(i=0; i<n; ++i)! // PrintStudent??;! }!! 24

25 Linked Lists l Linked List It consists of a sequence of data items such that in each item there is a pointer or a reference to link the next item. The memory blocks of elements may not be adjacent

26 Linked Lists l Linked List struct Node{! int data;! Node *next;! };! data next void ShowNode(const Node* p){! printf("%d\n", p->data);! }! // Default argument function Only in C++! // The defaults of data and pnext are 0 and NULL respectively.! Node* NewNode(int data = 0, Node *pnext = NULL){! Node x = {data, pnext};! Node *p = (Node *)malloc(sizeof(node));! *p = x;! return p;! }! 26

27 Linked Lists l Linked List // Creation! Node *phead = NULL;! for( int i=5; i>=0; --i)! phead = NewNode(i, phead);!! // Output! Node *p = phead;! while(p!= NULL) {! ShowNode(p);! p = p->next;! }! // deletion all! p = phead;! while(p!= NULL) {! Node *ptmp = p;! }! p = p->next;! free(ptmp); phead p !// Release each item! 27

28 Linked Lists l Linked List // Output! void ShowList(const Node *p){! while(p!= NULL) {! ShowNode(p);! p = p->next;! }! }!! // deletion all! Node* DelList(Node *p){! while(p!= NULL) {! Node *ptmp = p;! p = p->next;! free(ptmp);!! }! return NULL;! }! // Creation! Node *phead = NULL;! for( int i=5; i>=0; --i)! phead = NewNode(i, phead);!! ShowList(pHead);!! phead = DelList(pHead);! 28

29 Linked Lists l Linked List // Front Insertion! Node * PushFront( Node* p0, Node *pnew){! if(p0!= NULL)! pnew->next = p0;! return pnew;! }!!! int main(){! Node *phead = NULL;! }! phead pnew for( int i=5; i>=0; --i)! phead = FrontIns( phead, NewNode(i, NULL));! ShowList(pHead);! phead = DelList(pHead);! p0 29

30 Linked Lists l Linked List // Front erasing! Node * PopFront( Node* phead){! if(phead!= NULL){! Node *ptmp = phead;! phead = phead->next;! free(ptmp);! }! return phead;! phead }! ptmp! // deletion all! Node* DelList(Node *p){! while( (p = PopFront(p))!= NULL );! return p;! }! 30

31 Linked Lists l Exercises Back insertion Node * PushBack( Node* phead, Node *pnew);! phead: The pointer points the first node of list pnew: The pointer points the new node; The return is the header of list Back erasing Node * PopBack( Node* phead);! phead: The pointer points the first node of list The return is the header of list. If the list is empty, the return is NULL. 31

32 Linked Lists l Exercises Search Node * Search( Node* phead, int key);! phead: The pointer points the first node of list key: The search key The return is the pointer of the searched node if the node exists in the list. Otherwise the return is NULL remove the specified node Node * remove( Node* phead, int key);! phead: The pointer points the first node of list key: The target key The return is the header of list. If the list is empty, the return is NULL. 32

33 File System in OS l Microsoft Windows Filename DriveID : /DirctoryName/MainFileName.ExtensionName EX: Ø C:/test.txt Ø D:/MyDoc/Reports/ ppt Directories or Folders Storage Devices Files Files Directories or Folders Files 33

34 File System in OS l Microsoft Windows Filename DriveID Ø A and B are used for floppy disk drivers Ø C is the master boot device DirctoryName MainFileName ExtensionName = type name 34

35 File System in OS l Microsoft Windows Path Absolute path = full path, Relative path Ø a path relative to the working directory Ø EX: if the the working directory is D:/ICP/2011 p ABC.h è D:/ICP/2011/ABC.h p /0515/test.dat è D:/ICP/2011/0515/test.dat p../xyz.txt è D:/ICP/xyz.txt (.. represents the parent diretory) p../../t_t.b è D:/T_T.b 35

36 File System in OS l Microsoft Windows FAT32 (File Allocation Table 32-bit) Max. file size: 4 GB Max. length of MainFileName: 255 (in Windows), 8+3(DOS) Protection: Read-Only, Hidden and System exfat (Extended File Allocation Table) Max. file size: 64 ZebiB = 2 70 bytes Max. length of MainFileName: 255 Protection: Read-Only, Hidden and System NTFS (New Technology File System) Max. file size: 16 TB Max. length of MainFileName: 255 UTF-16 code units Protection: Read-Only, Hidden, System and Windows Access Control List(ACL) 36

37 File System in OS l UNIX family UNIX, FreeBSD, Linux, MAC OS X Tree structure file system Each node is either file or directory root etc usr volumes DVD USB-Disk data1 data2 37

38 File I/O in C l I/O redirection Redirect the standard I/O to file I/O < input redirection > output redirection EX: MyProg.exe < input.txt > ouput.txt #include <stdio.h>! void main(){! int x = 0, y = 0;! do{! scanf("%d %d", &x, &y);! printf("x = %d, y = %d, x * y = %d\n", x, y, x*y);! }while( x>=0);! }! 38

39 File I/O in C l Stream I/O 1. Open file FILE * file;! file = fopen( "C:/MyDoc/data.txt", "r" );! // Do not use the back-slash in filename, why?! // file = fopen( "C:\MyDoc\data.txt", "r" );! 2. Input (read) 3. Output (write) 4. Close file int x[10];! fread( &x, sizeof( int ), 10, file );! char *s = "Hello world";! fwrite( s, 1, strlen(s), file );! fclose(file);! 39

40 File I/O in C l Stream I/O Open file FILE *fopen( const char *filename, const char *mode );! mode can be: Ø "r Opens for reading. The file must be existed. Ø "w Opens an empty file for writing. If the given file exists, its contents are destroyed. Ø "a Opens for appending; creates the file first if it doesn t exist. n Writing always at the end of the file (appending) Ø "r+ = r with writing. (The file must exist.) Ø "w+ = w with reading Ø "a+ = a with reading. 40

41 File I/O in C l Stream I/O Open file FILE *fopen( const char *filename, const char *mode );! mode with translation type : Ø mode + t : Open in text mode. (In MS Windows) n It inserts \r (0x0D) before \n (0x0A) in write-out data. n It discard \r (0x0D) before \n (0x0A) in read-in data. n EX: "rt", "wt", "at", "r+t", "w+t", "a+t" Ø mode + b : Open in binary mode. n EX: "rb", "wb", "ab", "r+b", "w+b", "a+b Ø t + mode and b + mode are wrong usages and fopen returns NULL n EX: bw, ta, Ø The default translation type depends on the OS. n You should explicitly specify the translation type at any time. 41

42 l File I/O in C Stream I/O fgec/fputc!int fgetc(file *);!! int fputc(int, FILE*);! è returns the character read/written if OK; Otherwise EOF fgets/fputs char * fgetc(char *, FILE *);! è returns the result string if OK; Otherwise NULL int fputc(const char *, FILE*);! è returns non-negative value if OK; Otherwise 42

43 l File I/O in C Stream I/O fread/fwrite size_t fread(void *buffer, size_t size,! size_t count, FILE *stream);!!size_t fwrite(const void *buffer, size_t size,! size_t count, FILE *stream);! buffer : Storage location for data size: Item size in bytes count: Maximum number of items read/written stream: Pointer to the FILE structure è returns the number of full items actually read/written 43

44 File I/O in C l Stream I/O #include <stdio.h>! #include <string.h>! int main(){! FILE *pf = fopen("a.txt", "wt"); // try "wb"! if(pf!= NULL){! char s[] = "Hello\nworld!\n";! fwrite(s, 1, strlen(s), pf);! }! fclose(pf);!! pf = fopen("a.txt", "rt"); // try "rb"! int c = 0;! while( (c = fgetc(pf))!= EOF)! printf("%02x ", c);! printf("\n");! fclose(pf);! }! 44

45 File I/O in C l Stream I/O Other Stream I/O functions: long ftell( FILE *stream );! Ø Gets the current position of a file pointer. int fseek( FILE *stream, long offset, int origin);! Ø Moves the file pointer to a specified location. Ø Origin = SEEK_CUR è current position + offset SEEK_ENDè files size(in byte) + offset SEEK_SET è offset int feof( FILE *stream );! Ø Returns true if the cursor is at the end of file

46 File I/O in C l printf & scanf for file int fprintf(file *pfile, const char *format [, arg]...);! int fscanf(file *pfile, const char *format [, arg]...);! #include <stdio.h>! int main(){! FILE *pf = fopen("a.txt", "wb");! for(int i=0; i<10; ++i)!! fprintf(pf, "LINE[%d]\n", i);! fclose(pf);!! pf = fopen("a.txt", "rb");! char A[255] = {0};! while(fscanf(pf, "%s", A)!= EOF)! printf("%s\n", A);! fclose(pf);! }! 46

47 Exercise: PackBits l PackBits A data compression algorithm Invented by Apple Fast, simple and lossless It's suitable for image files, but unsuitable for text files Run-length encoding (RLE) 將一連串相同重複的 byte 改以兩個 byte 來表示, 第一個 byte 代表重複次數, 後第二個 byte 是資料 EX: Ø 0A 0A 0A 0A 0A è 05 0A TIFF and TGA image file use PackBits to compress the image data 47

48 Exercise: PackBits l Algorithm int n =0; while( n < MaxSize ) Read the next source byte into B. if (0 B 127){ read the next B+1 bytes. n += B + 1; } else if (-127 B -1){ read the next byte -B+1 times. n += (-B + 1) } } EX AA AA AA A AA AA AA AA A 22 AA AA AA AA AA AA AA AA AA AA è FE AA A FD AA A 22 F7 AA 48

49 Exercise: PackBits l Requirement: Using PackBits to compress a BMP or TIF file And send the compressed file to other classmate. Uncompressing the compressed file 49