CS631 - Advanced Programming in the UNIX Environment. Dæmon processes, System Logging, Advanced I/O

CS631 - Advanced Programming in the UNIX Environment Slide 1 CS631 - Advanced Programming in the UNIX Environment Dæmon processes, System Logging, Advanced I/O Department of Computer Science Stevens Institute of Technology Jan Schaumann jschauma@stevens.edu https://www.cs.stevens.edu/~jschauma/631/

Dæmon processes CS631 - Advanced Programming in the UNIX Environment Slide 2 So... what s a dæmon process anyway?

Dæmon characteristics CS631 - Advanced Programming in the UNIX Environment Slide 3 Commonly, dæmon processes are created to offer a specific service. Dæmon processes usually live for a long time are started at boot time terminate only during shutdown have no controlling terminal

Dæmon characteristics CS631 - Advanced Programming in the UNIX Environment Slide 4 The previously listed characteristics have certain implications: do one thing, and one thing only no (or only limited) user-interaction possible resource leaks eventually surface consider current working directory how to create (debugging) output

Writing a dæmon CS631 - Advanced Programming in the UNIX Environment Slide 5 fork off the parent process change file mode mask (umask) create a unique Session ID (SID) change the current working directory to a safe place close (or redirect) standard file descriptors open any logs for writing enter actual dæmon code

Writing a dæmon CS631 - Advanced Programming in the UNIX Environment Slide 6 int daemon(int nochdir, int noclose) { int fd; switch (fork()) { case -1: return (-1); case 0: break; default: _exit(0); } if (setsid() == -1) return (-1); if (!nochdir) (void)chdir("/"); } if (!noclose && (fd = open(_path_devnull, O_RDWR, 0))!= -1) { (void)dup2(fd, STDIN_FILENO); (void)dup2(fd, STDOUT_FILENO); (void)dup2(fd, STDERR_FILENO); if (fd > STDERR_FILENO) (void)close(fd); } return (0);

Dæmon conventions CS631 - Advanced Programming in the UNIX Environment Slide 7 prevent against multiple instances via a lockfile allow for easy determination of PID via a pidfile configuration file convention /etc/name.conf include a system initialization script (for /etc/rc.d/ or /etc/init.d/) re-read configuration file upon SIGHUP relay information via event logging

A central logging facility CS631 - Advanced Programming in the UNIX Environment Slide 8 There are three ways to generate log messages: via the kernel routine log(9) via the userland routine syslog(3) via UDP messages to port 514

A central logging facility CS631 - Advanced Programming in the UNIX Environment Slide 9

syslog(3) CS631 - Advanced Programming in the UNIX Environment Slide 10 #include <syslog.h> void openlog(const char *ident, int logopt, int facility); void syslog(int priority, const char *message,...); openlog(3) allows us to set specific options when logging: prepend ident to each message specify logging options (LOG CONS LOG NDELAY LOG PERRO LOG PID) specify a facility (such as LOG DAEMON, LOG MAIL etc.) syslog(3) writes a message to the system message logger, tagged with priority. A priority is a combination of a facility (as above) and a level (such as LOG DEBUG, LOG WARNING or LOG EMERG).

Nonblocking I/O CS631 - Advanced Programming in the UNIX Environment Slide 11 Recall from our lecture on signals that certain system calls can block forever: read(2) from a particular file, if data isn t present (pipes, terminals, network devices) write(2) to the same kind of file open(2) of a particular file until a specific condition occurs read(2) and write(2) of files that have mandatory locking enabled certain ioctls(2) some IPC functions (such as sendto(2) or recv(2)) See eintr.c from that lecture.

Nonblocking I/O CS631 - Advanced Programming in the UNIX Environment Slide 12 Recall from our lecture on signals that certain system calls can block forever: read(2) from a particular file, if data isn t present (pipes, terminals, network devices) write(2) to the same kind of file open(2) of a particular file until a specific condition occurs read(2) and write(2) of files that have mandatory locking enabled certain ioctls(2) some IPC functions (such as sendto(2) or recv(2)) Nonblocking I/O lets us issue an I/O operation and not have it block forever. If the operation cannot be completed, return is made immediately with an error noting that the operating would have blocked (EWOULDBLOCK or EAGAIN).

Nonblocking I/O CS631 - Advanced Programming in the UNIX Environment Slide 13 Ways to specify nonblocking mode: pass O NONBLOCK to open(2): open(path, O RDRW O NONBLOCK); set O NONBLOCK via fcntl(2): flags = fcntl(fd, F GETFL, 0); fcntl(fd, F SETFL, flags O NONBLOCK);

Nonblocking I/O CS631 - Advanced Programming in the UNIX Environment Slide 14 $ cc -Wall nonblock.c -o block $ cc -DNONBLOCK -Wall nonblock.c -o nonblock $./nonblock >/dev/null wrote 100000 bytes [...] $./block ( sleep 3; cat >/dev/null ) [...] $./nonblock ( sleep 3; cat >/dev/null ) [...] $ (./nonblock cat >/dev/null ) 2>&1 more [...] $ nc -l 8080 >/dev/null & $./nonblock nc hostname 8080 [...]

Resource Locking CS631 - Advanced Programming in the UNIX Environment Slide 15 Ways we have learned so far to ensure only one process has exclusive access to a resource: open file using O CREAT O EXCL, then immediately unlink(2) it create a lockfile if file exists, somebody else is using the resource use of a semaphore What are some problems with each of these?

Advisory Locking CS631 - Advanced Programming in the UNIX Environment Slide 16 #include <fcntl.h> int flock(int fd,int operation); Returns: 0 if OK, -1 otherwise applies or removes an advisory lock on the file associated with the file descriptor fd operation can be LOCK NB and any one of: LOCK SH LOCK EX LOCK UN locks entire file

Advisory Locking CS631 - Advanced Programming in the UNIX Environment Slide 17 $ cc -Wall flock.c 1$./a.out Shared lock established - sleeping for 10 seconds. [...] Giving up all locks. 2$./a.out Shared lock established - sleeping for 10 seconds. Now trying to get an exclusive lock. Unable to get an exclusive lock. [...] Exclusive lock established. 1$./a.out [blocks until the other process terminates]

Advisory Record Locking CS631 - Advanced Programming in the UNIX Environment Slide 18 Record locking is done using fcntl(2), using one of F GETLK, F SETLK or F SETLKW and passing a struct flock { short l_type; /* F_RDLCK, F_WRLCK, or F_UNLCK */ off_t l_start; /* offset in bytes from l_whence */ short l_whence; /* SEEK_SET, SEEK_CUR, or SEEK_END */ off_t l_len; /* length, in bytes; 0 means "lock to EOF" */ pid_t l_pid; /* returned by F_GETLK */ } Lock types are: F RDLCK Non-exclusive (read) lock; fails if write lock exists. F WRLCK Exclusive (write) lock; fails if any lock exists. F UNLCK Releases our lock on specified range.

CS631 - Advanced Programming in the UNIX Environment Slide 19 Advisory Record locking #include <unistd.h> int lockf(int fd, int value, off t size); Returns: 0 on success, -1 on error value can be: F ULOCK unlock locked sections F LOCK lock a section for exclusive use F TLOCK test and lock a section for exclusive use F TEST test a section for locks by other processes

Advisory Record locking CS631 - Advanced Programming in the UNIX Environment Slide 20 Locks are: released if a process terminates released if a filedescriptor is closed (!) not inherited across fork(2) inherited across exec(2) released upon exec(2) if close-on-exec is set

Advisory Record locking CS631 - Advanced Programming in the UNIX Environment Slide 21 Locks are associated with a file and process pair, not with a filedescriptor!

Mandatory locking CS631 - Advanced Programming in the UNIX Environment Slide 22 not implemented on all UNIX flavors chmod g+s,g-x file possible to be circumvented: $ mandatory-lock /tmp/file & $ echo foo > /tmp/file2 $ rm /tmp/file $ mv /tmp/file2 /tmp/file

Asynchronous I/O CS631 - Advanced Programming in the UNIX Environment Slide 23

Synchronous blocking I/O CS631 - Advanced Programming in the UNIX Environment Slide 24

Synchronous non-blocking I/O CS631 - Advanced Programming in the UNIX Environment Slide 25

Asynchronous blocking I/O CS631 - Advanced Programming in the UNIX Environment Slide 26

Asynchronous non-blocking I/O CS631 - Advanced Programming in the UNIX Environment Slide 27

Asynchronous I/O CS631 - Advanced Programming in the UNIX Environment Slide 28 System V derived async I/O limited to STREAMS enabled via ioctl(2) uses SIGPOLL BSD derived async I/O limited to terminals and networks enabled via fcntl(2) (O ASYNC, F SETOWN) uses SIGIO and SIGURG POSIX aio(3) kernel process manages queued I/O requests notification of calling process via signal or sigevent callback function calling process can still choose to block/wait

CS631 - Advanced Programming in the UNIX Environment Slide 29 Memory Mapped I/O #include <sys/types.h> #include <sys/mman.h> void *mmap(void *addr, size t len, int prot, int flags, int fd, off t offset); Returns: pointer to mapped region if OK Protection specified for a region: PROT READ region can be read PROT WRITE region can be written PROT EXEC region can be executed PROT NONE region can not be accessed flag needs to be one of MAP SHARED MAP PRIVATE MAP COPY which may be OR d with other flags (see mmap(2) for details).

Memory Mapped I/O CS631 - Advanced Programming in the UNIX Environment Slide 30

Memory Mapped I/O CS631 - Advanced Programming in the UNIX Environment Slide 31 Exercise: write a program that benchmarks this performance and run it on the systems you have access to.

Memory Mapped I/O CS631 - Advanced Programming in the UNIX Environment Slide 32 http://cvsweb.netbsd.org/bsdweb.cgi/src/bin/cp/utils.c?rev=head

Reading CS631 - Advanced Programming in the UNIX Environment Slide 33 https://www.ibm.com/developerworks/linux/library/l-async/index.html http://menehune.opt.wfu.edu/kokua/more_sgi/007-2478-008/sgi_html/ch08.html http://lse.sourceforge.net/io/aionotes.txt https://en.wikipedia.org/wiki/streams