Open System Services. Developer's Tips. Marc Fuentes and Milind Dattawadkar

Transcription

1 Open System Services (OSS) Update and OSS Developer's Tips Marc Fuentes and Milind Dattawadkar 2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice

2 AGENDA Recent enhancements AF_UNIX Release 2 OSS Limits Lifting POSIX User Threads Migration considerations Upcoming enhancements 64-bit OSS Processes Additional OSS Limits Lifting FLOSS and Open Source Utilities 2

3 AF_UNIX Release 2 Overview Completely re-architectured implementation of AF_UNIX sockets on OSS Many fewer moving parts API compatibility with AF_UNIX Release 1 (the earlier implementation) AF_UNIX Release 2 totally replaces AF_UNIX Release 1 Benefits Increased reliability and robustness Increased performance Availability AF_UNIX Release 2 is installed automatically when H06.16/J06.05 or later SUT is installed. $ZPMON.#ZPLS remains in the OSS Monitor s Database for fall-back to earlier versions 3

4 OSS Limits Lifting Overview More OSS file opens (64,000 opens per CPU) 48,000 disk opens; 16,000 sockets; 16,000 pipes More memory for file IO operations (> 100 MB per CPU) 60 MB disk cache; 32 MB sockets; 32 MB pipes Efficient memory allocation for file IO operations Resource usage EMS events Significant state changes and Failures Benefits Increased scalability Efficient use of memory Enhanced manageability Availability Automatically present and enabled when H06.18/J06.07 or later SUT is installed 4

5 POSIX User Threads Overview Thread-aware version of NSK public DLLs allow multi-threaded programs to safely execute non-blocking library I/O functions and functions that share context. Stack overflow detection provides protected memory: process signal stack, and protected thread stacks. It allows an OSS process to enable a signal handler to catch stack overflow traps (SIGSTK). Benefits Improved performance for threaded applications Improved robustness in applications (Stack overflow detection) Improved POSIX standard compliance Availability Available when H06.21/J06.10 or later SUT is installed Applications need to be rebuilt with T1280, the POSIX User Thread (PUT) library Certain applications may need modifications 5

6 AF_UNIX Release 2 features Relesase 1 (compatibility) and Release 2 (portability) mode behavioral differences (1 of 3) Release 1: getpeername() returns the relative pathname of the peer socket when bind() was given a relative pathname. Release 2: getpeername() returns the fully-qualified equivalent of the relative pathname of the peer socket when bind() was given a relative pathname. Release 1 Datagram sockets: getpeername() returns ENOENT when the peer socket file has been renamed Release 2 Datagram sockets: getpeername() returns the fully qualified name of the socket file given to connect() Release 1 Datagram sockets: getpeername() returns ENOENT when the peer socket file has been unlinked Release 2 Datagram sockets: getpeername() returns the fully qualified name of the socket file given to connect() 6

7 AF_UNIX Release 2 features Release 1 (compatibility) and Release 2 (portability) mode differences (2 of 3) Release 1 receivefrom()/receivemsg(): When the file to which a sending datagram socket is bound is removed and one of the send family of functions is called, the receiving client s call to recvfrom() or recvmsg() returns a null address (all fields in the address are zero). Release 2 receivefrom()/receivemsg(): When the file to which a sending datagram socket is bound is removed and one of the send family of functions is called, the receiving client s call to recvfrom() or recvmsg() returns the fully-qualified form of the address to which the sending socket was originally bound. Release 1 connect(): If listen() is called with a backlog <= 0, subsequent attempts to connect to the socket are rejected with ECONNREFUSED unless there is a pending accept() call on the socket. Release 2 connect(): If listen() is called with a backlog <= 0, subsequent attempts to connect to the socket block until there is a corresponding accept() call on the socket. 7

8 AF_UNIX Release 2 features Release 1 (compatibility) and Release 2 (portability) mode differences (3 of 3) Release 1 getsockname(): When a bound socket file is unlinked or renamed, a subsequent call to getsockname() fails with ENOENT. Release 2 getsockname(): When a bound socket file is unlinked or renamed, a subsequent call to getsockname() still returns the fully qualified name. 8

9 AF_UNIX Release 2 features Specifying Release 1 (compatibility mode) or Release 2 (portability mode) behavior Processes can use both Release 1 and Release 2 sockets concurrently Sockets created in one mode cannot interact with sockets created in a different mode (connect() fails with ECONNREFUSED, sendto()/sendmsg() fails with EPERM) The default mode is Release 1 (compatibility mode) Changing the mode only affects the calling process and child processes, it does not change the mode for all present/future processes on the system The =_AFUNIX_PROCESS_NAME File System Define is used to set the default mode for processes that inherit the define Processes can call socket_transport_name_set() to set the mode for the next socket the process creates. 9

10 AF_UNIX Release 2 Operations and Management Operations and usage LS2 processes may be stopped for replacement. SCF STOP SERVER $ZPMON.#ZLSnn If there are AF_UNIX sockets in the CPU, the operation fails: 1-> assume process $zpmon 2-> stop server #zls00 OSS E00030 Failed to stop server $ZLS00 3-> 10

11 AF_UNIX Release 2 features Selecting Release 1 behavior (compatibility mode) TACL (note that $ZPLS is not an actual process name under AF_UNIX Release 2) ADD DEFINE =_AFUNIX_PROCESS_NAME, CLASS MAP, FILE $ZPLS OSS shell add_define =_AFUNIX_PROCESS_NAME class=map file=\$zpls Programmatically if (DEFINESETATTR( CLASS<blank-padded-for11-bytes>, MAP,3) exit(-1); if(definesetattr( FILE<blank-paded-for-12-bytes, $ZPLS,5) exit(-1); if(2051 == DEFINEADD( =_AFUNIX_PROCESS_NAME<blank-padded-for3-bytes>,1)) if(defineadd( =_AF_UNIX_PROCESS_NAME<blank-padded-for-3-bytes>,0) exit(-1); Dynamically #define _XOPEN_SOURCE_EXTENDED 1 #include <sys/socket.h> If(socket_transport_name_set(AF_UNIX, $ZPLS )) exit(-1); 11

12 AF_UNIX Release 2 features Selecting Release 2 behavior (portability mode) TACL (note that $ZAFN2 is not an actual process name under AF_UNIX Release 2) ADD DEFINE =_AFUNIX_PROCESS_NAME, CLASS MAP, FILE $ZAFN2 OSS shell add_define =_AFUNIX_PROCESS_NAME class=map file=\$zafn2 Programmatically if(definesetattr( CLASS<blank-padded-for11-bytes>, MAP,3) exit(-1); if(definesetattr( FILE<blank-paded-for-12-bytes, $ZAFN2,6) exit(-1); if(2051 == DEFINEADD( =_AFUNIX_PROCESS_NAME<blank-padded-for3-bytes>,1)) if(defineadd( =_AF_UNIX_PROCESS_NAME<blank-padded-for-3-bytes>,0) exit(-1); Dynamically #define _XOPEN_SOURCE_EXTENDED 1 #include <sys/socket.h> if(socket_transportname_set(af_unix, $ZAFN2 )) exit(-1); 12

13 OSS Limits Lifting New EMS events Resources monitored POB open table, PXS buffers, socket open table, socket buffers, pipe open table, pipe buffers, disk opens, disk cache buffers Event types Limit warning (When 85% of the resource has been allocated) Limit error (When 100% of the resource has been allocated) Status normal (When the resource usage falls to or below 65%) Allocation error (When the resource cannot be allocated for some other reason) 13

14 OSS Limits Lifting New EMS events Limit warning event example :00:18 \YOSQA26.$ZPP01 TANDEM.OSS.H Resource PIPEFIFO BUFS is approaching the resource limit. Resource limit: Warning threshold: 85 Current percentage: 85 Current usage: Peak percentage: 100 Peak usage: Peak time: 28JAN09 18:55 Reset time: 28JAN09 16:05 Safe threshold: 65 14

15 OSS Limits Lifting New EMS events Limit error event example :00:33 \YOSQA26.$ZPP01 *TANDEM.OSS.H Limit error for PIPEFIFO BUFS resource. Resource limit: Current percentage: 100 Current usage: OSS PID: Program file: \YOSQA26.$SYSTEM.SYS52.OSSPS Attempted Allocation Amount:

16 OSS Limits Lifting New EMS events Status normal event example :02:08 \YOSQA26.$ZPP01 TANDEM.OSS.H Status returned to normal for PIPEFIFO BUFS resource. Resource limit: Warning threshold: 85 Safe threshold: 65 Status threshold: 0 Current percentage: 65 Current usage: Peak percentage: 100 Peak usage: Peak time: 28JAN09 18:55 Reset time: 28JAN09 16:05 16

17 OSS Limits Lifting New EMS events Allocation error event example :49:33 \YOSQA26.$ZPP01 *TANDEM.OSS.H Allocation error for PIPEFIFO BUFS resource. Resource limit: Current percentage: 0 Current usage: 0 Peak percentage: 100 Peak usage: Peak time: 28JAN09 18:55 Reset time: 28JAN09 16:05 OSS PID: Program file: \YOSQA26.$SYSTEM.SYS52.OSSPS Attempted Allocation Amount:

18 POSIX User Threads Migrating from the SPT (T1248) library to the PUT library (T1280) Compiler defines Include pthread.h in /usr/include instead of spthread.h #include <pthread.h> Define _PUT_MODEL_ This define is required for threaded applications using the PUT library. It may be specified in the source code as a #define or on the compiler command line with the D switch. Faster select() algorithm Define _PUT_SELECT_SINGLE_ to use a version of select() that is faster. This is equivalent to SPT_SELECT_SINGLE in the SPT library 18

19 POSIX User Threads Migrating from the SPT library to the PUT library Environment variables PUT_THREAD_AWARE_REGULAR_IO_DISABLE Disables thread-aware regular I/O behavior. Thread-aware APIs ignore the O_NONBLOCK flag for regular files. Setting this environment variable before running an application that used the PUT library causes thread-aware APIs to have process blocking behavior. See the Open System Services Programmer s Guide for details. PUT_PROTECTED_STACK_DISABLE Setting this environment variable causes the thread stack to be allocated from the process heap instead of being allocated on a separate protected stack 19

20 POSIX User Threads migration considerations Migrating from the SPT library to the PUT library Mixing the SPT library and the PUT library in the same process is not allowed, attempts to do so fail with runtime error 019 pthread_attr_default, pthread_mutexattr_default, pthread_condattr_default do not exist as global varables in the PUT library The PUT library functions return -1 and set errno rather than returning the errno value. unistd.h has a set of macros that influence HP s thread implementation. See the Open System Services Programmer s Guide for details. Thread-aware signals are always enabled in the PUT library. PUT-based applications have to statically link with the PUT library. 20

21 POSIX User Threads migration considerations Signal handling topics (1 of 3) PUT model applications cannot call sigaltstack() to establish a signal stack. They can call sigaltstack() to find the information about the signal stack. The Open System Services Programmer s Guide lists a set of functions that are not async-signal safe. These functions should not be used in signal handlers (this is similar to the SPT library). Job control signals and their corresponding actions are not supported (this is similar to the SPT library). The Real-time Signals Extension option is not supported (this is similar to the SPT library). signal() is deprecated, use sigaction() instead See the Open System Services Programmer s Guide for a list of APIs that support thread-aware signal handling 21

22 POSIX User Threads migration considerations Signal handling topics (2 of 3) Similar to the SPT library Some signals affect the entire process. See the Open System Services Programmer s Guide for details If a signal is delivered to a thread waiting on a condition variable, then upon return from the signal handler, the thread will resume waiting for the condition variable as if it were not interrupted. Do not use the raise() API for an exception, and do not use the longjmp() API to exit a synchronous signal handler. Simply returning from a signal handler is safe and is the recommended practice. Use exception handlers instead of the setjmp() and longjmp() APIs for threaded OSS applications. Do not use the option of the sigsetjmp() and siglongjmp() APIs that saves and restores the signal mask. Doing so can cause signals enabled by another thread to be masked. 22

23 POSIX User Threads migration considerations Signal handling topics (3 of 3) Do not deliver signals like SIGALRM and SIGCHLD using the kill command or the raise() function because these do not provide enough information about the process IDs for the parent and child processes. Instead, use the PUT Model library version of the alarm() API to raise alarms. 23

24 POSIX User Threads migration considerations APIs in the SPT library that do not exist in the PUT library The PUT model library does not provide analogs to spt_*x() or spt_*z() APIs, such as spt_readx() or spt_readz(), whose purpose is to be non-blocking alternatives to standard system library functions. Instead, use the standard API names. The PUT model library does not provide analogs to spt_*() functions, such as spt_fprintf(), whose purpose is to be non-blocking alternatives to the standard C run-time library functions. Instead, use the standard API names. The C run-time library functions are thread-aware if called from a PUT-based application. 24

25 POSIX User Threads migration considerations Pitfalls See the Open System Services Programmer s Guide for a list of threadaware APIs, a list of non-thread-safe APIs, non-thread-safe cancellation points, and a list of APIs that are potential cancellation points Do not mix modules compiled to use the PUT library with non threadaware modules when the modules perform I/O to the same files Look out for warnings of the form: function <function-name>" declared implicitly, reports that no explicit definition of <function-name>() appeared and therefore the compiler assumed defaults for all function attributes. Consequently, calls to <function-name>() are not aliased to the nonblocking variant and cause an unneeded and unwanted suspension of the entire process. 25

26 Thread-Aware I/O Availability Pre-POSIX User Thread called directly from: Thread-aware system I/O on non-disk file Thread-aware system I/O on disk file Thread-aware C I/O on nondisk file Thread-aware C I/O on disk file Thread-aware C++ I/O streaming SPT-based application Х Х Public DLL linked to SPT-based applications Х Х Х Х Х Post-POSIX User Thread called directly from: Thread-aware system I/O on non-disk file Thread-aware system I/O on disk file Thread-aware C I/O on nondisk file Thread-aware C I/O on disk file Thread-aware C++ I/O streaming Thread-aware public DLL functions SPT-based application Х Х Х Public DLL linked to SPT-based applications Х Х Х Х Х Х PUT-based application Public DLL linked to PUT-based applications 26

27 64 Bit OSS Processes Overview for C/C++ programs Allow OSS programs access to a much larger virtual address space for data through support of 64-bit addressing. SQL/MX Access from processes using 64-bit addressing. Run-time heap in 64-bit addressable virtual memory Benefits Default limit is 12GB NonStop OS limit is.5tb for heap and 64-bit flat segments. Practical limit determined by the amount of RAM and disk space for swap files. Increased scalability Future product plans, dates, and functionality are subject to change without notice. 27

28 Additional OSS Limits Lifting Overview More OSS processes per system 128,000 simultaneous OSS processes on a 16-processor system More OSS file opens (128,000 opens per CPU) 96,000 disk opens; 32,000 sockets; 32,000 pipes; 32,000 ttys More memory for file IO operations 120 MB disk cache; 64 MB sockets; 32 MB pipes Resource monitoring capabilities with SCF commands Resource monitoring capabilities with Measure Benefits Increased scalability Enhanced resource monitoring Future product plans, dates, and functionality are subject to change without notice. 28

29 FLOSS and Open Source Utilities Overview FLOSS - enables porting of open source tools to the NonStop platform At present, FLOSS version 0.9 is available at the Connect website ( This will be made available as a standard product on the RVU Provides select Open Source Tools on the NonStop platform FLOSS will be officially supported Tools made available on a business case by case basis Samba Update tools on the Connect website (BASH, Archival tools, Core utilities, etc.) Provide RPM and RPM based packages on the Connect website to ease installation and dependency management Benefits Enable portability of open source packages Updated tools 29 Future product plans, dates, and functionality are subject to change without notice.

30 S T E Pup the momentum of your technology evolution. HP Integrity NonStop modernization for greater infrastructure convergence