Usmux: Unix-domain socket multiplexing

Transcription

1 Usmux: Unix-domain socket multiplexing Steven Simpson October 14, 2017 Abstract Although Java programs are executed on a virtual machine, they can run as fast as programs compiled to the native architecture when just-in-time (JIT) compilation is applied, along with other optimizations involving profiling and in-lining. These are most effective in long-running executions, where the cost of initialization and analysis can be amortized. Consequently, Java programs can run efficiently over long periods as server processes and desktop applications, but perform significantly less well as short lived programs such as CGI scripts. A long-running Java application could be adapted for use by short-lived commands by making them connect to (say) a socket which the application is listening on. The short-lived commands are then simple clients, whose job is merely to relay I/O between the user and the application. The only built-in and portable mechanism to connect with such a Java process is to use network sockets, possibly restricted to the loopback address for security purposes. There are also third-party Java libraries which provide access to Unix-domain sockets, giving additional security (especially on multi-user machines), but these do not fit well with Java s native socket abstraction, which allows its Internet-specific nature to leak through. They also usually require JNI, and therefore introduce additional complexity in terms of installation. This report details an approach to supporting Unixdomain socket access to persistent Java processes, without using any JNI, and just a small amount of C in a separate wrapper or launcher program. This brings an additional benefit of launching the Java process in the background, with a synchronization point just after start-up to allow the process to abort execution if failure is detected early. Under the approach, the launcher creates a pair of named pipes in the filesystem, starts the Java process (informing it of the names of the two pipes), opens a Unix-domain socket, and multiplexes connections on the socket to the Java process via the pipes. A protocol is defined to permit the multiplexing. Specifications of the protocol and the Java API are independent of this particular use of sockets and pipes, so they could be re-applied on other platforms with alternative concepts, while allowing the Java code that uses them to remain portable between platforms. Introduction At first, Java appears inherently slow by its design. Java source code is first compiled into bytecode, an assembly language for a virtual processor called the Java Virtual Machine (JVM). The JVM then processes each (virtual- )machine code instruction to produce the behaviour represented by the source code, and it is this processing which is slower that an equivalent program compiled to a native machine code. The benefit is a write-once, runanywhere (WORA) environment making Java programs maximally portable. The additional layer of abstraction (the JVM must ultimately be written in native machine code) also provides greater integrity, increasing security, making dynamic loading of foreign code possible, and detecting programmer error. The creators of Java have also sought speed improvements along several avenues. Start-up times have been reduced by increasing the modularity of the standard library, and by permitting some lazy class initialization. Bytecode can be compiled to native code as it is loaded with a just-in-time (JIT) compiler installed in the JVM. The JVM may also monitor its process, looking for sections of code most frequently executed, in order to prioritize its optimization. It may also in-line sections of code to reduce the overhead of method invocation. Note that these optimizations apply at run time in the JVM, whereas the compiler mostly avoids them, as the effectiveness of these optimizations depend greatly on runtime characteristics not available during compilation. 1

2 Altogether, these optimizations can allow a Java process to run at least as fast as a natively compiled program, while still retaining the benefits of portability. However, each one also comes with an overhead, which must be amortized by its results being applicable over a sufficiently long period. For example, a Java program used as a short-lived command cannot easily benefit from JIT because JIT-ed code may only be executed a small number of times before exiting (and the next invocation will have to be re-jit-ed), nor from HotSpot because the process terminates before enough analysis has been performed. Consequently, Java performs best when used for long-running programs, such as desktop applications and server processes. Some environments are designed to work with shortlived processes. A command shell such as Bash repeatedly executes external commands, and the Common Gateway Interface (CGI) used by web servers to delegate requests to user programs mandates one process per request. These are therefore poor environments in which to use Java. Persistence for CGI programs Java isn t the only language to suffer great overheads due to repeated start-ups. Any per-process delay on a CGI program, in whatever language, makes CGI the bottleneck for any high-demand service. FastCGI 1 was devised to address this issue by using a regular TCP connection between the web server and the CGI program, which runs persistently and serves multiple requests, instead of running once per request. This is a good approach, but the Usmux project aims for a more general mechanism than one built to support CGI; it should be possible to implement a FastCGI-like mechanism over Usmux. [upcgi] Java support for Unix-domain sockets Another approach to allow a Java process to run persistently is to provide a library supporting Unix-domain sockets. Several implementations are summarized here: JUDS 2 [Investigate.] junixsocket 3 This is a JNI library that uses the the Java Sockets API, and supports RMI. Fitting in with the Sockets API is a feat, as much of it is based on Internetdomain sockets, and doesn t seem to fit well when a broader abstraction is required. J-BUDS This is an orphaned project. [Investigate.] gnu.net.local This is another oprhaned project. [Investigate.] Providing native support for Unix-domain sockets in Java seems to create more problems than it solves. It is necessarily platform-specific, so the Java code that uses the support must be wrapped behind a platformindependent abstraction if the program is to remain platform-independent itself. The fact that the Internetdomain nature of the Java Sockets API leaks through its own abstraction means that that API does not help to hide platform-specificity. Usmux Usmux is an approach to allow one-invocation-perrequest environments like CGI to be used with persistent Java processes. At the time of writing, the implementation is POSIX-specific, but the architecture is platform-independent. Implementations on other platforms may be possible. On a POSIX system, the usmux command takes a Java command as its trailing arguments. It also opens a Unix-domain socket, creates two named pipes, and injects the names of these pipes as a configuration string into the Java command, which is then executed in the background. It then accepts connections on the socket, and presents them as sessions to the Java process by creating another pair of named pipes per session, using its original pipes to notify the Java process of them, and relaying data between the pipes and the socket. As a result, the Java process can run persistently, but be invoked by multiple clients over a Unix-domain socket. On other platforms, a similar command could use other platform-specific features to interact with a server running the same Java software. Only the injected configuration string needs to change, and the Usmux Java library will dynamically load in the matching serverside implementation

3 Even under POSIX, other mechanisms are possible too. For example, usmux could create a pair of named pipes, and multiplex several sessions over them. (However, experience has shown that this is error-prone to code, and hampers over-all through-put.) The use of a Unix-domain socket to communicate with clients has several advantages. First, as its rendezvous point is part of the filesystem, the host s native access control can be applied to it. For more sophisticated control, it is possible to inform the server of which user or group is actually connecting. Finally, invocation of the client could exploit external authentication protocols such as SSH. Architecture Usmux aims to allow a client to initiate a session with a server. Each session is a pair of simplex streams carrying byte-oriented data. Upstream denotes travel from the client to the server, while downstream denotes travel in the opposite direction. Either peer may terminate either stream at any time, and the session is terminated when both streams are terminated. The basic Usmux architecture is shown in Figure 1. The Usmux daemon sits between the server and its clients, and relays messages between them. The daemon is also responsible for forking the server into the background. Various communication schemes may be employed between client and daemon, and between daemon and server. In some schemes, client and server communicate directly, with the daemon playing no part after start-up. Client-daemon schemes may involve a Unix-domain socket, an Internet-domain socket, or any other connection-oriented communication supported by the host platform(s). Daemon-server schemes may be similarly varied, but are constrained mainly by having to appear platform-independent in order to be accessible from Java. Effectively, the daemon acts as an adapter between the client s use of a platform-specific technology and the server s requirement for a platformindependent one. The session abstraction must necessarily be very primitive, as the only characteristic that a session has under any scheme is that it consists of two streams. However, the server may be able to make use of information about the connecting client, even though it is scheme-specific. When the client connects to the daemon with a Unix-domain socket, its details (process id, user id, group id) can be detected by the daemon, and be relayed to the server. When the client connects directly to the server via TCP, its host and port can be made available to the server. These are exposed to the server in a generic form as client meta-data, described in 2.1. When the daemon is involved in implementing sessions, the protocols involved in talking to the server allow the daemon to relay this information as opaque binary data. Figure 2 depicts one daemon-server scheme which can appear platform-independent to the server, in which all connections accepted by the daemon are multiplexed over two simplex named pipes. This protocol is described in 3.2, and is capable of relaying arbitrary client meta-data to the server. Alternatively, the daemon may establish a pair of pipes per session, and inform the server of each one, plus client meta-data, over a pipe pair dedicated for signalling, as shown in Figure 3. This scheme is described in??, and is also capable of relaying arbitrary client meta-data to the server. Under another alternative, the daemon serves only to fork the server into the background. The server then opens an Internet-domain socket, and accepts TCP connections directly from the client as sessions. It is essential that a Java implementation of the server component can operate as independently as possible from these various communication schemes between client, daemon and server. To this end, a Java API ( 2.2) is defined to provide an abstraction of sessions. This abstraction provides only an input stream and an output stream, with generic access to client meta-data. Client meta-data Meta-data provides information on connecting clients which are otherwise hidden from the server behind the Usmux daemon. The format of this information depends on the connecting mechanism. For example, if the client talks to the daemon via a Unix-domain socket, the platform may be able to provide the daemon with (say) the user id of the client process. This information is potentially useful to the server process, but has no fixed form, so it is relayed from daemon to server in a binary format, which the multiplexing scheme can regard as opaque. An extensible framework then allows multiple binary formats to be interpreted as various Java structures, by making the data self-describing by its initial bytes, length and/or context. (Under some schemes, the client talks directly to the 3

4 Figure 1: Basic architecture Figure 2: Multiplexing between daemon and server over named pipes Figure 3: Per-client pipe pairs between daemon and server 4

5 server, leaving the daemon uninvolved as soon as the server is established. Under these schemes, the data is likely already available to the server, and does not need a binary format.)?? descripbes how meta-data is transmitted in the multipipe protocol describes how meta-data is transmitted in the multiplex protocol. Two meta-data types are specified here. Unix-domain client meta-data When the client that instigates a session is communicating through a connection-oriented Unix-domain socket, the meta-data can be a UnixDomainSocketAttributes 4 object containing user id, group id and process id of the calling client. As binary data relayed between daemon and server, this is represented as 10-byte block, starting with the US-ASCII codes for UNIX (Figure 4). The next two bytes are the process id, followed by two bytes for the user id, and two bytes for the group id, all in big-endian format x55 0x4e 0x49 0x58 process id group id user id Figure 4: Binary meta-data format for a Unix-domain socket Internet-domain client meta-data When the client that instigates a session is communicating through a connection-oriented Internet-domain socket, the meta-data can be an InetSocketAddress 5 object containing the IP address and port number of the calling client. This type of meta-data has no binary representation, as the client is expected to connect to the server directly. Java API The Javadoc-generated documentation is available online 6. This section gives an overview of its use. 4 uk.ac.lancs.scc.usmux.unix.unixdomainsocketattributes 5 java.io.inetsocketaddress 6 ss/apis/usmux/overviewsummary.html The Java server should be able to operate with different schemes, depending on the platform available. It remains platform-independent by accepting an opaque string as an argument, and passing this to SessionServerFactory.makeServer. This method loads and instantiates an appropriate SessionServer implementation which can make use of information extracted from the string. Subsequently, the Java server merely needs to start() the SessionServer object, and invoke accept on it repeatedly to service sessions. These calls each yield a Session object, which provides the basic input and output streams of the session. Table 1 shows this main loop, where config is the opaque configuration string, and SessionThread is an application-specific class to process each session. Session also provides a getattributes method to extract scheme-specific information, especially about the client. For example, Table 2 shows how to get a structure providing the process id, user id and group id of the caller who connected to the Unix-domain socket, if that scheme was used. Multiplexing schemes Three multiplexing schemes are defined. Historically, the single-pipe scheme ( 3.2) was defined first. This attempts to multiplex sessions over a single pair of simplex channels, so communication must be divided into discrete messages, and peers must carefully signal availability of buffer space to avoid session activities impeding each other. The result appears to have poor performance. The multipipe scheme ( 3.1) attempts to improve performance by allowing each session to use a dedicated pair of pipes. An additional pair are used for signalling. The TCP scheme ( 3.3) simply allows the server process to use a regular TCP socket. However, to keep parity with other schemes, there is an additional step enabling the daemon to detect that the server has reached readiness, and then leave it running in the background. The scheme also supports simple access control. Multipipe scheme Under the multipipe scheme, each session operates over a dedicated duplex channel, while another duplex channel (the control channel) is used for signalling. 5

6 SessionServer server = SessionServerFactory.makeServer(config); server.start(); Session sess; while ((sess = server.accept())!= null) new SessionThread(sess).start(); Table 1: Main server loop UnixDomainSocketAttributes attrs = sess.getattributes(unixdomainsocketattributes.class); System.out.println("User: " + attrs.getuserid()); System.out.println("Group: " + attrs.getgroupid()); System.out.println("Process: " + attrs.getprocessid()); Table 2: Accessing Unix-domain socket attributes of the client The scheme uses a configuration string of the form multipipe:upfile;downfile, where upfile is the name of a named pipe that forms the upstream half of the control channel, and downfile is the name of the pipe that forms the downstream half. The downstream control channel is used initially to detect readiness of the server signaled by a single byte and to detect when the server is terminating acceptance of future connections the channel is closed. For each session, the daemon creates an additional pair of named pipes, and uses the upstream control channel to transmit their names to the server. The downstream pipe s name is printed on a line of its own, followed by the upstream pipe s name. One channel is then used to transmit upstream data of the session, and one downstream. The daemon creates all named pipes, and should delete them on termination if not yet passed to the server. However, the server should delete them itself as soon as it has opened them. The meta-data for each session is transmitted as a prefix to the upstream data, beginning with a 2-byte bigendian length. The downstream channel is used exclusively for application data. The single-pipe scheme The single-pipe scheme allows multiple sessions to share a single duplex channel, over which a multiplexing protocol is run. The scheme uses a configuration string of the form pipe:downfile;upfile, where upfile is the name of a named pipe that forms the upstream half of the channel, and downfile is the name of the pipe that forms the downstream half. The multiplexing protocol consists of several byteoriented messages transmitted over a reliable, fullduplex channel between two end-points. Each endpoint is the peer of the other. One end-point is the server process (or just server), while the other is the daemon process (or just daemon). Each session consists of a pair of streams, one in each direction between the two end-points. The protocol is designed to support connections accepted on a Unix-domain socket as sessions multiplexed over the pair of named pipes used by Usmux, and permits meta-data about each connection to be supplied to the Java process. However, it is not dependent on named pipes or Unix-domain sockets, so it can be applied to hosts that support neither of these concepts. To this end, the protocol regards the meta-data as an opaque block. Similarly, it is not tied to Java in any way, so the server process can be written in any language. The protocol has an initialization phase, in which the server process transmits a ready message to indicate its readiness to receive sessions. This phase also allows the server to indicate how much meta-data it will accept per session. After intitialization, sessions can be relayed over the channel. Each session has a handshake phase, during which meta-data is supplied to the server, and the server and daemon exchange references. A data phase follows, in which each end-point repeatedly updates its peer about the space available in its reception buffer, and transmits data according to space available in its peer s buffer. 6

7 Initialization phase The initialization phase consists of the daemon waiting for the server to send a RDY message. This message may contain a meta-data limit, the maximum number of bytes the server is prepared to receive as payload for future NEW messages sent to initiate each session. No other messages may be transmitted over the channel until RDY has been received. Handshake phase Each session begins with a handshake phase. The daemon initiates each session by transmitting a NEW message. This includes the session id that the daemon will use in all future messages concerning this session. Any additional data is considered to be meta-data, which the server may use in any way it sees fit. The server responds to each NEW message with an ACK message. This echoes back the daemon s session id, and includes the server s session id. From this point on, each message related to this session and transmitted by the daemon will use the server s session id, while the server will correspondingly use the daemon s session id. Data phase After a session has been established by a handshake phase, either party may transmit CTS messages according to the space it has available to receive data for the relevant session. The effect of multiple CTS messages is cummulative, so a peer can send upto the sum of the amounts specified by received CTS messages, minus that which it has already sent. An end-point can use this to ensure that its peer can send a complete message without blocking due to the receiver s buffer being full. To transmit data, a peer sends a STR message, whose payload starts by identifying the session id, and continues with the data. The sum of the message s data length (the payload length minus two) and of the data lengths of all prior STR messages for the same session must not exceed the sum of the clearances of all CTS messages received for the same session. When a peer has no more data to send, it must send an EOS message to close the session. A peer can also indicate that it will not use any more data by sending a STP message. Messages Each message has a type and a length (Figure 5), so unrecognized messages can be skipped. The message type is a two-byte big-endian field at the start of the message. The length is the two-byte big-endian field that follows it. The whole message need not be wordaligned. Message types are summarized in Table message type payload payload length Unused message types payload Figure 5: Message format The UNK and NOP message types must never be sent. The receiver may ignore them entirely. They are intended to be used by the daemon or the server internally to indicate that no complete message header has been received, or that the remainder of a message may be ignored. RDY Server ready The RDY message type (Figure 6) is the first one sent over the channel. It is sent only once, and by the server to the daemon. If its payload is at least two bytes, these are taken as a big-endian unsigned integer specifying the maximum number of meta-data bytes that the server will accept with each session. Any other payload, or an incomplete payload, may be ignored (RDY) payload length meta-data limit Figure 6: The RDY message server ready 7

8 Type Value Direction Meaning Parameters UNK 0 never sent Unknown None NOP 1 never sent No operation None RDY 2 server-to-daemon Server ready Meta-data limit NEW 3 daemon-to-server New session Session id and meta-data ACK 4 server-to-daemon Session acknowledged Session ids CTS 5 both Clear to send Session id and clearance STR 6 both Stream data Session id, data length and data EOS 7 both End of stream Session id STP 8 both Stop stream Session id Table 3: Message types Handshake message types Sessions are initiated by the daemon. For each session, it issues a NEW message (Figure 7), with at least two bytes of payload giving the daemon s session id, which the server must use in all future communication pertaining to this session (ACK) payload length daemon session id server session id Figure 8: The ACK message session acknowledged Data message types Any remaining payload is interpreted by the server as meta-data. For example, if the session is a relay for For each session, each end-point must only send the a Unix-domain socket, the meta-data may contain the amount of data it has been permitted by its peer. One process id, user id and group id of the calling process. end-point permits its peer to send data by sending CTS 2.1 lists some meta-data types. messages (Figure 9), which includes the peer s session id, and a 32-bit signed length. Each message instructs the receiver31to accumulate provided lengths, and to send 3 (NEW) payload length no more data than the accumulated value, including data already send on the session. daemon session id meta-data 5 (CTS) payload length peer s session id clearance clearance Figure 9: The CTS message clear to send Figure 7: The NEW message new session On receipt of a NEW message, the server must generate a session id of its own, and respond with an ACK message (Figure 8). The first two bytes of its payload form the daemon s session id; the next two form the server s session id, which the daemon must use in all future communication pertaining to this session. Additional payload bytes carry no meaning, and can be ignored. To send data, an end-point sends a STR message (Figure 10) to its peer. This includes the peer s session id, with the remaining payload being the stream data. When an end-point has no more data to send on its stream, it must send an EOS message (Figure 11), unless it has already received a STP message for the session. This includes the peer s session id. It must not send any further STR messages after that. When an end-point cannot use data received on a stream, it may send a STP message (Figure 12). This includes the peer s session id. The peer need not send any further STR or EOS messages on the session, and 8

9 (STR) payload length Figure 11: The EOS message end of stream TCP scheme The scheme uses a configuration string of the form pipetcp:sigfile;bindip;bindport;acl, where bindip is IP address to which the server should bind a TCP socket to, and listen on it, and bindport is the port number to which the socket shall be bound. acl is a semicolon-separated list of IP addresses, IP address prefixes, and hostnames (possibly with wildcards), from which the server should accept connections. Connections from entries prefixed with a dash - should be rejected. sigfile should be a named pipe. The server should open and close it to indicate that it is ready. The daemon should not return control to the user until this has happened. Extensions peer s session id Creating daemon-server communication stream data schemes The Java server does not need to communicate with the Usmux daemon in one particular way. Indeed, it does not need to involve the daemon at all. New schemes can be defined, and then implemented by deriving from SessionServerFactory, and providing a createserver method. This takes an opaque Figure 10: The STR message stream data configuration string, which the factory class either recognizes or ignores (to be passed to other factories) When the string 31 is recognized, its data can be used to 7 (EOS) payload length configure the SessionServer that the factory class must create. peer s session id The recommended format for a configuration string is one that appears URI-like, i.e. scheme:params The factory class can then either recognize the scheme and process the parameters to create a the end-point need not use them. SessionServer, or return null. When the factory class is packaged, its jar should include an entry 31 in the file named in Table 4 giving the 8 (STP) payload length factory s class name. This makes it available automatically to the static makeserver method that checks an peer s session id opaque configuration string against all registered factories. Figure 12: The STP message stop stream Creating client-daemon communication schemes Consider the meta-data that can be provided to the application through Session.getAttributes. Define a class to provide this data. 9

10 META-INF/services/uk.ac.lancs.scc.usmux.ServerSessionFactory Table 4: File holding registrations of server factories META-INF/services/uk.ac.lancs.scc.usmux.MetadataHandler Table 5: File holding registrations of meta-data handlers 10