External Data Representation (XDR) Prof. Chuan-Ming Liu Computer Science and Information Engineering National Taipei University of Technology Taipei, TAIWAN NTUT, TAIWAN 1
Introduction This chapter examines a de facto standard for external data representation and presentation as well as a set of library procedures used to perform data conversion This chapter describes the general motivations for using an external data representation and the details of one particular implementation NTUT, TAIWAN 2
Introduction (cont.) The next chapter shows how an external data representation standard helps simplify client and server communication It illustrates how a standard makes it possible to use a single, uniform remote access mechanism for client-server communication NTUT, TAIWAN 3
Representation of Data Fig. 20.1 shows little endian (if it stores the LSB of an integer at the lowest memory address) and big endian representation for 32-bit integers Programmers who create client and server software must contend with data representation because both endpoints must agree on the exact representation for all data sent across the communication channel between them; otherwise, conversion is required NTUT, TAIWAN 4
Figure 20.1 NTUT, TAIWAN 5
Asynchronous Conversion and the N-Squared Problem Designs that follow the paradigm of converting directly from one representation to the server s representation are said to use asymmetric data conversion because only one side needs to perform conversion NTUT, TAIWAN 6
Asynchronous Conversion and the N-Squared Problem If client-server software is designed to convert between the client s native data representation and the server s native data representation asymmetrically, the number of versions of the software grows as the square of the number of architectures (N- Squared Conversion Problem) NTUT, TAIWAN 7
Network Standard Byte Order The TCP/IP protocols use the same representation for all the data sent across a network in a protocol header (headers represent integer data in a standard, machine-independent form) Such designs employ symmetric data conversion NTUT, TAIWAN 8
Network Standard Byte Order (cont.) Symmetric data representation has an important consequence: only one version of protocol software is needed The standard representation used for data traversing the network is known as the external data representation NTUT, TAIWAN 9
Network Standard Byte Order (cont.) The chief advantage lies in flexibility: neither the client and the server needs to understand the architecture of the other The total programming effort required will be proportional to the number of machine architectures The chief disadvantage of symmetric conversion is computation overhead even if the client and the server both operate at the same architecture NTUT, TAIWAN 10
Network Standard Byte Order (cont.) Because the external representation may add information to the data or align it with word boundaries, the conversion may result in a larger stream of bytes than necessary Most programmers agree that using symmetric conversion is worthwhile NTUT, TAIWAN 11
Network Standard Byte Order (cont.) It simplifies programming, reduces errors, and increases interoperability among programs It also makes network management and debugging easier because the network manager can interpret the contents of packets without knowing the architectures of the sending and receiving machines NTUT, TAIWAN 12
A De Facto Standard External Data Representation XDR, Sun s external Data Representation has become a de facto standard for most client-server applications XDR specifies data formats for most of the data types that clients and servers exchange. For example, XDR specifies that 32-bit binary integers should be represented in big endian order NTUT, TAIWAN 13
XDR Data Types Fig. 20.2 lists the data types for which XDR defines a standard representation XDR allows the programmer to compose aggregate types from other types For example, XDR allows an array of structure, each of which can have multiple fields that can be an array, structure, or union Thus, XDR provides representation for most of the structure that a C programmer can specify NTUT, TAIWAN 14
Figure 20.2 NTUT, TAIWAN 15
Implicit Types The XDR standard specifies how a data object should be encoded for each of the data types listed in Fig. 20.2. However, the encodings contain only the data items and do not contain additional bits to identify their types. Thus, clients and servers using XDR must agree on the exact format of messages they will exchange. NTUT, TAIWAN 16
Software Support for Using XDR Programmers perform the conversions: Sending: insert a function call in the code to convert each data item in a message to external form Receiving: and insert a function call to convert each data item to internal form when a message arrives To eliminate potential conversion errors, an implementation of XDR includes library routines that perform the necessary conversions NTUT, TAIWAN 17
XDR Library Routines XDR library routines for a given machine can convert data items from the computer s native representation to the XDR standard representation and vice versa. Most implementations of XDR use a buffer paradigm that allows a programmer to create a complete message in XDR form. NTUT, TAIWAN 18
Building a Message One Piece at a Time Buffer paradigm allocate a buffer large enough to hold the external representation of a message and to add item (i.e., fields) one at a time In Linux, xdrmem_create allocates a buffer in a memory and inform XDR that it intends to compose an external representation in it initializes the memory so it represents an XDR stream that can be used to encode or decode NTUT, TAIWAN 19
Building a Message One Piece at a Time (cont.) The declarations and calls needed to create an XDR stream using C are shown in Fig. 20.2a Once a program has created an XDR stream, it can call individual XDR conversion routines to convert native data objects into external form For example, a program invokes xdr_int by passing it a pointer to an XDR stream and a pointer to an integer, as shown in Fig. 20.2b Fig. 20.3 illustrates how the call to xdr_int shown in the sample code adds 4 bytes of data to the XDR stream NTUT, TAIWAN 20
Figure 20.2a #include <rpc/xdr.h> //size of memory for encoding #define BUFSIZE 4000 XDR *xdrs; //pointer to an XDR stream char buf[bufsize];//memory area to hold XDR data xdrmem_create(xdrs, buf, BUFSIZE, XDR_ENCODE); NTUT, TAIWAN 21
Figure 20.2b int i; //integer in native representation //assume stream initialized for ENCODE i = 260; //assign integer value to be converted xdr_int(xdrs, &i); //convert integer and append to stream NTUT, TAIWAN 22
Figure 20.3 NTUT, TAIWAN 23
Conversion Routines in the XDR Library The table in Fig. 20.4 lists the XDR conversion routines The receiving application must calls xdrmem_create to create a memory buffer that will hold an XDR stream, and places the incoming message in a buffer area For example, if the receiver has established an XDR stream used for input (i.e., the call specified XDR_DECODE), it can extract a 32-bit integer and convert it to the native representation by calling xdr_int, as shown in Fig. 20.4 NTUT, TAIWAN 24
Figure 20.4 NTUT, TAIWAN 25
Conversion Routines in the XDR Library (cont.) Unlike the conversion, htons and ntohs that Linux provides, individual XDR conversion routines do not specify the direction of conversion. Instead, the direction is specified when creating the XDR stream. NTUT, TAIWAN 26
XDR Stream, I/O, and TCP The code fragments above create an XDR stream associated with a buffer in memory After items have been converted to external form and placed in the buffer, the application must call an I/O function like write to send it across a TCP connection It is possible to arrange to have XDR conversion routines send data across a TCP connection automatically each time they convert a data item to external form NTUT, TAIWAN 27
XDR Stream, I/O, and TCP (cont.) An application program first creates a TCP socket, and then calls fdopen to attach a standard I/O stream to the socket The application calls xdrstdio_create to create an XDR stream and connect it to the existing I/O descriptor Each time the application calls an XDR conversion routine, the conversion automatically performs a buffered read or write operation using the underlying descriptor NTUT, TAIWAN 28
Records, Record Boundaries, and Datagram I/O The XDR mechanism works well when connected to a TCP socket because both XDR and TCP implementations use the stream abstraction To make XDR work with UDP, the alterative design provides an application with a record-oriented interface To use the record-oriented interface, a program calls function xdrrec_create when creating a record-oriented XDR stream NTUT, TAIWAN 29
Records, Record Boundaries, and Datagram I/O (cont.) The call includes two arguments, inproc and outproc, that specify an input procedure and an output procedure When converting to external form, each conversion routine check the buffer If the buffer becomes full, the conversion routine calls outproc to send the existing buffer contents and make space for the new data NTUT, TAIWAN 30
Records, Record Boundaries, and Datagram I/O (cont.) Each time the application calls a conversion routine to convert from external form to the native representation, the routine checks the buffer to see if it contains data If the buffer is empty, the conversion routine calls inproc to obtain more data NTUT, TAIWAN 31
Records, Record Boundaries, and Datagram I/O (cont.) To use XDR with UDP, an application creates a record-oriented XDR stream It arranges for the input and output procedures associated with the stream to call read and write (or recv and send) Record-oriented streams allow an application to mark record boundaries NTUT, TAIWAN 32
Summary Client-server interaction can be asymmetric or symmetric Asymmetric conversion requires either the client or the server to convert between its own representation and the other machine s native representation Symmetric conversions uses a standard network representation and requires both the client and server to convert between the network standard and the local representation NTUT, TAIWAN 33
Summary (cont.) Client and server programs can use XDR routines to convert data to external form before sending it and to internal form after receiving it The conversion routine can be associated with input and output using TCP or UDP Although some protocols in the TCP/IP suite use ASN.1 (Abstract Syntax Notation One, by ISO) representation, most application programmers use XDR NTUT, TAIWAN 34