Assured and security Deep-Secure XDS An Extensible Structure for Trustworthy Document Content Verification Simon Wiseman CTO Deep- Secure 3 rd June 2013 This technical note describes the extensible Data Structure (XDS), which is a format specifically designed for presenting the business information found in all kinds of documents to the verification component of a high assurance guard. Document Content Verification Attackers are often able to gain control of a system by exploiting mistakes in the way applications handle unusual or malformed structures in documents or other data. So verifying that a document only contains structures that can be safely handled by applications is an important part of defending a system from attack. However, to be an effective defence, the verification system must itself be resistant to attacks involving malformed structures. This is difficult because most document formats are highly complex so it is hard to be sure that a verifier will work correctly under all conditions. Worse still, there are many document formats in common usage and a separate robust verifier will be required for each of them. Consequently it will take significant time to introduce support for new formats and the overall cost of this approach will be prohibitive. Converting all documents into a single common format before verification can reduce costs. Documents are first converted to the common format, the data is verified using a common verification component and then a new document is constructed in the appropriate format for delivery. This process is referred to as Transshipment. New formats can be supported without changing the security critical verification component, so this scales. However the solution is only effective if the format is simple enough to verify easily and flexible enough to handle the wide variety of information conveyed by common document formats. Deep- Secure have designed a format specifically to meet these goals. Called the extensible Data Structure (XDS), this is a way of encoding arbitrary structured data that is rich enough to be used to represent complex documents and yet simple enough that trustable software can be produced to check that an encoding adheres to some defined structures. Deep Secure is using XDS as an intermediate format in its next generation high assurance Transshipment Guards. The main complexity of a guard is in its 1 2013 Deep- Secure Ltd
protocol proxy software and the parsers and renderers for the complex data formats it must handle. To avoid this complex functionality becoming security critical, the parser and renderer are kept separate from the verifier. The parser converts the complex formats into an XDS representation that is handed to the security critical data verifier. If the verifier passes the data, it is then given to the renderer for conversion into an appropriate complex format needed for delivery. Essential Characteristics of a Common Format for Verification The common format needs to be capable of representing a wide range of documents, including word processing, spreadsheets, imagery, and structured data, so it must be extensible and general purpose. In a guard, the structure representing a document needs to be passed from source proxy to verifier and from verifier to destination proxy, but if the proxies and verifier share memory to hold the structure it is difficult to be sure they are unable to communicate in other ways. So the structure must be serialised to pass it as a byte stream from one component to another, which means it must be easy to produce trustable serialisation software for use within the security critical verifier. The verifier is security critical and so any configuration errors need to be trapped before any damage is done. Strong type enforcement within the verifier will achieve this, which means the data and the schemas that define what structures are acceptable need to support a variety of data types. The overriding requirement is for simplicity, since some security critical software will need to handle the common format. Is XML a Good Candidate? XML lacks the essential characteristics needed to act as a common format for trustworthy document content verification. Its main virtue is its extensibility, but in other regards it is problematic. The main issues are that XML, and its related toolset, is complicated to understand, use effectively and implement. Most of this complexity arises from it being serialised as a mark- up language, rather than a data structure, but its use of namespaces adds further complication. However the principle of using a tagged data structure for extensibility is sound and formed the basis of Deep- Secure s work to create XDS. Another major disadvantage of XML is that it only supports the string data type. This not only makes XML inefficient, because binary data must be encoded as text in some way, but it also means there is no opportunity for intrinsic type checking. The toolset associated with XML is also complex, in particular it is too difficult to assure the correctness of implementations of the path language and schema 2
definition languages for XML, making them unsuitable for use in a high assurance verifier. The XDS equivalents are similar but have been carefully designed to have simple well- defined semantics that can be implemented easily. So XML is not the candidate of choice for a common method of representing data for format verification. XDS Structures An XDS structure is made up of tags that logically form a tree 1. There are different types of tag, with all tags having a name and possibly some attributes. Tags are either: Empty, a Container, Text or Binary. Empty tags are leaf nodes in the tree and contain nothing. Container tags contain a, possibly empty, sequence of tags. Text tags contain a, possibly empty, Unicode text string. Binary tags contain a, possibly empty, byte sequence. Note that Empty tags, Container tags with an empty sequence of children, Text tags containing the empty string and Binary tags containing an empty sequence of bytes are all different and distinguished. There is no equivalent distinction in XML because of that language s mark- up roots. Each of a tag s attributes has a unique name and a typed value. The types are Unicode text string, binary (sequence of bytes), Boolean, unsigned integer (64 ), signed integer (64 ) and floating point numbers. Tags and attributes have simple case- sensitive names, with characters taken from the set A- Z, a- z, 0-9 and underscore. This limitation allows an implementation to avoid the complexities and expense of Unicode when handling these names. It is not expected that any application level string data will be encoded as tag or attribute names, rather it will be held as the values of attributes and Text tags. Representing an XDS Structure as Text Since XDS is a data structure capable of handling typed data it is difficult to show examples in a document such as this. Consequently a text encoding is also defined. This is primarily for use within documentation, but it could be used to create editable text representations of XDS structures used for configuration data or similar purposes. An XDS structure can be serialised to a sequence of either 7- bit ASCII characters or Unicode characters using one of the standard representations (UTF- 8, UTF- 16BE/LE, UTF- 32BE/LE) indicated by a Byte Order Mark. 1 Strictly XDS is defined in terms of an acyclic graph with a single root node, meaning that a tree that has common sub- trees need only store them once. 3
Empty tags are rendered as <TAG/> and Container tags as <TAG> </TAG>. Note that <TAG></TAG> is not equivalent to <TAG/> the former is a container tag with no children and the latter is an empty tag. Text tags are rendered as <TAG>text:text</TAG> and binary tags as <TAG>base64:binary</TAG>. Text tags may also be rendered as <TAG>text</TAG> where there is no ambiguity (that is, the text is not empty and does not start with a less- than character). With an ASCII encoding, characters in the text that have no representation in ASCII must be escaped using a hexadecimal representation of their Unicode character code, for example &20AC; must be used to represent the Euro character. Control characters, including tab, newline and carriage return, must also be escaped in all encodings. Similar escaping is used with Unicode encodings to represent the less- than character, in order to distinguish a less- than in the text from the less- than that terminates the text. Also, in any encoding, since ampersand is chosen as the herald of an escape sequence it must always be escaped as &26;. Character escaping is permitted even if not necessary, so for example &40; can be used to represent the @ character even though this is not necessary. Since tag and attribute names are from a simple character set these all render without escaping whatever the representation chosen for characters. The text inside text and binary tags may contain newlines that are ignored. Also, any leading or trailing whitespace surrounding the lines of text is ignored. Should any such whitespace be significant it must be escaped. Attributes are listed as name=value pairs after the tag name. String values are rendered as "string", Boolean values as true or false, unsigned integers as digits, signed integers as +digits or - digits and binary values as base64:binary. Each representation of the different types starts with a different set of characters, thus the type of the attribute s value can be determined from the first character of its representation. Below is the text representation of an example XDS structure: <DOC Width=320> <PARA>Title:&9;An Example Document &20;About XDS</PARA> <PARA>Author:&9;Deep-Secure</PARA> </DOC> Here the DOC tag is a Container and the two PARA tags are Text tags. The DOC tag s Width attribute is an unsigned integer. The first PARA tag contains text that is split across two lines. The newline and leading whitespace on the second line is ignored, but a space character before About is escaped as &20; and so is significant. The text also contains tab characters, escaped as &9;. 4
Comparison with XML In XDS an empty tag is distinguishable from the tag with no children. This is an important difference as it allows type checking to detect more errors when validating XDS against a schema and when evaluating path expressions. XML supports international characters in tag and attribute names, while XDS only allows simple ASCII alphanumeric names. This simplification allows implementations to be more efficient without needing to introduce complex mechanisms. It does not impact on the applicability of XDS as the names are intended to encode structure not application data. XDS attributes and values are typed, whereas XML only supports character strings. This not only makes the representation more efficient, by avoiding the need to store numeric values as strings, but also allows type checking to be effective. XML allows mixed content, where the sequence of elements contained in a tag can be a mixture of text and tags. The main problem with mixed content is that it makes the type system more complicated, as the type of a tag s element cannot be determined statically. It is relevant when XML is used as a mark- up language, as in XHTML, but is not a particularly useful construct in a data structure. XDS lacks any equivalent of XML namespaces. New attributes or new tags can be defined to extend structures, but different extensions may use the same names for different purposes. Thus XDS is not as easily extended as XML but the framework for representing arbitrary data formats can easily be defined to accommodate extensibility using attribute values, so there is no disadvantage here and the clear advantage is in the simplicity of the XDS design and implementation. Since XDS is a binary data structure rather than a text mark- up language it has no issues regarding the handling of whitespace, hence there is no equivalent of xml:space. XDS does not have the special control attributes xml:lang or xml:id as any information about language and any unique identifiers in a structure are part of the structure and represented using tags and attributes like any other data. XDS does not provide any equivalent of XML s CDATA construct, processing instructions or document type definitions. The textual representation of an XDS structure also differs from the way XML is represented. The character set used to represent XML is not known until part way through the document the charset attribute in the xml declaration partly governs the choice which complicates parsing. In XDS a Byte Order Mark at the start of the text always defines the encoding. 5
XML supports textual names, decimal values and hexadecimal values in escape sequences, whereas XDS only supports hexadecimal. This simplification means the parser for textual XDS is easier to test and represents no loss in capability. Leading and trailing whitespace is never significant in textual XDS, while it can be in XML and is a source of much error and confusion, and there is no equivalent of XML s CDATA sections. XDS attributes are typed and the representation of the value determines its type, whereas XML only supports the string type and schemas then impose constraints on the strings to give them a type. Comments in the textual representation of XDS are shown as <!>...</!>, while in XML they are <!- - - - >. XDS Binary Serialisation Applications are free to serialise XDS in any way they see fit, but a standard binary serialisation that represents an XDS structure as a byte stream is defined to allow independently developed system components to pass XDS between each another. All integers are serialised in Little Endian format, rather than Big Endian, to reflect the dominance of Intel processors. All characters are represented in Unicode using a 32 bit integer, despite Unicode only requiring 21 bits. This is on the assumption that the receiving application will represent strings as arrays of 32 bit integers to keep string processing simple. As it is common for an XDS structure to use tag and attribute names many times, the names are represented by 4 byte integers in the serialisation. The mapping table that translates the integers to the names is either known a priori to the sender and receiver or is sent once at the start of the structure. Tags are represented by the number of their name, a counted list of attributes, a one byte type code indicating what type of tag they are and the serialisation of the tag s contents, if any. Attributes are represented by the number of their name, a one byte type code indicating the type of the attribute s value and the value itself. The content of Text tags is a counted sequence of Unicode characters, while for Binary tags it is a counted sequence of bytes and for Container tags it is a counted list of Tags. Empty tags have no content. 6
The example discussed previously would be serialised as follows: (1) number of attribute names Width attribute name 1 in ASCII (2) number of tag names DOC tag name 1 in ASCII PARA tag name 2 in ASCII (1) DOC tag (1) attr count (1) Width attr U Unsigned type indicator 320 Attr value (unsigned integer) C Container tag indicator (2) count of child tags (2) PARA tag (0) attr count (no attributes) T Text tag indicator (36)Title:(tab)An Example Document About XDS (2) PARA tag (0) attr count (no attributes) T Text tag indicator (19)Author:(tab)Deep-Secure Canonical Representation Neither the binary nor textual representations of XDS are suitable for generating hashes that uniquely identify structures, because both are capable of representing the same document in different ways. Consequently a canonical form of the binary serialisation is defined that adds additional constraints that mean it is only possible to represent an XDS structure in one way. In the canonical form the numeric identifiers are allocated to tag and attribute names in alphabetical order and a tag s set of attributes are ordered into a sequence by their name. With these additional constraints there becomes only one way of representing an XDS structure as a sequence of bytes and hence hashes can be generated to uniquely identify a particular XDS structure. XDSPath XDSPath is a language for calculating values based on an XDS structure. An XDSPath expression defines how a sequence of tags or sequence of scalar values is to be derived from an XDS tag and some context. Superficially, XDSPath is very much like XML s XPath, but it dispenses with the notion of axes, arranges results as sequences not sets and is a strongly typed expression language. The simplest path expression is the name of a tag. Given a container tag this produces the sub- sequence of the tag s child tags that have the given name. For example, the expression PARA applied to the example document above will return a sequence of two PARA tags. Arithmetic expressions produce a scalar value given a tag. The expression can calculate a result from the values of the given tag s attributes using the usual arithmetic and string operators. For example, the expression @#Width returns 7
the unsigned integer value of the given tag s Width attribute. If the attribute has a different type the expression is invalid, but if the tag does not have an attribute with this name the result is the special Null value. If the example expression is applied to the example document above it returns a sequence of one unsigned integer whose value is 320. Two path expressions can be combined, using the / operator, so that the second is evaluated with each of the first s results in turn. The resulting sequence- of- sequences is concatenated to produce a single sequence as the overall result. For example, the expression PARA/text() applied to the example structure given above returns a sequence of two Unicode strings: Title:(tab)An Example Document About XDS and Author:(tab)Deep- Secure. A path expression can also be used to filter the results of another, using the syntax path[filter- path]. The filter is evaluated with each of the tags produced by the first path in turn to produce a Boolean value. The overall result is the sub- sequence of the tags produced by the first path for which the second path evaluated to True. For example, the expression PARA[length(text())>20]/text() applied to the example structure given above returns a sequence of one string Title:(tab)An Example Document About XDS. The XDS- Path expression language also supports parameters and external functions, and has many advanced features similar to those in XPath2, but the language design means it has clean simple semantics and can be implemented simply and efficiently. XDS- Schema XDS- Schema is a written language, based on regular expression syntax, for defining a set of conforming XDS structures. It serves the same purpose as XML Schema does for XML, but is more compact and readable. An XDS- Schema is a set of grammar rules that describe all XDS structures that conform to the specification. Each rule has a discrimination part that describes the name and attributes of a conforming tag. For content tags there are additional grammar rules that describe the structure of the tag s content. It is also possible to attach arbitrary XDSPath constraints to the discrimination part of a grammar rule. The path condition is evaluated against the tag and determines whether the rule applies. If a schema contains a choice rule, the discrimination part of each choice is evaluated to determine which choice to take. If no choices match, the structure does not conform to the schema. If more than one choice matches, the input data structure is considered ambiguous and non- conformant to the schema. This means the schema validator ignores content when considering which choice applies, but path conditions can be used to guide the validation explicitly if this is required. 8
The following schema is given as an example. The sample data structure shown above conforms to this schema. # Example schema main = TAG DOC : container, ATTR Width : uint / para*; para = TAG PARA : text; XDS- Transform Transformations to be applied to XDS data structures can be defined using the XDS- Transform language. This is an XDS structure that declaratively describes the transformation of one XDS document into another. An XDS- Transform consists of a list of templates that are selected by the input document tags as they are encountered. Each template describes an XDS fragment that is created in the destination document and directs the transformation of subsequently selected input tags. The XDS fragment description composes the transformed text, tag, attribute and binary objects and by copying sections of the input document. XDSPath is used throughout XDS- Transform for selecting and filtering the input document into the output document. The following template example changes the PARA tag's name of the earlier example whilst keeping the textual content the same: <TEMPLATE match="para"> <TAG name="mypara"> <COPYOF select="./text()" /> </TAG> <TEMPLATE> This would output something like: <DOC Width=320> <MYPARA>Title:&9;An Example Document &20;About XDS</MYPARA> <MYPARA>Author:&9;Deep-Secure</MYPARA> </DOC> Note that the select and match attribute are XDSPath based. Many places where literal values are used, such as the name attribute in the template example above, can be replaced with quick selector by using the '?' character as the first letter. Quick selectors generate literal values as the result of an XDSPath expression evaluated against the input document. The XDSPath expression is placed after the '?' character. Some limited flow control is also supplied by the IF, FOREACH and CHOOSE constructs. The operands for these also use XDS- Path expressions evaluated over the input. To attain high assurance in a verifier it must be kept simple, so it is unlikely that XDS- Transform will be used in such a verifier. However the sub- systems that 9
surround the verifier may well need to apply transformations to XDS data. For example a parser may extract all possible information from an input document, but in a particular deployment only a subset of the data may be required or permitted to pass through the guard. The parser could be made configurable as to what data to include, but this complicates its implementation and will never be fully general in the options it offers. The alternative is to apply a transformation after parsing to trim the data back to that needed, and this is one role of XDS- Transform. Summary The extensible Data Structure (XDS) has been devised as a means of representing the information found in all kinds of documents in a way that means simple software can verify its structure. XDS is similar to XML but there are significant differences, in particular the use of strong typing. Two languages accompany the data structure definition, XDSPath for searching XDS structures and calculating values based on the data and XDS- Transform for defining transformations from one structure to another. 10