ASN.1/PL. ASN.1 Type Definition Module. Source Code. ASN.1 Database System

Transcription

1 A Database System Based on ASN.1: Its System Architecture and Database Programming Language 3 Kaname HARUMOTO y Masahiko TSUKAMOTO z Shojiro NISHIO y y Department of Information Systems Engineering z Information Technology Research Laboratories Faculty of Engineering, Osaka University SHARP Corporation 2-1 Yamadaoka, Suita 565, Japan Ichinomoto, Tenri 632, Japan fharumoto,nishiog@ise.eng.osaka-u.ac.jp tuka@shpcsl.sharp.co.jp Abstract This paper proposes a database system based on ASN.1 (Abstract Syntax Notation One). Aiming at excluding the ambiguity of data representation in the heterogeneous computer environments, ASN.1 notation as well as its encoding rules is used in many advanced applications including those in communication networks. As the amount of data described by ASN.1 becomes very large, it is eagerly required to store and reuse data with ASN.1 types. However, the type denition capability of ASN.1 is too strong for the conventional database systems to represent straightforwardly the data specied by ASN.1, which makes it dicult to store and handle data described by ASN.1 directly. In this paper, we describe the system architecture and its database programming language of the ASN.1 database system, which we have designed and implemented to resolve the above diculty. 1 Introduction ASN.1 (Abstract Syntax Notation One) is the general term for two international standards specied by ISO (International Organization for Standardization), that is, the ASN.1 notation[1] and the encoding rules for ASN.1[2]. The ASN.1 notation is used for dening complex data structures exchanged in the environment of heterogeneous computer systems, and the encoding rules for ASN.1 specify how to represent data structures exchanged by application entities of the OSI 3 This researchwas supported in part by the Ohkawa Institute of Information and Telecommunication. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ADTI copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of ADTI Organizing Committee. To copy otherwise, or to republish, requires a fee and/or special permission. International Symposium on Advanced Database Technologies and Their Integration (ADTI'94) Nara, Japan, October 1994 (Open Systems Interconnection) basic reference model so that any computer system can translate the representation of the data structures without ambiguity into their own internal representation. Hence, ASN.1 can be considered as the data structure denition language, and was designed to have enough capability to describe data structures of all sorts of applications. As a result, it is used not only in OSI applications but also in a wide variety of application areas. Among suchapplications are: OSI applications such as MHS (Message Handling System), Directory, FTAM (File Transfer, Access and Management), and ODA (Open Document Architecture), OSI lower layer protocols, especially for their managed objects, such as IS-IS (Intermediate SystemtoIntermediate System Intra-Domain Routeing Information Exchange Protocol) and IDRP (Inter-Domain Routeing Protocol), TCP/IP applications such as SNMP (Simple Network Management Protocol), multimedia applications such as those using the MHEG (Multimedia and Hypermedia coding Expert Group) standard[10], and genome databases (see, e.g., [4]). As the application areas of ASN.1 have become remarkably large, there is a growing demand for storing ASN.1 data in the secondary storage for sharing and reusing these data sources. For example, MHS requires storages for exchanged messages, and DIB (Directory Information Base) stores information oered by DSA (Directory Service Agent) in the OSI Directory system. In implementing such data storage, if we employ directly one of the conventional database management

2 systems (DBMSs), general purpose programming languages such as C language are usually used together. However, in a practical sense, several deciencies are there in developing applications as described in the following: 1. Application software developers have to transform agiven ASN.1 type denition into a schema denition in the employed database[5]. Although a high performance implementation technique for Directory systems is shown in [8], such transformation process diers in each application software for improving its performance. Therefore, the transformation technique developed for an application cannot be generally reused for other applications. 2. If a general purpose programming language is used to implement the applications based on ASN.1, it is required to transform ASN.1 type denitions into data structure descriptions peculiar to the language. For this requirement, ASN.1 compilers, such as snacc[11] and APRICOT [7], which automatically transform data structures, are widely used in developing the applications of OSI communication software. However, the transformed denitions of the data structures usually become more complex than the original ASN.1 type denitions, thereby the application programmers have to grasp those complex structures while describing program codes. This decreases the ef- ciency of development of application programs. 3. We are involved in one of the most serious problems in database programming languages; an impedance mismatch between database query languages and application description languages regarding their description capabilities. The mismatch is principally due to the fact that the former is designed considering set-oriented operations, while the latter is designed for operations executed on singleton objects. Therefore, this mismatch makes it dicult to perform typechecking beforehand, and consequently run-time errors of applications frequently occur, which makes the application programs unreliable. In this paper, we discuss the architecture of a new database system called the ASN.1 database system, in which the data with the structures dened by ASN.1 are stored. We also propose a database programming language which directly handles ASN.1 data in the database. This ASN.1 database system resolves the above problems in the following ways: In the ASN.1 database system, data type denitions described by ASN.1 are directly used as the database schema denitions. Therefore, application programmers are not involved in the problem of transformation from ASN.1 type denitions to another form of database schema denitions. Our proposing database programming language can handle directly any data of arbitrary ASN.1 types. Using this language, in addition to the computation executed by usual programming languages, complicated operations including database operations or data transfer operations can be easily specied. Because the type system in the environment for developing applications is unied to ASN.1, the impedance mismatch problem is resolved. Therefore, strict type checking can be done at the time of compilation of application programs, and we can mostly avoid the errors during the execution of applications. In the following, we outline the ASN.1 standards in section 2, and then discuss the architecture of our proposing ASN.1 database system in section 3. In section 4, we describe a database programming language for our system. Our current implementation of the ASN.1 database system is shown in section 5. Finally, we summarize this paper in section 6. 2 Outline of ASN.1 ASN.1 is a data structure description language[1] with its basic encoding rules (BER)[2] standardized byiso. ASN.1 is originally designed for absorbing the dierence of data representations in heterogeneous computer environments and has been used to dene data structures of application protocol data units (APDUs) exchanged among OSI application entities. The notation of data structures dened by employing ASN.1 is called abstract syntax. BER is used as a transfer syntax for encoding ASN.1 data structures into the corresponding bit stream for transmission. Because encoded bit streams do not depend on a specic computer architecture or a specic programming language, their receivers can decode them into their own internal representations for data without ambiguity. A set of ASN.1 type denitions which are relevant to a specic application protocol is called a type denition module or simply a module. Using built-in types oered by ASN.1, types of data exchanged between applications are dened in a module. The built-in types of ASN.1 are divided into two categories; simple types and structured types. Simple types are used as primitive types in constructing more complex types, which include the following types. BOOLEAN: The values of this type are TRUE or FALSE.

3 INTEGER: The values of this type are arbitrary integers. REAL: The values of this type are arbitrary real numbers. ENUMERATED: The values of this type are symbolic constants associated with distinct integers. BIT STRING: The values of this type are arbitrarily long strings of bits. OCTET STRING: The values of this type are arbitrarily long strings of octets. OBJECT IDENTIFIER: The values of this type are sequences of integers. Each value of this type identies a certain object in the world. Structured types are constructed by using type constructors which combine simple or user-dened types. Data types with more complex structures can be de- ned by using these type constructors repeatedly. Type constructors include the following ones: SEQUENCE: Eachvalue of this type is an ordered sequence of values, one from each of the specied types. SET: Each value of this type is an unordered set of values, one from each of the specied types. SEQUENCE OF: Each value of this type is an ordered sequence of zero, one or more values of the specied type. SET OF: Each value of this type is an unordered set of zero, one or more values of the specied type. CHOICE: Each value of this type is a value of the type which is chosen from the specied types. Further, ASN.1 provides the almightytype \ANY," each of whose values belongs to an arbitrary ASN.1 type. ANY and CHOICE are often called meta types, because values of them do not have a certain xed type. Thus, using these meta types, heterogeneous data structures can be expressed in a single type. Meta types are one of the most prominent features of ASN.1 as compared with the conventional programming languages such as C and Pascal where such types cannot be represented straightforwardly. Figure 1 shows an example of a module denition, which generally denes data structures of the information on a person. The type Person is a SEQUENCE type whose elds are named name, age, sex, and belongto, respectively. The type VisibleString is PersonalRecords DEFINITIONS ::= BEGIN Person ::= SEQUENCE { name VisibleString, age INTEGER, sex ENUMERATED { male(0), female(1) }, belongto BelongTo } BelongTo ::= CHOICE { univ [0] University, comp [1] Company } University ::= SEQUENCE { name VisibleString, faculty VisibleString, department VisibleString } Company ::= SEQUENCE { name VisibleString, department VisibleString } Figure 1: An example of ASN.1 module. a built-in character string type of ASN.1. Note that the eld belongto has the type BelongTo, which is also dened as a CHOICE type in the module, whose actual type is either University or Company both of which are also dened in the module. 3 Architecture of ASN.1 Database System In the conventional declarative query languages such as SQL, it is easy to describe database operations because they do not require users to specify how toaccess the database, while their computation power is not sucient to specify other modules of application programs, such as user interface modules. Therefore, the current practice in database programming is to use a query language embedded in a host language which has enough computation power. If database programs for ASN.1 applications are described in this way, then there are several problems in developing these applications given as follows: There are three dierent data formats in the program: the external format on wire (i.e., encoded bit streams), the internal format used by the host language, and the format in the database system. This enforces application programmers to manage the conversions among these formats in the program. We cannot avoid an impedance mismatch between the host language and the query language in which database operations are described. This is due to

4 ASN.1 Type Definition Module schema ASN.1 Database System ASN.1/PL Source Code application 4 ASN.1 Database Programming Language In this section, we discuss our proposing database programming language ASN.1/PL for the ASN.1 database system mainly concerning its design issues, syntax, and type checking facility. 4.1 Design Issues of ASN.1/PL Figure 2: Architecture of ASN.1 database system. the fact that a type available in the host language is not always compatible with a certain type available in the database system. Thus, an access to the database should be described by query processing function calls with a query expression as an argument, or simply by embedding query expressions in the host language. In both cases, application programmers are required to be familiar with both query and host languages. Moreover, since the type checking mechanisms of a compiler of the host language will not work enough on such query expressions, most of potential syntax errors in a query expression cannot be found at the time of compilation. Even if there are no syntax errors, it is very hard to determine which type should be coerced to each query result. These errors may reveal themselves during the run-time of the application, which reduce the eciency of the development and the reliability of the application programs. These problems originate in the fact that dierent type systems are there in an application development environment. Aiming at resolving such deciency, we designed the ASN.1 database system based on the principle that the type system in the environment for developing ASN.1 applications should be uni- ed to ASN.1. The architecture of the system is illustrated in Figure 2. The ASN.1 database system handles ASN.1 type denition modules as its schema denitions, and processes source codes written in ASN.1/PL as application programs, where ASN.1/PL is a database programming language explained more precisely in the next section. Because the language ASN.1/PL directly handles the type system of ASN.1, the problems described above are resolved, and we can improve the eciency of application developments as well as the reliability of developed applications. As was described in the previous section, ASN.1/PL directly handles the ASN.1's type system, whichmakes programmers possible to write codes on the level of abstract syntax. Therefore, they are not required to care about how the data structures are represented internally or externally. Once the type system of the application development environment is unied, strict type checking can be done at the time of compilation of application programs. However, the type system of ASN.1 has its own peculiar property, i.e., meta types. This makes it dicult to check types in a straightforward way. Fortunately, Machiavelli[3, 9], which is the database programming language derived from language ML[6], can handle the type system including types which have compatible expressiveness with the meta types of ASN.1. Therefore we employmachiavelli's type checking algorithms so that we can check types of ASN.1/PL programs strictly. On the other hand, polymorphic languages such as Machiavelli are considered to be very useful for database programming[9], especially when data handled by applications have complex structures. As ASN.1 types usually have complicated nested structures in their practical applications, the notion of polymorphism is considered to be convenient in developing complex applications using ASN.1. Therefore, we introduce the notion of polymorphism into the language design of ASN.1/PL. 4.2 Syntax of ASN.1/PL To check types strictly, ASN.1/PL has been designed as a functional language and most of syntactical features are based on Machiavelli. In the following subsections, we describe the syntax of ASN.1/PL Types Built-in types of ASN.1 and the types dened in ASN.1 type denition modules are handled by ASN.1/PL as full-edged types. Furthermore, programmers can also dene new types in ASN.1/PL programs. Types are described by using their type names.

5 4.2.2 Values Any value handled by ASN.1/PL should belong to a certain ASN.1 type. Values are described in the ASN.1 value notations Variables In ASN.1/PL, a variable is used as a reference to a certain value, and belongs to the type of the value. Variables must be declared before their use in the following syntax: val var name [: type] =expression The variable expressed by var name of the type type is initialized to the value obtained by evaluating expression. Because the type can be inferred from the expression on the right side of the declaration, the type specication can be omitted. Avalue referred byavariable can be updated in the following update expression: var name := expression When this update expression is evaluated, a value which is referred to by the variable var name is updated to an evaluated value of the expression. If a value to which the variable refers is stored in the database, the update of the variable implies the update of the value in the database, which will generate a relevant transaction processing. Atype name of ASN.1, which appears in the place where an expression is expected to be described other than a type specication, is regarded as a predened variable which refers to the extent ofthistype Functions In ASN.1/PL, a program consists of declarations of functions, where a function declaration is described in the following syntax: fun fun name ([arg1 [: type1],...]) [: type] = expression This expression denes the function expressed by fun name with arguments arg1,..., whose evaluation is dened by expression. Here, arg1 represents a temporary argument of declaring function, type1 represents its type specication, and type represents the type specication of the result value. Note that type specications on temporary arguments and the return value can be omitted if they can be inferred from an expression of the body of the function. In this way, it is possible to declare polymorphic functions. For example, consider the function nameof declared as follows. fun nameof(x) = x.name This function is applicable to an argument whose type is among types SEQUENCE, SET, and CHOICE, each of which includes a name eld. Also, the type of its return value can vary according to the type of the name eld. An application of a function is described in the following syntax: fun name ([expression,...]) An evaluation of this statement invokes the function fun name accompanied with the arguments which are obtained by evaluating the subsequent expressions Expressions Values, variables, and function applications are expressions. In addition, the following computations can be described as expressions. (a) Operators. The operator \=", which evaluates the equivalence of values of two expressions, can be applied to the values of any type. Simple types of ASN.1 have their fundamental operators such as those for comparison, arithmetic, etc. For SE- QUENCE OF type and SET OF type, whose values respectively represent a list and a set, the operator \in" is prepared for evaluating if a speci- ed value is included in the list or the set. As the peculiar operator for CHOICE type, \isa" is prepared for evaluating if a specied type is chosen. (b) Field selections. Field selections on nested structures which are constructed by using SE- QUENCE, SET, and CHOICE are described by the following dot notation: expression. eld name This expression indicates a selection of a eld expressed by eld name from the value evaluated by expression which belongs to a type constructed by using one of the constructors SEQUENCE, SET, and CHOICE. (c) Conditional expressions. There are two types of conditional expressions, i.e., if expression and case expression. The syntax of if expression is given as follows: if expression then expression1 else expression2 An evaluation of this expression invokes the evaluation of expression1 if expression is evaluated to be TRUE; otherwise, invokes the evaluation of expression2. Note that expression1 and expression2 must be evaluated to have the same type. The syntax of case expression is given as follows:

6 case expression of [rel op1 ] expression1 => expression2, [rel op2 ] expression3 => expression4,... else => expression0 end The specications of rel op1, rel op2,... indicate relative operators including in and isa. If such a specication as rel op1, rel op2,... is omitted, then it is assumed that the operator \=" isspeci- ed in the corresponding place. Note that expression2, expression4,..., and expression0 must be evaluated to have the same type. This expression is indeed the abbreviation of the following expression: if expression rel op1 expression1 then expression2 else if expression rel op2 expression3 then expression4 else if... else expression0 (d) Iterative expressions. of iterative expressions: foreach expression. There are two types while expression and The syntax of while expression is given as follows: while expression1 do expression2 This expression indicates the repeated evaluation of expression2 while expression1 is evaluated to be TRUE. The syntax of foreach expression is given as follows: foreach (l var, expression1 ) expression2 This expression indicates to evaluate the expression2 repeatedly by making the loop variable l var refer one by one to an elementofexpression1 with SEQUENCE OF type or SET OF type. (e) Structured query expressions. ASN.1/PL adopts structured query expressions not as embedded ones but as full-edged ones, i.e., they are included in the syntax of ASN.1/PL. The syntax is given as follows: select expression1 [,...] from var name <- expression2 [,...] where expression3 Each of expressions in the from clause such as expression2 represents a target value of the query whose type is either SEQUENCE OF type or SET OF type, and var name indicates a temporary variable which refers one by one to an element of the target value. A condition for selecting elements out of target values is represented by expression3 in the where clause. Expressions in the select clause indicate a list of desired values. Note that in the select clause, any expression including function applications can be written. This feature comes from the adoption of structured query expressions as a part of the syntax of ASN.1/PL. This expression is evaluated to be a value of type SET OF. The type equals to the type of expression1 if the select clause has only one expression; otherwise, is SEQUENCE such that the nth eld has the type of the nth expression of expressions listed in the select clause Built-in Functions ASN.1/PL provides several types of built-in functions, including those for (1) communication such as send, receive, asn1 encode, and asn1 decode, (2) interaction with users such as input line and output, (3) duplicating values such as copy, and (4) converting the lifetime of values such as persistent copy and transient copy. 4.3 Example of ASN.1/PL Program An example program of ASN.1/PL is shown in Figure 3. This program is to retrieve data from the database, assuming that data on persons with ASN.1 types given in Figure 1 have already been stored. According to a command input by a user, it works as follows: If the command is list, then it lists up the names of all persons in the database. If the command is univ or comp, then it lists up the names of persons in the database each of whom belongs respectively to a university or a company. If the command is quit, then it quits the program execution. Otherwise, it prints all information about persons whose names are equal to the name given as the command. 4.4 Type inference mechanisms in ASN.1/PL To check strictly the types of the programs written in ASN.1/PL, the compiler employs the type inference algorithms which are originally provided in Machiavelli[3, 9]. Machiavelli is a polymorphic language which extends type inference algorithms of ML

7 fun nameof(x) = x.name; fun univpersons() = select x from x <- Person where x.belongto isa University; fun comppersons() = select x from x <- Person where x.belongto isa Company; fun whois(name) = select x from x <- Person where nameof(x) = name; fun main() = { val command = input_line(std_in); while not command = "quit" do { case command of "list" => foreach(p, Person) output(nameof(p)), "univ" => foreach(p, univpersons()) output(nameof(p)), "comp" => foreach(p, comppersons()) output(nameof(p)), else => foreach(p, whois(command)) output(p) end; command := input_line(std_in); } } Figure 3: An example of ASN.1/PL program. to make it appropriate for the database programming, especially with respect to eld selection. ASN.1 types are mapped into Machiavelli types in the following way: simple types : simple types SEQUENCE, SET : the record type SEQUENCE OF, SET OF : the set type CHIOCE : the selection type ANY : the dynamic type Further, we can easily nd the corresponding expression of Machiavelli to any ASN.1/PL expression which is possibly constructed by using variables, functions, eld selections, structured queries, and so on. Thus, we can make use of Machiavelli type inference algorithms for ASN.1/PL type inference mechanisms. By using these type inference algorithms, the types of all expressions and functions in a program are inferred. For example, the types of the argument and the return value of the function nameof in Figure 3 can be inferred from its body. That is, the typeofitsargument x is one of Person, University, and Company, and the type of the returned value is VisibleString. The type checking can be performed using such inferred type information. 5 Implementation of ASN.1 Database System In this section, we show the implementation of the ASN.1 database system. At the nal stage of our implementation of the system, its complete components are planned to be developed by ourselves. However, at the current stage, a commercially available C++based OODBMS (Object-Oriented Database Management System) ObjectStore 1 is employed just for convenience as a system component of the data storage. As compared with the conventional database management systems such as relational database systems, the OODBMS can represent to some extent the data structures of ASN.1 such as nested structure. Figure 4 illustrates the implementation of the system, where the system components in the gure are as follows: ASN.1 Compiler: A converter from an ASN.1 type denition module to a schema for OODBMS, i.e., C++ class denitions. PLtoC++ Compiler: A converter from an ASN.1/PL source code to a C++ source code. Compiler & Linker: The C++ compiler and linker oered by the OODBMS. Our ASN.1 compiler has been implemented by extending the existent ASN.1 compiler snacc. Snacc can transform types dened in ASN.1 modules into the corresponding C++ class denitions. In these classes, some methods including those for encoding and decoding are automatically generated. However, it does not designed for handling persistent data. We have extended it in some respects so that the transformed C++ class denitions can be used as a database schema. The PLtoC++ compiler has two functions. First, the compiler checks the existence of potential type errors in a given ASN.1/PL source code with the help of the type inference algorithms described in the previous section. If no type errors are detected, then the compiler converts the source code into a C++ source code. The output of these two compilers is compiled to an executable application by the C++ compiler which is oered by the OODBMS. In this way, in spite of using 1 ObjectStore is the trademark of OODBMS developed by Object Design, Inc.

8 ASN.1 Type Definition Module ASN.1 Compiler C++ Class Definitions Compiler & Linker ASN.1/PL Source Code PLtoC++ Compiler C++ Source Code Application OODB ASN.1 Database System Figure 4: Implementation of ASN.1 database system. the OODBMS, the programmers are not required to take account of how ASN.1 data are actually stored in the OODB. 6 Conclusions In this paper, we proposed the ASN.1 database system which employs ASN.1 modules as database schema, and the database programming language ASN.1/PL which handles ASN.1 types as its primitive types. Since ASN.1/PL programmers are not required to be conscious of transformation of data structures, it becomes easy for them to develop database applications whose data types are described by ASN.1. Furthermore, the impedance mismatch between the host language and the database system can be remarkably reduced. As a result, static type checking can be employed for application programmers to handle easily complicated data structures. Consequently, ASN.1 applications can be eciently and easily developed using ASN.1/PL for the ASN.1 database. References [1] ISO IS 8824, Information processing systems Open Systems Interconnection Specication of Abstract Syntax Notation One (ASN.1), [2] ISO IS 8825, Information processing systems Open Systems Interconnection Specication of Basic Encoding Rules for Abstract Syntax Notation One (ASN.1), [3] Buneman, O.P. and Ohori, A.: \Polymorphism and Type Inference in Database Programming", ACM Transactions on Database Systems (to appear). [4] Couteau, J.: \Genome Databases", Science, Vol. 254, pp. 201{207, [5] Masunaga, Y.: \Consideration of Standard Database Storage Structure of Multimedia Documents" (in Japanese), IPSJ SIG Notes, Vol. 93, No. 45 (93-DBS-93, 93-OS-59), pp. 83{92, [6] Milner, R., Tofte, M. and Harper, R.: The Denition of Standard ML, The MIT Press, [7] Nakakawaji, T., Katsuyama, K., Miyauchi, N., and Mizuno, T.: \Development and Evaluation of APRICOT (Tools for Abstract Syntax Notation One)", Proceeding of the 2nd international symposium on interoperable information systems, INTAP, pp. 55{62, [8] Nishiyama, S., Obana, S., Horiuchi, H., and Suzuki, K.: \Design of DBMS for High Performance OSI Directory Based on Extensible DBMS Technique and Its Evaluation" (in Japanese), Transaction of Information Processing Society of Japan, Vol. 34, No. 6, pp , [9] Ohori, A., Buneman, O.P., and Breazu-Tannen, V.: \Database Programming in Machiavelli: a Polymorphic Language with Static Type Inference", Proceedings of ACM-SIGMOD conference '89, pp. 46{57, [10] Price, R.: \MHEG: An Introduction to the future International Standard for Hypermedia Object Interchange", Proceedings of ACM Multimedia 93, pp. 121{128, [11] Sample, M.: Snacc 1.1: A High Performance ASN.1 to C/C++ Compiler, snacc reference manual, 1993.