Modeling, authoring and formatting hypermedia documents in the HyperProp system

Transcription

1 Multimedia Systems 8: (2000) Multimedia Systems c Springer-Verlag 2000 Modeling, authoring and formatting hypermedia documents in the HyperProp system Luiz Fernando G. Soares, Rogério F. Rodrigues, Débora C. Muchaluat Saade Departamento de Informática, PUC-Rio, R. Marquês de São Vicente 225, Rio de Janeiro, Brazil; lfgs@inf.puc-rio.br;{rogerio,debora}@telemidia.puc-rio.br SPRINGER-VERLAG, (2000). This is the author's version of the work. It is posted here by permission of Springer-Verlag for your personal use. Not for redistribution. The definitive version was published in in Multimedia Systems, {VOL8, ISS , (03/2000)}. The original publication is available at Abstract. This paper discusses multimedia and hypermedia modeling, authoring and formatting tools, presenting the proposals of the HyperProp system and comparing them to related work. It also highlights several research challenges that still need to be addressed. Moreover, it stresses the importance of document logical structuring and considers the use of compositions in order to represent context relations, synchronization relations, derivation relations and task relations in hypermedia systems. It discusses temporal and spatial synchronization among multimedia objects and briefly presents the HyperProp graphical authoring and formatting tools. Integration between the proposed system and the WWW is also addressed. Key words: Nested context model HyperProp system Multimedia/hypermedia formatting Document logical structuring Spatio-temporal synchronization 1 Introduction Hypermedia systems have been addressed in three different levels in the literature: storage, specification (authoring) and execution (formatting). This paper focuses mainly on authoring and formatting. The specification level aims at defining hypermedia application requirements and associated constraints. The formatting level refers to the development of protocols and schemes for document presentation, dealing with intra-media temporal synchronization, and inter-media temporal and spatial synchronization. Hypermedia systems may, and sometimes have to, introduce several desirable facilities, such as: allowing a structured authoring; allowing the explicit definition of temporal and spatial relationships among their components (including those with non-deterministic times); supporting a rich set of capabilities for each media component (e.g., presentation duration flexibility, alternatives for different presentations, behavior changes during presentation, etc.); supporting relations anchoring on internal points of their components (fine Correspondence to: L.F.G. Soares granularity), and not only on its start or end points (coarse granularity); allowing quality-of-service (QoS) definition required by media components (e.g., jitters, bandwidth, etc.); and allowing a presentation environment description (e.g., network delay, supported devices, etc.). In order to achieve these goals, hypermedia systems must be supported by powerful conceptual models. The greater the expressiveness of a model, the greater the document authoring complexity will be; authoring that must allow the complete document specification as easily (or friendly) as possible. Similarly, the greater the complexity of the spatiotemporal (spatial and temporal) formatting algorithm will be. The formatting algorithm must consider all the environment non-determinism and allow correcting the presentation on the fly, when unpredictable events occur (such as network delay, user interaction, etc.). It must also take advantage of media segment QoS specifications and environment characteristics in order to trigger some actions, for instance, doing pre-fetch of the content of objects. In this paper, we consider the development of multimedia and hypermedia authoring and formatting tools, examining the current state of the art. All the aforementioned facilities are discussed. Solutions proposed in the Hyper- Prop system are presented and compared with related work throughout the text. Moreover, we discuss a set of research challenges that need to be addressed before the full potential of multimedia output technology can effectively be used for sharing information. The paper is organized as follows. In Sect. 2, general issues related to hypermedia document authoring are discussed, stressing the importance of document logical structuring, and showing how to use compositions to support them. Section 3 presents the HyperProp hypermedia conceptual model and discusses how it addresses requirements for authoring documents with spatio-temporal constraints. Section 4 discusses several issues and solutions for defining hypermedia application requirements and associated constraints during authoring and formatting phases. Authoring tools with graphical interfaces make the task of document specification easier. The HyperProp solution for a graphical editor is introduced in Sect. 5. Section 6 describes the temporal and spatial-formatting machine of the system.

2 Multimedia Systems 8: (2000) Multimedia Systems c Springer-Verlag 2000 Modeling, authoring and formatting hypermedia documents in the HyperProp system Luiz Fernando G. Soares, Rogério F. Rodrigues, Débora C. Muchaluat Saade Departamento de Informática, PUC-Rio, R. Marquês de São Vicente 225, Rio de Janeiro, Brazil; lfgs@inf.puc-rio.br;{rogerio,debora}@telemidia.puc-rio.br Abstract. This paper discusses multimedia and hypermedia modeling, authoring and formatting tools, presenting the proposals of the HyperProp system and comparing them to related work. It also highlights several research challenges that still need to be addressed. Moreover, it stresses the importance of document logical structuring and considers the use of compositions in order to represent context relations, synchronization relations, derivation relations and task relations in hypermedia systems. It discusses temporal and spatial synchronization among multimedia objects and briefly presents the HyperProp graphical authoring and formatting tools. Integration between the proposed system and the WWW is also addressed. Key words: Nested context model HyperProp system Multimedia/hypermedia formatting Document logical structuring Spatio-temporal synchronization 1 Introduction Hypermedia systems have been addressed in three different levels in the literature: storage, specification (authoring) and execution (formatting). This paper focuses mainly on authoring and formatting. The specification level aims at defining hypermedia application requirements and associated constraints. The formatting level refers to the development of protocols and schemes for document presentation, dealing with intra-media temporal synchronization, and inter-media temporal and spatial synchronization. Hypermedia systems may, and sometimes have to, introduce several desirable facilities, such as: allowing a structured authoring; allowing the explicit definition of temporal and spatial relationships among their components (including those with non-deterministic times); supporting a rich set of capabilities for each media component (e.g., presentation duration flexibility, alternatives for different presentations, behavior changes during presentation, etc.); supporting relations anchoring on internal points of their components (fine granularity), and not only on its start or end points (coarse granularity); allowing quality-of-service (QoS) definition required by media components (e.g., jitters, bandwidth, etc.); and allowing a presentation environment description (e.g., network delay, supported devices, etc.). In order to achieve these goals, hypermedia systems must be supported by powerful conceptual models. The greater the expressiveness of a model, the greater the document authoring complexity will be; authoring that must allow the complete document specification as easily (or friendly) as possible. Similarly, the greater the complexity of the spatiotemporal (spatial and temporal) formatting algorithm will be. The formatting algorithm must consider all the environment non-determinism and allow correcting the presentation on the fly, when unpredictable events occur (such as network delay, user interaction, etc.). It must also take advantage of media segment QoS specifications and environment characteristics in order to trigger some actions, for instance, doing pre-fetch of the content of objects. In this paper, we consider the development of multimedia and hypermedia authoring and formatting tools, examining the current state of the art. All the aforementioned facilities are discussed. Solutions proposed in the Hyper- Prop system are presented and compared with related work throughout the text. Moreover, we discuss a set of research challenges that need to be addressed before the full potential of multimedia output technology can effectively be used for sharing information. The paper is organized as follows. In Sect. 2, general issues related to hypermedia document authoring are discussed, stressing the importance of document logical structuring, and showing how to use compositions to support them. Section 3 presents the HyperProp hypermedia conceptual model and discusses how it addresses requirements for authoring documents with spatio-temporal constraints. Section 4 discusses several issues and solutions for defining hypermedia application requirements and associated constraints during authoring and formatting phases. Authoring tools with graphical interfaces make the task of document specification easier. The HyperProp solution for a graphical editor is introduced in Sect. 5. Section 6 describes the temporal and spatial-formatting machine of the system. Correspondence to: L.F.G. Soares

3 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system 119 The integration of HyperProp with the WWW is discussed in Sect. 7. Section 8 contains our final remarks. Book Chapter Section B 1 C 1 2 Structured authoring of hypermedia documents S 1.1 S 1.2 C 2 An authoring environment should offer good editing and browsing tools for defining the content of its components and their granularity. The content is the set of information units of the component, and the content granularity specifies the subset of information units that can be marked and used for defining anchors. The exact notion of information unit and marked subset of information unit is part of the definition of a component, from here on called a node. For example, an information unit of a video node could be a frame, while an information unit of a text node could be a word or a character. Authoring tools must also be rich in their semantic capabilities to express relationship requirements among their objects. There are several types of relations that must be supported, among others reference relations: for example, those leading to a small note or those that jump to an entirely new section; context relations: for example, those that specify a hierarchical structure of a document, such as a book and its chapters, these chapters and their sections, and so on; synchronization relations: those that define both temporal and spatial ordering of objects; derivation relations: for example, those that bind data to the object from which it was created; and task relations: those that organize tasks in a cooperative work. Relations can be modeled, as in several authoring systems [13], by means of links alone. However, only one hypermedia model entity cannot have the semantics needed to model all kinds of relations. Composition is another model entity that can be used for structuring documents, depending on which relation it models. The structured definition of documents is desirable as it carries built-in concepts of modularity, encapsulation and abstraction. The composition concept is introduced as a container of nodes media objects (media segments) or even other compositions, recursively and possibly relationships among them (represented by links). Compositions should allow the support for both top-down and bottom-up design. Some relations can be derived from the logical structure defined by compositions, while others must be defined by other model entities (links, for example). There are several structurebased authoring models, depending on which type of the aforementioned relations they are interested in structuring. Compositions for context structuring They are usually defined in structured hypertext models. For example, they are found in the context nodes of HyperProp [6]. Figure 1 shows an example of their use. In the figure, a book (composition B 1 ) is composed by chapters C 1, C 2 and C 3. Composition C 1 (chapter C 1 ) contains sections S 1.1 and S 1.2 and a media object O 1, etc. Other relations, for example, reference relations, may be represented by links. For instance, a reference relationship between object O 1 and chapter C 3 is represented by link l 1. O 1 l 1 C 3 O 1 S 3.1 Fig. 1. Context structuring. Nodes are represented by rectangles, compositions are represented by rectangles containing other rectangles and links are represented by filled arrows Compositions for spatio-temporal presentation structuring In this case, as stated in [37], compositions are used for representing both temporal and spatial ordering of its objects. In other words, they are entities that embed a set of nodes with their spatial and temporal presentation relationships. These compositions are found in several conceptual models, for example CMIF [29], SMIL [17], I-HTSPN [39], Madeus [19], MOAP [35] and IMAP 1 [36]. Explicit temporal relationships among nodes can be defined in other entities, as the sync arcs of CMIF, the Petri net transitions of I-HTSPN, the scenario tuples (and actions) of MOAP and IMAP, and the temporal constraints of Madeus. Temporal relationships among nodes can also be derived implicitly by the composition type. For example, in CMIF and SMIL, relationships are given implicitly by the composition type that can be parallel or sequential, where all components must be presented in parallel or in sequence, respectively. In models where compositions structure temporal presentations, context relations as well as reference relations are usually given by links. There is no way to distinguish the semantic of these two types of relations through that unique model entity. This was one of the problems raised by Halasz in his seven issues on Notecards [13]. Compositions for grouping correlated versions of nodes Here, compositions are used for grouping objects in order to maintain and manipulate a history of changes in these objects, that is, they group nodes that represent versions of the same entity at some level of abstraction. The grouped versions can even be (if it is permitted) the previously mentioned types of compositions (for context or presentation relations), allowing to explore and manage several alternate configurations for a network of nodes (related by context or presentation purposes). Compositions for version control can be found in the mobs of CoVer [11], version groups of HyperPro 2 [25] and version and variant contexts of HyperProp [32, 33]. Compositions for task maintenance Conceptual hypermedia models that allow cooperative work usually use compositions for structuring cooperative tasks that com- 1 Actually, IMAP derived from the MOAP model. However, there are some conceptual differences between them, which will be highlighted throughout the paper. Although the IMAP model appears as IMD in some references, we will not make distinction between them. 2 Note that HyperPro and HyperProp are distinct systems and NCM is the conceptual data model of HyperProp.

4 120 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system such a variety of users. Some of these works are reported in [14, 16]. 3 The NCM Fig. 2. NCM class hierarchy pound the cooperative environment. Examples of such compositions are the tasks of CoVer and VerSE [12], and the private bases of HyperProp [32, 33]. In all these models, tasks are a set of interrelated document components and subtasks, recursively. Composition as a class from which other composition types can be derived as subclasses This is the case of compositions defined in models that have more than one structuring purpose, or in meta-models for hypermedia authoring conceptual models. Examples of these compositions can be found in the already mentioned CoVer and HyperPro, but especially in NCM nested context model, the conceptual model of the HyperProp system [6, 32, 33]. Figure 2 shows the NCM class hierarchy. In NCM, user contexts are used for context structuring; version and variant contexts for derivation structuring; private bases and public hyperbase for cooperative work structuring; and trails for guided-tours structuring and for maintenance of user navigation history. In NCM, temporal and spatial relations are defined by links. Reference relations are considered to be a particular case of synchronization relations triggered by a user interaction. It is worth noting that the context class of NCM might also be specialized in parallel-composition and sequentialcomposition subclasses (not shown in the figure), with the same functionality defined in CMIF and SMIL. Despite several work discussing compositions and their many different types of use, several open issues still remain. Many systems allow the creation of document structure but give authors no support in determining what would be the most effective structure to be created. Support for automatic generation of document structure is in the beginning, for example, automatic composition extraction from spatio-temporal specifications. Even a more difficult issue is the structured-document automatic generation. For instance, based on a subject and the knowledge of a user task, a structured-document can be generated by combining appropriate media items into coherent groups with derived temporal and layout presentation specifications. The goal in this case is to (semi-) automatically generate presentations, thus reducing the effort required by the publisher to cater for Authoring tools are based on conceptual models that must be rich in their semantic capabilities to represent complex multimedia objects and also to express their relationship requirements. Hypermedia models are also the core of formatting tools. This section is dedicated to the NCM presentation, providing the basis for the discussion about authoring and formatting facilities held throughout the following sections. In NCM (refer to Fig. 2), an entity is an object that has as attributes 3 a unique identifier, an access control list and an entity descriptor. The unique identifier has the usual meaning. For each entity attribute, the access control list has an entry that associates a user or user group to its access rights for the attribute. The entity descriptor contains information determining how the entity should be presented in time and space. The value of this attribute is usually a descriptor object, as will be defined in Sect Content and composite nodes A node is an entity that has as additional attributes a content and an anchor ordered list. The content identifies a set of information units. Each element in the anchor list is called an anchor of the node. An anchor identifies a region. The identification can be done indirectly, in which case an anchor specifies an anchor object that has an additional attribute for a region identification. A region is defined by a marked subset of information units of the node content. The exact definition of node content and region depends on the node class, as mentioned in Sect. 3. References to an anchor can be made through its unique identifier or through a node identification and the anchor position in the list of anchors of the node. NCM distinguishes two basic classes of nodes, called content node and composite node. Intuitively, a content node contains data whose internal structure is application dependent, modeling traditional hypermedia nodes. NCM requires that the special symbol λ must be a valid region value, representing the whole node content. Content nodes may be specialized in other classes (text, graphic, audio, video, etc.), as required by applications. Section 7 gives an example of a specialization of the text node class that will allow WWW documents to incorporate NCM features, called HTML node. The logical structuring of a document is made by introducing the composition concept. In NCM, a composite node C has, as content, a list S of links and nodes, which may be content or composite nodes, recursively. Moreover, an anchor of C must be a sublist of S only containing nodes. Similar to content nodes, in the case of composite nodes, the special symbol λ is a sublist of S containing all its nodes. We say that an entity E in S is a component of C and that E is contained in C. We also say that A is recursively contained 3 We will frequently use the name of the attribute of an entity to refer to the attribute value. When the context does not allow this simplification, we will explicitly use the term attribute value.

5 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system 121 in C if and only if A is contained in C or A is contained in a node recursively contained in C. An important restriction is made: a node cannot be recursively contained in itself. As a node can be recursively contained in different compositions (for example, node O 1 in compositions C 1 and C 3 in Fig. 1) and composite nodes can be nested to any depth, it is necessary to introduce the concept of perspective. Intuitively, the perspective of a node identifies a sequence of nested composite nodes through which a given node instantiation is being observed. Formally, a perspective of a node N is a sequence P = (N m,...,n 1 ), with m 1, such that N 1 = N, N m is a node not contained in any composite node, and for i [1,m), N i+1 is a composite node and N i is contained in N i+1. For example, in Fig. 1, node O 1 has two different perspectives: P 1 =(B 1,C 1,O 1 ) and P 2 =(B 1,C 3,O 1 ). Composite nodes are objects whose semantics must be well known by the model. A conceptual model should not only represent data-structural concepts, but also define data operations to manipulate and modify structures. Thus, every composite node C has the following methods: Insert node, Remove node, Create link, Destroy link, Display structure (usually defined by the composite node descriptor, this method displays the structure of nodes and links graph contained in the composite node; moreover, the method is usually an editor providing all other operations for composite nodes), Select node (that instantiates the node, if not instantiated yet, and calls its show 4 method), Select link (that instantiates the link, if not instantiated yet, and calls its show method), Select father node (that instantiates the composite node that contains node C through the current perspective, if not instantiated yet, and calls its show method), and methods to change the node state defined for versioning purposes [33]. Parameters like a large number of nodes and links can hinder document navigation mechanisms. Disoriented users need context information to reestablish the sense of orientation. In particular, temporal context information is needed to answer questions like How did I get here?. These questions are answered introducing the concept of trail. Given a composite node C, atrail T for C is a composite node containing an ordered list of nodes and trails, such that all nodes are recursively contained in C and all trails are trails for C. Moreover, T has as attributes current, whose value indicates the current entity (in the list) of T, and view, whose value associates each occurrence of a node N in the content of T with a sequence of nested nodes (C,...,N). We also say that T is associated to C. Trails come equipped with the following additional methods: next (to jump to the next entity of the trail content the ordered list in relation to the current entity), previous (to move to the previous entity), home (to return to the first entity in the list), and last (to go to the last entity in the list). Note that a node or a trail may occur more than once in the ordered list of T. Moreover, each node occurrence is associated with a sequence of nested composite nodes from the point of view of C. Trails are useful for lining up 4 Every entity should have a method, generically named show, to present/edit its content. This method is defined by the entity descriptor or a descriptor object. hypermedia documents and for implementing guided tours. As in Intermedia [40], a special system private base trail keeps track of all navigation made during a work session, so that a user can move at random from node to node and go back step by step. Besides the aforementioned methods, every trail must have methods to enable or disable actions of the other methods. Thus, we have enable: that enables the methods next, previous, home and last, and disables the methods insert node and remove node. disable: that disables the methods next, previous, home and last, and enables the methods insert node and remove node. The reason to have methods enable and disable is to prevent a trail from being simultaneously used for navigation (through the trail) and for maintaining navigation history. NCM defines a context node C as a composite node whose content is a set of links, content nodes and context nodes. As a set, the content cannot have replicated components, different from a trail. Context nodes have an additional attribute called descriptor set. A descriptor set contains for each node contained in C a descriptor object or a null value. The notion of descriptor, as already mentioned, will be defined in Sect Unfortunately, the definitions of the NCM classes are mutually recursive, hindering the presentation in a linear text. This work is a typical example where the hypertext paradigm would be very useful. 3.2 Links and events An event is an occurrence in time that can be instantaneous or can occur over some time period 5.Apresentation event is defined by the presentation of a marked set of information units (an anchor) of a node in a given perspective. A selection event is defined by the selection of a marked set of information units of a node in a given perspective. An attribution event is defined by the changing of an attribute of a node in a given perspective. In NCM, an event can be in one of the following states: sleeping, preparing, prepared, occurring, paused and aborted. Every event has an associated attribute, named occurrences (OC). OC counts how many times an event transits from occurring to prepared state during a document presentation. Presentation events also have an attribute, named repeat, which indicates how many times the event should be presented in sequence. Moreover, a presentation event has other attributes defined by the descriptor object, among them a tuple,t min,t opt,t exp,t max,f cost >, where the minimum allowed event presentation duration t min, the maximum allowed duration t max, the duration that would give the best quality of presentation t opt and the cost of shrinking or stretching the event presentation duration f cost are defined. The expected value t exp is initially set to be equal to the optimum value. As will be seen in Sect. 6, it is a role of the spatio-temporal formatter to choose an expected 5 We will follow as much as possible the definitions used in Perez-Luque and Little s Temporal Reference Framework [26].

6 122 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system Abort Abort Prepare Start (n) Pause Sleeping Preparing Prepared Occurring Paused Stop Stop Fig. 3. HyperProp (NCM) presentation event state machine Aborted Resume duration based on this specification and other information, such as the description of the platform where the document will be presented. Intuitively, taking a presentation event as an example (see Fig. 3), it starts in the sleeping state. It goes to the preparing state while some (pre-)fetching procedure of its information units is executed. At the end of the procedure, the event goes to the prepared state and its repeat attribute is initialized. At the beginning of the information unit presentation it goes to the occurring state. If the presentation is temporarily suspended, the event stays in the paused state, while the situation lasts. At the end of the presentation, the event comes back to the prepared state, when the OC attribute value is incremented and the repeat attribute value is decremented. After being decremented, if the repeat attribute contains a value greater than zero, the event presentation automatically restarts. If the presentation is abruptly finished, the event goes to the aborted state and immediately after it comes back to the prepared state, without incrementing the OC attribute value and setting to zero the value of the repeat attribute. Obviously, events like selection and attribution may stay in the occurring state only during an infinitesimal time. The HyperProp event state machine is executed under the document spatio-temporal formatter responsibility. An NCM link is an m : n relationship composed by m source end points, n target end points and a meeting point. Since a node may pertain to more than one composite node, the source and target end points of a link are identified by pairs of the form < (N k,...,n 1 ),α,type>, such that 1. N 1 is a node in the node list (N k,...,n 1 ), called anchor node of this link; 2. N i+1 is a composite node and N i is contained in N i+1, for all i [1,k), with k>0; 3. α is an anchor of N 1 or the identification of an attribute of N 1 ; 4. type specifies the event associated to α (presentation, selection or attribution). In the case of an attribution event, α is an anchor of N 1 or specifies an attribute of N 1. Otherwise (selection and presentation), it specifies an anchor of N N k, called head node of the link, is a component of the composite node or the composite node itself (that contains the link). A meeting point attribute value is a specialization of the script class, called meeting-point object. A script has an attribute (called content) that contains an ordered operation sequence. A meeting point is a script that contains only one operation composed by a condition and an action. Conditions are related to source end points, while actions are operations that should be applied to target end points. When a condition is satisfied, its associated action is fired. C 1 C 2 C 3 A T l 1 V V Fig. 4. Link definition example l 2 l 3 I Link l 1 is defined in composition C 1 as: Source end points Target end points Ev 1 =<(C 3,T),, selection> Ev 2 =<(C 2,A), λ, presentation> where is an anchor of T Ev 3 =<(C 2,V), λ, presentation> Meeting point Condition: (state(ev 1 )=occurring, state(ev 1 )=prepared) Action:(start(Ev 2 ) start(ev 3 )) Meeting-point conditions evaluate boolean values and can be binary simple or compound. Every binary simple condition is expressed by two unary conditions: a previous condition, to be satisfied immediately before time instant t when the condition is evaluated, and a current condition, to be satisfied at time t. A binary simple condition is satisfied when both unary conditions are satisfied (true). Either the previous or the current condition can receive the true value, if they are not relevant in the binary simple condition evaluation. Comparison operators used in the evaluation of unary conditions are: =, /=, <,, >,. Comparisons are made concerning states of an event E. Attributes associated to an event E, like the OC (occurrences) attribute, and valid values for these attributes. Node attributes, in the case of an attribution event, and valid values for these attributes. Any logical condition expression based on the operators (and), (or), ( (not) and (±delay) defines a compound condition. The logical operators, and have the usual meanings of the propositional logic. An expression C (±delay) is satisfied at a time t if condition C was satisfied at time [t (±delay)]. As in the event presentation duration specification, the delay is specified as a tuple <t min,t opt,t exp,t max,f cost >. Meeting-point actions can be simple or compound. Some simple actions for presentation events are shown in Fig. 3 in handwriting letters. A compound action is defined by an action expression based on the operators (parallel) and (sequential), defining the execution order of each element of the action. Moreover, every simple or compound action has an optional attribute called wait. This attribute determines the time that has to be waited before executing the action. As in the event presentation duration specification, the wait attribute is specified as a tuple <t min,t opt,t exp,t max,f cost >. For a link definition example, see Fig. 4. NCM links for a node can be defined in any composite node of the node perspective. Thus, links that anchor on a node are dependent on the context (nesting of compositions) a node is within. In order to define these contextual links, we have: given a node N 1 in a perspective P =(N m,...,n 1 ), with m>0, we say that link l is a contextual link in P for N 1 if and only if there is a composite node N i in P containing l, for i [1,m], such that 1. if i>1, then (N i 1,...,N 1 ) is a node list of one of the end points of l; 2. else, (N 1 ) is a node list of one of the end points of l. It is also important to define the notion of visible link to determine which links actually touch the node instantiation

7 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system 123 (anchor at the node or inside a component recursively contained in the node). Given a node N 1 and a perspective P =(N m,...,n 1 ), we say that link l is visible from P by N 1 if and only if there is a composite node N i in P containing l, for i [1,m], such that 1. if i>1 then (N i 1,...,N 1 ) is a sublist of the node list of one of the end points of l; 2. else, (N 1 ) is a sublist of the node list of one of the end points of l. For example, in Fig. 4, node V in perspective P 1 = (C 1,C 2,V) has visible link l 1 and in perspective P 2 = (C 1,C 3,V) node V has visible link l 2. As another example, node C 3 in perspective P 3 =(C 1,C 3 ) has two visible links l 1 and l 3 and one contextual link which is l Spatial synchronization and temporal behavior specification In NCM, the definition of the content of a document component (content or composite node) as well as the definition of its anchors are contained in what we call data objects DO. The creation of such objects is usually made by specific media editors. A data object is created either as a totally new object or as a local version of other data objects or as a local version of already created persistent objects, called storage objects SO. Data objects created from other objects are considered data versions. The aggregation of a data object and a descriptor (layout specification) in order to present a component is called a representation object RO and is considered a representation version of the data object. Thus, storage, data and representation objects are related through versioning operations, as discussed in [33]. Intuitively, the plane of storage objects, represented by the lower plane of Fig. 5, groups all entities stored in a server, similar to a WWW server. The upper plane, which represents the plane of representation objects, collects all entities used during a user work session. The middle plane, which represents the plane of data objects, groups data objects used by the client to create representation objects. It is worth mentioning in some implementations the middle plane can be an abstract plane and may not have any instantiated object. In NCM, when presenting a node, the descriptor explicitly defined on the fly by the end user bypasses descriptors defined during the authoring phase. These, in turn, have the following precedence order: first, that defined in the link (in prepare and start actions) used for reaching the node; second, that defined in the composite node that contains the node, if it is the case; third, that defined within the node (descriptor attribute); and finally, the default descriptor defined by the node class. Figure 5 shows the association of a data object node and a descriptor. This association is represented by dashed lines connecting data objects in the middle plane to representation objects drawn in the upper plane. Note that one node can be combined to different descriptors, originating different representation objects of the same node. Figure 5 shows this feature with the association of descriptors D e1 and D e2 to Representation Object Plane Data Object Plane Storage Object Plane Y Nodes Y Z Nodes Fig. 5. Object planes Z E A A E Z Y C C A 1 Y E A Y E A E E A E A A 2 C C C Dez D ey D ey Dez D e2 D e1 D e2 D e1 Y E A 1 E A 1 A 2 D ee D ec Descriptors D ee D ec C A 2 Descriptors node A, creating representation objects A 1 and A 2.Inthe figure, A has two different representations, for example, one for each different way to access it. One can reach A by depth navigation (through composite node C ) using the descriptor D e1, or it can be accessed by the link that touches it using descriptor D e2. More precisely, a descriptor is an entity that has as attributes a start specification and a collection of event descriptions. A start specification contains all information needed to start a node presentation. In particular, it defines methods for displaying and editing a node. These methods can be any program or even procedures defined in the descriptor itself. A start specification can also have an ordered list of operations that must be executed when preparing the node for presentation. This list defines all parameters needed for creating a representation object from a data object. The procedures specified in the creation of a representation object act on content or composite nodes and their events. The behavior specification procedures provide functions to implement a user interface used for creating and editing events in the representation objects. Examples of these procedures are the ones found in the temporal and spatial views that will be discussed in Sect. 5. The temporal analysis procedures facilitate the automatic creation of the presentation schedule, providing timing information about the object being presented, as, for example, the time interval since an event has occurred. The controlling procedures permit the system to control the behavior of objects, for example, increasing or decreasing the speed of a video during the document presentation. These procedures are usually part of the editors and viewers specified in the descriptor start specification. For all types of nodes, descriptors should define the controlling procedures that can be used for changing the node presentation behavior. Node descriptors should

8 124 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system also define procedures to perform actions (prepare, start, stop, etc.), as defined in links. The list of operations defined in a descriptor is a specialization of the script class, containing an ordered sequence of operations. Each operation is composed by a condition and a set of actions. When a condition is satisfied, it fires the associated set of actions. Operation list conditions are similar to link meeting-point conditions, but their entry scope is limited to events and attributes of the representation object created using the descriptor. Operation list actions are dependent on the node class to which the descriptor will be associated. Actions should always correspond to a presentation behavior change of the representation object, created using the descriptor (controlling procedures). For example, in the case of an audio, a behavior change action could be increase the volume to X db. An event description in a descriptor consists of the tuple < α, type, duration, rep, oper, case >. Parameter α identifies an anchor of the data object to which the descriptor will be associated. Type specifies the event type associated to the anchor (presentation, selection) or has a special value ξ. Duration specifies the presentation duration of the event. In other words, duration specifies the values to initialize the tuple parameter, t min, t opt, t exp, t max, f cost, of the corresponding presentation duration attributes of the event. Rep specifies a value to initialize the repeat attribute of the event (as discussed in Sect. 3). Oper specifies an operation list object. Case indicates if the operation list can be executed. If the type parameter specifies a presentation or selection event, the operation list is executed when the event state changes from prepared to occurring. If the type parameter is ξ, the operation list must be executed if the event is in the occurring state and when the descriptor receives a message from another object specifying the anchor (for example, the activate event action of a link, not discussed in this paper). We should highlight that behavior changes are programmed in operation lists, as in [3]. Event descriptions can be enabled or disabled assigning values to the case parameter. An operation list can have temporal synchronization constraints, as for example, 5 s after a certain condition is satisfied, action X must be executed. Behavior changes can alter the event duration and thus have to be taken into account by the formatter, especially in the elastic time computation (as will be seen in Sect. 6). Examples are: Increase playback speed, pause 10 s, etc. 3.4 Versioning and cooperative work The notion of composite node can be used for supporting cooperative work and for abstracting the planes of Fig. 5. In order to be able to do that, the class of context nodes is specialized in other classes. A similar concept can be found in [11, 25]. A user context node U is a context node whose content attribute only contains content nodes, user context nodes and links. User context nodes have the semantics to model context-structuring relationships, as defined in Sect. 2. In NCM, only content nodes and user context nodes are subject to versioning, as seen in dark-gray background in Fig. 2. In order to address the problem of maintaining the history of a document, we introduce the classes of version context nodes and variant context nodes, as detailed in [33]. In order to support cooperative work, NCM has three more specializations of the context node class. The public hyperbase is defined as a special type of context node that groups content nodes and user context nodes. All nodes in the public hyperbase PH must be storage objects and if a composite node C is in PH, then all nodes in C must also belong to PH. Intuitively, the public hyperbase groups all entities stored in a server, similar to a WWW server, and represents the nodes of the lower plane of Fig. 5. NCM also defines a private base as a special type of context node, whose components are user context nodes, content nodes, trails, and private bases. Briefly, a DO-private base DOPB is a data object that collects all the nodes of the middle plane of Fig. 5. An RO-private base ROP B is a representation object derived from a DOPB. The data objects of DOPB will be associated to their descriptors, in order to create the representation objects that will be contained in ROP B. Representation objects contained in ROP B will be created according to versioning operations. Intuitively, ROP B collects all entities used during a user work session, represented by the nodes of the upper plane of Fig. 5. Reference [33] discusses the versioning support of NCM, defining all its class attributes and methods, including methods (operations) for version control and the redefinition of the display structure methods for all the subclasses derived from the context class specialization. 4 Authoring and formatting issues on conceptual models This section discusses some issues and compares related work solutions for defining hypermedia application requirements and associated constraints during authoring phase. HyperProp solutions are stressed throughout the section. The structured authoring issue was already discussed apart in Sect. 2, but complementing punctual issues regarding this matter are also discussed here. 4.1 Compositions and structured authoring Relation inheritance in compositions Relation inheritance in compositions means that relationships among nodes can be defined in any composition that recursively contains these nodes. In some models, composite nodes are defined as a logical grouping of nodes only. Depending on the model, composite nodes can be nested (that is, a composition can recursively contain other compositions), as in CMIF, Hyper-G [2] and Chimera [1]. In all these models, references represented by links are stored in an independent repository that sometimes is a single one (CMIF, Hyper-G and DHM). Relation inheritance does not make sense in this case.

9 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system 125 S 1.1 S 1.2 C 1 C 2 B 1 B 2 nodes V and A 2 must start at the same time, without the background audio. Therefore, the presentation depends on the external context, that is, on the navigation that led to presentation of the composite node. O 1 l 1 C Trails and guided tours Fig. 6. Relation inheritance O 1 S 3.1 In other models, compositions group nodes and links (relationships), but a relationship is stored in an ancestor of the linked nodes, which is the lowest in the hierarchy, as it is the case of the sync arcs of AHM [15]. Relation inheritance does not make sense in this case either. In several other models (MOAP, I-HTSPN and Madeus, for example), relationships are defined between components contained in a composition. They allow a composition as an end point of a relationship, but not a component inside a composition. This will prevent relation inheritance. In other models, compositions contain not only nodes, but also links, and offer the possibility of defining relations depending on the context of discussion. When links are defined in a composite node, we can have a structured reuse of relations implicitly given by the composition, as well as of relations explicitly defined inside the composition. Indeed, relation inheritance in composition nesting (as exemplified in NCM) is very important in order to allow a more general composition (and thus structure) reuse and, therefore, making the authoring process easier. As an example, suppose the composition representing chapter C 1 of Fig. 1. For a new book given to a reader (composition B 2 ), it could be desirable to introduce a relationship between sections S 1.2 and S 1.1 of C 1, to give a hint of related matters; for another advanced reader, the book (composition B 1 ) should be delivered without that relation, as in Fig. 1. Note that B 2 could be defined as a composition containing B 1 and the introduced relation, reusing all the structure of B 1, as shown in Fig Compositions with different entry points Some models (MOAP, I-HTSPN and Madeus, for example) allow a composition as an end point of a relationship, but not a component inside a composition. Different entry points in a composition are desirable since they allow different presentations of nodes that are recursively contained in the composition. This is also an important characteristic to allow structure reuse and, therefore, making the authoring process easier. NCM is an example of a model that allows such facility, since a link can go into nested compositions, as specified by the node list of an end point of the link. For example, in Fig. 7, the presentation of composition C 2 can be started through links l 1 or l 3, coming from other parts of the document. When C 2 starts through link l 1, nodes V (video), A 1 (background audio) and A 2 (voice node) must start at the same time. However, if C 2 starts through link l 3, Parameters like a large number of nodes and links, lots of changes in the document, bad response time to user actions, insufficient visual differences among nodes and links, and visually disoriented users are combined to hinder document navigation mechanisms. Disoriented users need context information to reestablish the sense of orientation. In particular, two kinds of information are needed: spatial context to answer the question Where can I go now? and temporal context to answer to How did I get here?. The first question is treated in [22] and is briefly discussed in Sect. 5, the second one is answered introducing the concept of trail (in particular, guided tours). In some systems, as for example some HTML authoring systems, a guided tour is implemented using links, that is, inserting a tour anchor in each node, an anchor that leads to the next node in the tour. In these systems, it is impossible to have alternative guided tours. In HyperProp, guided tours are created using trails, overcoming the mentioned problem. The great majority of WWW browsers do not differentiate the creation and navigation phases of a trail. In these browsers, when a user navigates on the trail that maintains the navigation history, new entries are created leading to a loop in the next forward navigations. In order to avoid this problem, some systems, also exemplified by several WWW browsers like Nescape and Internet Explorer, do not insert new entries during trail navigation, but loses information of the trail from a certain node, if a navigation is made to another node different from that specified in the trail. The HyperProp solution comes from the enable and disable methods of a trail, used for preventing the trail from being simultaneously used for navigation and for maintaining navigation history (as discussed in Sect. 3.1). If a user enables the navigation through the system private base trail, a copy of this trail is created, through which the navigation will occur. During navigation on this copy, the system private base trail is updated in order to maintain the navigation history. If a user navigates to a node out of the copy trail, this trail is simply destroyed. 4.2 Temporal synchronization Relations based on event states We usually need to test platform-dependent activities in order to start the presentation of an object. For example, we may need to test if a video node was got from a remote server before initiating its presentation in parallel with a local audio. HyperProp presents a solution through its event-state machine, where any event state, or event-state change, can be used in a condition of a link. None of the other related work presents a solution to the problem.

10 126 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system C 1 C 2 C 3 A 1 A 2 l 1 T 1 T 2 V N 1 Fig. 7. Example of hypermedia document Multipoint relations l 2 l 3 N 2 N 3 A 2 l 5 l 4 V Although relations are usually directional, they can be followed in both directions in almost all conceptual model implementations. However, few models allow multipoint relationships, that is, relations with several source and destination end points. I-HTSPN and NCM present a general solution for the specification of these multipoint relations, as seen in Sect Multipoint relations are important not only because they allow to test conditions on several events, but also because they provide the facilities described in the next two items Context for a link Multipoint relations are an elegant solution for the problem of defining a context for a link, as stated by [15]: In hypermedia it is useful for the author to be able to specify which parts of the presentation should remain and which should be replaced when a link is followed. A source context for a link is that part of a hypermedia presentation affected by initiating a link, and a destination context is that part of the presentation which is played on arriving at the destination of a link. In HyperProp, context for a link is a natural consequence of a multipoint relation and not a new attribute for a 1 : 1 relation Relations based on combinations of presentation and selection events Some models (CMIF, AHM, Madeus, etc.) define temporal and spatial relationships only among presentation events; relationships triggered by selection events are defined in another model entity (usually a link) completely separate from spatio-temporal relationships. In these models, it is not possible to merge conditions spatio-time dependent and userinteraction dependent to trigger a navigation. For example, the situation play an audio explanation when a user selects a text button during a video presentation cannot be represented. Note that this is also a typical case of multipoint relation. This situation is not rare and models should support them, as in NCM, IMAP and I-HTSPN Indefinite relationships As defined by [26], indefinite temporal relationships are those temporal relationships between time instants or time l 6 intervals that are not explicitly or unambiguously given. Usually they are expressed as disjunctions of the basic temporal relationships (for example, at the same time or after ). Several models, exemplified by NCM and IMAP, present some support to indefinite relations. In [26, 37], all possible indefinite relations for instant- and interval-based temporal relations are discussed. If all those relationships are needed is an issue to be tested. The event presentation duration as well as the delay and wait parameters (Sect. 3.2) used in NCM are specified by a tuple <t min,t opt,t exp,t max,f cost >. The parameters t min, t opt, t exp, t max, can be either a non-negative real number or the unspecified value (φ), or the unpredictable value (?). The possibility of having comparison operators in unary conditions (equal, unequal, less than, less than or equal, greater than, and greater than or equal), and of having any logical condition expression based on the operators (and), (or), (not) and (delay) in compound conditions, and moreover, of having the event presentation duration and the delay and wait parameters defined by an interval, gives HyperProp the capability of supporting indefinite relations. 4.3 Spatial synchronization and behavior specification An authoring environment should allow the definition of each component s expected behavior when presented. Several authoring systems (Firefly [3], CMIF, I-HTSPN, Hyper- Prop, SMIL, IMAP, etc.) define the layout of their components presentation separated from the associated data object. This will allow a better reuse of objects. For example, using distinct layouts, one can define different presentations for the same media object. A text media segment can be presented as text, or it can be synthesized as audio, depending on the layout. Note that as different layouts can lead to different event durations, different quality of presentations and different platform requirements, the definition of all these issues should also be part of the layout and not part of the media object. The presentation of an object should be created by the spatio-temporal formatter from its corresponding layout specification and data. Spatial synchronization can optionally be defined in the same relations used for defining temporal synchronization. For example, a link synchronization relation can specify not only the moment that a given presentation event should start but also how the presentation (layout) inside the current scenario must occur. The scenario tuples (and actions) of MOAP and IMAP, the spatial constraints of Madeus, and the links of NCM are examples. Optionally, spatial synchronization can be defined in other entities different from links. This is the case of Firefly, CMIF, SMIL, I-HTSPN, and also NCM. In all these latter models, the spatial synchronization is established by the placement of objects in an absolute spatial coordinate system, based on a paradigm analogous to the timeline for temporal synchronization, with all the disadvantages reported in several works in the literature. The spatial synchronization of IMAP is richer than the other models, as it allows event constraint-based spatial relationships, generalizing the constraint instant-based and interval-based paradigm for temporal relationships, present in all mentioned models, including NCM.

11 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system 127 Some systems allow a joint definition of the spatial synchronization, specifying the layout in an entity that can be shared by several media objects. This is the case of CMIF, where an event is assigned to a channel, which is an abstraction of a set of properties shared by other objects of the same media type. The use of CMIF channels can be very desirable. For example, it ensures that the system s audio driver will not need to be reinitialized for every data block sent to the device. It also allows special effects to be introduced when changing from an object presentation to another, which would be difficult to achieve with individual layout entities. Alternative layouts for the same media object are also important to allow platform QoS adaptation, user choice and different presentations for a document depending on the navigation start point. Alternative layouts are allowed by several systems and models like NCM and SMIL. Therefore, given a media object, an associated layout should be specified on the fly by the end user, or in the document specification, as NCM does. Presentation choices only in the document specification give the author the responsibility for designing and implementing alternate presentations and (s)he often does not know the diverse needs of users. An authoring model should also allow defining the layout in a media object attribute, or in a link (or any other entity specifying spatio-temporal or reference relation) that has the event as its destination anchor. Composite nodes may also have, for each node they contain, an attribute that can be used for storing the layout identifier for that node. The layout specification should also specify behavior changes, that is, how a presentation of an object evolves in time. This is the case, for example, of Firefly and NCM, where behavior changes and spatial synchronization are handled by operation lists, contained in the descriptor in the case of NCM. However, both Firefly and HyperProp only treat discrete behavior changes, that is, they identify time instants where behavior changes must be made, but do not allow continuous behavior changes. In [24], good ideas about the matter are given. We intend to incorporate these ideas in HyperProp. 5 Graphical interfaces for authoring An author (who is usually a non-technical person) needs tools for a high-level complete description of all aspects of a multimedia application. Several systems (CMIFed, HyperProp, I-HTSPN, Madeus, etc.) offer authoring tools with graphical interfaces, making the task of document authoring easier. A graphical authoring environment should support both top-down and bottom-up constructions. Constraints among media items (events) should preferably be defined directly among those items. Graphical authoring tools are usually based on different views that allow the specification of all types of relations previously defined. Each view favors one type of relation definition and all views must work together. This section briefly presents the HyperProp authoring environment, which is based on three different views, as shown in Figs Fig. 8. Structural view Fig. 9. Temporal view The first view shown in Fig. 8, called structural view, supports browsing and editing the logical context and task structure of documents, providing features for editing nodes and links and grouping them into compositions. In this view, nodes are represented by icons (content nodes) and rectangles (composite nodes), links are represented by lines and the containment relationship is represented by the inclusion of rectangles and icons (nodes) and lines (links) in another rectangle (composite node). Usually, the structural view has a very complex structure, requiring sophisticated algorithms to build the view. The main motivation is to balance local detail and global context with respect to a node in focus. Local detail is necessary to give information about the navigation possibilities, depending on the focus in the document structure. Global context is important to give information about the focus position within the overall structure. There are several techniques to display graph structures [30] 1. Presenting all nodes and links of the graph, which allows users to have a global view of the structure. The drawback is that maps become confusing and do not help user navigation when the number of nodes and links increases. 2. Using scroll and zoom to view portions of the graph, which makes readable maps, but loses the overall structure. 3. Using two or more views, one with a global view and another with a small zoomed portion of the graph, which has the advantage of displaying local details and the overall structure of the information space, but forces the user to mentally integrate the views.

12 128 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system 4. Using filtering mechanisms, such as fisheye views, to build a global view, which only displays what is more interesting to the user at a certain moment, maintaining map legibility. The difficulty is to find out what is more interesting to the user at that moment. The last option is the most desirable solution, but the most complex. Filters to hide information, and possibly landmarks (special nodes that always appear), will generally be necessary. Maintaining the legibility of maps is the main purpose. The user will only visualize what is interesting to him, depending on his position (focus) in the document structure and what is important regarding the overall structure of the hypermedia document. In order to build those filters to present structural views of private bases and the public hyperbase, HyperProp uses an extension of the fisheye-view strategy for conceptual models that offer composite nodes [22, 23]. Another important feature of the fisheye-view strategy is that it allows users to choose the amount of information shown, tailoring the map according to their interests. Thus, users can choose to see more or less detail in the structural view. Another desirable characteristic is to be at two foci at once in the structural view. This can be needed, for example, because some small structure is being copied from one place to somewhere else, or because a relation is being authored between two parts of the document. Given a compound graph, there are several works proposing different types of layout [7, 8, 34]. However, a tool integrating filtering and automatic graph layout design oriented to hypermedia users is still missing. Moreover, even the partial solutions already presented remains to be tested by a large number of naive users. The second view, called temporal view, is responsible for supporting the specification of temporal relationships among components of a document, defining their relative position in time. As shown in Fig. 9, in this view, nodes are represented by rectangles, whose lengths indicate the mean optimum duration of their presentation in a time axis and whose relative positions establish their temporal synchronization. A remarkable difference between the temporal view and a timeline is that the time shown in the temporal view is not explicitly specified by the author. It is derived from temporal constraints, just as an approximation of the real-time presentation [23]. Temporal views are very useful to display related presentation events. However, when relationships can be defined among selection and presentation events, it is difficult to represent the situation. This happens because a user interaction time instant is runtime dependent. The HyperProp poor solution [23] for this case is done by defining a region, called limbo, where nodes that do not have their presentation time instant exactly determined can be placed. Those nodes are dragged from the structural view and dropped into the limbo. All events related to nodes in the limbo are assumed to have occurred in the past, but one cannot determine when. Placing objects in the limbo permits manipulating relationships, even though it is not precisely known when its source events have occurred. As far as we know, there is not an efficient graphical tool yet that permits the definition of non-deterministic temporal relationships. Fig. 10. Spatial view Temporal views usually display the mean optimum duration of an object presentation. However, duration is usually specified as varying within an interval. In [20], the duration of an object is represented by an elastic time-box with a spring symbolizing the variation. Again, as far as we know, there is not a tool yet that permits a good representation of time variations when several objects are involved. Relations can be defined in the temporal view, like links in NCM, sync arcs in CMIF and temporal constraints in Madeus. However, unless relations are stored in a separate repository, the composition that contains it should be defined using the structural view. This is one of the reasons why these two views must work together. Finally, the third view, called spatial view, supports the definition of spatial relationships among components of a document, establishing their presentation characteristics in a given device, at a specific instant of time. As shown in Fig. 10, in this view, rectangles represent spatial characteristics of objects at a given specific time instant, such as their position in a video monitor or their volume level in an audio device. In CMIFed, the player shows the effect of mapping the abstract document to a particular platform. It acts indeed as a document spatio-temporal formatter. However, sometimes the author needs an abstract spatial view independent of the platform, like the spatial view of HyperProp. The spatial view should allow spatial relationship definition among its objects, like the temporal view allows temporal relationship definition. Obviously, spatio-temporal relations must be defined using the two views in an integrated way. Moreover, the relation may have to be placed inside a composition, so the structural view must also be integrated with the two other views. Indeed, the three views should be related to each other. In HyperProp, the focused object in the structural view is the basis for the time chain shown in the temporal view. For each point of time in the latter view, the spatial view shows how components will be presented in the space defined by the output devices. Links and nodes defined in the temporal view are immediately updated in the structural view, and vice-versa. The same integration can be found in CMIFed and Madeus. The spatial and temporal views can also have a very complex structure, requiring sophisticated algorithms to build the view. Filtering may also be needed in this case. An efficient design of these two views for a large number of nodes is a great problem to be faced. Other open issues can be mentioned. There is not an efficient tool that guides authors to define temporal and spatial consistent documents for a given particular platform. Indeed,

13 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system 129 the complete design of a graphical authoring tool is an open issue. The few existent proposals are not definitive. They have to be tested for large document authoring and with a large number of naive authors, so that we can have feedback for better designs. Inconsistencies report Presentation specification Environment specification Structure for Execution Plan Pre-Compile Time Time Pre-Compiler Chains Compile Time Structure for Execution Compiler Plan Executor Run Time Controllers interface Actions, Get & Set attribute values Viewers Controllers Viewers interface Editors/... Viewers 6 Spatio-temporal formatting in the HyperProp system The spatio-temporal formatter, or simply formatter, is responsible for controlling the document presentation based on its specification and on the platform (or environment) description. The main idea is to build an execution plan (a schedule) to guide the formatter in its task. The execution plan should contain information about actions that must be fired when an event-state change is signalized. Platform specifications would typically be supplied by someone such as a system or network administrator rather than an author of a particular document. Before discussing the HyperProp formatter, we have to define several concepts that will be used throughout this section 1. We say that a presentation event has a predictable duration when it is possible to determine the exact time its state will change from occurring to prepared, in relation to the time its state has last changed from prepared to occurring. Otherwise, we say that the event has an unpredictable duration. 2. We say that an event is predictable in relation to another event E when it has a predictable duration and it is possible to know, a priori, its expected start time, that is, the time its state changes from prepared to occurring, in relation to event E (in relation to any time instant of the occurring period of E). 3. An event is called unpredictable when there is no event E for which it can be predictable. 4. The sequence of event occurrences in time is called time chain. 5. A time chain is called partial time chain when all its events, except the first one, are predictable in relation to at least one event of the same partial time chain. 6. If an event E is predictable in relation to another event that pertains to a partial time chain PTC, then E also pertains to PTC. As a corollary, we have that every selection event is unpredictable and begins a new PTC. Needless to say, in HyperProp, we can have 1. events that have an unpredictable duration, but whose duration can be limited within a finite interval; and 2. events with predictable duration that can be chosen within an interval. As previously mentioned, hypermedia systems can be addressed at three different levels: storage, specification (authoring) and execution (formatting). Each one of these levels is based on a conceptual data model, which can be different from those of the other levels. In practice, choosing a single model for a particular application is not often sufficient. One needs both the flexibility of more abstract models and the accurate control over the presentation offered by more Authoring Subsystem interface Storage Subsystem interface Multimedia objects Server Events signaling and attribute values report Temporal and Spatial Formatter Fig. 11. The HyperProp spatio-temporal formatter architecture presentation-dependent models. As a result, one needs tools to convert documents from one model into another. Authoring models are usually based on higher level abstractions, as they are closer to users. The great expressiveness of authoring models must be offered to users in an easy and comprehensible way. These are some of the reasons that led HyperProp to use an event-driven model to specify the logical structure and presentation characteristics of a document. The input structure for the spatio-temporal formatter will be extracted from this model. As the formatter is closer to the operational machine, a lower level model for parallel-state machines may be the most adequate for tasks like scheduling. Almost all aforementioned systems follow different models for each one of their levels. For example, Firefly uses a time instant (event-driven) constraint-based model during authoring, which is translated by a compiler (called scheduler) to PTCs structured as timelines, which feed the runtime formatter. Firefly tries to avoid timeline problems by concatenating PTCs to support unpredictable events. The CMIF formatter (called player) has a compiler that generates, from the document specification, a directed graph of timing dependencies, which feeds the player. The graph used by CMIF is similar to a Petri net. The I-HTSPN toolkit has a compiler (called MHEG translator) that converts the I-HTSPN specification (based on extended-time Petri nets) into an MHEG representation. An MHEG machine should control the document presentation. HyperProp uses an object-oriented time instant event-driven model (NCM) in the authoring phase, which is translated to a graph (similar to a Time Stream Petri Net [39]) to model the execution plan. Other data models may coexist between authoring and execution phases. For example, in HyperProp, NCM can be converted to an RT-LOTOS specification for consistency checking, as described in [31]. The HyperProp spatio-temporal formatter architecture, which is very similar to the Firefly formatter architecture, is composed of four elements: the pre-compiler, the compiler, the executor and the viewer controllers (or simply controllers), as shown in Fig. 11. Although the pre-compiler performs formatting tasks, it is indeed a support for the authoring environment. Its main function is to check the temporal and spatial consistency of each document PTC. The pre-compilation is an incremental compilation done during authoring time. It can help authors

14 130 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system find errors, giving an instantaneous feedback whenever any inconsistency is detected, like mismatched temporal relationships (temporal inconsistency), or conflicts in a device use (spatial inconsistency) in an abstract and ideal platform. A document is called intrinsically consistent if no internal synchronization constraints can lead to inconsistent situations when the presentation is performed over an ideal platform. However, even if a document is intrinsically consistent, its presentation can lead to a deadlock situation, because the analysis is only based on the intrinsic (specified) duration of object presentations. Therefore, we define a document as extrinsically consistent if its presentation in a real and resource-limited platform is also consistent. In this case, the verification is performed over the composition of the document specification and the platform specification, testing properties that are not intrinsic to the document, but depend on the behavior of its presentation over a specific platform. During pre-compilation time, only intrinsic consistency can be checked. This is the kind of inconsistency check made in the temporal view discussed in Sect. 5. Considering the complete document presentation specification and the platform description, the compiler is responsible for generating the data structure used by the spatiotemporal formatter to build the execution plan. As stated in Sect. 3.2, in the event presentation duration specified by a tuple <t min,t opt,t exp,t max,f cost >, the expected value t exp is initially set to be equal to the optimum value. At compile time, the expected duration should be computed based on the cost minimization in order to warrant the spatial and temporal consistency of the document. This computation mechanism is called elastic time computation in [20]. The compiler also had better evaluate, from the platform specification, a tuple <t min,t exp,t max > for the preparing duration of a presentation event (that is, the time the event spends in the preparing state), where t min is the minimum preparing duration, t max is the maximum preparing duration and t exp is the mean preparing duration. This will be very useful for pre-fetching decisions, as will be explained later. In HyperProp, the Multimedia Objects Server (NCM server) is responsible for the persistent storage of the public hyperbase, descriptor objects, version contexts and public trails. When a user asks for the presentation of a document, the compiler gets from the NCM server all composite nodes recursively contained in the composition that models the whole document. From the links contained in these compositions, expressions in their meeting points are compiled in order to obtain the concerned events and create PTCs with them. For each event in a time chain, the compiler creates a reference to each meeting point where it is used. Thereby, when a change in an event state occurs, the executor knows all links that must be evaluated. All PTCs together make the presentation state machine that feeds the executor. The execution plan created by the compiler will guide the formatter in adjusting object duration on the fly, as well as in pre-fetching components content, in order to improve the presentation quality and to reduce the probability of temporal and spatial inconsistencies. Representation objects are created by the executor from the corresponding descriptors and data objects. However, the executor does not create representation objects directly. Based on the execution plan, the executor starts a controller and passes all information needed to create and control each object presentation. This information includes the data object, the descriptor and a list of all events that are conditions in the meeting points of the data object contextual links (as discussed in Sect. 3.2). This information is obtained from the data structure built by the compiler. The executor knows the type of controller to be started based on information contained in the descriptor (start specification). The controller creates the representation object getting its corresponding data object from the NCM server. Through the associated descriptor, the controller obtains the operation list that determines the behavior changes that must happen during the object presentation. The concept of a controller for each representation object, with a well-defined interface to the executor, will allow incorporating commercial viewers into the system, such as Word, Netscape, etc. It will only be necessary to implement controllers for these viewers in order to exchange messages according to the defined interface. From the viewers, controllers receive signals of changes in event states to be reported to the executor. The executor then updates the appropriate event state machine and evaluates all links associated with the event. If there is a condition in a link meeting point that is satisfied, the corresponding actions are fired. When the controller reports a change in an event state to the executor, the executor checks the expected time in the execution plan to see if some adjustment is needed. Adjustments may be done through messages sent to controllers, for example, telling them to accelerate the presentation rate. These elastic time adjustments are not implemented yet and are addressed as future work. The formatter specification and implementation are presented in details in [27, 28]. 6.1 Media object pre-fetching Based on the execution plan, the formatter can pre-fetch a component content in order to improve the presentation quality and to reduce the probability of temporal and spatial inconsistencies. In order to get the media object data content, the formatter must negotiate the QoS it needs with the whole platform environment (operating system, network, hypermedia server, etc.). Unfortunately, complete QoS guarantees is still a broad open issue. Based on QoS guarantees, if any are obtained, the formatter can estimate the worst delay to obtain a data content. This, however, can be much larger than the real delay, so the formatter can make use of other estimations. For example, CMIF initially estimates the delay based on heuristics involving data size. When a pre-fetching action is executed, the actual time it takes is saved and used as an estimate for following presentations. When a document presentation has a lot of user interactions, pre-fetching is less effective. Again, when a presentation is played several times, statistics can be stored about which user-dependent relations are taken most often, to guide the selection of presentations to be first prepared. Based on all the mentioned factors, heuristics should be developed to guarantee the QoS of a document presentation and minimize elastic time adjustments necessary to main-

15 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system 131 tain extrinsic consistency. Good heuristic proposals are still needed. 6.2 Object placement In order to guarantee the QoS of a presentation, a hypermedia server must distribute its objects for minimizing its access delay and maximizing its effective bandwidth. Some researches approach the problem focusing on a specific matter. For example, [9] proposes an object placement strategy based on object access probabilities and on object sizes. The problem is how to estimate such probabilities. Note that heuristics to estimate these probabilities are closely related with the ones used in pre-fetching. As another example, [37] proposes an indexing scheme based on multi-dimensional (spatial) data structures for efficient handling of queries related to spatio-temporal relationships. Other database works discuss the efficient storage of versions of the same object [4]. Indeed, object placement must involve all issues related to versioning, cooperative work, spatio-temporal synchronization, document access frequency, etc. A general algorithm or heuristic is thus necessary and is still an open issue. Also the definition of a complete query language that allows not only content queries but also structure queries is necessary. Again, there are some proposals for specific topics, for example, for querying version structures, querying spatio-temporal structures and querying context structures. Structure query is closely related to the object placement issue and also needs a solution. Note also that structure queries are closely related with the automatic generation of documents mentioned in Sect Elastic time computation In order to guarantee document consistency, the expected duration of an object presentation should be computed based on cost minimization. The elastic time computation may be done during compile time. Some possible algorithms are discussed in [3, 20]. However, the problem is much more complex when this computation must be done in real time. This can happen when the executor has to shrink or stretch an object presentation to compensate for some unpredictable platform delay. When this is done, the expected duration for remaining objects to be presented should be computed again, and in real time. That is why the few systems that implement elastic time adjustment only do it at compile time. In the current version, HyperProp does not implement pre-fetching of media objects, object placement heuristics to guarantee QoS of a presentation and elastic time computation at execution time. 7 HyperProp and the WWW As mentioned in Sect. 6, the goal to specify a well-defined interface between the formatter controllers and the executor is to allow incorporating commercial editors/viewers to present the content of NCM nodes. Following this approach, a controller for the Netscape browser was implemented, in order to present HTML nodes defined in NCM. This will allow WWW documents to incorporate NCM features, through the mapping of an HTML page to an NCM HTML node. This section briefly discusses the HyperProp/WWW integration. An NCM HTML node is a specialization of the text node class. This class has a URL that specifies the location of an HTML file as its content attribute. Each region of an HTML node anchor has two parameters: name and position. Name corresponds to the text defined by the anchor and position indicates the number of characters from the beginning of the HTML file to the region defined by the anchor. The node descriptor contains a descriptor object whose attributes are used for dynamically building a cascading style sheet [21] to present the node content. The definition of an HTML node anchor region depending on the number of characters might seem a problem for anchor maintenance. Indeed, changes in the node content may lead to inconsistent anchors. However, based on the name attribute, the system automatically checks these inconsistencies and disables links touching these anchors. The system authoring tool also offers mechanisms for verifying and automatically correcting anchor inconsistencies. Figure 12a presents an example of an HTML node (all its attributes) and Fig. 12b presents the content of the file identified by the URL. Although not show in the figure, it is worth noting that in the content of an HTML node there may be other embedded anchors and links (WWW anchors and links). However, NCM anchors and links are separated from the content, as illustrated in Fig. 12a. The NCM controller for the Netscape browser was implemented using Java and JavaScript languages and using the LiveConnect facility of Netscape for integrating these languages. We also implemented in Java a version of the structural view and a version of the HyperProp spatio-temporal formatter described in Sects. 5 and 6, respectively. From these implementations it became possible to download the HyperProp authoring and formatting environment as an applet and to obtain a first prototype of the HyperProp system integrated with the WWW. The integration between open hypermedia systems (OHSs) and the WWW has been motivated by trying to solve WWW s limitations, using more sophisticated and powerful hypermedia models, while exploring the Web large distribution and standards. Among other features, the integration between Hyper- Prop and the WWW already allows separation of links from data content in WWW documents; creation of references in HTML pages where write access is not granted; definition of compositions to structure WWW documents (allowing the addition of contextual links and reuse of structures of nodes and links in these documents); visualization of NCM documents in HTML browsers; definition of links between NCM and WWW documents, generating NCM/WWW integrated documents; visualization of WWW, NCM and NCM/WWW documents structure, allowing users to know which links reference a certain node;

16 132 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system separation of presentation characteristics from data content (allowing different presentations for the same object); and user navigation through graphical views, and navigation through guided tours and trails (that maintain the document navigation history). A detailed discussion about the integration of HyperProp with the WWW can be found in [27]. Several other proposals for integrating OHSs and the WWW can be found in the literature, from which we can highlight DLS [5], Chimera [1], Hyper-G [2], HyperDisco [38] and DHM/WWW [10]. The DLS client interface was designed to support the use of Netscape for communication with DLS servers. The nature of the client interface and the way it communicates depend on the particular platform being used. Hyper-G requires a special browser (and also a special text document format, HTF) in order for users to achieve all benefits of the system, including the ability to create links and collections. DHM/WWW and HyperProp/WWW present a platformindependent solution based on the Netscape browser. DHM also presents two other platform-dependent solutions, one using Netscape plugins and another using Internet Explorer. None of the related work offers a graphical tool for helping authoring and user navigation as illustrated by the Hyper- Prop structural view. We are currently integrating the following NCM facilities to the WWW: support for version control, for cooperative work and for the definition of n-ary relations that could also specify spatial and temporal synchronization among nodes. Our goal is to put all these facilities together in the Web of today, without having to change the implementations of WWW clients and servers. Other efforts have been made to support spatio-temporal synchronization in the Web, exemplified by the SMIL recommendation [17]. Also, efforts have been made to provide version control and cooperative work, exemplified by the WEBDAV group proposal [18]. Another problem to be faced is to implement a platformand browser-independent solution for the NCM and WWW integration. Unfortunately, this is not possible yet, as evidenced by all related work of other OHSs. However, a natural tendency is that WWW browsers become more opened, and the Java language offers more features, allowing a more general solution in the near future. Finishing this section, it is worth mentioning that we are currently integrating SMIL with NCM. The main goal is to have a proposal for an extended SMIL incorporating NCM facilities. 8 Final remarks This paper considered some issues on the development of multimedia and hypermedia authoring and formatting tools. It presented the HyperProp solutions examining the current state of the art and discussing a set of research challenges that still need to be addressed. The first prototype of the HyperProp hypermedia system was completely developed in C++. We are currently working on a Java prototype. All NCM classes, a graphical authoring tool (structural view), a spatio-temporal formatter and mechanisms for version control are already implemented in a name: hyperprop.html content: telemidia/projects/hyperprop/hyperprop.html anchors: [0]: default anchor (whole node) [1]: (Global View, 1080) [2]: (Objectives, 1144) [3]: (Partners and Sponsors, 1207) [4]: (Project History, 1281) [5]: (Staff, 1349) descriptor: Descriptor Default 1 name: Descriptor Default 1 position: top: left: color: font-family: font-size: font-variant: absolute; 0px; 20px; black; arial; 14px; small-caps; <html> <head><title>hyperprop Project</title></head> <body background="../../images/fundo2.jpg" bgcolor="#ffffff"> <TABLE CELLSPACING=0 CELLPADDING=0 > <TR ALIGN=CENTER VALIGN=CENTER> <TD width=100%><center> <IMG SRC="../../images/telemidi.gif" ALT="TeleMídia Lab" HEIGHT=49 WIDTH=480> </CENTER></TD> </TABLE> <TABLE CELLSPACING=0 CELLPADDING=0 WIDTH="100%"> <TR ALIGN=CENTER VALIGN=CENTER> <TD ALIGN=CENTER VALIGN=CENTER WIDTH="100%"> <IMG SRC="../../images/bar.jpg" HEIGHT=4 WIDTH=330> </TD> <TD ALIGN=CENTER VALIGN=CENTER WIDTH="30%"> <IMG SRC="../../images/projlogo.gif" HEIGHT=48 WIDTH=151> </TD> </TR> </TABLE> <p><h3 align="center"> Hyperprop Project - Open Hypermedia System and Applications </CENTER></H3> <p>the HyperProp project intends to build an open hypermedia system to incorporate facilities for the treatment of multimedia and hypermedia documents, in applications distributed or not. <p>to become more familiar with HyperProp project, you can select one of the below topics: <p><img src="../../images/blt_blue.gif" space="30">global View <p><img src="../../images/blt_blue.gif" space="30">objectives <p><img src="../../images/blt_blue.gif" space="30">partners and Sponsors <p><img src="../../images/blt_blue.gif" space="30">project History <p><img src="../../images/blt_blue.gif" space="30">staff <p></body> </html> b Fig. 12. a Example of an HTML node and its descriptor. b HTML content identified by the URL, without embedded anchors and links the new prototype, allowing the integration with the WWW. An XML DTD for NCM allows declarative specifications of NCM documents. Our next step is to implement the temporal and spatial views in order to have all three views working together, as it is provided by the first prototype of HyperProp. The definition of spatial and temporal multipoint relations is also a feature to be integrated into the WWW. We are also working on real-time consistency checking and elastic time algorithms to be included in our formatter.

17 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system 133 Acknowledgements. This work was supported by the Brazilian Telecommunication Enterprise (Embratel) and Brazilian Research Foundations (CNPq and FINEP). References 1. Anderson K (1997) Integrating open hypermedia systems with the World Wide Web. In: Bernstein M, Carr L, Osterbye K (eds) Proceedings of the Eighth ACM International Hypertext Conference, 1997, Southampton, UK. ACM, New York, NY, pp Andrews K, Kappe F, Maurer H (1995) Serving information to the Web with Hyper-G. Comput Networks ISDN Syst 27(6): Buchanan M, Zellweger P (1993) Automatic temporal layout mechanisms. In: Rangan P (ed) Proceedings of ACM Multimedia, 1993, Anaheim, Calif. ACM, New York, NY, pp Carey M, DeWin D, Richardson J, Shekita E (1986) Object and file management in the EXODUS Extensible Database System. In: Kaufmann M (ed) Proceedings of the Twelfth Int. Conf. On Very Large Data Bases, 1986, Kyoto, Japan. Morgan Kaufmann, Los Altos, Calif., pp Carr L, Roure D, Hall W, Hill G (1995) The Distributed Link Service: a tool for publishers, authors and readers. In: Goldstein I, Vezza A (eds) Proceedings of the Fourth International World Wide Web Conference, 1995, Boston, Mass. O Reilly and Associates, pp Casanova M, Tucherman L, Lima M, Rodriguez N, Soares L (1991) The Nested Context Model for hyperdocuments. In: Walker J (ed) Proceedings of Hypertext, 1991, San Antonio, Texas. ACM, New York, NY, pp Fades P, Feng Q (1996) Multilevel visualization of clustered graphs. In: North S (ed) Proceedings of Graph Drawing, 1996, California, USA. Springer, Berlin, pp Eades P, Feng Q, Lin X (1996) Straight-line drawing algorithms for hierarchical graphs and clustered graphs. In: North S (ed) Proceedings of Graph Drawing, 1996, California, USA. Springer, Berlin, pp Ghandeharizadeh S, Ramos L, Asad Z, Qureshi W (1991) Object placement in parallel hypermedia systems. In: Lohman G, Sernadas A, Camps R (eds) Proceedings of Hypertext, 1991, Texas, USA, pp Gronbaek K, Bouvin N, Sloth L (1997) Designing Dexter-based hypermedia services for the World Wide Web. In: Bernstein M, Carr L, Osterbye K (eds) Proceedings of the Eighth ACM International Hypertext Conference, 1997, Southampton, UK. ACM, New York, NY, pp Haake A (1992) Cover: A contextual version server for hypertext applications. In: Lucarella D, Nanard J, Nanard M, Paulini P (eds) Proceedings of the European Conference on Hypertext, 1992, Milan, Italy. ACM, New York, NY, pp Haake A, Hicks D (1996) VerSE: towards hypertext versioning styles. In: Marshall C, Bernstein M, Poltrock S (eds) Proceedings of Hypertext, Washington, DC. ACM, New York, NY, pp Halasz F (1988) Reflections on NoteCards: seven issues for the next generation of hypermedia systems. Commun ACM 31(7): Hardman L, Bulterman D (1995) Towards the generation of hypermedia structure. In: Proceedings of the First International Workshop on Intelligence and Multimodality in Multimedia Interfaces, 1995, Edinburgh, UK. ftp://ftp.cui.nl/pub/mmpapers/immi1.ps.gz 15. Hardman L, Bulterman D, Rossum G van (1993) The Amsterdam Hypermedia Model: extending hypertext to support real multimedia. Hypermedia J 5(1): Hardman L, Worring M, Bulterman D (1998) Integrating the Amsterdam Hypermedia Model with the standard reference model for intelligent multimedia presentation systems. Comput Stand Interfaces 18(6 7): Hoschka P, et al. - Synchronized Multimedia Work Group of the World Wide Web Consortium (1998) Synchronized Multimedia Integration Language (SMIL) 1.0 Specification - W3C Recommendation. Available in IETF WEBDAV Working Group (1999) WWW Distributed Authoring and Versioning (webdav). Available in: Jourdan M, Layaida N, Roisin C, Sabry-Ismail L, Tardif L (1998) Madeus, an authoring environment for interactive multimedia documents. In: Effelsberg W, Smith B (eds) Proceedings of the ACM Multimedia Conference, 1998, Bristol, UK. ACM, New York, NY, pp Kim M, Song J (1995) Multimedia documents with elastic time. In: Zellweger P (ed) Proceedings of ACM Muttimedia, 1995, San Francisco, Calif. ACM, New York, NY, pp Lie H, Bos B (1999) Cascading Style Sheets level 1- W3C Recommendation. Available in Muchalual-Saade D, Soares L (1996) Fisheye views for compound graphs. Technical Report of the TeIeMídia Laboratory. PUC-Rio, Rio de Janeiro, Brazil 23. Muchaluat-Saade D, Soares L, Costa F, Souza G (1997) Graphical structured editing of multimedia documents with temporal and spatial constraints. In: Thalmann N, Thalmann D (eds) Proceedings of the Multimedia Modeling - MMM, 1997, Singapore. IEEE Computer Society, Los Alamitos, Calif., pp Nang J, Kang S (1997) A new multimedia synchronization specification method for temporal and spatial events. In: Georganas N (ed) Proceedings of the IEEE International Conference on Multimedia Computing and Systems - ICMCS, 1997, Ottawa, Canada. IEEE Computer Society, Los Alamitos, Calif., pp Osterbye K (1992) Structural and cognitive problems in providing version control for hypertext. In: Lucarella D, Nanard J, Nanard M, Paulini P (eds) Proceedings of the European Conference on Hypertext -ECHT, 1992, Milan, Italy. ACM, New York, NY, pp Pérez-Luque M, Little T (1996) A temporal reference framework for multimedia synchronization. IEEE J Sel Areas Commun (Special Issue: Synchronization Issues in Multimedia Communication) 14(1): Rodrigues R, Muchaluat-Saade D, Soares L (1998) Composite nodes, contextual links and graphical structural views on the WWW. J Braz Comput Soc (Special Issue on World Wide Web) 5(2): Rodrigues R, Scares L, Souza G (1997) Authoring and formatting of documents based on event-driven hypermedia models. In: Sarikaya B (ed) Proceedings of the IEEE Conference on Protocols for Multimedia Systems and Multimedia Networking, 1997, Santiago, Chile. IEEE Computer Society, Los Alamitos, Calif., pp Rossum G van, Jansen J, Mullender K, Bulterman D (1993) CM1IFed: a presentation environment for portable hypermedia documents. In: Rangan P (ed) Proceedings of ACM Multimedia, 1993, Anaheim, Calif. ACM, New York, NY, pp Sarkar M, Brown M (1994) Graphical fisheye views. Commun ACM 37(12): Santos C, Soares L, Souza G, Courtiat J (1998) Design methodology and formal validation of hypermedia documents. In: Effelsberg W, Smith B (eds) Proceedings of ACM Multimedia, 1998, Bristol, UK. ACM, New York, NY, pp Scares L, Casanova M, Rodriguez N (1995) Nested composite nodes and version control in an open hypermedia system. Int J Inf Syst (Special Issue on Multimedia Information Systems) 20(6): Scares L, Souza G, Rodrigues R, Muchaluat-Saade D (1999) Versioning Support in the HyperProp System. J Multimedia Tools Appl 8(3): Sugiyama K, Misue K (1991) Visualization of structural information: automatic drawing of compound digraphs. IEEE Trans Syst Man Cybern 21(4): Vazirgiannis M, Mourlas C (1993) An object-oriented model for interactive multimedia applications. Comput J 36(1): Vazirgiannis M, Stamati I, Trafalis M, Hatzopoulos M (1998) Interactive multimedia scenario: modeling and rendering. In: Proceedings of Multimedia track of ACM Conference - SAC, ftp://fpt.dbnet.ece.ntua.gr/pub/papers/publish/1998/vtsh98.ps.gz 37. Vazirgiannis M, Theodoridis Y, Sellis T (1998) Spatio-temporal composition and indexing for large multimedia applications. Multimedia Syst 6(4): Wiil U, Leggett J (1997) Workspaces: the HyperDisco approach to Internet distribution. In: Bernstein M, Carr L, Osterbye K (eds) Pro-

18 134 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system ceedings of the Eighth ACM International Hypertext Conference, 1997, Southampton, UK. ACM, New York, NY, pp Willrich R, Saqui-Sannes P, Sénac P, Diaz M (1996) Hypermedia document design using the HTSPN model. In: Courtiat J, Diaz M, Sénac P (eds) Proceedings of the Third International Conference on Multi- Media Modeling - MMM, 1996, Toulouse, France. World Scientific, Singapore, pp Yankelovich N, Meyrowitz N, Dam A van (1985) Reading and writing the electronic book. IEEE Comput 18(10): Luiz Fernando G. Soares received the B.Sc. degree in electrical engineering from the Catholic University of Rio de Janeiro, PUC-Rio, Brazil. He received, from the same university, the M.Sc. degree in electrical engineering in 1979 and the Ph.D. degree in computer science in He has worked as a researcher at COBRA (Brazilian Computers S/A) and at the IBM Scientific Center in Rio de Janeiro. He is currently a Professor at the computer science department of the Catholic University of Rio de Janeiro, Brazil. His research interests are in multimedia and hypermedia systems and in high-speed networks, in which areas he has several books and papers published. Débora C. Muchaluat Saade is a PhD candidate in Computer science at the Catholic University of Rio de Janeiro, PUC-Rio, Brazil. She received the B.Sc. degree in computer engineering in 1992 and the M.Sc. degree in computer science in 1996, both from the Catholic University of Rio de Janeiro. She has worked as a system engineer at the Exxon Oil Company from 1992 to 1994, as a researcher at the IBM Scientific Center in Rio de Janeiro from 1993 to 1994 and as a system engineer in the Cindam Bank from 1994 to She has been a researcher at the TeleMídia Laboratory in the Department of Computer Science of the Catholic University of Rio de Janeiro since She is currently a professor of the computer networks specialization course at the same university. Her research interests include visualization of hypermedia structures, version control mechanisms and temporal and spatial synchronization models for multimedia and hypermedia systems and high-speed computer networks. Rogério F. Rodrigues is a PhD candidate in computer Science at the Catholic University of Rio de Janeiro, PUC-Rio, Brazil. He received the B.Sc. degree in computer engineering in 1994 and the M.Sc. degree in computer science in 1997, both from the PUC-Rio University. He has worked as a system analyst at the Braziliam Oil Company (Petrobras) from 1994 to He has been a researcher at the TeleMídia Laboratory in the Department of Computer Science of the Catholic University of Rio de Janeiro since He is currently a professor of the Computer Networks specialization course at the same university. His research interests include temporal and spatial synchronization models for multimedia and hypermedia systems, open hypermedia systems, version control mechanisms in hypermedia models and high-speed networks.