Multimedia Systems 8: 118–134 (2000) Multimedia Systemsc© Springer-Verlag 2000

Modeling, authoring and formatting hypermedia documentsin the HyperProp systemLuiz Fernando G. Soares, Rogerio F. Rodrigues, Debora C. Muchaluat Saade

Departamento de Informatica, PUC-Rio, R. Marques de Sao Vicente 225, 22453-900 - Rio de Janeiro, Brazil;e-mail: lfgs@inf.puc-rio.br;{rogerio,debora}@telemidia.puc-rio.br

Abstract. This paper discusses multimedia and hypermediamodeling, authoring and formatting tools, presenting the pro-posals of the HyperProp system and comparing them to re-lated work. It also highlights several research challenges thatstill need to be addressed. Moreover, it stresses the impor-tance of document logical structuring and considers the useof compositions in order to represent context relations, syn-chronization relations, derivation relations and task relationsin hypermedia systems. It discusses temporal and spatial syn-chronization among multimedia objects and briefly presentsthe HyperProp graphical authoring and formatting tools. In-tegration between the proposed system and the WWW isalso addressed.

Key words: Nested context model – HyperProp system– Multimedia/hypermedia formatting – Document logicalstructuring – Spatio-temporal synchronization

1 Introduction

Hypermedia systems have been addressed in three differ-ent levels in the literature: storage, specification (authoring)and execution (formatting). This paper focuses mainly onauthoring and formatting. The specification level aims atdefining hypermedia application requirements and associatedconstraints. The formatting level refers to the developmentof protocols and schemes for document presentation, dealingwith intra-media temporal synchronization, and inter-mediatemporal and spatial synchronization.

Hypermedia systems may, and sometimes have to, intro-duce several desirable facilities, such as: allowing a struc-tured authoring; allowing the explicit definition of temporaland spatial relationships among their components (includingthose with non-deterministic times); supporting a rich setof capabilities for each media component (e.g., presentationduration flexibility, alternatives for different presentations,behavior changes during presentation, etc.); supporting rela-tions anchoring on internal points of their components (fine

Correspondence to: L.F.G. Soares

granularity), and not only on its start or end points (coarsegranularity); allowing quality-of-service (QoS) definition re-quired by media components (e.g., jitters, bandwidth, etc.);and allowing a presentation environment description (e.g.,network delay, supported devices, etc.).

In order to achieve these goals, hypermedia systems mustbe supported by powerful conceptual models. The greaterthe expressiveness of a model, the greater the document au-thoring complexity will be; authoring that must allow thecomplete document specification as easily (or friendly) aspossible. Similarly, the greater the complexity of the spatio-temporal (spatial and temporal) formatting algorithm will be.The formatting algorithm must consider all the environmentnon-determinism and allow correcting the presentation onthe fly, when unpredictable events occur (such as networkdelay, user interaction, etc.). It must also take advantage ofmedia segment QoS specifications and environment charac-teristics in order to trigger some actions, for instance, doingpre-fetch of the content of objects.

In this paper, we consider the development of multime-dia and hypermedia authoring and formatting tools, exam-ining the current state of the art. All the aforementionedfacilities are discussed. Solutions proposed in the Hyper-Prop system are presented and compared with related workthroughout the text. Moreover, we discuss a set of researchchallenges that need to be addressed before the full poten-tial of multimedia output technology can effectively be usedfor sharing information. The paper is organized as follows.In Sect. 2, general issues related to hypermedia documentauthoring are discussed, stressing the importance of docu-ment logical structuring, and showing how to use composi-tions to support them. Section 3 presents the HyperProp hy-permedia conceptual model and discusses how it addressesrequirements for authoring documents with spatio-temporalconstraints. Section 4 discusses several issues and solutionsfor defining hypermedia application requirements and asso-ciated constraints during authoring and formatting phases.Authoring tools with graphical interfaces make the task ofdocument specification easier. The HyperProp solution for agraphical editor is introduced in Sect. 5. Section 6 describesthe temporal and spatial-formatting machine of the system.

©SPRINGER-VERLAG, (2000). This is the author's version of the work. It is posted here by permission

of Springer-Verlag for your personal use. Not for redistribution. The definitive version was published in

in Multimedia Systems, {VOL8, ISS 0942-4962, (03/2000)}. The original publication is available at

http://springerlink.com - http://dx.doi.org/10.1007/s005300050155

Multimedia Systems 8: 118–134 (2000) Multimedia Systemsc© Springer-Verlag 2000

Modeling, authoring and formatting hypermedia documentsin the HyperProp systemLuiz Fernando G. Soares, Rogerio F. Rodrigues, Debora C. Muchaluat Saade

Departamento de Informatica, PUC-Rio, R. Marques de Sao Vicente 225, 22453-900 - Rio de Janeiro, Brazil;e-mail: lfgs@inf.puc-rio.br;{rogerio,debora}@telemidia.puc-rio.br

Abstract. This paper discusses multimedia and hypermediamodeling, authoring and formatting tools, presenting the pro-posals of the HyperProp system and comparing them to re-lated work. It also highlights several research challenges thatstill need to be addressed. Moreover, it stresses the impor-tance of document logical structuring and considers the useof compositions in order to represent context relations, syn-chronization relations, derivation relations and task relationsin hypermedia systems. It discusses temporal and spatial syn-chronization among multimedia objects and briefly presentsthe HyperProp graphical authoring and formatting tools. In-tegration between the proposed system and the WWW isalso addressed.

Key words: Nested context model – HyperProp system– Multimedia/hypermedia formatting – Document logicalstructuring – Spatio-temporal synchronization

1 Introduction

Hypermedia systems have been addressed in three differ-ent levels in the literature: storage, specification (authoring)and execution (formatting). This paper focuses mainly onauthoring and formatting. The specification level aims atdefining hypermedia application requirements and associatedconstraints. The formatting level refers to the developmentof protocols and schemes for document presentation, dealingwith intra-media temporal synchronization, and inter-mediatemporal and spatial synchronization.

Hypermedia systems may, and sometimes have to, intro-duce several desirable facilities, such as: allowing a struc-tured authoring; allowing the explicit definition of temporaland spatial relationships among their components (includingthose with non-deterministic times); supporting a rich setof capabilities for each media component (e.g., presentationduration flexibility, alternatives for different presentations,behavior changes during presentation, etc.); supporting rela-tions anchoring on internal points of their components (fine

Correspondence to: L.F.G. Soares

granularity), and not only on its start or end points (coarsegranularity); allowing quality-of-service (QoS) definition re-quired by media components (e.g., jitters, bandwidth, etc.);and allowing a presentation environment description (e.g.,network delay, supported devices, etc.).

In order to achieve these goals, hypermedia systems mustbe supported by powerful conceptual models. The greaterthe expressiveness of a model, the greater the document au-thoring complexity will be; authoring that must allow thecomplete document specification as easily (or friendly) aspossible. Similarly, the greater the complexity of the spatio-temporal (spatial and temporal) formatting algorithm will be.The formatting algorithm must consider all the environmentnon-determinism and allow correcting the presentation onthe fly, when unpredictable events occur (such as networkdelay, user interaction, etc.). It must also take advantage ofmedia segment QoS specifications and environment charac-teristics in order to trigger some actions, for instance, doingpre-fetch of the content of objects.

In this paper, we consider the development of multime-dia and hypermedia authoring and formatting tools, exam-ining the current state of the art. All the aforementionedfacilities are discussed. Solutions proposed in the Hyper-Prop system are presented and compared with related workthroughout the text. Moreover, we discuss a set of researchchallenges that need to be addressed before the full poten-tial of multimedia output technology can effectively be usedfor sharing information. The paper is organized as follows.In Sect. 2, general issues related to hypermedia documentauthoring are discussed, stressing the importance of docu-ment logical structuring, and showing how to use composi-tions to support them. Section 3 presents the HyperProp hy-permedia conceptual model and discusses how it addressesrequirements for authoring documents with spatio-temporalconstraints. Section 4 discusses several issues and solutionsfor defining hypermedia application requirements and asso-ciated constraints during authoring and formatting phases.Authoring tools with graphical interfaces make the task ofdocument specification easier. The HyperProp solution for agraphical editor is introduced in Sect. 5. Section 6 describesthe temporal and spatial-formatting machine of the system.

L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system 119

The integration of HyperProp with the WWW is discussedin Sect. 7. Section 8 contains our final remarks.

2 Structured authoring of hypermedia documents

An authoring environment should offer good editing andbrowsing tools for defining the content of its componentsand their granularity. The content is the set of informationunits of the component, and the content granularity specifiesthe subset of information units that can be marked and usedfor defining anchors. The exact notion of information unitand marked subset of information unit is part of the definitionof a component, from here on called anode. For example,an information unit of a video node could be a frame, whilean information unit of a text node could be a word or acharacter.

Authoring tools must also be rich in their semantic ca-pabilities to express relationship requirements among theirobjects. There are several types of relations that must besupported, among others

– reference relations: for example, those leading to a smallnote or those that jump to an entirely new section;

– context relations: for example, those that specify a hier-archical structure of a document, such as a book and itschapters, these chapters and their sections, and so on;

– synchronization relations: those that define both temporaland spatial ordering of objects;

– derivation relations: for example, those that bind data tothe object from which it was created; and

– task relations: those that organize tasks in a cooperativework.

Relations can be modeled, as in several authoring systems[13], by means oflinks alone. However, only one hyper-media model entity cannot have the semantics needed tomodel all kinds of relations.Compositionis another modelentity that can be used for structuring documents, depend-ing on which relation it models. The structured definitionof documents is desirable as it carries built-in concepts ofmodularity, encapsulation and abstraction.

The composition concept is introduced as a container ofnodes – media objects (media segments) or even other com-positions, recursively – and possibly relationships amongthem (represented by links). Compositions should allow thesupport for both top-down and bottom-up design. Some re-lations can be derived from the logical structure defined bycompositions, while others must be defined by other modelentities (links, for example). There are several structure-based authoring models, depending on which type of theaforementioned relations they are interested in structuring.

Compositions for context structuring – They are usu-ally defined in structured hypertext models. For example,they are found in thecontext nodesof HyperProp [6]. Fig-ure 1 shows an example of their use. In the figure, a book(compositionB1) is composed by chaptersC1, C2 andC3.CompositionC1 (chapterC1) contains sectionsS1.1 andS1.2and a media objectO1, etc. Other relations, for example, ref-erence relations, may be represented by links. For instance,a reference relationship between objectO1 and chapterC3is represented by linkl1.

Book

C2

C3

C1

S1.1 S1.2

O1

O1S3.1

Chapter

B1

Section

l1

Fig. 1. Context structuring. Nodes are represented byrectangles, compo-sitions are represented byrectangles containing other rectanglesand linksare represented byfilled arrows

Compositions for spatio-temporal presentation struc-turing – In this case, as stated in [37], compositions are usedfor representing both temporal and spatial ordering of its ob-jects. In other words, they are entities that embed a set ofnodes with their spatial and temporal presentation relation-ships. These compositions are found in several conceptualmodels, for example CMIF [29], SMIL [17], I-HTSPN [39],Madeus [19], MOAP [35] and IMAP1 [36].

Explicit temporal relationships among nodes can be de-fined in other entities, as thesync arcsof CMIF, the Petri nettransitions of I-HTSPN, thescenariotuples(andactions) ofMOAP and IMAP, and the temporal constraints of Madeus.Temporal relationships among nodes can also be derived im-plicitly by the composition type. For example, in CMIF andSMIL, relationships are given implicitly by the compositiontype that can beparallel or sequential, where all componentsmust be presented in parallel or in sequence, respectively.

In models where compositions structure temporal presen-tations, context relations as well as reference relations areusually given by links. There is no way to distinguish thesemantic of these two types of relations through that uniquemodel entity. This was one of the problems raised by Halaszin his seven issues on Notecards [13].

Compositions for grouping correlated versions ofnodes –Here, compositions are used for grouping objectsin order to maintain and manipulate a history of changes inthese objects, that is, they group nodes that represent ver-sions of the same entity at some level of abstraction. Thegrouped versions can even be (if it is permitted) the previ-ously mentioned types of compositions (for context or pre-sentation relations), allowing to explore and manage severalalternate configurations for a network of nodes (related bycontext or presentation purposes). Compositions for versioncontrol can be found in themobsof CoVer [11], versiongroupsof HyperPro2 [25] and versionand variant contextsof HyperProp [32, 33].

Compositions for task maintenance –Conceptual hy-permedia models that allow cooperative work usually usecompositions for structuring cooperative tasks that com-

1 Actually, IMAP derived from the MOAP model. However, there aresome conceptual differences between them, which will be highlightedthroughout the paper. Although the IMAP model appears as IMD in somereferences, we will not make distinction between them.

2 Note that HyperPro and HyperProp are distinct systems and NCM isthe conceptual data model of HyperProp.

120 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system

Fig. 2. NCM class hierarchy

pound the cooperative environment. Examples of such com-positions are thetasksof CoVer and VerSE [12], and thepri-vate basesof HyperProp [32, 33]. In all these models, tasksare a set of interrelated document components and subtasks,recursively.

Composition as a class from which other compositiontypes can be derived as subclasses – This is the case ofcompositions defined in models that have more than onestructuring purpose, or in meta-models for hypermedia au-thoring conceptual models. Examples of these compositionscan be found in the already mentioned CoVer and HyperPro,but especially in NCM –nested context model, the concep-tual model of the HyperProp system [6, 32, 33].

Figure 2 shows the NCM class hierarchy. In NCM,usercontextsare used for context structuring;versionandvariantcontextsfor derivation structuring;private basesandpublichyperbasefor cooperative work structuring; andtrails forguided-tours structuring and for maintenance of user navi-gation history. In NCM, temporal and spatial relations aredefined by links. Reference relations are considered to bea particular case of synchronization relations triggered by auser interaction.

It is worth noting that the context class of NCM mightalso be specialized in parallel-composition and sequential-composition subclasses (not shown in the figure), with thesame functionality defined in CMIF and SMIL.

Despite several work discussing compositions and theirmany different types of use, several open issues still re-main. Many systems allow the creation of document struc-ture but give authors no support in determining what wouldbe the most effective structure to be created. Support forautomatic generation of document structure is in the begin-ning, for example, automatic composition extraction fromspatio-temporal specifications. Even a more difficult issue isthe structured-document automatic generation. For instance,based on a subject and the knowledge of a user task, astructured-document can be generated by combining appro-priate media items into coherent groups with derived tem-poral and layout presentation specifications. The goal in thiscase is to (semi-) automatically generate presentations, thusreducing the effort required by the publisher to cater for

such a variety of users. Some of these works are reported in[14, 16].

3 The NCM

Authoring tools are based on conceptual models that mustbe rich in their semantic capabilities to represent complexmultimedia objects and also to express their relationship re-quirements. Hypermedia models are also the core of format-ting tools. This section is dedicated to the NCM presentation,providing the basis for the discussion about authoring andformatting facilities held throughout the following sections.

In NCM (refer to Fig. 2), anentity is an object that has asattributes3 a unique identifier, an access control list and anentity descriptor. Theunique identifierhas the usual mean-ing. For each entity attribute, theaccess control listhas anentry that associates a user or user group to its access rightsfor the attribute. Theentity descriptorcontains informationdetermining how the entity should be presented in time andspace. The value of this attribute is usually a descriptor ob-ject, as will be defined in Sect. 3.3.

3.1 Content and composite nodes

A nodeis an entity that has as additional attributes a contentand an anchor ordered list. Thecontentidentifies a set of in-formation units. Each element in the anchor list is called ananchorof the node. An anchor identifies aregion. The iden-tification can be done indirectly, in which case an anchorspecifies ananchor objectthat has an additional attributefor a region identification. A region is defined by a markedsubset of information units of the node content. The exactdefinition of node content and region depends on the nodeclass, as mentioned in Sect. 3. References to an anchor canbe made through its unique identifier or through a node iden-tification and the anchor position in the list of anchors of thenode.

NCM distinguishes two basic classes of nodes, calledcontent node and composite node. Intuitively, acontent nodecontains data whose internal structure is application depen-dent, modeling traditional hypermedia nodes. NCM requiresthat the special symbolλ must be a valid region value, rep-resenting the whole node content. Content nodes may bespecialized in other classes (text, graphic, audio, video, etc.),as required by applications. Section 7 gives an example of aspecialization of the text node class that will allow WWWdocuments to incorporate NCM features, called HTML node.

The logical structuring of a document is made by intro-ducing the composition concept. In NCM, acomposite nodeC has, as content, a listS of links and nodes, which maybe content or composite nodes, recursively. Moreover, ananchor ofC must be a sublist ofS only containing nodes.Similar to content nodes, in the case of composite nodes, thespecial symbolλ is a sublist ofS containing all its nodes. Wesay that an entityE in S is a component ofC and thatE iscontained inC. We also say thatA is recursively contained

3 We will frequently use the name of the attribute of an entity to referto the attribute value. When the context does not allow this simplification,we will explicitly use the termattribute value.

L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system 121

in C if and only if A is contained inC or A is contained ina node recursively contained inC. An important restrictionis made: a node cannot be recursively contained in itself.

As a node can be recursively contained in different com-positions (for example, nodeO1 in compositionsC1 andC3in Fig. 1) and composite nodes can be nested to any depth,it is necessary to introduce the concept of perspective. In-tuitively, the perspective of a node identifies a sequence ofnested composite nodes through which a given node instan-tiation is being observed. Formally, aperspectiveof a nodeN is a sequenceP = (Nm, . . . , N1), with m ≥ 1, suchthat N1 = N , Nm is a node not contained in any com-posite node, and fori ∈ [1, m), Ni+1 is a composite nodeand Ni is contained inNi+1. For example, in Fig. 1, nodeO1 has two different perspectives:P1 = (B1, C1, O1) andP2 = (B1, C3, O1).

Composite nodes are objects whose semantics must bewell known by the model. A conceptual model should notonly represent data-structural concepts, but also define dataoperations to manipulate and modify structures. Thus, ev-ery composite nodeC has the following methods:Insertnode, Remove node, Create link, Destroy link, Display struc-ture (usually defined by the composite node descriptor, thismethod displays the structure of nodes and links – graph– contained in the composite node; moreover, the methodis usually an editor providing all other operations for com-posite nodes),Select node(that instantiates the node, if notinstantiated yet, and calls its show4 method),Select link(thatinstantiates the link, if not instantiated yet, and calls its showmethod),Select father node(that instantiates the compositenode that contains nodeC through the current perspective,if not instantiated yet, and calls its show method), and meth-ods to change the node state defined for versioning purposes[33].

Parameters like a large number of nodes and links canhinder document navigation mechanisms. Disoriented usersneed context information to reestablish the sense of orien-tation. In particular, temporal context information is neededto answer questions like ”How did I get here?”. These ques-tions are answered introducing the concept of trail.

Given a composite nodeC, a trail T for C is a compos-ite node containing an ordered list of nodes and trails, suchthat all nodes are recursively contained inC and all trails aretrails for C. Moreover,T has as attributescurrent, whosevalue indicates the current entity (in the list) ofT , andview,whose value associates each occurrence of a nodeN in thecontent ofT with a sequence of nested nodes (C, . . . , N ).We also say thatT is associated toC. Trails come equippedwith the following additional methods:next (to jump to thenext entity of the trail content – the ordered list – in relationto the current entity),previous(to move to the previous en-tity), home(to return to the first entity in the list), andlast(to go to the last entity in the list).

Note that a node or a trail may occur more than oncein the ordered list ofT . Moreover, each node occurrenceis associated with a sequence of nested composite nodesfrom the point of view ofC. Trails are useful for lining up

4 Every entity should have a method, generically namedshow, topresent/edit its content. This method is defined by the entity descriptoror a descriptor object.

hypermedia documents and for implementingguided tours.As in Intermedia [40], a specialsystem private base trailkeeps track of all navigation made during a work session,so that a user can move at random from node to node andgo back step by step.

Besides the aforementioned methods, every trail musthave methods to enable or disable actions of the other meth-ods. Thus, we have

– enable: that enables the methodsnext, previous, homeand last, and disables the methodsinsert nodeand re-move node.

– disable: that disables the methodsnext, previous, homeandlast, and enables the methodsinsert nodeandremovenode.

The reason to have methodsenableand disable is to pre-vent a trail from being simultaneously used for navigation(through the trail) and for maintaining navigation history.

NCM defines acontext nodeC as a composite nodewhose content is a set of links, content nodes and contextnodes. As a set, the content cannot have replicated compo-nents, different from a trail. Context nodes have an additionalattribute called descriptor set. Adescriptor setcontains foreach node contained inC a descriptor object or a null value.The notion of descriptor, as already mentioned, will be de-fined in Sect. 3.3. Unfortunately, the definitions of the NCMclasses are mutually recursive, hindering the presentation ina linear text. This work is a typical example where the hy-pertext paradigm would be very useful.

3.2 Links and events

An eventis an occurrence in time that can be instantaneousor can occur over some time period5. A presentation eventis defined by the presentation of a marked set of informa-tion units (an anchor) of a node in a given perspective. Aselection eventis defined by the selection of a marked setof information units of a node in a given perspective. Anattribution eventis defined by the changing of an attributeof a node in a given perspective.

In NCM, an event can be in one of the followingstates:sleeping, preparing, prepared, occurring, pausedandaborted. Every event has an associated attribute, namedoc-currences(OC). OC counts how many times an event tran-sits from occurring to prepared state during a document pre-sentation. Presentation events also have an attribute, namedrepeat, which indicates how many times the event shouldbe presented in sequence. Moreover, a presentation eventhas other attributes defined by the descriptor object, amongthem a tuple, tmin, topt, texp, tmax, fcost >, where the min-imum allowed event presentation durationtmin, the maxi-mum allowed durationtmax, the duration that would givethe best quality of presentationtopt and the cost of shrink-ing or stretching the event presentation durationfcost aredefined. The expected valuetexp is initially set to be equalto the optimum value. As will be seen in Sect. 6, it is arole of the spatio-temporal formatter to choose an expected

5 We will follow as much as possible the definitions used in Perez-Luqueand Little’s Temporal Reference Framework [26].

122 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system

Sleeping Preparing Prepared Occurring Paused

Stop Resume

Aborted

AbortAbortPrepare Start (n) Pause

Stop

Fig. 3. HyperProp (NCM) presentation event state machine

duration based on this specification and other information,such as the description of the platform where the documentwill be presented.

Intuitively, taking a presentation event as an example(see Fig. 3), it starts in the sleeping state. It goes to thepreparing state while some (pre-)fetching procedure of itsinformation units is executed. At the end of the procedure,the event goes to the prepared state and its repeat attributeis initialized. At the beginning of the information unit pre-sentation it goes to the occurring state. If the presentation istemporarily suspended, the event stays in the paused state,while the situation lasts. At the end of the presentation, theevent comes back to the prepared state, when theOC at-tribute value is incremented and the repeat attribute value isdecremented. After being decremented, if the repeat attributecontains a value greater than zero, the event presentation au-tomatically restarts. If the presentation is abruptly finished,the event goes to the aborted state and immediately afterit comes back to the prepared state, without incrementingthe OC attribute value and setting to zero the value of therepeat attribute. Obviously, events like selection and attribu-tion may stay in the occurring state only during an infinites-imal time. The HyperProp event state machine is executedunder the document spatio-temporal formatter responsibility.

An NCM link is an m : n relationship composed bymsource end points,n target end points and a meeting point.Since a node may pertain to more than one composite node,the source and target end points of a link are identified bypairs of the form< (Nk, . . . , N1), α, type >, such that

1. N1 is a node in the node list (Nk, . . . , N1), calledanchornodeof this link;

2. Ni+1 is a composite node andNi is contained inNi+1,for all i ∈ [1, k), with k > 0;

3. α is an anchor ofN1 or the identification of an attributeof N1;

4. type specifies the event associated toα (presentation, se-lection or attribution). In the case of an attribution event,α is an anchor ofN1 or specifies an attribute ofN1. Oth-erwise (selection and presentation), it specifies an anchorof N1.

5. Nk, called head nodeof the link, is a component ofthe composite node or the composite node itself (thatcontains the link).

A meeting point attribute value is a specialization of thescript class, called meeting-point object. A script has an at-tribute (called content) that contains an ordered operationsequence. Ameeting pointis a script that contains only oneoperation composed by a condition and an action. Conditionsare related to source end points, while actions are operationsthat should be applied to target end points. When a conditionis satisfied, its associated action is fired.

Li nk l1 is defined in composition C1 as:

Source end points Target end pointsEv1=<(C3,T), ∝, selection>

where∝ is an anchor of T

Ev2=<(C2,A), λ, presentation>

Ev3=<(C2,V), λ, presentation>

Meeting point

Condition: (state(Ev1)=occurring, state(Ev1)=prepared)

Action: (start(Ev2) | start(Ev3))

C2

V

A

l1

C1

C3

T

V

l2

Il3

Fig. 4. Link definition example

Meeting-point conditions evaluate boolean values andcan be binary simple or compound. Everybinary simplecondition is expressed by two unary conditions: a previouscondition, to be satisfied immediately before time instanttwhen the condition is evaluated, and a current condition, tobe satisfied at timet. A binary simple condition is satisfiedwhen both unary conditions are satisfied (true). Either theprevious or the current condition can receive the true value,if they are not relevant in the binary simple condition evalu-ation. Comparison operators used in the evaluation of unaryconditions are: =,/=, <, ≤, >, ≥. Comparisons are madeconcerning

– states of an eventE.– Attributes associated to an eventE, like theOC (occur-

rences) attribute, and valid values for these attributes.– Node attributes, in the case of an attribution event, and

valid values for these attributes.

Any logical condition expression based on the operators∧(and), ∨ (or), ¬ ( (not) and⊕(±delay) defines a com-pound condition. The logical operators∧, ∨ and¬ have theusual meanings of the propositional logic. An expressionC ⊕ (±delay) is satisfied at a timet if condition C wassatisfied at time [t − (±delay)]. As in the event presenta-tion duration specification, thedelay is specified as a tuple< tmin, topt, texp, tmax, fcost >.

Meeting-point actions can be simple or compound. Somesimple actions for presentation events are shown in Fig. 3in handwriting letters. A compound action is defined byan action expression based on the operators| (parallel)and → (sequential), defining the execution order of eachelement of the action. Moreover, every simple or com-pound action has an optional attribute calledwait. This at-tribute determines the time that has to be waited beforeexecuting the action. As in the event presentation dura-tion specification, thewait attribute is specified as a tuple< tmin, topt, texp, tmax, fcost >.

For a link definition example, see Fig. 4.NCM links for a node can be defined in any composite

node of the node perspective. Thus, links that anchor on anode are dependent on the context (nesting of compositions)a node is within. In order to define thesecontextual links, wehave: given a nodeN1 in a perspectiveP = (Nm, . . . , N1),with m > 0, we say that linkl is a contextual link inPfor N1 if and only if there is a composite nodeNi in Pcontainingl, for i ∈ [1, m], such that

1. if i > 1, then (Ni−1, . . . , N1) is a node list of one of theend points ofl;

2. else, (N1) is a node list of one of the end points ofl.

It is also important to define the notion ofvisible link todetermine which links actually touch the node instantiation

L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system 123

(anchor at the node or inside a component recursively con-tained in the node). Given a nodeN1 and a perspectiveP = (Nm, . . . , N1), we say that linkl is visible fromP byN1 if and only if there is a composite nodeNi in P con-taining l, for i ∈ [1, m], such that

1. if i > 1 then (Ni−1, . . . , N1) is a sublist of the node listof one of the end points ofl;

2. else, (N1) is a sublist of the node list of one of the endpoints of l.

For example, in Fig. 4, nodeV in perspectiveP1 =(C1, C2, V ) has visible link l1 and in perspectiveP2 =(C1, C3, V ) node V has visible link l2. As another exam-ple, nodeC3 in perspectiveP3 = (C1, C3) has two visiblelinks l1 and l3 and one contextual link which isl3.

3.3 Spatial synchronizationand temporal behavior specification

In NCM, the definition of the content of a document compo-nent (content or composite node) as well as the definition ofits anchors are contained in what we calldata objects– DO.The creation of such objects is usually made by specific me-dia editors. A data object is created either as a totally newobject or as a local version of other data objects or as a localversion of already created persistent objects, calledstorageobjects– SO. Data objects created from other objects areconsidereddata versions.

The aggregation of a data object and a descriptor (layoutspecification) in order to present a component is called arep-resentation object– RO and is considered arepresentationversionof the data object. Thus, storage, data and represen-tation objects are related through versioning operations, asdiscussed in [33].

Intuitively, the plane of storage objects, represented bythe lower plane of Fig. 5, groups all entities stored in aserver, similar to a WWW server. The upper plane, whichrepresents the plane of representation objects, collects allentities used during a user work session. The middle plane,which represents the plane of data objects, groups data ob-jects used by the client to create representation objects. It isworth mentioning in some implementations the middle planecan be an abstract plane and may not have any instantiatedobject.

In NCM, when presenting a node, the descriptor explic-itly defined on the fly by the end user bypasses descriptorsdefined during the authoring phase. These, in turn, have thefollowing precedence order: first, that defined in the link (inprepare and start actions) used for reaching the node; second,that defined in the composite node that contains the node, ifit is the case; third, that defined within the node (descriptorattribute); and finally, the default descriptor defined by thenode class.

Figure 5 shows the association of a data object node anda descriptor. This association is represented by dashed linesconnecting data objects in the middle plane to representationobjects drawn in the upper plane. Note that one node canbe combined to different descriptors, originating differentrepresentation objects of the same node. Figure 5 shows thisfeature with the association of descriptorsDe1 and De2 to

Representation ObjectPlane

Data ObjectPlane

Nodes

Storage ObjectPlane

Descriptors

DescriptorsNodes

Y CA

Z E

Dey

De2

De1

DeEDez

Dec

Y

CA

E

A

E

Y’ C’A’

Z’ E’

Dey

De2

De1

DeEDez

Dec

C’A’

E’

A’

E’

Y”

C”

A1”

Z” E”

Y”

E”

A1”

A2”

A2”

C”

A1”

E”

A2”

Y’

Fig. 5. Object planes

nodeA′, creating representation objectsA′′1 andA′′

2 . In thefigure, A′ has two different representations, for example,one for each different way to access it. One can reachA′by depth navigation (through composite nodeC ′) using thedescriptorDe1, or it can be accessed by the link that touchesit using descriptorDe2.

More precisely, adescriptor is an entity that has as at-tributes a start specification and a collection of event de-scriptions.

A start specificationcontains all information needed tostart a node presentation. In particular, it defines methodsfor displaying and editing a node. These methods can beany program or even procedures defined in the descriptoritself. A start specification can also have an ordered list ofoperations that must be executed when preparing the nodefor presentation. This list defines all parameters needed forcreating a representation object from a data object.

The procedures specified in the creation of a represen-tation object act on content or composite nodes and theirevents. Thebehavior specification proceduresprovide func-tions to implement a user interface used for creating andediting events in the representation objects. Examples ofthese procedures are the ones found in the temporal andspatial views that will be discussed in Sect. 5. Thetem-poral analysis proceduresfacilitate the automatic creationof the presentation schedule, providing timing informationabout the object being presented, as, for example, the timeinterval since an event has occurred. Thecontrolling proce-durespermit the system to control the behavior of objects,for example, increasing or decreasing the speed of a videoduring the document presentation. These procedures are usu-ally part of the editors and viewers specified in the descriptorstart specification. For all types of nodes, descriptors shoulddefine the controlling procedures that can be used for chang-ing the node presentation behavior. Node descriptors should

124 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system

also define procedures to perform actions (prepare, start,stop, etc.), as defined in links.

The list of operations defined in a descriptor is a special-ization of the script class, containing an ordered sequence ofoperations. Each operation is composed by a condition anda set of actions. When a condition is satisfied, it fires theassociated set of actions. Operation list conditions are simi-lar to link meeting-point conditions, but their entry scope islimited to events and attributes of the representation objectcreated using the descriptor.

Operation list actions are dependent on the node classto which the descriptor will be associated. Actions shouldalways correspond to a presentation behavior change of therepresentation object, created using the descriptor (control-ling procedures). For example, in the case of an audio, abehavior change action could be ”increase the volume to XdB”.

An event descriptionin a descriptor consists of the tuple< α, type, duration, rep, oper, case >. Parameterα identi-fies an anchor of the data object to which the descriptor willbe associated.Type specifies the event type associated tothe anchor (presentation, selection) or has a special valueξ.Duration specifies the presentation duration of the event.In other words,duration specifies the values to initializethe tuple parameter,tmin, topt, texp, tmax, fcost, of the cor-responding presentation duration attributes of the event.Repspecifies a value to initialize the repeat attribute of the event(as discussed in Sect. 3).Oper specifies an operation listobject.Case indicates if the operation list can be executed.If the type parameter specifies a presentation or selectionevent, the operation list is executed when the event statechanges from prepared to occurring. If thetype parameteris ξ, the operation list must be executed if the event is in theoccurring state and when the descriptor receives a messagefrom another object specifying the anchor (for example, theactivate event actionof a link, not discussed in this paper).

We should highlight that

– behavior changes are programmed in operation lists, asin [3].

– Event descriptions can be enabled or disabled assigningvalues to the case parameter.

– An operation list can have temporal synchronization con-straints, as for example, 5 s after a certain condition issatisfied, action X must be executed.

– Behavior changes can alter the event duration and thushave to be taken into account by the formatter, espe-cially in the elastic time computation (as will be seenin Sect. 6). Examples are: ”Increase playback speed”,”pause 10 s”, etc.

3.4 Versioning and cooperative work

The notion of composite node can be used for supportingcooperative work and for abstracting the planes of Fig. 5.In order to be able to do that, the class of context nodes isspecialized in other classes. A similar concept can be foundin [11, 25].

A user context nodeU is a context node whose contentattribute only contains content nodes, user context nodes

and links. User context nodes have the semantics to modelcontext-structuring relationships, as defined in Sect. 2.

In NCM, only content nodes and user context nodes aresubject to versioning, as seen in dark-gray background inFig. 2. In order to address the problem of maintaining thehistory of a document, we introduce the classes of versioncontext nodes and variant context nodes, as detailed in [33].

In order to support cooperative work, NCM has threemore specializations of the context node class. Thepublichyperbaseis defined as a special type of context node thatgroups content nodes and user context nodes. All nodes inthe public hyperbasePH must be storage objects and if acomposite nodeC is in PH, then all nodes inC must alsobelong toPH. Intuitively, the public hyperbase groups allentities stored in a server, similar to a WWW server, andrepresents the nodes of the lower plane of Fig. 5.

NCM also defines a private base as a special type of con-text node, whose components are user context nodes, contentnodes, trails, and private bases. Briefly, a DO-private baseDOPB is a data object that collects all the nodes of themiddle plane of Fig. 5. An RO-private baseROPB is a rep-resentation object derived from aDOPB. The data objectsof DOPB will be associated to their descriptors, in orderto create the representation objects that will be containedin ROPB. Representation objects contained inROPB willbe created according to versioning operations. Intuitively,ROPB collects all entities used during a user work ses-sion, represented by the nodes of the upper plane of Fig. 5.Reference [33] discusses the versioning support of NCM,defining all its class attributes and methods, including meth-ods (operations) for version control and the redefinition ofthe display structure methods for all the subclasses derivedfrom the context class specialization.

4 Authoring and formatting issues on conceptual models

This section discusses some issues and compares relatedwork solutions for defining hypermedia application require-ments and associated constraints during authoring phase. Hy-perProp solutions are stressed throughout the section. Thestructured authoring issue was already discussed apart inSect. 2, but complementing punctual issues regarding thismatter are also discussed here.

4.1 Compositions and structured authoring

4.1.1 Relation inheritance in compositions

Relation inheritance in compositions means that relation-ships among nodes can be defined in any composition thatrecursively contains these nodes.

In some models, composite nodes are defined as a logicalgrouping of nodes only. Depending on the model, compositenodes can be nested (that is, a composition can recursivelycontain other compositions), as in CMIF, Hyper-G [2] andChimera [1]. In all these models, references represented bylinks are stored in an independent repository that sometimesis a single one (CMIF, Hyper-G and DHM). Relation inher-itance does not make sense in this case.

L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system 125

C2

C3

C1

S1.1 S1.2

O1

O1 S3.1

B1

l1

B2

Fig. 6. Relation inheritance

In other models, compositions group nodes and links(relationships), but a relationship is stored in an ancestor ofthe linked nodes, which is the lowest in the hierarchy, as it isthe case of thesync arcsof AHM [15]. Relation inheritancedoes not make sense in this case either.

In several other models (MOAP, I-HTSPN and Madeus,for example), relationships are defined between componentscontained in a composition. They allow a composition as anend point of a relationship, but not a component inside acomposition. This will prevent relation inheritance.

In other models, compositions contain not only nodes,but also links, and offer the possibility of defining relationsdepending on the context of discussion. When links are de-fined in a composite node, we can have a structured reuse ofrelations implicitly given by the composition, as well as ofrelations explicitly defined inside the composition. Indeed,relation inheritance in composition nesting (as exemplifiedin NCM) is very important in order to allow a more gen-eral composition (and thus structure) reuse and, therefore,making the authoring process easier. As an example, sup-pose the composition representing chapterC1 of Fig. 1. Fora new book given to a reader (compositionB2), it could bedesirable to introduce a relationship between sectionsS1.2andS1.1 of C1, to give a hint of related matters; for anotheradvanced reader, the book (compositionB1) should be deliv-ered without that relation, as in Fig. 1. Note thatB2 could bedefined as a composition containingB1 and the introducedrelation, reusing all the structure ofB1, as shown in Fig. 6.

4.1.2 Compositions with different entry points

Some models (MOAP, I-HTSPN and Madeus, for example)allow a composition as an end point of a relationship, butnot a component inside a composition. Different entry pointsin a composition are desirable since they allow differentpresentations of nodes that are recursively contained in thecomposition. This is also an important characteristic to allowstructure reuse and, therefore, making the authoring processeasier. NCM is an example of a model that allows suchfacility, since a link can go into nested compositions, asspecified by the node list of an end point of the link.

For example, in Fig. 7, the presentation of compositionC2 can be started through linksl1 or l3, coming from otherparts of the document. WhenC2 starts through linkl1, nodesV (video),A1 (background audio) andA2 (voice node) muststart at the same time. However, ifC2 starts through linkl3,

nodesV and A2 must start at the same time, without thebackground audio. Therefore, the presentation depends onthe external context, that is, on the navigation that led topresentation of the composite node.

4.1.3 Trails and guided tours

Parameters like a large number of nodes and links, lots ofchanges in the document, bad response time to user actions,insufficient visual differences among nodes and links, andvisually disoriented users are combined to hinder documentnavigation mechanisms. Disoriented users need context in-formation to reestablish the sense of orientation. In particu-lar, two kinds of information are needed: spatial context toanswer the question ”Where can I go now?” and temporalcontext to answer to ”How did I get here?”. The first ques-tion is treated in [22] and is briefly discussed in Sect. 5, thesecond one is answered introducing the concept of trail (inparticular, guided tours).

In some systems, as for example some HTML authoringsystems, a guided tour is implemented using links, that is,inserting a ”tour anchor” in each node, an anchor that leadsto the next node in the tour. In these systems, it is impossibleto have alternative guided tours. In HyperProp, guided toursare created using trails, overcoming the mentioned problem.

The great majority of WWW browsers do not differen-tiate the creation and navigation phases of a trail. In thesebrowsers, when a user navigates on the trail that maintainsthe navigation history, new entries are created leading to aloop in the next forward navigations. In order to avoid thisproblem, some systems, also exemplified by several WWWbrowsers like Nescape and Internet Explorer, do not insertnew entries during trail navigation, but loses informationof the trail from a certain node, if a navigation is madeto another node different from that specified in the trail.The HyperProp solution comes from theenableanddisablemethods of a trail, used for preventing the trail from beingsimultaneously used for navigation and for maintaining nav-igation history (as discussed in Sect. 3.1). If a user enablesthe navigation through the system private base trail, a copyof this trail is created, through which the navigation will oc-cur. During navigation on this copy, the system private basetrail is updated in order to maintain the navigation history.If a user navigates to a node out of the copy trail, this trailis simply destroyed.

4.2 Temporal synchronization

4.2.1 Relations based on event states

We usually need to test platform-dependent activities in or-der to start the presentation of an object. For example, wemay need to test if a video node was got from a remote serverbefore initiating its presentation in parallel with a local au-dio. HyperProp presents a solution through its event-statemachine, where any event state, or event-state change, canbe used in a condition of a link. None of the other relatedwork presents a solution to the problem.

126 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system

C2

A1

V

A2

N1

l1

l3

C1

C3

N2

V

A2

N3

l6

l2

T1T2

l4

l5

Fig. 7. Example of hypermedia document

4.2.2 Multipoint relations

Although relations are usually directional, they can be fol-lowed in both directions in almost all conceptual model im-plementations. However, few models allow multipoint re-lationships, that is, relations with several source and des-tination end points. I-HTSPN and NCM present a generalsolution for the specification of these multipoint relations,as seen in Sect. 3.2. Multipoint relations are important notonly because they allow to test conditions on several events,but also because they provide the facilities described in thenext two items.

4.2.3 Context for a link

Multipoint relations are an elegant solution for the problemof defining a context for a link, as stated by [15]: ”In hyper-media it is useful for the author to be able to specify whichparts of the presentation should remain and which shouldbe replaced when a link is followed. A source context fora link is that part of a hypermedia presentation affected byinitiating a link, and a destination context is that part of thepresentation which is played on arriving at the destinationof a link”. In HyperProp, context for a link is a natural con-sequence of a multipoint relation and not a new attribute fora 1 : 1 relation.

4.2.4 Relations based on combinations of presentationand selection events

Some models (CMIF, AHM, Madeus, etc.) define tempo-ral and spatial relationships only among presentation events;relationships triggered by selection events are defined in an-other model entity (usually a link) completely separate fromspatio-temporal relationships. In these models, it is not pos-sible to merge conditions spatio-time dependent and user-interaction dependent to trigger a navigation. For example,the situation ”play an audio explanation when a user selectsa text button during a video presentation” cannot be rep-resented. Note that this is also a typical case of multipointrelation. This situation is not rare and models should supportthem, as in NCM, IMAP and I-HTSPN.

4.2.5 Indefinite relationships

As defined by [26], indefinite temporal relationships arethose temporal relationships between time instants or time

intervals that are not explicitly or unambiguously given.Usually they are expressed as disjunctions of the basic tem-poral relationships (for example, ”at the same time or after”).Several models, exemplified by NCM and IMAP, presentsome support to indefinite relations. In [26, 37], all possibleindefinite relations for instant- and interval-based temporalrelations are discussed. If all those relationships are neededis an issue to be tested.

The event presentationdurationas well as thedelayandwait parameters (Sect. 3.2) used in NCM are specified by atuple< tmin, topt, texp, tmax, fcost >. The parameterstmin,topt, texp, tmax, can be either a non-negative real numberor the unspecified value (φ), or the unpredictable value (?).

The possibility of having comparison operators in unaryconditions (equal, unequal, less than, less than or equal,greater than, and greater than or equal), and of having anylogical condition expression based on the operators∧ (and),∨ (or), ¬ (not) and⊕(delay) in compound conditions, andmoreover, of having the event presentation duration and thedelay and wait parameters defined by an interval, gives Hy-perProp the capability of supporting indefinite relations.

4.3 Spatial synchronization and behavior specification

An authoring environment should allow the definition ofeach component’s expected behavior when presented. Sev-eral authoring systems (Firefly [3], CMIF, I-HTSPN, Hyper-Prop, SMIL, IMAP, etc.) define the layout of their compo-nents’ presentation separated from the associated data object.This will allow a better reuse of objects. For example, usingdistinct layouts, one can define different presentations for thesame media object. A text media segment can be presentedas text, or it can be synthesized as audio, depending on thelayout. Note that as different layouts can lead to differentevent durations, different quality of presentations and differ-ent platform requirements, the definition of all these issuesshould also be part of the layout and not part of the mediaobject. The presentation of an object should be created bythe spatio-temporal formatter from its corresponding layoutspecification and data.

Spatial synchronization can optionally be defined in thesame relations used for defining temporal synchronization.For example, a link synchronization relation can specify notonly the moment that a given presentation event should startbut also how the presentation (layout) inside the currentscenario must occur. Thescenariotuples (and actions) ofMOAP and IMAP, the spatial constraints of Madeus, andthe links of NCM are examples. Optionally, spatial synchro-nization can be defined in other entities different from links.This is the case of Firefly, CMIF, SMIL, I-HTSPN, and alsoNCM. In all these latter models, the spatial synchronizationis established by the placement of objects in an absolutespatial coordinate system, based on a paradigm analogousto the timeline for temporal synchronization, with all thedisadvantages reported in several works in the literature.

The spatial synchronization of IMAP is richer than theother models, as it allows event constraint-based spatialrelationships, generalizing the constraint instant-based andinterval-based paradigm for temporal relationships, presentin all mentioned models, including NCM.

L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system 127

Some systems allow a joint definition of the spatial syn-chronization, specifying the layout in an entity that can beshared by several media objects. This is the case of CMIF,where an event is assigned to achannel, which is an ab-straction of a set of properties shared by other objects of thesame media type. The use of CMIF channels can be verydesirable. For example, it ensures that the system’s audiodriver will not need to be reinitialized for every data blocksent to the device. It also allows special effects to be intro-duced when changing from an object presentation to another,which would be difficult to achieve with individual layoutentities.

Alternative layouts for the same media object are alsoimportant to allow platform QoS adaptation, user choice anddifferent presentations for a document depending on the nav-igation start point. Alternative layouts are allowed by severalsystems and models like NCM and SMIL.

Therefore, given a media object, an associated layoutshould be specified on the fly by the end user, or in the doc-ument specification, as NCM does. Presentation choices onlyin the document specification give the author the responsi-bility for designing and implementing alternate presentationsand (s)he often does not know the diverse needs of users.An authoring model should also allow defining the layoutin a media object attribute, or in a link (or any other entityspecifying spatio-temporal or reference relation) that has theevent as its destination anchor. Composite nodes may alsohave, for each node they contain, an attribute that can beused for storing the layout identifier for that node.

The layout specification should also specify behaviorchanges, that is, how a presentation of an object evolvesin time. This is the case, for example, of Firefly and NCM,where behavior changes and spatial synchronization are han-dled byoperation lists, contained in the descriptor in the caseof NCM. However, both Firefly and HyperProp only treatdiscrete behavior changes, that is, they identify time instantswhere behavior changes must be made, but do not allowcontinuous behavior changes. In [24], good ideas about thematter are given. We intend to incorporate these ideas inHyperProp.

5 Graphical interfaces for authoring

An author (who is usually a non-technical person) needstools for a high-level complete description of all aspectsof a multimedia application. Several systems (CMIFed, Hy-perProp, I-HTSPN, Madeus, etc.) offer authoring tools withgraphical interfaces, making the task of document author-ing easier. A graphical authoring environment should sup-port both top-down and bottom-up constructions. Constraintsamong media items (events) should preferably be defined di-rectly among those items.

Graphical authoring tools are usually based on differentviews that allow the specification of all types of relationspreviously defined. Each view favors one type of relationdefinition and all views must work together.

This section briefly presents the HyperProp authoringenvironment, which is based on three different views, asshown in Figs. 8–10.

Fig. 8. Structural view

Fig. 9. Temporal view

The first view shown in Fig. 8, calledstructural view,supports browsing and editing the logical context and taskstructure of documents, providing features for editing nodesand links and grouping them into compositions. In this view,nodes are represented by icons (content nodes) and rectan-gles (composite nodes), links are represented by lines andthe containment relationship is represented by the inclusionof rectangles and icons (nodes) and lines (links) in anotherrectangle (composite node).

Usually, the structural view has a very complex struc-ture, requiring sophisticated algorithms to build the view.The main motivation is to balance local detail and globalcontext with respect to a node in focus. Local detail is nec-essary to give information about the navigation possibilities,depending on the focus in the document structure. Globalcontext is important to give information about the focus po-sition within the overall structure.

There are several techniques to display graph structures[30]

1. Presenting all nodes and links of the graph, which al-lows users to have a global view of the structure. Thedrawback is that maps become confusing and do not helpuser navigation when the number of nodes and links in-creases.

2. Using scroll and zoom to view portions of the graph,which makes readable maps, but loses the overall struc-ture.

3. Using two or more views, one with a global view andanother with a small zoomed portion of the graph, whichhas the advantage of displaying local details and theoverall structure of the information space, but forces theuser to mentally integrate the views.

128 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system

4. Using filtering mechanisms, such as fisheye views, tobuild a global view, which only displays what is moreinteresting to the user at a certain moment, maintainingmap legibility. The difficulty is to find outwhat is moreinteresting to the user at that moment.

The last option is the most desirable solution, but the mostcomplex. Filters to hide information, and possibly landmarks(special nodes that always appear), will generally be neces-sary. Maintaining the legibility of maps is the main purpose.The user will only visualize what is interesting to him, de-pending on his position (focus) in the document structure andwhat is important regarding the overall structure of the hy-permedia document. In order to build those filters to presentstructural views of private bases and the public hyperbase,HyperProp uses an extension of the fisheye-view strategy forconceptual models that offer composite nodes [22, 23]. An-other important feature of the fisheye-view strategy is thatit allows users to choose the amount of information shown,tailoring the map according to their interests. Thus, userscan choose to see more or less detail in the structural view.Another desirable characteristic is to be at two foci at oncein the structural view. This can be needed, for example, be-cause some small structure is being copied from one placeto somewhere else, or because a relation is being authoredbetween two parts of the document.

Given a compound graph, there are several works propos-ing different types of layout [7, 8, 34]. However, a toolintegrating filtering and automatic graph layout design ori-ented to hypermedia users is still missing. Moreover, eventhe partial solutions already presented remains to be testedby a large number of naive users.

The second view, calledtemporal view, is responsible forsupporting the specification of temporal relationships amongcomponents of a document, defining their relative positionin time. As shown in Fig. 9, in this view, nodes are repre-sented by rectangles, whose lengths indicate the mean opti-mum duration of their presentation in a time axis and whoserelative positions establish their temporal synchronization.A remarkable difference between the temporal view and atimeline is that the time shown in the temporal view is notexplicitly specified by the author. It is derived from tem-poral constraints, just as an approximation of the real-timepresentation [23].

Temporal views are very useful to display related presen-tation events. However, when relationships can be definedamong selection and presentation events, it is difficult to rep-resent the situation. This happens because a user interactiontime instant is runtime dependent. The HyperProp poor so-lution [23] for this case is done by defining a region, calledlimbo, where nodes that do not have their presentation timeinstant exactly determined can be placed. Those nodes aredragged from the structural view and dropped into the limbo.All events related to nodes in the limbo are assumed to haveoccurred in the past, but one cannot determine when. Placingobjects in the limbo permits manipulating relationships, eventhough it is not precisely known when its source events haveoccurred. As far as we know, there is not an efficient graph-ical tool yet that permits the definition of non-deterministictemporal relationships.

Fig. 10. Spatial view

Temporal views usually display the mean optimum dura-tion of an object presentation. However, duration is usuallyspecified as varying within an interval. In [20], the durationof an object is represented by an elastic time-box with aspring symbolizing the variation. Again, as far as we know,there is not a tool yet that permits a good representation oftime variations when several objects are involved.

Relations can be defined in the temporal view, like linksin NCM, sync arcs in CMIF and temporal constraints inMadeus. However, unless relations are stored in a separaterepository, the composition that contains it should be definedusing the structural view. This is one of the reasons whythese two views must work together.

Finally, the third view, calledspatial view, supports thedefinition of spatial relationships among components of adocument, establishing their presentation characteristics ina given device, at a specific instant of time. As shown inFig. 10, in this view, rectangles represent spatial character-istics of objects at a given specific time instant, such as theirposition in a video monitor or their volume level in an audiodevice.

In CMIFed, the player shows the effect of mapping theabstract document to a particular platform. It acts indeed asa document spatio-temporal formatter. However, sometimesthe author needs an abstract spatial view independent of theplatform, like the spatial view of HyperProp.

The spatial view should allow spatial relationship defi-nition among its objects, like the temporal view allows tem-poral relationship definition. Obviously, spatio-temporal re-lations must be defined using the two views in an integratedway. Moreover, the relation may have to be placed inside acomposition, so the structural view must also be integratedwith the two other views.

Indeed, the three views should be related to each other.In HyperProp, the focused object in the structural view isthe basis for the time chain shown in the temporal view. Foreach point of time in the latter view, the spatial view showshow components will be presented in the space defined bythe output devices. Links and nodes defined in the temporalview are immediately updated in the structural view, andvice-versa. The same integration can be found in CMIFedand Madeus.

The spatial and temporal views can also have a very com-plex structure, requiring sophisticated algorithms to build theview. Filtering may also be needed in this case. An efficientdesign of these two views for a large number of nodes is agreat problem to be faced.

Other open issues can be mentioned. There is not an ef-ficient tool that guides authors to define temporal and spatialconsistent documents for a given particular platform. Indeed,

L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system 129

the complete design of a graphical authoring tool is an openissue. The few existent proposals are not definitive. Theyhave to be tested for large document authoring and with alarge number of naive authors, so that we can have feedbackfor better designs.

6 Spatio-temporal formatting in the HyperProp system

The spatio-temporal formatter, or simply formatter, is re-sponsible for controlling the document presentation basedon its specification and on the platform (or environment)description. The main idea is to build an execution plan(a schedule) to guide the formatter in its task. The execu-tion plan should contain information about actions that mustbe fired when an event-state change is signalized. Platformspecifications would typically be supplied by someone suchas a system or network administrator rather than an authorof a particular document.

Before discussing the HyperProp formatter, we have todefine several concepts that will be used throughout thissection

1. We say that a presentation event has apredictable du-ration when it is possible to determine the exact timeits state will change from occurring to prepared, in rela-tion to the time its state has last changed from preparedto occurring. Otherwise, we say that the event has anunpredictable duration.

2. We say that an event ispredictable in relation to anothereventE when it has a predictable duration and it is pos-sible to know, a priori, its expected start time, that is,the time its state changes from prepared to occurring, inrelation to eventE (in relation to any time instant of theoccurring period ofE).

3. An event is calledunpredictablewhen there is no eventE for which it can be predictable.

4. The sequence of event occurrences in time is calledtimechain.

5. A time chain is calledpartial time chainwhen all itsevents, except the first one, are predictable in relation toat least one event of the same partial time chain.

6. If an eventE is predictable in relation to another eventthat pertains to a partial time chainPTC, thenE alsopertains toPTC.

As a corollary, we have that every selection event is unpre-dictable and begins a new PTC.

Needless to say, in HyperProp, we can have

1. events that have an unpredictable duration, but whoseduration can be limited within a finite interval; and

2. events with predictable duration that can be chosenwithin an interval.

As previously mentioned, hypermedia systems can be ad-dressed at three different levels: storage, specification (au-thoring) and execution (formatting). Each one of these levelsis based on a conceptual data model, which can be differentfrom those of the other levels. In practice, choosing a sin-gle model for a particular application is not often sufficient.One needs both the flexibility of more abstract models andthe accurate control over the presentation offered by more

Pre-Compiler Compiler

Executor ViewersControllers

Events signaling andattribute values

report

Structure forExecution Plan

Viewersinterface

Actions,Get & Setattributevalues

Structurefor

ExecutionPlanPresentation

specification

Temporal and Spatial Formatter

Run TimeCompile TimePre-Compile Time

MultimediaobjectsServer

...Editors/Viewers

Storage Subsysteminterface

Environmentspecification

AuthoringSubsystem

interface

Inconsistenciesreport

TimeChains

Controllersinterface

Fig. 11. The HyperProp spatio-temporal formatter architecture

presentation-dependent models. As a result, one needs toolsto convert documents from one model into another.

Authoring models are usually based on higher level ab-stractions, as they are closer to users. The great expressive-ness of authoring models must be offered to users in aneasy and comprehensible way. These are some of the rea-sons that led HyperProp to use an event-driven model tospecify the logical structure and presentation characteristicsof a document. The input structure for the spatio-temporalformatter will be extracted from this model. As the formatteris closer to the operational machine, a lower level model forparallel-state machines may be the most adequate for taskslike scheduling.

Almost all aforementioned systems follow different mod-els for each one of their levels. For example, Firefly usesa time instant (event-driven) constraint-based model duringauthoring, which is translated by a compiler (called sched-uler) to PTCs structured as timelines, which feed the runtimeformatter. Firefly tries to avoid timeline problems by con-catenating PTCs to support unpredictable events. The CMIFformatter (called player) has a compiler that generates, fromthe document specification, a directed graph of timing de-pendencies, which feeds the player. The graph used by CMIFis similar to a Petri net. The I-HTSPN toolkit has a compiler(called MHEG translator) that converts the I-HTSPN speci-fication (based on extended-time Petri nets) into an MHEGrepresentation. An MHEG machine should control the doc-ument presentation. HyperProp uses an object-oriented timeinstant event-driven model (NCM) in the authoring phase,which is translated to a graph (similar to a Time Stream PetriNet [39]) to model the execution plan.

Other data models may coexist between authoring andexecution phases. For example, in HyperProp, NCM canbe converted to an RT-LOTOS specification for consistencychecking, as described in [31].

The HyperProp spatio-temporal formatter architecture,which is very similar to the Firefly formatter architecture,is composed of four elements: the pre-compiler, the com-piler, the executor and the viewer controllers (or simplycontrollers), as shown in Fig. 11.

Although thepre-compilerperforms formatting tasks, itis indeed a support for the authoring environment. Its mainfunction is to check the temporal and spatial consistency ofeach document PTC. The pre-compilation is an incrementalcompilation done during authoring time. It can help authors

130 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system

find errors, giving an instantaneous feedback whenever anyinconsistency is detected, like mismatched temporal relation-ships (temporal inconsistency), or conflicts in a device use(spatial inconsistency) in an abstract and ideal platform.

A document is calledintrinsically consistentif no in-ternal synchronization constraints can lead to inconsistentsituations when the presentation is performed over an idealplatform. However, even if a document is intrinsically con-sistent, its presentation can lead to a deadlock situation, be-cause the analysis is only based on the intrinsic (specified)duration of object presentations. Therefore, we define a doc-ument asextrinsically consistentif its presentation in a realand resource-limited platform is also consistent. In this case,the verification is performed over the composition of the doc-ument specification and the platform specification, testingproperties that are not intrinsic to the document, but dependon the behavior of its presentation over a specific platform.During pre-compilation time, only intrinsic consistency canbe checked. This is the kind of inconsistency check made inthe temporal view discussed in Sect. 5.

Considering the complete document presentation specifi-cation and the platform description, thecompiler is respon-sible for generating the data structure used by the spatio-temporal formatter to build the execution plan. As stated inSect. 3.2, in the event presentation duration specified by a tu-ple < tmin, topt, texp, tmax, fcost >, the expected valuetexp

is initially set to be equal to the optimum value. At compiletime, the expected duration should be computed based on thecost minimization in order to warrant the spatial and tempo-ral consistency of the document. This computation mecha-nism is calledelastic time computationin [20]. The compileralso had better evaluate, from the platform specification, atuple < tmin, texp, tmax > for the preparing duration of apresentation event (that is, the time the event spends in thepreparing state), wheretmin is the minimum preparing du-ration, tmax is the maximum preparing duration andtexp isthe mean preparing duration. This will be very useful forpre-fetching decisions, as will be explained later.

In HyperProp, the Multimedia Objects Server (NCMserver) is responsible for the persistent storage of the pub-lic hyperbase, descriptor objects, version contexts and pub-lic trails. When a user asks for the presentation of a docu-ment, the compiler gets from the NCM server all compositenodes recursively contained in the composition that modelsthe whole document. From the links contained in these com-positions, expressions in their meeting points are compiledin order to obtain the concerned events and create PTCs withthem. For each event in a time chain, the compiler createsa reference to each meeting point where it is used. Thereby,when a change in an event state occurs, the executor knowsall links that must be evaluated. All PTCs together make thepresentation state machine that feeds the executor.

The execution plan created by the compiler will guide theformatter in adjusting object duration on the fly, as well asin pre-fetching components’ content, in order to improve thepresentation quality and to reduce the probability of temporaland spatial inconsistencies.

Representation objects are created by theexecutorfromthe corresponding descriptors and data objects. However,the executor does not create representation objects directly.Based on the execution plan, the executor starts a controller

and passes all information needed to create and control eachobject presentation. This information includes the data ob-ject, the descriptor and a list of all events that are conditionsin the meeting points of the data object contextual links (asdiscussed in Sect. 3.2). This information is obtained fromthe data structure built by the compiler. The executor knowsthe type of controller to be started based on informationcontained in the descriptor (start specification).

The controller creates the representation object gettingits corresponding data object from the NCM server. Throughthe associated descriptor, the controller obtains the operationlist that determines the behavior changes that must happenduring the object presentation. The concept of a controllerfor each representation object, with a well-defined interfaceto the executor, will allow incorporating commercial viewersinto the system, such as Word, Netscape, etc. It will only benecessary to implement controllers for these viewers in orderto exchange messages according to the defined interface.

From the viewers, controllers receive signals of changesin event states to be reported to the executor. The executorthen updates the appropriate event state machine and evalu-ates all links associated with the event. If there is a conditionin a link meeting point that is satisfied, the correspondingactions are fired. When the controller reports a change inan event state to the executor, the executor checks the ex-pected time in the execution plan to see if some adjustmentis needed. Adjustments may be done through messages sentto controllers, for example, telling them to accelerate thepresentation rate. These elastic time adjustments are not im-plemented yet and are addressed as future work. The format-ter specification and implementation are presented in detailsin [27, 28].

6.1 Media object pre-fetching

Based on the execution plan, the formatter can pre-fetcha component content in order to improve the presentationquality and to reduce the probability of temporal and spatialinconsistencies. In order to get the media object data content,the formatter must negotiate the QoS it needs with the wholeplatform environment (operating system, network, hyperme-dia server, etc.). Unfortunately, complete QoS guarantees isstill a broad open issue.

Based on QoS guarantees, if any are obtained, the for-matter can estimate the worst delay to obtain a data content.This, however, can be much larger than the real delay, sothe formatter can make use of other estimations. For exam-ple, CMIF initially estimates the delay based on heuristicsinvolving data size. When a pre-fetching action is executed,the actual time it takes is saved and used as an estimate forfollowing presentations.

When a document presentation has a lot of user inter-actions, pre-fetching is less effective. Again, when a pre-sentation is played several times, statistics can be storedabout which user-dependent relations are taken most often,to guide the selection of presentations to be first prepared.

Based on all the mentioned factors, heuristics should bedeveloped to guarantee the QoS of a document presentationand minimize elastic time adjustments necessary to main-

L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system 131

tain extrinsic consistency. Good heuristic proposals are stillneeded.

6.2 Object placement

In order to guarantee the QoS of a presentation, a hyper-media server must distribute its objects for minimizing itsaccess delay and maximizing its effective bandwidth. Someresearches approach the problem focusing on a specific mat-ter. For example, [9] proposes an object placement strat-egy based on object access probabilities and on object sizes.The problem is how to estimate such probabilities. Note thatheuristics to estimate these probabilities are closely relatedwith the ones used in pre-fetching. As another example, [37]proposes an indexing scheme based on multi-dimensional(spatial) data structures for efficient handling of queries re-lated to spatio-temporal relationships. Other database worksdiscuss the efficient storage of versions of the same object[4].

Indeed, object placement must involve all issues relatedto versioning, cooperative work, spatio-temporal synchro-nization, document access frequency, etc. A general algo-rithm or heuristic is thus necessary and is still an open issue.

Also the definition of a complete query language thatallows not only content queries but also structure queries isnecessary. Again, there are some proposals for specific top-ics, for example, for querying version structures, queryingspatio-temporal structures and querying context structures.Structure query is closely related to the object placement is-sue and also needs a solution. Note also that structure queriesare closely related with the automatic generation of docu-ments mentioned in Sect. 2.

6.3 Elastic time computation

In order to guarantee document consistency, the expected du-ration of an object presentation should be computed basedon cost minimization. The elastic time computation may bedone during compile time. Some possible algorithms are dis-cussed in [3, 20]. However, the problem is much more com-plex when this computation must be done in real time. Thiscan happen when the executor has to shrink or stretch anobject presentation to compensate for some unpredictableplatform delay. When this is done, the expected duration forremaining objects to be presented should be computed again,and in real time. That is why the few systems that implementelastic time adjustment only do it at compile time.

In the current version, HyperProp does not implementpre-fetching of media objects, object placement heuristics toguarantee QoS of a presentation and elastic time computationat execution time.

7 HyperProp and the WWW

As mentioned in Sect. 6, the goal to specify a well-definedinterface between the formatter controllers and the execu-tor is to allow incorporating commercial editors/viewers topresent the content of NCM nodes. Following this approach,

a controller for the Netscape browser was implemented, inorder to present HTML nodes defined in NCM. This will al-low WWW documents to incorporate NCM features, throughthe mapping of an HTML page to an NCM HTML node.This section briefly discusses the HyperProp/WWW integra-tion.

An NCM HTML node is a specialization of the text nodeclass. This class has a URL that specifies the location of anHTML file as its content attribute. Each region of an HTMLnode anchor has two parameters: name and position. Namecorresponds to the text defined by the anchor and positionindicates the number of characters from the beginning of theHTML file to the region defined by the anchor. The nodedescriptor contains a descriptor object whose attributes areused for dynamically building a cascading style sheet [21] topresent the node content. The definition of an HTML nodeanchor region depending on the number of characters mightseem a problem for anchor maintenance. Indeed, changes inthe node content may lead to inconsistent anchors. However,based on the name attribute, the system automatically checksthese inconsistencies and disables links touching these an-chors. The system authoring tool also offers mechanisms forverifying and automatically correcting anchor inconsisten-cies. Figure 12a presents an example of an HTML node (allits attributes) and Fig. 12b presents the content of the fileidentified by the URL. Although not show in the figure, itis worth noting that in the content of an HTML node theremay be other embedded anchors and links (WWW anchorsand links). However, NCM anchors and links are separatedfrom the content, as illustrated in Fig. 12a.

The NCM controller for the Netscape browser was im-plemented using Java and JavaScript languages and using theLiveConnect facility of Netscape for integrating these lan-guages. We also implemented in Java a version of the struc-tural view and a version of the HyperProp spatio-temporalformatter described in Sects. 5 and 6, respectively. Fromthese implementations it became possible to download theHyperProp authoring and formatting environment as an ap-plet and to obtain a first prototype of the HyperProp systemintegrated with the WWW. The integration between openhypermedia systems (OHSs) and the WWW has been mo-tivated by trying to solve WWW’s limitations, using moresophisticated and powerful hypermedia models, while ex-ploring the Web large distribution and standards.

Among other features, the integration between Hyper-Prop and the WWW already allows

– separation of links from data content in WWW docu-ments;

– creation of references in HTML pages where write accessis not granted;

– definition of compositions to structure WWW documents(allowing the addition of contextual links and reuse ofstructures of nodes and links in these documents);

– visualization of NCM documents in HTML browsers;– definition of links between NCM and WWW documents,

generating NCM/WWW integrated documents;– visualization of WWW, NCM and NCM/WWW doc-

uments structure, allowing users to know which linksreference a certain node;

132 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system

– separation of presentation characteristics from data con-tent (allowing different presentations for the same ob-ject); and

– user navigation through graphical views, and navigationthrough guided tours and trails (that maintain the docu-ment navigation history).

A detailed discussion about the integration of HyperPropwith the WWW can be found in [27]. Several other proposalsfor integrating OHSs and the WWW can be found in theliterature, from which we can highlight DLS [5], Chimera[1], Hyper-G [2], HyperDisco [38] and DHM/WWW [10].

The DLS client interface was designed to support theuse of Netscape for communication with DLS servers. Thenature of the client interface and the way it communicatesdepend on the particular platform being used. Hyper-G re-quires a special browser (and also a special text documentformat, HTF) in order for users to achieve all benefits of thesystem, including the ability to create links and collections.DHM/WWW and HyperProp/WWW present a platform-independent solution based on the Netscape browser. DHMalso presents two other platform-dependent solutions, oneusing Netscape plugins and another using Internet Explorer.None of the related work offers a graphical tool for helpingauthoring and user navigation as illustrated by the Hyper-Prop structural view.

We are currently integrating the following NCM facilitiesto the WWW: support for version control, for cooperativework and for the definition of n-ary relations that could alsospecify spatial and temporal synchronization among nodes.Our goal is to put all these facilities together in the Webof today, without having to change the implementations ofWWW clients and servers. Other efforts have been madeto support spatio-temporal synchronization in the Web, ex-emplified by the SMIL recommendation [17]. Also, effortshave been made to provide version control and cooperativework, exemplified by the WEBDAV group proposal [18].

Another problem to be faced is to implement a platform-and browser-independent solution for the NCM and WWWintegration. Unfortunately, this is not possible yet, as evi-denced by all related work of other OHSs. However, a nat-ural tendency is that WWW browsers become more opened,and the Java language offers more features, allowing a moregeneral solution in the near future.

Finishing this section, it is worth mentioning that we arecurrently integrating SMIL with NCM. The main goal is tohave a proposal for an extended SMIL incorporating NCMfacilities.

8 Final remarks

This paper considered some issues on the development ofmultimedia and hypermedia authoring and formatting tools.It presented the HyperProp solutions examining the currentstate of the art and discussing a set of research challengesthat still need to be addressed.

The first prototype of the HyperProp hypermedia systemwas completely developed in C++. We are currently workingon a Java prototype. All NCM classes, a graphical author-ing tool (structural view), a spatio-temporal formatter andmechanisms for version control are already implemented in

name: hyperprop.html

content:http://imperatriz.telemidia.puc-rio.br/telemidia/projects/hyperprop/hyperprop.html

anchors:[0]: default anchor (whole node)[1]: (Global View, 1080)[2]: (Objectives, 1144)[3]: (Partners and Sponsors, 1207)[4]: (Project History, 1281)[5]: (Staff, 1349)

descriptor: Descriptor Default 1

name: Descriptor Default 1

position: absolute;top: 0px;left: 20px;color: black;font-family: arial;font-size: 14px;font-variant: small-caps;

a

<html><head><title>HyperProp Project</title></head>

<body background="../../images/fundo2.jpg" bgcolor="#FFFFFF">

<TABLE CELLSPACING=0 CELLPADDING=0 ><TR ALIGN=CENTER VALIGN=CENTER><TD width=100%><CENTER><IMG SRC="../../images/telemidi.gif" ALT="TeleMídia Lab"HEIGHT=49 WIDTH=480></CENTER></TD></TABLE>

<TABLE CELLSPACING=0 CELLPADDING=0 WIDTH="100%"><TR ALIGN=CENTER VALIGN=CENTER><TD ALIGN=CENTER VALIGN=CENTER WIDTH="100%"><IMG SRC="../../images/bar.jpg" HEIGHT=4 WIDTH=330></TD><TD ALIGN=CENTER VALIGN=CENTER WIDTH="30%"><IMG SRC="../../images/projlogo.gif" HEIGHT=48 WIDTH=151></TD></TR></TABLE>

<p><h3 align="center">Hyperprop Project - Open Hypermedia System and Applications</CENTER></H3><p>The HyperProp project intends to build an open hypermediasystem to incorporate facilities for the treatment of multimediaand hypermedia documents, in applications distributed or not.<p>To become more familiar with HyperProp project, you can selectone of the below topics:<p><img src="../../images/blt_blue.gif" space="30">Global View<p><img src="../../images/blt_blue.gif" space="30">Objectives<p><img src="../../images/blt_blue.gif" space="30">Partners andSponsors<p><img src="../../images/blt_blue.gif" space="30">Project History<p><img src="../../images/blt_blue.gif" space="30">Staff<p></body></html>

b

Fig. 12. aExample of an HTML node and its descriptor.b HTML contentidentified by the URL, without embedded anchors and links

the new prototype, allowing the integration with the WWW.An XML DTD for NCM allows declarative specifications ofNCM documents. Our next step is to implement the temporaland spatial views in order to have all three views working to-gether, as it is provided by the first prototype of HyperProp.The definition of spatial and temporal multipoint relations isalso a feature to be integrated into the WWW. We are alsoworking on real-time consistency checking and elastic timealgorithms to be included in our formatter.

L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system 133

Acknowledgements.This work was supported by the Brazilian Telecommu-nication Enterprise (Embratel) and Brazilian Research Foundations (CNPqand FINEP).

References

1. Anderson K (1997) Integrating open hypermedia systems with theWorld Wide Web. In: Bernstein M, Carr L, Osterbye K (eds) Proceed-ings of the Eighth ACM International Hypertext Conference, 1997,Southampton, UK. ACM, New York, NY, pp 157-166

2. Andrews K, Kappe F, Maurer H (1995) Serving information to theWeb with Hyper-G. Comput Networks ISDN Syst 27(6): 919-926

3. Buchanan M, Zellweger P (1993) Automatic temporal layout mech-anisms. In: Rangan P (ed) Proceedings of ACM Multimedia, 1993,Anaheim, Calif. ACM, New York, NY, pp 341-350

4. Carey M, DeWin D, Richardson J, Shekita E (1986) Object and filemanagement in the EXODUS Extensible Database System. In: Kauf-mann M (ed) Proceedings of the Twelfth Int. Conf. On Very LargeData Bases, 1986, Kyoto, Japan. Morgan Kaufmann, Los Altos, Calif.,pp 91-100

5. Carr L, Roure D, Hall W, Hill G (1995) The Distributed Link Service:a tool for publishers, authors and readers. In: Goldstein I, Vezza A (eds)Proceedings of the Fourth International World Wide Web Conference,1995, Boston, Mass. O’Reilly and Associates, pp 647-656

6. Casanova M, Tucherman L, Lima M, Rodriguez N, Soares L (1991)The Nested Context Model for hyperdocuments. In: Walker J (ed) Pro-ceedings of Hypertext, 1991, San Antonio, Texas. ACM, New York,NY, pp 193-201

7. Fades P, Feng Q (1996) Multilevel visualization of clustered graphs. In:North S (ed) Proceedings of Graph Drawing, 1996, California, USA.Springer, Berlin, pp 101-112

8. Eades P, Feng Q, Lin X (1996) Straight-line drawing algorithms forhierarchical graphs and clustered graphs. In: North S (ed) Proceedingsof Graph Drawing, 1996, California, USA. Springer, Berlin, pp 113-128

9. Ghandeharizadeh S, Ramos L, Asad Z, Qureshi W (1991) Objectplacement in parallel hypermedia systems. In: Lohman G, SernadasA, Camps R (eds) Proceedings of Hypertext, 1991, Texas, USA, pp243-254

10. Gronbaek K, Bouvin N, Sloth L (1997) Designing Dexter-based hy-permedia services for the World Wide Web. In: Bernstein M, Carr L,Osterbye K (eds) Proceedings of the Eighth ACM International Hy-pertext Conference, 1997, Southampton, UK. ACM, New York, NY,pp 146-156

11. Haake A (1992) Cover: A contextual version server for hypertext ap-plications. In: Lucarella D, Nanard J, Nanard M, Paulini P (eds) Pro-ceedings of the European Conference on Hypertext, 1992, Milan, Italy.ACM, New York, NY, pp 43–52

12. Haake A, Hicks D (1996) VerSE: towards hypertext versioning styles.In: Marshall C, Bernstein M, Poltrock S (eds) Proceedings of Hyper-text, 1996. Washington, DC. ACM, New York, NY, pp 224-234

13. Halasz F (1988) Reflections on NoteCards: seven issues for the nextgeneration of hypermedia systems. Commun ACM 31(7): 836-852

14. Hardman L, Bulterman D (1995) Towards the generation of hyperme-dia structure. In: Proceedings of the First International Workshop onIntelligence and Multimodality in Multimedia Interfaces, 1995, Edin-burgh, UK. ftp://ftp.cui.nl/pub/mmpapers/immi1.ps.gz

15. Hardman L, Bulterman D, Rossum G van (1993) The Amsterdam Hy-permedia Model: extending hypertext to support real multimedia. Hy-permedia J 5(1): 47-69

16. Hardman L, Worring M, Bulterman D (1998) Integrating the Amster-dam Hypermedia Model with the standard reference model for intelli-gent multimedia presentation systems. Comput Stand Interfaces 18(6–7): 497-508

17. Hoschka P, et al. - Synchronized Multimedia Work Group of the WorldWide Web Consortium (1998) Synchronized Multimedia IntegrationLanguage (SMIL) 1.0 Specification - W3C Recommendation. Availablein http://www.w3.org/TR/REC-smil

18. IETF WEBDAV Working Group (1999) WWW Distributed Authoringand Versioning (webdav). Available in:http://www.ietf.org/html.charters/webdav-charter.htmI

19. Jourdan M, Layaida N, Roisin C, Sabry-Ismail L, Tardif L (1998)Madeus, an authoring environment for interactive multimedia docu-ments. In: Effelsberg W, Smith B (eds) Proceedings of the ACM Mul-timedia Conference, 1998, Bristol, UK. ACM, New York, NY, pp 267–278

20. Kim M, Song J (1995) Multimedia documents with elastic time. In:Zellweger P (ed) Proceedings of ACM Muttimedia, 1995, San Fran-cisco, Calif. ACM, New York, NY, pp 143-154

21. Lie H, Bos B (1999) Cascading Style Sheets level 1 - W3C Recom-mendation. Available in http://www.w3.org/TR/REC-CSS1

22. Muchalual-Saade D, Soares L (1996) Fisheye views for compoundgraphs. Technical Report of the TeIeMıdia Laboratory. PUC-Rio, Riode Janeiro, Brazil

23. Muchaluat-Saade D, Soares L, Costa F, Souza G (1997) Graphicalstructured editing of multimedia documents with temporal and spa-tial constraints. In: Thalmann N, Thalmann D (eds) Proceedings ofthe Multimedia Modeling - MMM, 1997, Singapore. IEEE ComputerSociety, Los Alamitos, Calif., pp 279-295

24. Nang J, Kang S (1997) A new multimedia synchronization specifi-cation method for temporal and spatial events. In: Georganas N (ed)Proceedings of the IEEE International Conference on Multimedia Com-puting and Systems - ICMCS, 1997, Ottawa, Canada. IEEE ComputerSociety, Los Alamitos, Calif., pp 236-243

25. Osterbye K (1992) Structural and cognitive problems in providing ver-sion control for hypertext. In: Lucarella D, Nanard J, Nanard M, PauliniP (eds) Proceedings of the European Conference on Hypertext -ECHT,1992, Milan, Italy. ACM, New York, NY, pp 33-42

26. Perez-Luque M, Little T (1996) A temporal reference framework formultimedia synchronization. IEEE J Sel Areas Commun (Special Issue:Synchronization Issues in Multimedia Communication) 14(1): 36-51

27. Rodrigues R, Muchaluat-Saade D, Soares L (1998) Composite nodes,contextual links and graphical structural views on the WWW. J BrazComput Soc (Special Issue on World Wide Web) 5(2): 31-44

28. Rodrigues R, Scares L, Souza G (1997) Authoring and formatting ofdocuments based on event-driven hypermedia models. In: Sarikaya B(ed) Proceedings of the IEEE Conference on Protocols for Multime-dia Systems and Multimedia Networking, 1997, Santiago, Chile. IEEEComputer Society, Los Alamitos, Calif., pp 74-83

29. Rossum G van, Jansen J, Mullender K, Bulterman D (1993) CM1IFed:a presentation environment for portable hypermedia documents. In:Rangan P (ed) Proceedings of ACM Multimedia, 1993, Anaheim, Calif.ACM, New York, NY, pp 183-188

30. Sarkar M, Brown M (1994) Graphical fisheye views. Commun ACM37(12): 73-84

31. Santos C, Soares L, Souza G, Courtiat J (1998) Design methodologyand formal validation of hypermedia documents. In: Effelsberg W,Smith B (eds) Proceedings of ACM Multimedia, 1998, Bristol, UK.ACM, New York, NY, pp 39-48

32. Scares L, Casanova M, Rodriguez N (1995) Nested composite nodesand version control in an open hypermedia system. Int J Inf Syst (Spe-cial Issue on Multimedia Information Systems) 20(6): 501-519

33. Scares L, Souza G, Rodrigues R, Muchaluat-Saade D (1999) Version-ing Support in the HyperProp System. J Multimedia Tools Appl 8(3):325-339

34. Sugiyama K, Misue K (1991) Visualization of structural information:automatic drawing of compound digraphs. IEEE Trans Syst Man Cy-bern 21(4): 876-892

35. Vazirgiannis M, Mourlas C (1993) An object-oriented model for inter-active multimedia applications. Comput J 36(1): 78-86

36. Vazirgiannis M, Stamati I, Trafalis M, Hatzopoulos M (1998) In-teractive multimedia scenario: modeling and rendering. In: Pro-ceedings of Multimedia track of ACM Conference - SAC, 1998.ftp://fpt.dbnet.ece.ntua.gr/pub/papers/publish/1998/VTSH98.ps.gz

37. Vazirgiannis M, Theodoridis Y, Sellis T (1998) Spatio-temporal com-position and indexing for large multimedia applications. MultimediaSyst 6(4): 284-298

38. Wiil U, Leggett J (1997) Workspaces: the HyperDisco approach toInternet distribution. In: Bernstein M, Carr L, Osterbye K (eds) Pro-

134 L.F.G. Soares et al.: Modeling, authoring and formatting hypermedia documents in the HyperProp system

ceedings of the Eighth ACM International Hypertext Conference, 1997,Southampton, UK. ACM, New York, NY, pp 146-156

39. Willrich R, Saqui-Sannes P, Senac P, Diaz M (1996) Hypermedia doc-ument design using the HTSPN model. In: Courtiat J, Diaz M, SenacP (eds) Proceedings of the Third International Conference on Multi-Media Modeling - MMM, 1996, Toulouse, France. World Scientific,Singapore, pp 151-166

40. Yankelovich N, Meyrowitz N, Dam A van (1985) Reading and writingthe electronic book. IEEE Comput 18(10): 15-30

Luiz Fernando G. Soares receivedthe B.Sc. degree in electrical engineeringfrom the Catholic University of Rio deJaneiro, PUC-Rio, Brazil. He received,from the same university, the M.Sc. de-gree in electrical engineering in 1979and the Ph.D. degree in computer sci-ence in 1983. He has worked as a re-searcher at COBRA (Brazilian Comput-ers S/A) and at the IBM Scientific Centerin Rio de Janeiro. He is currently a Pro-

fessor at the computer science department of the Catholic University of Riode Janeiro, Brazil. His research interests are in multimedia and hypermediasystems and in high-speed networks, in which areas he has several booksand papers published.

Rogerio F. Rodrigues is a PhDcandidate in computer Science at theCatholic University of Rio de Janeiro,PUC-Rio, Brazil. He received the B.Sc.degree in computer engineering in 1994and the M.Sc. degree in computer sci-ence in 1997, both from the PUC-RioUniversity. He has worked as a systemanalyst at the Braziliam Oil Company(Petrobras) from 1994 to 1995. He hasbeen a researcher at the TeleMıdia Lab-oratory in the Department of Computer

Science of the Catholic University of Rio de Janeiro since 1996. He iscurrently a professor of the Computer Networks specialization course atthe same university. His research interests include temporal and spatialsynchronization models for multimedia and hypermedia systems, open hy-permedia systems, version control mechanisms in hypermedia models andhigh-speed networks.

Debora C. Muchaluat Saade isa PhD candidate in Computer scienceat the Catholic University of Rio deJaneiro, PUC-Rio, Brazil. She receivedthe B.Sc. degree in computer engineer-ing in 1992 and the M.Sc. degree incomputer science in 1996, both from theCatholic University of Rio de Janeiro.She has worked as a system engineer atthe Exxon Oil Company from 1992 to1994, as a researcher at the IBM Scien-tific Center in Rio de Janeiro from 1993to 1994 and as a system engineer in the

Cindam Bank from 1994 to 1995. She has been a researcher at the TeleMıdiaLaboratory in the Department of Computer Science of the Catholic Uni-versity of Rio de Janeiro since 1995. She is currently a professor of thecomputer networks specialization course at the same university. Her re-search interests include visualization of hypermedia structures, version con-trol mechanisms and temporal and spatial synchronization models for mul-timedia and hypermedia systems and high-speed computer networks.