Tangible Objects: Connecting Informational and Physical space

Tangible Objects: ConnectingInformational and Physical space

Peter Bøgh AndersenDepartment of computer science,Aalborg University

Palle NowackDepartment of computer science,Aalborg UniversityMaersk Institute,University of Southern Denmark.

1 Introduction: Where do you want to go today?

Information technology has changed the role space and time plays in ourlives. The Internet connects people that are geographically far apart, Tim-buktu is only a mouse click away, virtual reality replaces the real space with alife-like simulacrum, and augmented reality smears an informational coatingover real space.

As many writers have noticed, the relation between our representations andthe physical reality we live in is changing fundamentally. In this paper, weshall try to describe this change by means of three dimensions: conceptual,physical and informational dimensions.

Our main example will be a train station, so we start by observing the sta-tion from our three perspectives. From the conceptual perspective we all havean informal idea about the artefacts, events and activities in play at the sta-tion. From the physical perspective actual trains are entering and leaving cer-tain platforms to certain times (delayed and on-schedule). The informationalperspective is given by the signs, screens, timetables, etc. that are available topassengers and staff; information about the physical events. The conceptualperspective ties together the available information and the physical phenom-ena and facilitates understanding and planning of our activities.

Currently the boundaries between the perspectives are fixed according tothe available technology, but in the near future, we will see much more per-sonalized and mobile applications of the same information provision. That isthe world of pervasive computing. Pervasive computing is a variant of aug-mented reality. A computer (informational) model of the station has beenconstructed and the idea is to superimpose a view of this model over themental eyeglasses of the passenger, so that he can “see” the phenomena of thestation from another more informative perspective. The major thing requiredis that the system displaying the informational model “knows” the locationand orientation of the passenger and the various trains, coaches, platforms etc.

Peter Bøgh Andersen & Palle Nowack 2

What happens is very simple: we have two spatial coordinate systems, onedelineating the real space containing trains, platforms etc., and the other oneholding the computer model of the station. The user’s real position in the firstcoordinate system is fed into the computer, and the station is displayed to theuser as it would have looked, had the model been located in the user’s realcoordinate system.

Thus, in augmented reality, a computer model is displayed on top of ourview of reality, as it would have looked, had it really been at the chosen loca-tion. A classical example of augmented reality is the technique of projectingthe hidden wiring of a machinery onto the spectacles of the repair mechanicas they would appear to him, had they been visible. Virtual reality uses thesame technique of aligning two coordinate systems; only in this case themodel is displayed, as it would have looked to the user had he been located inthe model coordinate system.

From the point of view of software engineering, the new thing is that thephysical relations between the model, its reference and the user are systemati-cally exploited.

This paper presents an analysis of these phenomena, and makes two points:

(1) We suggest various additions and refinements to the modelling tech-niques of software engineering in order to cope with the changed rela-tionship between representations and physical space. In our approachwe apply the conceptual framework associated with tangible objects asdescribed by (May et al. 2001).

(2) We pinpoint the kind of change computer based signs will undergo as aconsequence of this new technology: a whole new set of indexical signswill emerge.

However, before we do that, we will present existing everyday examples ofthe same interaction between representation and physical space to get a firstshot at possible solutions.

2 Indexical signs

As indicated by the introduction, virtual and augmented reality implies achange of the relation between space/time, interpretations and representa-tions. In this section we shall start to characterize this change by discussingexisting examples of the phenomenon. We shall use semiotics as a frame ofreference.

Representations or signs are physical objects or processes (representamens)that are used in a special way: they are taken to stand for something otherthan themselves (their object), and they cause a reaction — an interpretant

Tangible objects 3

that relates the representamen to its object— in the perceiver. This reactioncan be the production of a new sign (e.g. a verbal comment) but it can alsoconsist in performing an action or refraining from performing an action, orforming an abstract concept or rule.

The relation between object and representamen can be of three types:iconic (the representamen is similar to the object), indexical (the represen-tamen and object inter into a causal relation) or symbolic (the representamenis related to its object by pure convention).

The signs we are interested in contain strong indexical elements because ofthe increased interaction between the model and its surrounding physicalspace.

In some traditional indexes, the representamen is attached permanently to aphysical location; its object is determined by its location whereas its inter-pretant is determined by convention. In Fig. 2.1, the representamen is inter-preted as an assertion, “This place is called Århus”, but the object referred tois determined by the physical location of the sign. If located in Aarhus Cen-tral Station, the proposition is true, but if the sign is moved to HamburgHauptbahnhof, the proposition expressed by the self-same sign is false.

Fig. 2.1. Naming. “Aarhus H” = “This place is called “Århus”.


Representamen Object

Interpretant

Fig. 2.2. Analysis of indexical sign

Interpretants are of different types; for example, the sign “Struer” in Fig. 2.3should be taken not as a statement but as a promise: “This train will be goingto Struer”. But again the object the railroad company promises to take toStruer, i.e. the train, is determined by the location of the sign: the object is thetrain the sign is attached to.

Fig. 2.3. Promising. “Struer” = “This train will be going to Struer”.

Finally, the sign “crossing prohibited” in Fig. 2.4 is a prohibition “You areforbidden to cross here” where the location part of the sentence, here, is de-termined by the location of the sign.

Tangible objects 5

Fig. 2.4. Regulating and controlling. “Crossing prohibited” = “You are forbidden to crosshere”. Physical and symbolic control.

Signs can not only be attached to locations but also to times. Fig 2.5 is takenas a promise that “At this time trains will be leaving for Struer from theseplatforms” where the time is fixed by the real time and the location is deter-mined by the placement of the monitors. Thus, the physical space we are in-terested in is really the space/time continuum.

Fig. 2.5. Attaching signs to time. “11.18, Struer Regionaltog…” = “At this time trains areleaving for Struer from these platforms”.


Thus, there is nothing new in exploiting the physical relation between objectand representamen. We have always decorated our surroundings with repre-sentamens relevant to particular locations.

The new thing is that this relationship is no longer static but dynamic. Mo-bile technology can sense its own location, and, given information about otherobjects at this location, it can present representamens of these objects to theuser.

Object (trains) Representamen (monitors)

Interpretant (promises, prohibitions, etc.)

Time and space

Fig. 2.6. Indexical signs.

In static signs like Fig. 2.2, the representamen so to speak incorporates part ofthe object, since the interpretation is based on the co-occurrence of both ofthem.

In dynamic signs like Fig. 2.5, the representamen has access to its changingspatial and temporal context and can adapt accordingly, exchanging one ob-ject for another, dependent upon the time and its location. Furthermore, someobjects are able to physically change their representamen. These three causalpaths are shown in Fig. 2.6.

Some dynamic indexes only incorporates a causal interaction between ob-ject and representamen. This is the case in process control. For example, therudder angle display on a ship bridge can describe the rudder angle because asensor attached to the rudder keeps sending signals to the display (Fig. 2.7).Conversely, the steering wheel (representamen) can change the rudder angle(the object denoted by the wheel position).

Tangible objects 7

Object (rudder angle) Representamen (rudder angle dislay)

Interpretant (assertion about rudder angle)

Fig. 2.7. The rudder angle influences the rudder angle display

In the next section we shall look at central concepts from systems design anddiscuss how they can be changed in order to cope with the changed relation-ship between representamen and physical context.

3 Models and domains

Most software development methodologies (e.g. OOA&D as described byMathiassen et al., 2001) emphasize the importance of software models; weconstruct models of the software implementation, the usage context, theproblem domain, and sometimes even of the development context, before weembark on the actual programming. Models are signs denoting various do-mains; they support our understanding, and they can be applied as a commu-nicative vehicle for controlling the software development processes. In Fig.3.1 (adapted from Jacobson et al, 1999) we illustrate the role of modelling ina software development process. For the case of simplicity, we have chosento focus on two of the human roles involved in a software development effort,the user and the developer. Other examples could include the customer/client,the system maintainer, the system administrator, etc. (we refer to Jacobsen etal. 1998 for a more in-depth discussion).

In the following we explain and exemplify four different domains onewould need to model in software development. As an example we considerthe task of developing an information system for a railway station as intro-duced in Section 2.


User DeveloperSystem

Application Domain Development Domain

Problem Domain

Solution Domain

AD Model

PD Model

SD Model

DD Model

User DeveloperSystem

Application Domain Development Domain

Problem Domain

Solution Domain

AD Model

PD Model

SD Model

DD Model

Fig. 3.1. Models and domains

Starting from the left side of the illustration, a user (or more precisely a userorganization) is supposed to interact in a certain way with the system to beconstructed; we term this the application domain. In the example the user istypically a passenger, although the station staff also could be users of (parts)of the system. As a part of the analysis effort, it is common to construct mod-els denoting the users’ interactions with the system, e.g. by describing use-cases (cf. RUP, Jacobson et al., 1999). An example would be a passengerwalking up to the signpost (the screen) in Figure 2.5 looking for informationabout trains leaving from this particular platform. He checks the screen in or-der to make sure that the destination (Struer) and the departure time (11.18) isrelevant for his demands. Another example would be a passenger sitting in atrain entering the station. From his window he can see the station as depictedin Figure 2.1. He checks whether he has reached his destination, and he alsohas the possibility of checking the current time. Hence, a use case would de-scribe how the passenger interacts with the station information system.

The problem domain is the part of the real world the system is supposed tocontrol, administer, regulate, etc. The problem domain can be modelled withspecial signs, such as entity-relations diagrams or object-oriented class dia-grams. Because we want to communicate with the user about the correctnessof our models, it is important that they reflect the user’s own understanding(e.g. use his/her vocabulary and conceptual relations). Hence, the models ofthe domains are depicted as “the way the user think” about the domains. Inthe example the objects in the problem domain would comprise differenttypes of trains (regional, intercity, or express trains), the current time, depar-ture times, arrival times, platforms, tracks, destinations, different types ofcoaches (standard, business, rest/silent, smoking, non-smoking).

Tangible objects 9

Similarly, we can focus on the role of the developer, and identify the ac-tivities the developer is engaged in, and his perspective on the system to beconstructed. The developer interacts with the system in the development do-main. Typically, software development methodologies model exactly thisdomain: activities (requirements elicitation, analysis, design, implementation,test, deployment), mental category of activity (construction, understanding,change, reuse), modelling tools (e.g. UML). The software development ap-proach chosen for this specific project together with the developer’s previouswork experience reflects the conceptual framework for the development pro-ject. Typically, the development domain is not modelled explicitly becausethe education and experience of the developers are considered to be suffi-ciently coherent to facilitate a manageable process. However, when dealingwith changing staff, maintenance, or further development (additions and/orchanges) of legacy systems, the importance of having an explicit developmentdomain model becomes significant in order to ensure the overall big pictureand coherence in the project.

The concepts the developer uses to think about the system to be developedare in what we call the solution domain. Examples include architectures,frameworks, patterns, components, libraries, programs, procedures, variables,if-statements, etc. The models of the solution domain are what we usually callthe software design. Hence in the example the solution domain would com-prise the design (the model of the implementation) of the railway station in-formation system.

4 Models and referent systems

In the following we outline object-oriented modelling , which is a crucial partof any object-oriented software engineering methodology. The modelling tra-dition is closely associated with the so-called Scandinavian school of object-orientation, cf. DELTA (Holbæk-Hansen et al., 1975) and BETA (Kristensenet al., 1983).

One normally distinguishes between a referent system (RS) and a modelsystem (MS). This distinction is a result of applying a perspective; we selectwhat to consider in the referent system and what to consider in the modelsystem. Any system can be either a referent system or a model system: it is arole played by the system. The purpose of modelling is to get an understand-ing of the referent system through the application and examination of themodel system. The model system exposes certain properties of the referentsystem. A referent system typically is implicit, whereas a model is explicit.


A software development project includes several such modelling activities,cf. the domains described in the previous section: problem domain, applica-tion domain, development domain, and solution domain. For each domain weidentify what to consider and what not to consider as being part of the systemto be modelled.

Modeling

Interpreting

DescriptionsConcepts

Abstraction Specification

Phenomena Representation

Referent System Model System

Fig. 4.1. Referent and model system

A fundamental human role is the modeller: the person that defines and appliesthe modelling perspective. This implies selecting the referent system and themodel system, and selecting the relation between them. Typically in a largeproject, it is not the case that the person constructing the actual model is theonly person interpreting the model. The models created must yield the “cor-rect” understanding, when applied to interpret the referent system, i.e. themodeller role can be divided into the model-constructer and the model-interpreter. In large system development projects, the systems analyst willbuild the object-oriented model of the problem domain, and then he will handover these models to the system designers, who will design the solution.However for the case of simplicity, these roles are often described as a singlerole.

As part of the perspective applied when observing a system from a referentpoint-of-view, we consider phenomena and concepts in the referent system.Phenomena can be classified/categorized into concepts, and concepts can beexemplified by phenomena. Concepts can be related in whole-part hierar-chies: a concept can be decomposed into a set of simpler concepts, and a setof concepts can be aggregated to form a new concept. Concepts can also berelated in is-a hierarchies: a concept can be a specialization of another con-cept, and a concept can be a generalization of another concept

Another fundamental human role is thus “the abstracter”: the person thatapplies the concept formation processes generalization/specialization and de-

Tangible objects 11

composition/aggregation, as well as the abstraction processes of exemplifica-tion and classification.

In the model system we describe concepts by means of classes and we rep-resent phenomena by objects. When a number of phenomena in the referentsystem can be classified into a concept, the corresponding class can be ap-plied to generate the corresponding objects.

The third fundamental human role is “the generator“: the person or tool thatgenerates objects based on the class descriptions. The generator also manipu-lates (combines/decomposes) class descriptions to form new class descrip-tions. The generator can also reverse-generate by forming class descriptionsbased on objects.

As examples we consider the problem and solution domains.When doing analysis we try to identify phenomena and concepts from the

users’ point of view. In the example we could e.g. construct the (not veryclever) model of the problem domain shown in Fig. 4.2.

Location Train

Station Platform Coach Event

Departure Arrival

Fig.4.2. Classes and associations in the railway model written in UML notation. The ar-rows have the following standardized meaning: large arrows denote a generalization rela-

tionship, where the specialized concepts points to the generalized concept; small arrows de-note concept associations (other than generalization and aggregation).

When doing design we try to identify the required concepts, and create corre-sponding phenomena. As part of an object-oriented design, we almost alwayscarry over a modified model of the problem domain into the implementation(typically termed the “the model component”). For example, we could chooseto implement the Train and Events phenomena as active objects (simple proc-esses or perhaps agents), and we could choose to create the location hierarchyas a relational database, whereas we would design the Coach phenomena tobe represented in the system as a traditional (non-active) object. More im-portantly, we design the patterns of interactions between the different repre-sentamens (objects, components, agents, processes, database records), and we


organize the design of the system to yield properties such as maintainability,flexibility, and reusability.

Referent systems can be found in different types of domains. We can dis-tinguish between focus domains and activity domains (generalized fromOOA&D, Mathiassen et al., 2001). The two previous examples (analysis anddesign of the train system) are examples of focus domain models.

F o c u s D o m a i n

S o f t w a r e S y s t e m


A c t i v i t y D o m a i n


A c t i v i t y D o m a i n

A c t i v i t y D o m a i n M o d e l

F o c u s D o m a i n M o d e l

A c t i v i t y D o m a i n M o d e l

F o c u s D o m a i n M o d e l

Fig. 4.3. Focus and activity domains

When modelling the focus domain we focus on the conceptual framework ofthe referent system (e.g. the problem domain or the solution domain). Whenmodelling the activity domain (e.g. the application domain and the develop-ment domain) we focus on the human processes associated with the focusdomain: how do we as humans interact with the physical representamen (thecomputerized model) of the focus domain?

Every referent system is only a single perspective on the real/perceivedworld. The modelling perspective guides this selection process.

Let us now apply the semiotic framework introduced in Section 2 to themodelling concepts above.

The software engineering concept of a model is clearly a representamen,something tangible used to refer to something else. In one interpretation, thereferent system is a conceptual phenomenon, i.e. it is the particular way wechoose to relate the model to reality: the selections we make, the perspectivewe apply, and the concepts we form. Thus, the referent system is the inter-pretant of the model. If the model is an UML diagram, such as Fig. 4.2, theinterpretant consists of the UML concepts, such as class, object, association,etc. If the model is a diagram of a piece of software, then the interpretantcould be design patterns, and the object the actual existing code. As men-tioned in the beginning of this section, the role of representamen (model) andinterpretant (referent system) are roles that can be played by any phenome-

Tangible objects 13

non. For example, a computer system can be the object of a design patternmodel, but itself be a representamen (a model) referring to a train station.

5 Suggestions for enhancements

The tradition described above has two problems in dealing with the new tech-nology. Both of them relate to an unclear relation to physical reality.

1. It is not clear whether the referent system is a physical object or a men-tal construct. The occurrence of concepts in the referent system indi-cates the latter interpretation if we believe that concepts are not “outthere”, whereas the term phenomena supports the former interpretation.But these two interpretations are incompatible — how can physicalphenomena and concepts form a system? However, “phenomena” canalso mean “our perception of the things ‘out there’”, but in this case thephysical world is not represented in the model at all.

2. The location of the model does not play an important role in the frame-work. The choice of the device on which to locate the model is mainlyseen as an implementation issue concerned with efficiency and stability.In particular, change of the location of model is outside the framework.

These two features of the tradition constitute a major problem since the newtechnology is characterized by systematically exploiting the physical relationbetween the referent and the model, and between the model and its location.

In Fig. 5.1 we have added a physical object to the referent system and themodel, we have connected both object and model to the fabric of time andspace, so that both know their own location and the time, and we have addedthe possibility of object and representamen physically influencing one another(compare to Fig. 2.6).

Object (physical referent) Representamen (model)

Interpretant (refrerent system)

Time and space

Fig. 5.1. Referent and model system with indexical properties added.


These amendments are necessary in order to cope with the new situation (thissituation has been normal for many years in process control where computermodels can sense the controlled plant via sensors and change it via actuators).However, the diagram can be generalized to acquire even more explanatorypower. Adding time and space does make the model sensitive to the physicalspace in which it is located, but there are other contexts that are just as rele-vant for understanding current changes in technology.

One of these can be called the informational space. It consists of all logicalpathways that connect models and which allow them to interact. Models arenot only able to move physically because the devices they reside on moves,they can also “move” in the informational space from one device to another.

The most clear example is mobile agents (White 1996). The user creates asmall piece of software and instructs it to gather information or perform tasksof a certain kind. The agent is frozen (its current state variables are recorded)and it is sent out in the world to do its job.

In the proposal for Mobile Agents, a special habitat at the destination end isnecessary. The Mobile Agent enters this habitat, comes to life, and beginsexecuting.

If the information is not found at the current place, the agent can travel onto other places — or, if the tasks is deemed large, it can spawn children andsend them out in different directions in order to perform searches in parallel.

Finally, it is not only the model and the physical object that interact with acontextual space, this is also true of the conceptual part of the triangle, theinterpretant. It too interacts with a larger social context of related concepts.

In summary, the complete model will contain representamen, object andinterpretant each of which can interact with physical, informational, and con-ceptual contexts and is able to move in the corresponding spaces (Fig. 5.2).Thus, a model can move both in physical space, because its device moves,and in informational space via the associations it contracts with other models.Based on this speculative approach of trying to combine object-oriented mod-elling with computer semiotics, we end up with a model that resembles themodel of tangible objects described in May et al., 2001.

Tangible objects 15

Object (physical referent)

Representamen (model)

Interpretant (Referent system)

Physical context

Informational context

Conceptual context

Fig. 5.2. Basic model of signs and their context

6 Self-reference

A third new feature is inherent in the notion of embedded software, i.e. soft-ware that is embedded in a physical object it controls. Normally we assumethat the model is disjunct from its reference system — the model so to speakstands at a distance and represents its object. In embedded technology, themodel may be located inside the object it denotes1. This means that some em-bedded systems tend to be self-referential since they are a part of the objectsthey represent (Fig. 6.1).

Fig. 6.1. Magellan 750M Mobile navigation system denoting its own position. “I amhere”.

1 The observation is due to Bent Bruun Kristensen.


Although this creates a rather abnormal relation between model and referencesystem, seen from an OOA point of view, the phenomenon is well-known inprocess-control. For example, the rate of turn indicator on a ship bridge signi-fies the speed with which the ship, and thereby the indicator itself, turns, andthe radar shows the position of the ship, including the radar itself. Since themodel-representamen represents its object, and since the object includes themodel, it follows that the model must represent itself either explicitly or im-plicitly.

The kind of mobile technology subsumed under the heading “tangible ob-jects” may also invite self-reference. Since, by definition, tangible objectsenter into changing informational, physical and social spaces, and must beable to adapt their behavior meaningfully to the changing contexts, they needto be able to distinguish representamens of their environment from repre-sentamens of their internal state. Dynamic evolving systems, such as www-servers, already have to be aware of and analyze their informational environ-ment, e.g. which helpers and plug-ins are currently available on the machinethey are running on. With tangible objects, the ability to analyze the physicaland social environment is added. This means that the system-environmentdistinction gains importance (Luhmann 1984). The system must differentiatebetween itself and the environment: where am I?

7 Using the Model: Associations and Habitats

In the following sections we shall show how to use the model defined above.The scenario lies in the future, although possibly not very distant.

7.1 Scenario

As Joe wakes up Tuesday morning, his alarm device informs him that thetrain towards Struer is planned to run as scheduled. The night before he usedhis wireless PDA to request a notification of this information (the represen-tamen moves from Timetable System to the PDA. The PDA is sensitive to thetemporal context and holds a model of Joe’s travel schedule).

As he showers he thinks about the upcoming meeting in Struer, and whencooking his breakfast he activates the memo-recording system of the houseand records a number of voice messages that clarify points in his presentation(the representamen moves from the sound medium to the recording system).

The house alarm system notifies him that the city bus to the train stationwill be arriving 5 minutes later than scheduled. He has an extra doughnut,selects a different tie, and as he exits the building he is notified that he forgot

Tangible objects 17

his PDA. He returns and picks it up, and as he exits again, the interpretedvoice memos are downloaded as text to the PDA (bus information moves tothe house alarm that is sensitive to the temporal context; the House System issensitive to the spatial location of PDA and Joe; the memo system is sensitiveto the location of the PDA).

On the bus he looks over the slides for his presentation and makes a fewcorrections according to the memos from the shower. The PDA beeps as theyapproach the station; time to get off (the PDA is sensitive to its location).

As Joe enters the station through the main entrance, he slips his PDA intothe pocket. Don’t want to fall over peoples’ legs as he did the last time hetried to read the morning paper while running to the train.

However, the PDA is still working, and as he approaches a large screenwith departure information, the display changes, and the train for Struer (forwhich he has a reservation) is displayed together with other departures (thedisplay is sensitive to the physical location of the PDA and creates an infor-mational pathway to it when it is close; it adapts its display to the travel in-formation it receives from the PDA).

He gets the PDA out of the pocket, and a small map showing his current lo-cation, the departure platform, and a path for getting there is displayed. In thelower right corner a timer is counting downwards: he still has 8 minutes to getthere (the PDA is sensitive to the social significance of its physical locationand the actions normally associated to this space and time).

Down with the PDA again and he walks towards the train. At one point hetakes the wrong direction, and the PDA notices it and displays the correctway. He ignores it, as he just wanted to get another coffee to go on the train.

Finally he reaches the platform, and the PDA now shows him where hiscarriage will be located when the train arrives (same as before).

He waits, the train arrives, and as he enters, the ticket information is veri-fied with the local train system (the ticket system is sensitive to the physicallocation of the PDA and creates an informational pathway to it).

As he finds his seat (on his own!) the PDA is updated with a map of thetrain and a city map of Struer. Joe falls asleep.

7.2 Associations

In the scenario we have assumed the following:

• Models and Referent systems: the PDA holds models of the travelschedule. The Time Table System and the PDA hold models of the so-cial use of specific locations.


• Physical context: the PDA is sensitive to its temporal and spatial loca-tion, the House System to the temporal context.

• Object: The House System is sensitive to the physical location of thePDA and Joe. The Memo System is sensitive to the location of thePDA. The Railroad Display and Ticket System are sensitive to proxi-mate PDAs.

• Informational context: the following pathways are created: PDA –Timetable System, House Alarm System – Bus System, Memo-System– PDA, Display System – PDA, Ticket System – PDA.

Fig. 7.1 summarizes the associations in physical and informational space as-sumed by the scenario.

Time

PDA House System

Memo System

Time table system

Display System

Ticket System

Bus System

Space

JoeMemo system

Fig. 7.1. Summary of physical and informational associations in the scenario. Full ar-rows: physical associations, dashed arrows: informational associations.

We see that we can have physical associations alone, as when the HouseSystem discovers that Joe but not the PDA is leaving the house. We can alsohave informational associations alone, as when the House System accessesthe Bus System and warns Joe that the bus is late. And we can have both atthe same time, where the Display and Ticket Systems sense the presence of aPDA and request travel information from it.

7.3 Habitats

We have also seen that some of the devices must hold a model of activitiesappropriate to particular locations and times. For example, the Display Sys-

Tangible objects 19

tem must know that when travellers are located in the Station Hall, they areprobably on the way to catch a train and therefore may need informationabout arrival and departure times. Furthermore, the PDA too must contain amodel of what happens in Station Halls, otherwise it would not be able tochoose to display a map with a path to the correct platform.

A space is thus more than just GPS information about latitude and longi-tude. Similarly, sensible use of time coordinates requires a model of whattakes place in that period of time, e.g. a model of the travel schedule.

Adopting the terminology of tangible objects (May et al., 2001) we shallcall such units habitats. A habitat is thus a segment of space/time that has as-sociated to it a set of socially standardized action possibilities. A model of ahabitat must specify the spatio-temporal coordinates and the associated actionpotentials of the habitat.

Non-computerized habitats are already a part of our everyday life. Most ofthe spaces that surround us are designed for a small set of actions, and ex-clude many others: dining rooms are for eating, bedrooms for sleeping, andkitchens for cooking.

In a railroad station signs control the movements and activities of the pas-sengers. Fig. 7.2. requires passenger to stand alone to the right, Fig. 7.3 de-fines the space as an eating place, and the signs in Fig. 7.4 distinguish be-tween toilets (to the left) and the ticket counter (to the right).

Fig. 7.2. Sign post located above escalator. Only single persons standing to the right areallowed.

Thus, habitats are really signs that use architecture and signposts to denotespecific activities and where the interpretant consists in performing exactlythose activities.

According to our general model of tangible objects, habitats can be com-puterized and in this capacity contain an executable model of its activities.Furthermore, it can be sensitive to the objects denoted by this model: peopleand devices, and it can sense the time to differentiate between actions appro-priate for some times but not others.

Habitats could be seen as a “docking” facility for movable devices. Whatactually will happen depends upon the collection of models residing in thehabitat and those brought along with the movable device. If the habitat offers


a service the device does not support, this service is disabled, and similarlywith services allowed by the PDA but not supported by the habitat.

Fig. 7.3. This place is for eating Fig. 7.4. This place is for buying tickets

We have the same situation in non-computerized habitats. On the one hand,my desire and ability to eat cannot be satisfied at the ticket counter: here onlyticket-selling and information giving can take place. On the other hand, sell-ing tickets requires me to able to buy them which I cannot do if I have forgotmy wallet.

With the words of Lind(2000) one can say that I bring with me a set of ca-pabilities and the habitat offers a set of opportunities. Only those capabilitiesthat are supported by corresponding opportunities can be realized.

The railroad station hall is a habitat where the supported actions includepassengers moving to the right platform at the right time. Therefore Joe’sPDA was able to download map information of this kind and display it tohim. There is a match between the capabilities of the PDA and opportunitiesoffered by the railroad station.

Joe’s PDA also has capabilities to display his ticket to the correct authori-ties, but the hall does not hold corresponding opportunities so ticket controldoes not happen here. However, the habitat at the platform where passengersare supposed to embark and disembark trains does offer a facility for in-specting Joe’s PDA when triggered by spatial contiguity.

Tangible objects 21

8 The conceptual context

What does it mean for a tangible object to interact with its physical, informa-tional, and conceptual context?

The first two ones are easy: a tangible object interacts with its physicalcontext if it can receive and process information about the physical propertiesof the context, e.g. spatial properties like position. A tangible object interactswith its informational context when it communicates properly with informa-tion services. But what does it mean that it interacts with its conceptual con-text, and what is, by the way, a conceptual context?

The first thing to note is that the object does not interact with concepts atall. It is the user that does so. The conceptual part of the tangible object is theinterpretant, so the context must contain other Interpretants with which it caninteract. This again means that habitats must be signs. This is clearly the casewith Fig. 7.3 and 7.4. In both cases, posters, sign-posts and decoration can beconventionally interpreted as assertions that here the passenger can getsomething to eat or buy tickets.

Conceptual interaction thus means interaction between the interpretationassigned to the habitat and that assigned to the tangible object. If Fig. 7.3 says“Eating is possible here” the appropriate response of the user is to buy a hot-dog but he can also just sit down at a table. If Fig. 7.3 says “Eating is man-datory here”, the latter is not allowed without buying something.

Thus, the proper functioning of the tangible object requires the habitats tobe interpretable signs too. If the platform is a habitat for ticket control, then ofcourse it must sense the physical presence of the PDA and it must be able toestablish informational contact to it. But, in addition, it must present informa-tion to the user that it is in fact a place for ticket control, since otherwise theactions of his PDA would be unintelligible to him.

9 Conclusion

Let us now return to the OOA/OOD methodology from Section 3-4. In thesesystem development methods, the accessibility space and the physical spacewere two unrelated entities. The accessibility space, for example, is used todesign object-oriented systems: which objects have access to which otherobjects? In which ways do they interact? Physical space, on the other hand, isa matter of implementation: on which machines should which objects reside?In principle and according to theory, informational space should be definedindependently of physical space (although practice shows another picture).

The new thing is that physical space acquires significance for informationalspace. Physical space is no longer merely a means for realizing informational


space, but determines in a significant way which paths should exist in the in-formational space. The connection between the Ticket System and the PDAwas triggered by physical contiguity and the download from the Memo Sys-tem to the PDS was triggered by the PDA leaving the house. The map shownat Joe’s PDA in the railroad station is downloaded from the Railroad Systemqua habitat for passengers looking for the correct train, but the connectionwas only opened by Joe being physically present in the habitat.

One way of capturing this changed role of physical space in relation to ac-cessibility space is to say that physical space has become part of the repre-sentamen that produces the interpretation, in the same way as the physical lo-cation of the signpost “Århus Central Station” in Fig. 2.2 contribute to theinterpretation of the letters on the signpost: one location makes the statementof the signpost true, another one makes it false.

Consider again Joe’s leaving his house without his PDA and the HouseSystem warning him. The warning sound only makes sense because Joe issimultaneously performing a physical movement that may land ham in an un-desirable situation if no warning is issued. It is the combination of Joe’sphysical movement and the symbolic warning that makes sense. Had thewarning sounded when Joe was watching television, he would had consideredit an error.

Thus, the revolution in computer technology we may see in the next yearscan be summed up in one sentence:

• Physical space begins to contribute to the interpretation of informa-tional space

Or even shorter with a concept from Section 2:

• Computer systems become indexical signs.

10 Acknowledgements

Thanks to Daniel May and Bent Bruun Kristensen for ideas and exciting dis-cussions.

11 References

Holbæk-Hansen, E., P. Håndlykken, K. Nygaard (1975). System Descriptionand the Delta Language. Norwegian Computing Center, Publ. No. 523.

Jacobsen, E. E., B. B. Kristensen, & P. Nowack (1998) Models, Domains,and Abstractions in Software Development. Proceedings of International

Tangible objects 23

Conference on Technology of Object-Oriented Languages and Systems(TOOLS Asia’98), Bejing, China,.

Jacobson, I., G. Booch, J. Rumbaugh (1999). The Unified Development Proc-ess. Addison-Wesley.

Kristensen, B.B., O.L. Madsen, B. Møller Pedersen, K. Nygaard (1983). Ab-straction Mechanisms in the BETA Programming Language. In Proc. 10thACM Symposium. Principles of Programming Languages.

Lind, M. (2000). Actions, functions and failures in dynamic environments.Center for Human Machine Interaction. CHMI-8-00.http://www.cs.auc.dk/~pba/ReportList

Luhmann, N. (1984). Soziale Systeme. Frankfurt am Main: Suhrkamp.Mathiassen, L., A. Munk-Madsen, P. A. Nielsen, J. Stage (2001). Object-

Oriented Analysis & Design. Marko.May, D.C., B. B. Kristensen & P. Nowack (2001). Tangible Objects - Mode-

ling in Style. Technical Report R-01-5004 (ISSN 1601-0590), AalborgUniversity.

White, J. (1996). Mobile Agents White Paper.http://www.genmagic.com/agents/Whitepaper/whitepaper.html

Tangible Objects: Connecting Informational and Physical space

Documents

Transcript of Tangible Objects: Connecting Informational and Physical space