ORIGINAL ARTICLE

Thomas Binder Æ Giorgio De Michelis

Michael Gervautz Æ Giulio Jacucci

Kresimir Matkovic Æ Thomas Psik Æ Ina Wagner

Supporting configurability in a mixed-media environmentfor design students

Received: 1 November 2003 / Accepted: 7 May 2004 / Published online: 29 July 2004� Springer-Verlag London Limited 2004

Abstract In many environments, the landscape of spaceand artefacts is evolving and changing with the tasks athand. Integrating digital media and computation inthese environments has to take into account the fact thatpeople will configure space functions and tools accord-ing to the situation, organising use in unexpected ways.In this article, we present and discuss how the issue ofconfigurability is dealt with, in a series of field trials withdesign students. The aim of these trials was to construct,for architecture and interaction design students, amixed-media environment for inspirational learning.The results include physical infrastructure in space andin furniture as integral parts of the interaction technol-ogy and the creation of composite representations called‘‘mixed objects’’, which blend digital and non-digitalmedia in one design artefact. Configurability has to besupported at different levels (infrastructures, artefacts,functions) and across the physical and digital realms.

Keywords Physical interfaces Æ Mixed mediaenvironments Æ Configurability Æ Boundaryobjects Æ Mixed objects

1 Introduction and background

Computing environments in the future will be populatedby a rich and diverse set of devices and networks, many

of them integrated with the physical landscape of spaceand artefacts. Since Weiser introduced the term ‘‘ubiq-uitous computing’’ and issued his call for computationaltechnologies, being at the same time available and con-figurable for the user in her everyday environment andcalmly fading into the background of attention, it hasbeen clear that new modes of interaction and presence ofinteractive systems have to be sought for [1]. Earlyattempts to take the desktop metaphor of graphicalinterface design back to the real desktops and white-boards by exploring new semantics of interaction waspioneered by Weiser’s group as well as by Buxton andothers [2, 3, 4]. The idea to have a new and more com-plex set of physical handles to digital media promised anextended bandwidth in the interaction between peopleand technology, and, in line with Engelbart’s pioneeringwork on direct manipulation for graphical user inter-faces, a new set of generic interface building blockswould open up a new realm for design of interactiontechnologies.

In parallel to the work of Weiser, Wellner andcolleagues argued for a new and broader interpreta-tion of augmented reality, turning computationalaugmentation into an enhancement of practices wellestablished with the interaction of more mundaneartefacts [5]. Fuelled by ethnographical studies of work,researchers such as Mackay et al. [6] suggested aug-mented environments, where computational resourceswere brought into play as extensions of, for example,the paper flight strips that traffic controllers used tocontrol airplanes as they pass through different trafficsectors. Such an approach is not in opposition to thedevelopment of new interaction modalities but it shiftsthe balance from generic interaction scheme to situ-ated embodiment of interactional possibilities. Ishiiand Ullmer [7] forged these two approaches into awider program for tangible interaction. With theambition to create seamless interfaces between ‘‘peo-ple, bits and atoms,’’ Ishii and Ullmer have expandedthe new field of design to include an integratedreshaping of desks, boards and rooms.

T. BinderInteractive Institute, Sweden

G. De MichelisDISCo, University of Milano–Bicocca, Italy

M. GervautzImagination Computer Services GesmbH, Austria

G. Jacucci (&)University of Oulu, FinlandE-mail: giulio.jacucci@hiit.fi

K. Matkovic Æ T. Psik Æ I. WagnerVienna University of Technology, Austria

Pers Ubiquit Comput (2004) 8: 310–325DOI 10.1007/s00779-004-0294-7

The growing number of experimental ubicom instal-lations has helped to shift the attention of interactivesystems away from individual work settings and towardslarger collaborative environments that were traditionallythe realm of other designers. After some years, duringwhich automatically generated context informationcreated high hopes for how computational technologiescould be made to match the complexity of user behav-iour [8], we are increasingly seeing suggestions such asHP’s Cooltown project for open infrastructures and end-user configurable systems, which may have a lowerintensity of computational monitoring, but, on the otherhand, appear more easily extendable to widespread real-life settings [9].

This new type of extendable systems with openboundaries provides traditional HCI research withimportant new challenges [10, 11]. In many environ-ments, the landscape is evolving and changing with thetasks at hand, and users will most likely make use offunctions and tools in unexpected ways and expect to besupported in doing so. Newman et al. [12] argue ‘‘thatsystems should inherently support the ability of users toassemble available resources to accomplish their tasks.In a world of richly embedded and inter-connectabletechnologies, there will always be particular combina-tions of functionality for which no application has beenexpressly written’’ [12]. This view reflects our experienceswithin the IST Project Atelier1, which develops a set ofarchitectures and technologies in support of inspira-tional learning in two areas—architecture and interac-tion design. We observed how students configured andreconfigured their workspace, as well as the relationshipsbetween design representations. This motivated a designapproach that focuses on configurability as an impor-tant feature of mixed-media environments.

The article starts with an analysis of observed prac-tices of configuring. It then presents three designexamples that highlight different aspects and levels ofend-user configuration. The discussion part summarisesthe diversity of configurations, and investigates in par-ticular how, in augmented or tangible computing envi-ronments, configurations of digital media extend intothe physical setting and into the qualities of the objectscreated.

2 How design students work

For architects, configurability is connected to theproperties of a space. Flexibility connotes the possibilityof relatively simple changes to a design so as to adapt itto different or shifting social uses (e.g. moveable walls).Variability means that a designed space or artefact,without elaborate transformations, can accommodate a

variety of functions [13]. The backstage and the garagestand for spaces in which everything is possible. Butthere are also some quite elaborate examples of config-urability, such as a building under construction by Dillerand Scofidio, which has been conceptualised as ‘‘a fun-damentally updateable, technologically and profoundlyrearrangeable (physically)’’ building setup. The archi-tects used the metaphor of open-source code for mod-elling the building as a space ‘‘capable of beingrewritten, upgraded, reprogrammed, reconfigured toaccomplish previously unanticipated tasks’’ [14]. Thebuilding is to be interactive, not only in the exhibitionand informative parts, but also architecturally. A visitor,entering the building physically or online, can manipu-late parts of the facade. ‘‘Smart walls’’ made from liquidcrystals between conductive film and glass panes allowfor an electric current to regulate the level of transpar-ency and daylight conditions within the building.

Configuring and reconfiguring, although with muchmore mundane means, is also part of students’ practice.Students voice a strong need to adapt their workspace sothat they can exhibit, perform, engage in group work orwork alone, build models, have a nap, make coffee,interact with material and odd objects, etc. At thebeginning of a project, the architecture students set uptheir workspaces. As the project progresses, they becomedense with design material, which is exhibited on thesurrounding walls and on parts of the desk space.Sketches, plans, models, a panorama print of a site andthe computer are all assembled in one desk space(Fig. 1). One student has put two desks on top of eachother to make room for a desktop computer, turning thedesk into a three-dimensional space. Here, configuringspatial elements and tools is very different from thepredesigned mobile and flexible ‘‘individual worksta-tions’’ that have become part of office design [15]. Theseare highly personalised workspaces, whose features and

1 IST-2001-33064 Atelier—Architecture and Technologies forInspirational Learning Environments http://atelier.k3.mah.se/,part of the Disappearing Computer Initiative of the FET area ofthe IST research program. Fig. 1 Workspace of a diploma student

311

components grow over time, expressing students’ iden-tity as well as the progress of their work.

The concept of configuring can also be applied to theways in which students arrange and rearrange designmaterials. Project work proceeds through developing alarge number of design representations. These arecharacterised by the expressive use of materials and aresometimes animated and presented dynamically. As eachof these representations exhibits and clarifies particularaspects of the design, it is important to forge andmaintain connections between them. In many instances,students configure and reconfigure design materials so asto read and reread the configuration from differentpoints of view, and to be able to go back to a momentwhere a particular issue emerged. In the process ofconceptualising and detailing, the design representationsand their relationships change continuously. Arrangingand rearranging material in the workspace is an essentialpart of this process, with the physical landscape ofthings on the walls and tables being in constant move-ment. This resonated with earlier findings on workpractices in (landscape) architecture, where we arguedthat: ‘‘What emerges is that manipulating the presenceand absence of materials and bringing them intodynamic spatial relations in which they can confronteach other are not just a context or prerequisite fordoing the work; rather, they are an integral part ofaccomplishing the work itself. To manipulate the contextis to do the work. Typically, what is important is not justto create or change a document or other materials, butto do so in the presence of and in relation to others’’ [16].

Configuring as a practice is intricately linked to the factthat, in evolving environments, such as the architectureclass or the interaction design studio, the boundaries ofactivities are continually moving [17]. Our observationshelped identify two meanings of configurability:

– Adapting a space to a diversity of uses and identi-ties—which is achieved through e.g. appropriating aspace, personalising it and configuring it in support ofdiverse arrangements, such as solitary work, groupdiscussions, performing and presenting, buildingmodels

– Configurations of artefacts within the physicalspace—with artefacts changing their position in rela-tion to others and different configurations beingexpressive of conceptual, chronological or narrativelinks between them

Embedding digital media in physical environmentscannot be simply understood as an extension of practiceswe observed in the physical world. Things need to bedesigned so as to support ‘‘the ability to improvisa-tionally combine computational resources and devices inserendipitous ways’’ [12]. The following chapters exam-ine several experiments with tangible augmentation ofstudent design studios with the intent to better under-stand the requirements of end-user configuration inmixed-media environments.

3 Mixing media in the design studio

The first widely published examples of tangibly aug-mented environments made it clear that such environ-ments are a much more complex mixed-media settingthan the preceding individual workstations [18, 19].Hereby, they also made it evident that a new set ofbuilding blocks has to be provided to designers in orderto let such environments flourish. Myers et al. [20] arguein a review of past and present toolkits that we probablyhave to operate in a very open field for many years tocome. One of the obvious needs identifiable today is,however, that the design material for leading edge aug-mented environments have to integrate hardware andsoftware in ways that reflects the fact that the digital andthe physical domains are no longer separated in theseenvironments [20]. Greenberg and Chester’s suggestionfor a widget-like toolkit of Phidgets [21], giving designersbuilding blocks to control and combine the operation ofbasic interaction elements like contactors, servomotorsetc., has, in student trials, resulted in interesting newexamples of mixed-media interfaces. In a slightly dif-ferent direction, Ballagas et al. [22] have developed a kitof generalised tangible interaction devices for an intel-ligent room, making it possible to easily design moreaggregate interaction systems. Strommen [23] has sug-gested yet another type of toolkit, linking computationalagent behaviour to an actual embodiment of doll-likephysical artefacts that can be freely placed in the envi-ronment of use. Of particular interest to us is the workon embodiment of electronic tags pioneered by Wantet al. [24] that embodies basic computational function-alities in border objects that can operate both as handlesand as actual media in settings rich in both physical anddigital media.

The problem with most attempts to provide designerswith the components from which mixed-media envi-ronments can be made, however, is that they still appearto be products of engineering labs. As pointed out, forexample, by Djajadiningrat et al. [25], tangibility may benew to computer engineers, but it is certainly a well-known challenge to product designers. Designers willneed toolkits that open up rather than seal off theembodiment of interaction components. Furthermore,other voices from the designer community claim thateven basic computational functionalities have to bemade accessible and mouldable for exploration of theinteraction designer [26]. There may be different ways topursue the exploration of form and function of inter-action elements. Hutchinson et al. [27] report on aplayful and experimental strategy of probing for newinteraction experiences in the home setting by embody-ing basic technological possibilities in ways that can begrasped by collaborating families.

Although inspiring, this approach has the limitationthat the embodiment tends to become too casual and adhoc to form the basis for a genuine genre of interaction

312

design relevant, for example, in design education. In theAtelier project, we gave design students the opportunityto explore and appropriate new combinations andlinkages between digital and non-digital media in on-going projects. As in the work of Want et al. [24], wesupplied them with possibilities to tie digital media tophysical objects through the use of RFID tags. Simi-larly, we provided them with simple applications forassociating barcodes or particular manipulations oftouch sensors to computational resources. Sensors, tagsand barcodes provided a simple way to associate mediafiles with the environment of space and objects. Asso-ciations of physical input, digital media and outputcould be edited and loaded through configuration tablesthat are stored in a database. Associating a physicalinput to digital media can result in a diversity of con-figurations. For example, the input component attachedto a barcode scanner can be configured to read onebarcode as input or a series of barcodes. Moreover, themedia output can be configured to display one or moredigital files on one or more projection surfaces in thespace.

3.1 Mixed-media spaces: sensors, tags and projections

Trials were organised at two sites: at the Master Class ofArchitecture, Academy of Fine Arts in Vienna, and theInteraction Design Studio, Malmo School of Art andCommunication. At each site, groups of students (ineach trial between 10 and 20 students in small groups)were carrying out their practical projects in the Atelierenvironment. The trials lasted between two and four -weeks. The results of the first cycle of trials were dif-ferent configurations of physical inputs and digitaloutputs, and a variety of projection setups. Studentscame up with a wide range of ideas of how to integrateinteractivity in physical objects. They used the space as aresource, e.g. by recreating elements of remote places inthe studio. Multiple projection surfaces were arrangedthrough ‘‘bricolage’’ in the space. Students used bar-codes for creating associations between media files and

parts of a physical model of a building. In Fig. 2 (left),the site represented by the model is a large garden withfruit trees and an old house. The model was animatedwith sound and projected images of interiors, of the oldhouse and of different perspectives onto the surroundinggarden. Touch sensors were also integrated in parts ofthe model (Fig. 2 right) to create interactive models ornavigation artefacts for presentations.

In preparing a project presentation, one of thearchitecture students plotted out her CAD plans withbarcodes on them. In one of her printouts, she hadintegrated the barcodes into a diagrammatic represen-tation (Fig. 3 right corner). She presented her workusing multiple interactive artefacts that triggered theplaying of sound and visual media on a projected screen(Fig. 3 upper right). Barcodes were integrated intoposters, which displayed plans and diagrams (Fig. 3upper left). A physical model of the section of the sta-dium and its surrounding environment was made inter-active with touch sensors (Fig. 3 bottom).

3.2 Configuring spatial elements

The use of multiple projections invited students to usethe space as a resource. Students arranged their ownprojection setups, recreating aspects of a remote place,by, for example, using projection screens and hangingposters in the shape of this place. For example, a studentprojected images of two residential buildings with twoback-projections, which he arranged facing each other.He visualised the transformation of the balconies intoseating arrangements for viewing a soccer game in thespace in-between, performing these changes while theclass was watching. The detailed plans of his interven-tions in the buildings were projected onto the spacebetween them (Fig. 4).

One of our interventions in the physical space was agrid in support of configuring the workspace for differ-ent activities (Fig. 5). The grid facilitated isolatingsmaller partitions of the room to be used for smallergroups and installing diverse projection setups. Most

Fig. 2 a Barcodes placed ontomodels. b Touch sensorsintegrated into model

313

importantly, it supported students in furnishing theirproject spaces in whatever way they wanted, and inrearranging them whenever activities changed. These

arrangements could be performed in a varied topogra-phy in the space, with the possibility to experience thingsfrom above or below.

3.3 A new multimodality in design representations

The design students’ experiments revealed to us theimportance of supporting the spatiality of configura-tions. Students created immersive installations byarranging projections spatially and they introduced anew multimodality in their representations where dif-ferent media were explicitly connected: the physicalmodel through touch sensors, the barcodes printed inposters, tags in cards, all were connected with pictures,sounds and video on multiple projectors. New hybridforms of design representations emerged in the work ofthe students; e.g. mixing issues of scale by shifting be-tween plan drawings and full-scale imagery. The possi-bilities to mix static and dynamic representations werealso explored in projections of live images on scalemodels.

Tagging physical models or drawings with barcodesassociated to digital media was particularly frequent.This bears similarities to the use of barcodes on papercards to activate digital presentations, as suggested withthe paper palette [28]. But unlike the paper palette, thestudents did not develop their use of barcodes for apreformatted type of presentation like a slide show. In-stead, they integrated barcodes as a graphical element intheir drawings and plans, and adopted stage-like prac-tices of projecting images, often onto non-standardisedsurfaces (physical objects, customised projectionscreens), in order to convey particular immersive per-spectives on their project.

Both physical and digital materials were collected andexplored during the students’ design work in a variety ofways. Digital images collected on field trips were usedfor projection on various mockup materials, and imagescould be used for both establishing a context or forcapturing particular interesting textures. In this way,new constellations of physical and digital material wereconcurrently sought for involving relinking and retag-

Fig. 5 Physical infrastructurefor projection spaces

Fig. 3 Configurations of sensors, artefacts, digital media andprojections

Fig. 4 Creating spatial elements with projections

314

ging materials. In the first trial, this was poorly sup-ported by the technology we provided to the students, asthey were not able to make such reconfigurations ontheir own.

In general, the experiments indicated that designwork in mixed-media environments provided the stu-dents with a new and fertile setting for exploring trulymixed-media representations where the most promisingresults were obtained when digital and non-digitalmaterials were forged together into new hybrid objects,often with dynamic behaviour. The use of tags andsensors to invoke digital media meant that existingpractices of sketching and modelling with physicalmaterials were enhanced without having the students toswitch to GUI-type interfaces.

While the experiments with our students werepromising, it also became clear during the evaluationsessions that the students found it difficult to work withthe environment we provided. They complained aboutthe complexity of coupling digital and physical config-urations. They also found it problematic that severalweeks of work remained distributed in the environmentand were not easily portable outside the environment,and could not be documented properly in their portfolioof projects.

In a second round of trials, we addressed theseproblems by developing prototypes that provided basicformats for collecting, configuring and documentingmixed-media representations. We made available morecomplex applications to mix physical and digital objects,physical infrastructures supporting configurations ofspace and configurable furniture. The next section de-scribes the ‘‘Tangible Archive’’; configurable furniturewe used in the trials of the second cycle. The sectionentitled ‘‘Mixed object table’’ describes more complexapplications we developed to mix physical and digitalobjects.

4 A tangible archive

When designers work, they collect inspirational materi-als as different media. This collection of materials formsan important resource base for much of the design work.As we have already mentioned, designing can be seen asa kind of bricolage where different materials are broughttogether to explore and envision design possibilities.Some of the authors have, in a previous project, writtenabout the problems in organising digital materials inways that facilitate a designerly exploration, and havesuggested the ‘‘Wunderkammer’’ as a metaphor for suchcollections [29]. The problem encountered when workingwith mixed-media representations is even more urgent ifthe availability of collected materials cannot be madetangible in the design environment. In the initial trialswith design students, we provided them with the taggingof selected design materials, but as already mentioned,the collection and storing of these materials becamea major obstacle as it depended on our support.

Furthermore, some of the most promising experimentsmade by the students involved setting up and exploringdynamic immersive environments for their design rep-resentations in a scale of 1:1. However, the studentsfound it tedious and often too difficult to sketch earlyideas in this way.

To overcome this problem, we developed, togetherwith a group of master students, what we called theTangible Archive. The Tangible Archive can be seen as aconfigurable working area where designers can store,organise and manipulate design material in differentmedia gathered for a particular project or set of projects.The basic principle is, as in the earlier experiments, touse the tagging of physical objects as a way to connect todigital material and computational resource, but in thearchive with an emphasis on providing components andconfigurable component assemblies that embody moregeneral affordances and interaction schemes. As inGronbæk et al. [30], we search for components, whichcan form the glue in mixed-media objects and extend ourconception of hypermedia as well as provide a basis fornew configurations of the design environment. Our goalwas, however, not to accomplish new types of augmen-tation that can provide a new tooling of conventionaldesign practices as demonstrated, for example, in theWorkspace project [31]. For us, the Tangible Archive isnot meant to be a fixed setup but, rather, a collection ofdesign patterns from which the design students cangradually transform their collected design materials intoexploratory full-scale mockups.

In addition to the work previously cited on tagging,we were inspired by the mediaBlocks project by Ullmeret al. [32], where an environment for manipulatingdigital video can be prototyped by a combination ofcontainer-like media blocks and appliance-like manipu-lators. But unlike Ullmer et al. [32], we did not want totarget a particular kind of media manipulation. On theone hand, we wanted to provide a very open frame forthe exploration of design materials. On the other hand,we attempted to provide the starting point also forinteraction semantics, somewhat in the direction of theconcept of DataTiles suggested by Rekimoto et al. [33].The DataTiles provide a basic set of interactionsemantics combining the (designable) physical form oftiles with an (exemplary) systemic scheme for how tilescan be combined to form more complex interactionpatterns [33]. We did not take it as far in terms ofemergent functionality, but wanted also to address issuesof physical embodiment. In the first version of theTangible Archive developed by a group of master stu-dents, a bookshelf with trays could be filled with phys-ical objects tagged with RFID tags. One or more digitalfiles were associated to each tagged object, and interac-tion points equipped with tag readers made it possible todisplay or print digital material associated to the taggedobject placed on the tag reader. What commands wereexecuted depended on special tagged command cardsplaced at an additional tag reader. The command cardsalso made it possible to copy and delete digital material

315

from the tagged objects. Finally, a digital camera wasassociated with the archive to enable users to add newdigital material to tagged objects.

Compared to earlier student experiments, we foundthat this group, in a more generic sense, had been able totake advantage of affordances of the physical environ-ment, like organising tagged objects in particular trays.They also made concrete embodiments of mechanismsfor connecting, displaying and copying digital materialsassociated to physical objects.

In our research group, we developed the conceptfurther to see if we could use the tangible archive both asa family of design patterns and as a concrete toolkit forother design students to work with. We saw the tangiblearchive as consisting of an overall pattern for keepingand organising collected material that may eventually beturned into mockups of a final design. We suggested tothe students to start their work by establishing thispattern as an actual literal storage for the materials theycollect on field trips etc. Then, on a more fine-grainedlevel, the archive consists of other patterns that we la-belled zones. A zone is an area in the tangible archivewhich combines a set of input and output devices, suchas tag readers and projectors, in order to accommodatecertain activities, such as entering or organising mate-rial. Finally, we developed the first steps towards acollection of interaction semantics for individual inter-action points that could also form the starting point fordesign patterns.

Practically, the Tangible Archive consists of physicalobjects such as models, material samples, pictures or‘‘objets trouves’’ and digital files organised in a hyper-media database, either as individual files or hypermediadocuments or projects. The physical objects are linked tothe digital files by either barcodes or RFID tags. The

basic principle is that every digital file is associated to aphysical object. The Tangible Archive has, as a mini-mum, an organising zone, where digital media associ-ated to the physical objects can be displayed and copied,and an entrance zone, where new digital materials can beentered and linked to physical objects. The archiveframe (Fig. 6) is built up of modules of size 48·48 cmmade of either plywood, transparent Plexiglas or semi-transparent Plexiglas (for projection), all 4 mm in width.The modules can be combined to form cubes, shelvesand vertical or horizontal working or projection areas,in a scale appropriate for working in scale 1:1, accordingto the needs in the project. Modules are joined togethermanually by ready-made joints.

By supplying the students with such a practicalbuilding kit, we wanted to ease the process of trans-forming the initial archive into relevant stages forexploring and presenting design ideas. For the zones weprepared, special plates with holes for placing input andoutput devices and the individual interaction pointscould be moulded on top of a set of basic building ele-ments, consisting of small tagged objects (to be con-nected to other objects) and matrices for the plate holesto signal which objects could go where. The underlyingtechnology, including the infrastructure and hypermediadatabase, was the same as in previous experiments.

4.1 Basic interaction principles—tags and tag readers

In the archive, digital media associated to physical ob-jects can be displayed and manipulated by moving thetag on the physical object to a tag reader. The resultingaction depends on the configuration of the tag reader. Inthe archive, all RFID tag readers are placed stationary

Fig. 6 The archive framepopulated with a diversity ofphysical objects

316

in the archive to form preconfigured action spots,whereas barcode readers are used when digital mediamust be invoked where the barcodes are placed.

Physical objects tagged with RFID tags are, in thearchive, associated to what we call a carousel of digitalmedia (corresponding to a hypermedia documentoptionally consisting of more than one file). The actionassociated to an RFID tag reader will be active, as longas the RFID tag is placed on the tag reader. Physicalobjects (such as print outs with thumbnail pictures andbarcodes) tagged with barcodes are, in the archive, onlyassociated to one digital file (corresponding to an objectin the hypermedia database). The action associated tothe barcode will be active until a new barcode is read bythe barcode reader.

4.2 Entering new materials in the archive

There are two types of entrances to the Tangible Archivethat makes it possible to bring in new digital media (newdigital media may also be generated in the archive bymanipulating other media, but this is not covered here).

The drop entrance is a dedicated work zone, wheremedia from USB-enabled devices, such as digital cam-eras, can be entered by connecting the device and placingan RFID-tagged object on an associated tag reader. Thedevice is treated as an external hard drive and the files onthe drive are stored in the hypermedia database andassociated to the tagged object as a carousel of files.These may later be copied (or deleted) and manipulatedin other zones.

The email entrance is a dedicated work zone madetangible by a printer printing barcodes (possibly alsowith thumbnail icons). E-mails with attachmentse-mailed to a particular address will be parsed by thee-mail entrance service so that individual media files(possibly with meta information) are associated toindividual barcodes. The files may later be copied (ordeleted) and manipulated in other zones.

4.3 The configurator deck of cards

The archive may be used by several people at the sametime. A part of the archive reserved for one particularactivity is called a zone. The simplest zone consists oftwo tag readers and a display. One tag reader identifieswhich media should be displayed/manipulated, and theother tag reader offers the possibility to copy what isdisplayed to the tag placed on this tag reader. There maybe a number of different ways to display media. Theycan be of different forms, like visual or audio. Media in,e.g. a carousel, may be displayed sequentially by placingand lifting the tag from the tag reader, or superimposed,by displaying all media in the carousel or from differentcarousels simultaneously. In order for the people usingthe archive to be able to configure it according to theirneeds, a configurator deck of cards is provided for each

possible type of work zone. The deck of cards has anindividual card for all the components that can becombined (Fig. 7). The card does not only represent aspecific piece of hardware (like a tag reader) but also anassociated service (like display sequentially). The cardsare graphically designed so that, when placed on or closeto the physical component, they give a direct indicationof the configuration. Each deck of cards also has anoverview card briefly stating the function and interrela-tions between components. At a later state, the confi-gurator deck of cards can be used for dynamicconfiguration of the archive. Barcodes on the cards andon the hardware can be subsequently read by a barcodereader to invoke the configuration.

4.4 From archive to mockup

Design students worked with the tangible archive in atwo-week design workshop. During the workshop, thestudents worked in groups to explore what we calledsemi-public places (a public library, a playground in thecity gardens and cafes on a public square) and they wereasked to design an interactive installation that conveyedwhat they found to be interesting qualities of the placesthey had studied.

They made video, audio and still-image recordings ofthe places they visited and they collected physical itemsfrom the area. After an introduction to the tangiblearchive, they built a first version of the archive for thecollected material.

The students used the archive frame to set the scenefor the exploration of their material. The group workingwith playgrounds made a table-like archive where col-lected digital materials were connected to tagged leavesgathered in a heap in the middle of the table. Images andvideos could be displayed on a sphere mounted above

Fig. 7 Tag reader integrated in the furniture (left); buildingdifferent forms of furniture with modules (upper right); theconfigurator deck of cards (lower right)

317

the heap and people were supposed to sit on the floor ina circle around the heap. The group working with cafescreated a closed cinema-like space where one couldwander along the cafes. The group working with thelibrary built a counter-like structure using barcodes andbarcode readers in ways similar to the way library staffregister books.

As the groups continued their work, they experi-mented with both the physical frame and the interactionzones to try out ideas. One group was particularlyinterested in the e-mail entrance and the possibilities ofusing this in connection with a public billboard forbuying and selling stuff between people in the city. Likein the original version of the tangible archive, they madeit up to the people actually present in front of the bill-board to pick up announcements e-mailed to the boardfrom the printer and put it on the board. Theannouncements were printed out at the board with abarcode connected to additional contact information.Another group originally working with the heap ofleaves made a variation of a display zone where twopeople placed at some distance to one another had tocollaborate to put together four pieces of an image wheneach of them only controlled two of the puzzle pieces.

In general, we found that the tangible archive enabledthe students to work in quick iterations with full-scalemockups of interactive installations. The students ten-ded to make smaller variations of the zones we hadprovided for them. They did not develop conceptuallynew zones (even though we encouraged them to do so),and, even with the setups we had demonstrated to them,they tended to work further with those that they alreadyhad a familiarity with. For example, we found that thestudents preferred working with barcodes and barcodereaders rather than with RFID tags. Even if this, tosome extent, may be explained by the fact that thebarcode technology in our setting was more reliable thanthe RFID tag readers, we found that the RFID tagreaders and tags also in the students’ view were moredifficult to embody in their design without considerableefforts to develop appropriate interaction schemes.

5 Mixed objects table

The Tangible Archive and its exploration in the studenttrials addressed the design students’ practice of workingwith mockups in the scale of 1:1. The initial trials alsoshowed a considerable interest among the students inworking with mixed media in connection with tabletop-scale models. Already in the first trials, the architecturestudents used digital imagery to capture and work with avisual texturing of physical 3D models. For the secondround of trials, we wanted to facilitate this way ofworking further by providing a more elaborated settingfor work with tabletop models. Others have beenworking with the augmentation of tabletop models. Forexample, Underkoffler and Ishii [34] have suggested a

luminous-tangible workbench where calculations basedon visual tracking of the physical model is made tangibleby the projection of shades indicating, for example,noise levels or sun shade in the surrounding of themodelled buildings. Schmudlach et al. [35] have beenworking with similar setups where digital technology isused to keep alignment between a digital and a physical3D model [35]. Our purpose was, however, not so muchto invoke computational resources for calculation or torelate to digital models, but rather, to enhance themanipulation of the direct visual appearance of thephysical 3D models. Work in this direction has beencarried out by Bandyopadhyay et al. [36] and by Raskaret al. [37], emphasising the visual possibilities in over-laying physical models with projected digital imagery.Their work does not, however, in the same way as ours,take its starting point in the way design students nor-mally work with tabletop-scale models. For our pur-poses, we developed the Mixed Objects Table. It is atable with a semi-transparent glass tabletop, which is aprojection screen. A mirror is mounted underneath,making a back-projection system. Users can combinethe table with different projection setups. In the Atelierspace at the Academy, for example, several movablelarge-size projection screens are available. Those screenscan enhance the table projection. Moreover, the MixedObjects Table can be coupled with the Texture Brush.The Texture Brush is a tool which makes it possible to‘‘paint’’ objects, such as models or parts of the physicalspace, applying textures, images or video, scaling androtating them. The Texture Brush uses a real brush, buta virtual ‘‘paint.’’ Finally, ARToolKit [38] opticalmarkers can be used in the Mixed Objects Table envi-ronment.

5.1 Table and projection setup

The table itself is a movable (it is equipped with wheels)piece of furniture with an integrated mirror. Two tagreaders and sockets for USB devices, e.g. for web cams,a scanner or a printer, have been integrated into itsframe. The table can be easily adapted to different uses.The simplest one is playing media by projecting them viaa mirror onto the table. Projecting the desktop turns thetable into a workspace, with the peripherals being di-rectly attached to it. Furthermore, it can be personalisedin a way which resembles Naoto Fukasawa’s design‘‘personal skies,’’ which has ‘‘two elements, a chair thatadapts chameleon-like to the clothing of the user, and ameans of personalizing the work environment by pro-jecting a personal ceiling above the desk (it could be animage of the sky in a choice of season or weather con-ditions, or of the home), sending a customized messageto the rest of the office like a screen saver’’ [15]. At theAcademy, students use the table for experimenting withobjects, such as scale models, in a projected environment(Fig. 8). More complex setups include additional pro-jection screens. Surrounding the table with various

318

projection screens hanging on the grid enhances thesimple projection setup, and invites the students toexperiment further. The most complex setup explored sofar is using the table with surrounding projections and aTexture Brush simultaneously.

5.2 Texture brush run-time configurations of textures

Students can position their physical models on the table,e.g. on top of a projected plan, and use the TextureBrush for ‘‘painting’’ the model using various textures.The Texture Brush uses a real brush, which is tracked byan infrared camera. The brush is equipped with a simplebutton as well. The system projects the virtual ‘‘paint’’only where the brush passes by and the button is pres-sed. In this way, the user has an impression of realpainting. The architecture students rarely use simplecolours for painting, but apply coloured textures,snapshots or even videos to their models. The TextureBrush is an application that creates mixed objects [39],where integration of the physical and the digital happenswithin one single object. This notion goes beyond simplyenriching a physical artefact by linking it with content indifferent media. In the case of the Texture Brush, thelink is such that the properties of the artefact itself canbe changed, by applying colour, inserting movement andcontext, and varying its dimension in relation to otherobjects in the physical space.

The students can configure the Texture Brush inmany ways. They can manipulate the brush size andshape by selecting these attributes from a menu bar,

located at the bottom of the projection area, which isconstantly displayed (Fig. 9). Working tools like‘‘polygon fill’’ that are known from applications likeAdobe Photoshop, have been integrated in the TextureBrush application. This allows the students to work withthe Texture Brush much in the same way they are usedto working with applications they know. They canchoose from a number of textures to use, includinganimated textures (videos), and the projection of thetextures can be moved, scaled and rotated. The wholeinteraction is done using the brush as only input device.

Tag readers and RFID tags and barcodes can be usedto load the textures into the system at run time. Anyimage or video stored in the hypermedia database can beused as a texture to paint the objects with. The texturebrush can be used to paint only one side of the model. Ifthe user wants to paint the model from more sides, moretexture brush systems have to be used simultaneously.Some of our students used two systems to paint themodels from the front side and from the top. There isalso an element of physical configurability, with thepossibility of placing physical objects on the table andvarying the background, using projection screens andvarying the ‘‘ground’’ on which the objects are placed byusing the back projection of the table.

5.3 Optical markers: configuring the virtual spacearound an object

Besides simple projections on the table (Fig. 8 left), thespace around a model can be further enriched using

Fig. 9 Configuring the TextureBrush for painting a model(left), on the border of the tableis the menu bar. A paintedmodel from two directions withbackground (right)

Fig. 8 The mixed objects table(left) projecting a CAD planonto the surface of the table.Two tag readers and one of theUSB slots with attachedwebcam (right)

319

ARToolkit optical markers. Optical markers are simpleprintouts which can be placed on the table. A web cam isused to capture the physical model and the markers. Theposition and orientation of the marker relative to thecamera can be estimated. Every marker represents a 3Dobject (such as a tree in Fig. 10), and when the positionand orientation of the marker is known, the objectcorresponding to the marker is added to the image.Students can place the real model on the table, paint itwith the real brush and virtual paint, place a few realmarkers (paper cards), project the ground-level imageonto the table and capture the whole setup with a webcam. What we get, in real-time projected on a display, isthe movie of the composed scene with the 3D objectspopping out of the markers. In this way, we have a realobject standing on the real table, but with the virtualtextures on the object (which have been painted using areal brush). Furthermore, we have a virtual playgroundfor our object, real markers, which do not make muchsense in the real world, and virtual 3D objects corre-sponding to the markers, visible only in the real-timemovie. The wall on the back side of this arrangementmay be simultaneously used for creating contexts againstwhich the model can be viewed. It is part of architects’practice to take pictures of these experimental setupsand to merge them with the other design material.

5.4 The ‘‘configuration poster’’

Configuration of the space with such a variety of pro-jection possibilities is not trivial. Students have to con-figure the system so that specific images are projectedonto the specific projection screen, or used as a texture inthe Texture Brush or as a tabletop image. We havedesigned a simple physical handle to configure the space.The configuration poster uses barcodes to specify thereceiver of a texture or any other input. This approach issimilar to the solution in the Tangible Archive example,but instead of using a dedicated tag reader and a set ofcommand tags, a poster displaying the possible con-nections between inputs and outputs using barcodes canbe used to configure the system. There is a barcode foreach command. In this way, the tag reader and the

barcode reader input can be configured to, for example,display the media file associated to a specific tag orbarcode on either the desktop or to use it as a texturewith the Texture Brush (Fig. 8b). Additional barcodeshave been added to specify printers and projections onthe cave corner as other output components. Also, othertag readers can be connected to other outputs—forexample, one tag reader is connected to the TextureBrush and the other is connected to the desktop pro-jection. These connections between input and outputpersist as long as they are not reconfigured by the stu-dents. The configuration and reconfiguration can beperformed at any time, dynamically changing the setupof the workspace, using only configuration poster, bar-code reader and barcodes that correspond to the imagesstored in the database. This allows students to adapt thesetup to their current needs, while they are working,making the configurability part of their work with thesetup. As the Mixed Objects Table uses the same hard-ware as the Tangible Archive (tag reader, display, etc.),with similar functionality being available, students canchoose the space configuration—table or archive—thatsuits their current working situation best.

5.5 Towards new modelling practices

The Mixed Object Table provided new ways of mixingdigital and physical objects. Students used the table andsurrounding projection screens as a stage for theirmodels. Nine groups of two students used the Atelierenvironment to experiment with virtual textures andbackgrounds for white models of buildings they built.Nearly 1,000 digital pictures and videos were enteredinto the database and were used to paint the whitemodels and background. The groups created severalconfigurations of textures and backgrounds for eachmodel, some of which were conceptual, others veryrealistic, like a video of moving water for a pool (Fig. 9right). The students often were using the Texture Brushfrom two different directions running two separateapplications (Fig. 9 right). After having painted andexperimented for several hours, the student took digitalpictures inside and outside the ‘‘painted’’ models from

Fig. 10 Using optical markersfor inserting a virtual tree into aphysical model

320

different directions. These were then projected on largescreens during final presentations, creating immersivespaces with multiple projections. The students wereenthusiastic about the texture brush but mentionedseveral difficulties in the interviews. They lamented thatthey needed to adjust projectors and surfaces every timethey started working again after another group (tapewas used to mark position and height of projectors). Aseach model was of different size and was painted fromdifferent directions, each group had to rearrange thetable and projectors into different positions (and also theparameters on the projectors were changed due tothe difference in the size of the picture). This resulted inthe impossibility for some groups to restore the sessions,juxtaposing exactly the painted texture and the model.In the presentations, some groups had to paint themodel from the start, and some could show only thepictures they took.

6 Discussion

We began with a notion of configurability, which wasshaped both by the architectural concept of adaptabilityof a space to a diversity of uses and identities, as well asby our observations of ‘‘dynamic spatial relations ofdesign materials’’ as an important aspect of designpractice. The three examples we provided explore dif-ferent aspects of configurability of a mixed-media envi-ronment: associations of inputs, media and outputs;spatiality and integration with artefacts; configuringfurniture and work zones (Tangible Archive); and real-time configuration of mixed objects (Mixed ObjectsTable).

Our first example shows how, using physical inter-faces such as sensors, tags, barcodes and projectionsetups, a space can be configured to form mixed-mediastages. Physical interfaces are integrated in diagrams andphysical models. Projection setups, exposed in the space,create spatial elements that provide a stage for enactingscenarios, performing presentations or travellingthrough media material. In contrast to a dedicatedmeeting setting with one projector and one projectionsurface, our experimentations reveal ways of using thespace as a resource for varying purposes.

The Tangible Archive is an example of a configurableplatform–furniture made of plywood or Plexiglas mod-ules. The furniture can be used as a surface for doingwork (with work zones being reserved for particularactivities), as shelves for storing materials or for pro-jections. Cards are used for defining and loading specificconfigurations for the different work zones. The MixedObjects Table is a platform for creating and manipu-lating mixed-media objects. As in the previous example,configuring is hardly distinguishable from proper use.Configuring and selecting textures to be painted andvirtual objects to be moved on optical markers happensas part of the experimentation process.

In all the described examples, configurability includesinterventions in the physical landscape of space andartefacts. The complex activity of configuring unfolds,and, therefore, should to be supported, on differentlevels and across different aspects of the environment:spatial arrangement (e.g. grid for fixing projection sur-faces), furniture (the Tangible Archive with its modules,the table), the landscape of artefacts (which can betagged, furnished with (hidden) sensors or (visible)barcodes), electronic components and devices (scanners,readers, connecting and plugging input and output de-vices), digital components and their interaction (digitalinfrastructure, associations of inputs, outputs and mediacontent in the database).

This large variety of methods can provoke confusionin participants who are unable to find a rationale to dealwith the new qualities of the space where they act, aswell as in the designers, who miss the compositionalgrammar for creating their devices and arrangements.Even the weaknesses of the space offered to users(recalled briefly in the evaluation sections above) can beattributed to the lack of a conceptualisation shaping thedesign of tangible computing environments. We were,therefore, somehow forced to enter into a discussion ofthe qualities that the artefacts we were designing hadand/or should have. On the one hand, this discussionhas created a deeper understanding of what we are doingin Atelier; on the other, it indicates new possibilities forthe design for configurability that we have not yet pur-sued in our research. In this section, we report briefly thefirst outcomes of this discussion.

6.1 Mixed-boundary objects

Most of the experiments during the first and the secondtrial focus on improving and enriching the presentationof the outcomes of students’ design projects to teachers,visitors and other students. The artefacts we providedfor them were, in fact, used to create absorbing anddynamic environments where what they had done couldbe brought forth to their audience. The grid, the Tan-gible Archive, the Mixed Object Table, the TextureBrush, as well as the physical models and/or projectplans, enriched with barcodes and/or touch sensors areall examples of boundary objects [40] and/or allow thecreation of boundary objects. A boundary object isanything that can help people from different communi-ties to build a shared understanding. Boundary objectswill be interpreted differently by the different commu-nities, and it is an acknowledgement and discussion ofthese differences that enables a shared understanding tobe formed. While it should be immediately clear whymodels and plans are boundary objects, helping visitorsto understand what students do in their projects, con-sidering the artefacts we have created to support multi-media representations as boundary objects requiresadditional clarification. They are boundary objects,since they allow visitors to share with the students the

321

knowledge about their design space (and the constraintsand the opportunities it offers). Also, they support stu-dents in creating boundary objects to represent theirwork. Brian Marick [41] lists some interesting factsabout boundary objects:

– If x is a boundary object, people from differentcommunities of practice can use it as a common pointof reference for conversations. They can all agree thatthey are talking about x.

– But the different people are not actually talking aboutthe same thing. They attach different meanings to x.

– Despite different interpretations, boundary objectsserve as a means of translation.

– Boundary objects are plastic enough to adapt tochanging needs. And change they do, as communitiesof practice cooperate. Boundary objects are workingarrangements, adjusted as needed. They are not im-posed by one community, nor by appeal to outsidestandards.

Our artefacts support this mixture of commonalityand diversity, offering the possibility to move from onerepresentation to another, either changing the level ofabstraction, or changing the supporting medium or,finally, changing the viewpoint. Several different repre-sentations that users can access make reference to oneunique thing (the designed building and/or device, theplanned territory and/or space, etc.).

In our approach, boundary objects are intrinsicallymulti-affordances objects, where commonality is sup-ported by the emergence of one unique object anddiversity by the multiplicity of affordances throughwhich users can access and manipulate it. Consideringthe experiments we have done in Atelier, some of themdeeply adhere to this concept (e.g. the Texture Brush)whilst others have not yet fully developed it (in somecases, any representation seems to have its own life andits links with other representations of the same object arenot highlighted).

Our boundary objects, therefore, are often andshould always be, mixed objects, i.e. objects couplingphysical and digital qualities [39]. Mixed objects arecharacterised, at least, by having both physical anddigital affordances (e.g. the plans enriched with bar-codes), at most, by having mixed affordances (thebuilding model painted with a digital texture). Even theconcept of boundary becomes broader than in its ori-ginal definition by Leigh Star: it refers to the contactline not only between different communities, but alsobetween the physical and the digital, and, as a conse-quence, between the different (spatio–temporal) situa-tions of any participant.

6.2 Openness, multiplicity and continuity

It is possible to continue this discourse at a higher levelof abstraction, focussing on the qualities that mixed–

boundary objects should have. One of the authors of thisarticle, in [42], considers openness, multiplicity andcontinuity as indispensable qualities of what physical,digital and mixed artefacts are becoming in our everydaylife. These qualities appear to be particularly appropri-ate for mixed-boundary objects and settings withevolving physical landscapes and activities.

Openness refers to how accessible and learnable anartefact is, and to its capability of being combined withother artefacts. Moreover, openness refers to the capa-bility of an artefact (an affordance) to have different,potentially unlimited, ways of being used and perceived.Our experience of providing students with simple pro-totypes, helping to extend them and furnish them withmore complex functionality is an example of openness toappropriation and use. Another crucial aspect of open-ness is the possibility for an artefact to be combined withother artefacts. Integrating barcodes, tags and touchsensors in physical models and diagrams helped createinteractive and, in some cases, innovative combinationsof physical and digital objects being perceived and usedin many different ways. The Texture Brush and opticalmarkers, applied in combination with physical objectsand projections, resulted in rather intriguing kinds ofmixed-boundary objects.

Multiplicity refers to the capability of a space orartefact of being made of different components havingdifferent qualities. Multiplicity can be seen in the com-bination of input (sensors, tag and barcode readers,scanners, etc.) and output (displays, printers, projectors,etc.) devices characterising the work-space of the Atelierstudents, and/or in the multiplicity of affordances of-fered by mixed boundary objects.

Continuity refers to the capability of moving fromone affordance to another, from one representation toanother, without changing artefact, without interruptionin space and in time. In some sense, multiplicity andopenness are contradictory as multiplicity creates dis-tinctions and ‘‘thus boundaries between one functionand another, whereas openness breaks down all bor-derlines to encompass all functions in one whole’’ [43].One of the arguments that may guide such integrationsis that the new objects populating augmented placesneed to combine openness and multiplicity throughcontinuity. Continuity can be achieved by putting re-sources on the borders of objects so that the borders actas both separators and connectors [44].

The new types of sophisticated mixed boundary ob-jects we are experimenting with could be achieved byputting resources on their physical borders—on theborders between their physical and virtual components.One of the challenges we are still working on is to beable to support all activities in embodied interactionswithout the use of conventional GUIs. Another chal-lenge is to support participants in carrying out config-urations flexibly, accountably and with ease. Theintegrations of physical interfaces with existing andevolving landscapes of physical objects we described,were aesthetically appealing and inspirational, but some

322

of them were handcrafted and co-developed ‘‘on-the-fly.’’ Although grid and sliding frames supported thespatial configuration of projections, this still requiredartistry and bricolage (which the architecture studentsare quite capable of). The furniture we designed is withwheels and modular, but more flexible ways need to befound of integrating it with electronic components.Integrating sensors into artefacts has improved since ourfirst trials, from wired prototypes to wireless compo-nents, but still need to be customised into (sometimes)fragile solutions. End-user configuration at all the levelswe have identified poses a series of challenges forarchitectural, industrial and interaction design (includ-ing the need for robust and open electronics compo-nents).

6.3 Configuring as staging mixed places

The potential of physical interfaces reaches beyond‘‘mere embodiment.’’ They provide people with themeans for producing configurations that change spati-ality and interactivity, and a physical landscape in waysthat help users experience, explore, present and perform.At the beginning of the field trials, the space was non-intentional and had to be appropriated. This was alsopart of the pedagogy, assuming that a perfectly fur-nished space is often not the best solution for creativework. Students need to appropriate the space, strugglewith its constraints and find their own interpretation andsetup. This is why they found the space almost com-pletely empty, apart from an infrastructure of networks,furniture, grids for projections, tag/barcode readers,computers and other electronic equipment. They wereasked to bring their own artefacts—pictures, videomaterial, scale models, diagrams and collages. Withthese resources at hand, students configured and rec-onfigured space and artefacts to accommodate diverseactivities—from browsing through pictures, to discuss-ing a design concept or performing a scenario of use. Wecan understand these configurations as forming anevolving set of temporary and, in some ways, ephemerallayers onto this neutral, almost empty environment.

This captures the aim of current research on enrich-ing the physical space, which is not solely to provide newfunctionalities for participants, but to transform theirsocial experience at its roots. The space metaphor doesmore than provide a resource for analysing humanbehaviour and designing for it; it also shapes the lan-guage through which we speak about ourselves. Just as itallows designers to describe relationships in socialinteraction, it provides a common framework for thoseactually engaged in interaction. Harrison and Dourish[45] argue that a place is a portion of space ‘‘invested[through language practice; authors’ remark] withunderstandings of behavioural appropriateness, culturalexpectations and so forth.’’ When people share an en-riched portion of space and a language to talk abouttheir experience, they transform the former into a place.

So what could mixed places be? They provide qualities,some of which cannot be found in physical space and areutterly new. One quality of mixed places that emergesfrom our trials is the capability of being reconfigureddynamically and radically. The configurability of a spacedepends on its layout, the design of the infrastructureand the design of the artefacts that populate it. Withregard to the configurability of artefacts, we have arguedthat they would have to combine openness and multi-plicity through continuity, producing the right interplaybetween infrastructures, mixed objects and activity. Asto the configurability of a space, we could learn fromgood architectural design that often plays with anambiguity in the relationship between spatial configu-ration and functional program, where ‘‘the allocation offunctions or uses is malleable, they are fitted into thespatial configuration. While some of them find amplespace, others might have to be squeezed in, overlap,extend into neighbouring spaces, thereby creating‘‘natural’’ connections or meeting ‘‘fixed’’ boundaries.This not only allows to suspend or transgress the usualhierarchy of functions and rooms. Also, the boundariesbetween interior and exterior space are designed aspermeable and fluent’’ [13].

Acknowledgements We are grateful to our co-researchers at theMalmo School of Arts & Communication, the Interactive Institute,the Consorzio Milano Ricerche and University of Milano–Bicocca,the University of Oulu, the Academy of Fine Arts and the Tech-nical University in Vienna, and Imagination Computer Services, aswell as to students and teaching staff in Malmo and Vienna.Finally, we would like to acknowledge Infotech Oulu for sup-porting this research at the University of Oulu. The authors wish tothank the three anonymous reviewers and the meta-reviewer fortheir careful reading of the preliminary version of the paper and forthe many suggestions they gave to improve its final version.

References

1. Weiser M (1999) The computer for the 21st century. ACMSIGMOBILE mobile computing and communications review,vol 3(3). ACM Press, New York, pp 3–11

2. Weiser M (1993) Some computer science issues in ubiquitouscomputing, Commun ACM 36(7):75–84

3. Fitzmaurice G, Ishii H, Buxton W (1995) Bricks: laying thefoundations for graspable user interfaces. In: Proceedings ofthe conference on human factors in computing systems(CHI’95), Denver, Colorado, May 1995. ACM Press, NewYork, pp 442–449

4. Rekimoto J (1998) A multiple device approach for supportingwhiteboard-based interactions. In: Proceedings of the confer-ence on human factors in computing systems (CHI‘98), LosAngeles, California, April 1998. ACM Press, New York, pp344–351

5. Wellner P, Mackay W, Gold R (1993) Computer-augmentedenvironments: back to the real world. Commun ACM 36(7):24–26

6. Mackay WE, Fayard A, Frobert L, Medini L (1998) Rein-venting the familiar: exploring an augmented reality designspace for air traffic control. In: Proceedings of the conferenceon human factors in computing systems (CHI‘98), Los Angeles,California, April 1998. ACM Press, New York, pp 558–565

323

7. Ishii H, Ullmer B (1997) Tangible bits: towards seamlessinterfaces between people, bits and atoms. In: Pemberton S (ed)Proceedings of the conference on human factors in computingsystems (CHI’97), Atlanta, Georgia, March 1997. ACM Press,New York, pp 234–241

8. Salber D, Dey AK, Abowd G (1999) The context toolkit:aiding the development of context-enabled applications. In:Proceedings of the conference on human factors in computingsystems (CHI’99), Pittsburgh, Pennsylvania, May 1999. ACMPress, New York, pp 434–441

9. Kindberg T, Barton J, Morgan J, Becker G, Caswell D, DebatyP, Gopal G, Frid M, Krishnan V, Morris H, Schettino J, SerraB, Spasojevic M (2002) People, places, things: Web presence forthe real world. Mobile Networ Appl 7(5):365–376

10. Bellotti V, Back MB, Edwards WK, Grinter R, Henderson A,Lopes C (2002) Making sense of sensing systems: five questionsfor designers and researchers. In: Proceedings of the conferenceon human factors in computing systems (CHI 2002), Minne-apolis, Minnesota, April 2002. ACM Press, New York, pp 415–422

11. Grudin J (2002) Group dynamics and ubiquitous computing.Commun ACM 45(12):74–78

12. Newman MW, Sedivy JZ, Neuwirth C, Edwards K, Hong J,Izadi S, Marcelo K, Smith T, Sedivy J, Newman M (2002)Designing for serendipity: supporting end-user configuration ofubiquitous computing environments. In: Proceedings of theconference on designing interactive systems: processes, prac-tices, methods, and techniques (DIS 2002), London, England,June 2002. ACM Press, New York, pp 147–156

13. Lainer R, Wagner I (eds) (1998) Connecting qualities of socialuse with spatial qualities. In: Proceedings of the 1st interna-tional workshop on cooperative buildings (CoBuild’98),Darmstadt, Germany, February 1998. Springer, Berlin Hei-delberg New York, pp 191–203

14. Leeser T (2002) Rewriting the museum. In: Frame 24 magazineJanuary–February 2002

15. Antonelli P (ed) (2001) Workspheres: design and contemporarywork styles. The Museum of Modern Art, New York

16. Buscher M, Mogensen P, Shapiro D, Wagner I (1999) TheManufaktur: supporting work practice in (landscape) archi-tecture. In: Bodker S, Kyng M, Schmidt K (eds) Proceedingsof the 6th European conference on computer supportedcooperative work (ECSCW’99), Copenhagen, Denmark,September 1999. Kluwer, Dordrecht, The Netherlands, pp21–40

17. Dourish P (2001) Where the action is: the foundations ofembodied interaction. MIT Press, Cambridge, Massachusetts

18. Ullmer B, Ishii H (1997) The metaDESK: models and proto-types for tangible user interfaces. In: Proceedings of the sym-posium on user interface software and technology (UIST’97),Banff, Alberta, Canada, October 1997. ACM Press, New York,pp 223–232

19. Streitz N, Geißler A, Holmer J, Konomi T, Muller-Tomfelde S,Reischl C, Rexroth W, Seitz P, Steinmetz PR (1999) i-LAND:an interactive landscape for creativity and innovation. In:Proceedings of the conference on human factors in computingsystems (CHI’99), Pittsburgh, Pennsylvania, May 1999, pp120–127

20. Myers B, Hudson S, Pausch R (2000) Past, present, future ofuser interface tools. ACM Trans Comput—Hum Interacti7(1):3–28

21. Greenberg S, Chester F (2001) Phidgets: easy development ofphysical interfaces through physical widgets. In: Proceedings ofthe 14th annual ACM symposium on user interface softwareand technology (UIST 2001), Orlando, Florida, November2001. ACM Press, New York, pp 209–218

22. Ballagas R, Ringel M, Stone M, Borchers J (2003) iStuff: aphysical user interface toolkit for ubiquitous computing envi-ronments. In: Proceedings of the conference on human factorsin computing systems (CHI 2003), Fort Lauderdale, Florida,April 2003. ACM Press, New York, pp 537–544

23. Strommen E (1998) When the interface is a talking dinosaur:learning across media with ActiMates Barney. In: Proceedingsof the conference on human factors in computing systems, LosAngeles, California, April 1998. ACM Press, New York, pp288–295

24. Want R, Fishkin K, Gujar A, Harrison BL (1999) Bridgingphysical and virtual worlds with electronic tags. In: Proceedingsof the conference on human factors in computing systems(CHI’99), Pittsburgh, Pennsylvania, May 1999. ACM Press,New York, pp 370–377

25. Djajadiningrat JP, Overbeeke CJ, Wensveen SAG (2000)Augmenting fun and beauty: a pamphlet. In: Proceedings of theconference on designing augmented reality environments(DARE 2000), Elsinore, Denmark, April 2000, pp 131–134

26. Martin H, Gaver B (2000) Beyond the snapshot from specu-lation to prototypes in audiophotography. In: Proceedings ofthe conference on designing interactive systems: processes,practices, methods, and techniques (DIS 2000), New York City,August 2000. ACM Press, New York, pp 55–65

27. Hutchinson H, Mackay W, Westerlund B, Bederson B, DruinA, Plaisant C, Beaudouin-Lafon M, Conversy S, Evans H,Hansen H, Roussel N, Eiderback B, Lindquist B, Sundblad Y(2003) Technology probes: inspiring design for and with fami-lies. In: Proceedings of conference on human factors in com-puting systems (CHI 2003), Fort Lauderdale, Florida, April2003. CHI Letters 5(1):17–24

28. Nelson L, Ichimura S, Pedersen ER, Adams L (1999) Palette: apaper interface for giving presentations. In: Proceedings ofconference on human factors in computing systems (CHI’99),Pittsburgh, Pennsylvania, May 1999. ACM Press, New York,pp 354–361

29. Buscher M, Kompast M, Lainer R, Wagner I (1999) Thearchitects Wunderkammer: aesthetic pleasure and engagementin electronic spaces. Digital Creativity 10(1):1–17

30. Gronbæk K, Kristensen JF, Ørbæk P, Agger Eriksen M (2003)Physical hypermedia: organising collections of mixed physicaland digital material. In: Proceedings of the 14th ACM con-ference on hypertext and hypermedia (Hypertext 2003), Not-tingham, UK, August 2003. ACM Press, New York, pp 10–19

31. Buscher M, Kramp G, Krogh P (2003) In formation: supportfor flexibility, mobility, collaboration, and coherence. Pers U-biquit Comput 7:136–146

32. Ullmer B, Ishii H, Glas D (1998) mediaBlocks: physical con-tainers, transports, and controls for online media. In: Pro-ceedings of the 25th annual conference on computer graphicsand interactive techniques (SIGGRAPH’98), Orlando, Florida,July 1998. ACM Press, New York, pp 379–386

33. Rekimoto J, Ullmer B, Oba H (2001) DataTiles: a modularplatform for mixed physical and graphical interactions. In:Proceedings of the conference on human factors in computingsystems (CHI 2001), Seattle, Washington, March/April 2001.ACM Press, New York, pp 269–276

34. Underkoffler J, Ishii H (1998) Illuminating light: an opticaldesign tool with a luminous-tangible interface. In: Proceedingsof the conference on human factors in computing systems(CHI’98), Los Angeles, California, April 2003. ACM Press,New York, pp 542–549

35. Schmudlach K, Hornecker E, Ernst H, Bruns FW (2000)Bridging reality and virtuality in vocational training. In: Pro-ceedings of the conference on human factors in computingsystems (CHI 2000), The Hague, The Netherlands, April 2000.ACM Press, New York, pp 137–138

36. Bandyopadhyay D, Raskar R, Fuchs H (2001) Dynamic shaderlamps: painting on movable objects. In: Proceedings of theIEEE and ACM international symposium on augmented reality(ISAR 2001), New York, October 2001, pp 207–216

37. Raskar R, Welch G, Chen W (1999) Table-top spatially-aug-mented reality: bringing physical models to life with projectedimagery. In: Proceedings of the 2nd IEEE and ACM interna-tional workshop on augmented reality (IWAR’99), San Fran-cisco, California, October 1999, pp 64–73

324

38. Kato H, Billinghurst M, Poupyrev I, Imamoto K, Tachibana K(2000) Virtual object manipulation on a table-top AR envi-ronments. In: Proceedings of the IEEE and ACM internationalsymposium on augmented reality (ISAR 2000), Munich, Ger-many, October 2000, pp 111–119

39. De Michelis G (2004) Mixed objects. Appliance Des J(to ap-pear)

40. Star SL (1989) The structure of ill-structured solutions:boundary objects and heterogeneous distributed problemsolving. In: Gasser L, Huhns MN (eds) Distributed artificialintelligence, vol 2. Pitman, London, pp 37–54

41. Marick B (2001) Boundary objects. Available at http://www.visibleworkings.com/analogyfest/marick-boundary-objects.pdf

42. De Michelis G (1998) Aperto molteplice continuo, gli artefattialla fine del Novecento. Dunod, Milano

43. De Michelis G (2003) The Swiss pattada: designing the ultimatetool (with original drawings by Marco Susani). Interactions10.3:44–53

44. Brown JS, Duguid P (1994) Borderline issues: social andmaterial aspects of design. Hum Comput Interact 9(1):3–36

45. Harrison S, Dourish P (1996) Re-Place-ing space: the roles ofplace and space in collaborative systems. In: Proceedings of theACM conference on computer supported cooperative work(CSCW’96), Boston, Massachusetts, November 1996, pp 67–76

325